Analyze PHPUnit test data in New Relic

If you use PHPUnit to manage and run your unit tests, the New Relic PHP agent can automatically capture the test summary results and send them to New Relic as an event where you can query and visualize test data at a glance. This feature was released in version 6.7.0 of the PHP Agent and supports PHPUnit versions 3.7 to 8.x

Enable PHPUnit test events

To enable PHPUnit test events:

1. Find or add the newrelic.phpunit_events.enabled setting in your newrelic.ini file and set it to true.
2. Restart your web server (apache, PHP-FPM, Nginx, etc) for the setting change to take effect.

View available attributes

When enabled, the PHP Agent detects PHPUnit commands and populates New Relic with two event types that contain data for the test suite (named TestSuite) and individual tests (named Test). You can query the data with NRQL queries and build a dashboard of information important to you.

To query your test events, use FROM TestSuite and FROM Test when specifying your SELECT statements:

SELECT uniqueCount(name) FROM TestSuite

Example of a PHPUnit test dashboard created by querying TestSuite and Test events.

Example queries

A dashboard of test summary data allows you to quickly see a snapshot of overall test success as well as dig into failing suites to determine their cause. These examples illustrate the kinds of widgets you can create with both TestSuite and Test event types.

Percent success

What percent of suites or tests are passing?

SELECT percentage(count(*), WHERE successful IS true) FROM TestSuite

SELECT percentage(count(*), WHERE outcome = 'passed') FROM Test

What percent of suites and tests are passing?

Test outcome

What is the breakdown of test outcomes?

SELECT count(*) FROM Test FACET outcome

What is the breakdown of test outcomes?

Test failures

What percent of the time does each test pass?

SELECT percentage(count(*), WHERE outcome = 'failed') FROM Test FACET name

What percent of the time does each test pass?

Test suite failures

What percent of the time does each suite pass, and is that consistent over time?

SELECT histogram((passedCount / testCount)*100, 100, 10) FROM TestSuite FACET name

What percent of the time does each suite pass, and is that consistent over time?

Duration

How long does each test suite take to run, and is that consistent over time?

SELECT histogram(duration*1000, 10, 20) FROM TestSuite FACET name

How long does each test suite take to run, and is that consistent over time?

Linked facets

Because a PHPUnit test suite is linked to individual tests via its run ID, you can use FACET widgets to filter results for a specific test run.

For example, if you created a widget with the most recent failing test suites and linked it to the current dashboard, you could click on a test and the surrounding widgets would update with information for only that test suite run.

FACET widget (top-left) linked to the current dashboard allows you to drill into failing test suites.

In the above example, you can see that by clicking on run 5bb37ccee2a1dbc7, we learn that one of two tests, testFoo, failed. Here are the NRQL queries that made up this example:

Recent unsuccessful suites:

SELECT latest(timestamp), latest(name) FROM TestSuite WHERE successful IS false FACET runId

Last unsuccessful suite:

SELECT host, name, duration * 1000 AS 'duration (ms)', assertionCount, testCount, passedCount, failedCount, incompleteCount, skippedCount, errorCount, riskyCount, warningCount FROM TestSuite WHERE successful IS false LIMIT 1

Last unsuccessful test:

SELECT host, name, testSuiteName, duration * 1000 AS 'duration (ms)', outcome, assertionCount, message FROM Test WHERE outcome != 'passed' LIMIT 1

PHPUnit event attributes

TestSuite and Test events contain the following attributes you can query against:

TestSuite event attributes

TestSuite events include the following attributes:

Attribute	Description
duration	The number of seconds it took for the test suite to run.
assertionCount	The total number of assertions the test suite made.
errorCount	The number of errors reported.
warningCount	The number of tests with warnings. Note that PHPUnit includes these in the passedCount as well.
failedCount	The number of failed tests.
incompleteCount	The number of incomplete tests.
passedCount	The number of passed tests.
riskyCount	The number of tests marked by PHPUnit as risky.
skippedCount	The number of tests that were skipped.
testCount	The number of tests that ran.
runId	A unique identifier that ties the test suite to the individual tests. For example, 861d12cea0d5b923.
successful	A boolean that is true if there were no failures or errors during the test suite run.
name	The name of the test suite.

Test event attributes

Test events include the following attributes:

Attribute	Description
duration	The number of seconds it took for the test to run.
message	Any message associated with the test outcome. For example: Failed asserting that false is true or This test depends on 'StackTest::testFailure' to pass..
assertionCount	The number of assertions the test made.
outcome	The outcome of the test. Options include passed, failed, skipped, risky, warning, and incomplete.
runId	A unique identifier that ties the test to its test suite. For example: 861d12cea0d5b923.
name	The name of the test.
testSuiteName	The name of the parent test suite.

For more help

Additional documentation resources include:

• Building dashboards (how to build and view customized dashboards)
• Using NRQL (how to use NRQL to explore your data)