If you use PHPUnit to manage and run your unit tests, the New Relic PHP agent can automatically capture the test summary results and send them to Insights as an event where you can query and visualize test data at a glance. This feature was released in version 6.7.0 of the PHP Agent and supports PHPUnit versions 3.7 to 5.x
Enable PHPUnit test events
To enable PHPUnit test events for use in Insights:
- Find or add the
newrelic.phpunit_events.enabledsetting in your newrelic.ini file and set it to
- Restart your web server (apache, PHP-FPM, Nginx, etc) for the setting change to take effect.
Exception messages are collected and sent with events. Also, if you use PHPUnit's
--disallow-test-output option, which flags tests that print output as "risky," the test event will include the offending output.
View available attributes
When enabled, the PHP Agent detects PHPUnit commands and populates New Relic Insights with two event types that contain data for the test suite (named
TestSuite) and individual tests (named
Test). You can query the data with NRQL queries and build a dashboard of information important to you.
To query your test events, use
FROM TestSuite and
FROM Test when specifying your
SELECT uniqueCount(name) FROM TestSuite
A dashboard of test summary data allows you to quickly see a snapshot of overall test success as well as dig into failing suites to determine their cause. These examples illustrate the kinds of widgets you can create with both
Test event types.
- Percent success
What percent of suites or tests are passing?
SELECT percentage(count(*), WHERE successful IS true) FROM TestSuite
SELECT percentage(count(*), WHERE outcome = 'passed') FROM Test
- Test outcome
What is the breakdown of test outcomes?
SELECT count(*) FROM Test FACET outcome
- Test failures
What percent of the time does each test pass?
SELECT percentage(count(*), WHERE outcome = 'failed') FROM Test FACET name
- Test suite failures
What percent of the time does each suite pass, and is that consistent over time?
SELECT histogram((passedCount / testCount)*100, 100, 10) FROM TestSuite FACET name
How long does each test suite take to run, and is that consistent over time?
SELECT histogram(duration*1000, 10, 20) FROM TestSuite FACET name
Because a PHPUnit test suite is linked to individual tests via its run ID, you can use
FACET widgets to filter results for a specific test run.
For example, if you created a widget with the most recent failing test suites and linked it to the current dashboard, you could click on a test and the surrounding widgets would update with information for only that test suite run.
In the above example, you can see that by clicking on run
5bb37ccee2a1dbc7, we learn that one of two tests,
testFoo, failed. Here are the NRQL queries that made up this example:
Recent unsuccessful suites:
SELECT latest(timestamp), latest(name) FROM TestSuite WHERE successful IS false FACET runId
Last unsuccessful suite:
SELECT host, name, duration * 1000 AS 'duration (ms)', assertionCount, testCount, passedCount, failedCount, incompleteCount, skippedCount, errorCount, riskyCount, warningCount FROM TestSuite WHERE successful IS false LIMIT 1
Last unsuccessful test:
SELECT host, name, testSuiteName, duration * 1000 AS 'duration (ms)', outcome, assertionCount, message FROM Test WHERE outcome != 'passed' LIMIT 1
PHPUnit event attributes
Test events contain the following attributes you can query against:
TestSuite events include the following attributes:
The number of seconds it took for the test suite to run.
The total number of assertions the test suite made.
The number of errors reported.
The number of tests with warnings. Note that PHPUnit includes these in the
The number of failed tests.
The number of incomplete tests.
The number of passed tests.
The number of tests marked by PHPUnit as risky.
The number of tests that were skipped.
The number of tests that ran.
A unique identifier that ties the test suite to the individual tests. For example,
A boolean that is
trueif there were no failures or errors during the test suite run.
The name of the test suite.
Test events include the following attributes:
The number of seconds it took for the test to run.
Any message associated with the test outcome. For example: Failed asserting that false is true or This test depends on 'StackTest::testFailure' to pass..
The number of assertions the test made.
The outcome of the test. Options include passed, failed, skipped, risky, warning, and incomplete.
A unique identifier that ties the test to its test suite. For example:
The name of the test.
The name of the parent test suite.
For more help
Additional documentation resources include: