Analyze PHPUnit test data in Insights

If you use PHPUnit to manage and run your unit tests, the New Relic PHP agent can automatically capture the test summary results and send them to Insights as an event where you can query and visualize test data at a glance. This feature was released in version 6.7.0 of the PHP Agent and supports PHPUnit versions 3.7 onward.

Enable PHPUnit test events

To enable PHPUnit test events for use in Insights:

  1. Find or add the newrelic.phpunit_events.enabled setting in your newrelic.ini file and set it to true.
  2. Restart your web server (apache, PHP-FPM, Nginx, etc) for the setting change to take effect.

Exception messages are collected and sent with events. Also, if you use PHPUnit's --disallow-test-output option, which flags tests that print output as "risky," the test event will include the offending output.

View available attributes

When enabled, the PHP Agent detects PHPUnit commands and populates New Relic Insights with two event types that contain data for the test suite (named TestSuite) and individual tests (named Test). You can query the data with NRQL queries and build a dashboard of information important to you.

To query your test events, use FROM TestSuite and FROM Test when specifying your SELECT statements:

SELECT uniqueCount(name) FROM TestSuite
PHPUnit_dashboard.png
New Relic Insights: Example of a PHPUnit test dashboard created by querying TestSuite and Test events.

Example queries

A dashboard of test summary data allows you to quickly see a snapshot of overall test success as well as dig into failing suites to determine their cause. These examples illustrate the kinds of widgets you can create with both TestSuite and Test event types.

Percent success

What percent of suites or tests are passing?

SELECT percentage(count(*), WHERE successful IS true) FROM TestSuite
SELECT percentage(count(*), WHERE outcome = 'passed') FROM Test
PHPUnit_passing_dials.png
What percent of suites and tests are passing?
Test outcome

What is the breakdown of test outcomes?

SELECT count(*) FROM Test FACET outcome
PHPUnit_test_outcomes.png
What is the breakdown of test outcomes?
Test failures

What percent of the time does each test pass?

SELECT percentage(count(*), WHERE outcome = 'failed') FROM Test FACET name
PHPUnit_test_failure.png
What percent of the time does each test pass?
Test suite failures

What percent of the time does each suite pass, and is that consistent over time?

SELECT histogram((passedCount / testCount)*100, 100, 10) FROM TestSuite FACET name
PHPUnit_passing_histogram.png
What percent of the time does each suite pass, and is that consistent over time?
Duration

How long does each test suite take to run, and is that consistent over time?

SELECT histogram(duration*1000, 10, 20) FROM TestSuite FACET name
PHPUnit_duration_histogram.png
How long does each test suite take to run, and is that consistent over time?

Linked facets

Because a PHPUnit test suite is linked to individual tests via its run ID, you can use FACET widgets to filter results for a specific test run.

For example, if you created a widget with the most recent failing test suites and linked it to the current dashboard, you could click on a test and the surrounding widgets would update with information for only that test suite run.

PHPUnit_facet_on_runId.png
New Relic Insights: FACET widget (top-left) linked to the current dashboard allows you to drill into failing test suites.

In the above example, you can see that by clicking on run 5bb37ccee2a1dbc7, we learn that one of two tests, testFoo, failed. Here are the NRQL queries that made up this example:

Recent unsuccessful suites:

SELECT latest(timestamp), latest(name) FROM TestSuite WHERE successful IS false FACET runId

Last unsuccessful suite:

SELECT host, name, duration * 1000 AS 'duration (ms)', assertionCount, testCount, passedCount, failedCount, incompleteCount, skippedCount, errorCount, riskyCount, warningCount FROM TestSuite WHERE successful IS false LIMIT 1

Last unsuccessful test:

SELECT host, name, testSuiteName, duration * 1000 AS 'duration (ms)', outcome, assertionCount, message FROM Test WHERE outcome != 'passed' LIMIT 1

PHPUnit event attributes

TestSuite and Test events contain the following attributes you can query against:

TestSuite event attributes

TestSuite events include the following attributes:

Attribute Description
duration

The number of seconds it took for the test suite to run.

assertionCount The total number of assertions the test suite made.
errorCount The number of errors reported.
warningCount The number of tests with warnings. Note that PHPUnit includes these in the passedCount as well.
failedCount The number of failed tests.
incompleteCount The number of incomplete tests.
passedCount The number of passed tests.
riskyCount The number of tests marked by PHPUnit as risky.
skippedCount The number of tests that were skipped.
testCount The number of tests that ran.
runId A unique identifier that ties the test suite to the individual tests. For example, 861d12cea0d5b923.
successful A boolean that is true if there were no failures or errors during the test suite run.
name The name of the test suite.
Test event attributes

Test events include the following attributes:

Attribute Description
duration The number of seconds it took for the test to run.
message

Any message associated with the test outcome. For example: Failed asserting that false is true or This test depends on 'StackTest::testFailure' to pass..

assertionCount The number of assertions the test made.
outcome The outcome of the test. Options include passed, failed, skipped, risky, warning, and incomplete.
runId A unique identifier that ties the test to its test suite. For example: 861d12cea0d5b923.
name The name of the test.
testSuiteName The name of the parent test suite.

For more help

Additional documentation resources include:

Join the discussion about PHP in the New Relic Online Technical Community! The Technical Community is a public platform to discuss and troubleshoot your New Relic toolset.

If you need additional help, get support at support.newrelic.com.