Troubleshoot with NRIntegrationError events

Problem

You sent metric data points to the Metric API, and are not seeing what you expect when querying the data. Use the following checklist to determine the root cause:

  • Make sure you are querying the data correctly.
  • Check the HTTP status codes returned by the API. Issues like authorization failures can be diagnosed with HTTP status codes.
  • Query your account for NrIntegrationError events. New Relic's ingestion endpoints are asynchronous, meaning the endpoint verifies the payload after it returns the HTTP response. If any issues occur while verifying your payload, then an NrIntegrationError event will be created in your account. New Relic also uses NrIntegrationError events to notify customers when various rate limits have been reached.

Solution

View error details

To view details about the errors, run this NRQL query:

SELECT count(*) FROM NrIntegrationError WHERE newRelicFeature ='Metrics' facet category, message limit 100 since 24 hours ago 

The category indicates the type of error and the message provides more detailed information about the error. If the category is rateLimit, then you should also examine the rateLimitType field for more information on the type of rate limiting.

Category rateLimitType Description and solution
BadRequest (not set)

There is an issue with the JSON payload. These include JSON syntax errors, attribute names, or values that are too long.

Check the message field to determine the exact issue. Then review the JSON payload, and update it to ensure it meets the proper semantic guidelines.

RateLimit DatapointsPerMinute You are sending too many datapoints per minute. If you get this error, you can either send data less frequently, or submit an email to rate-limit-request@newrelic.com and request that your limits be raised.
RateLimit UniqueTimeseriesPerDay

This means you have an attribute with a high number of unique values, like containerId or URI. To resolve this, review any attributes that may be causing the issue and remove them.

RateLimit RequestsPerMinute Too many requests per minute are being sent. To resolve this, put more datapoints in each request, and send them less frequently.

Match errors to ingested payloads

When a NrIntegrationError event is created as a result of a syntax issue with the HTTP request payload, then the event contains the attributes apiKeyPrefix and requestId.

  • The apiKeyPrefix matches the first 6 characters of the API key used to send the data.
  • The requestId matches the requestId sent in the HTTP response.

To view these fields, run this NRQL query:

SELECT message, apiKeyPrefix, requestId FROM NrIntegrationError limit 100

To verify a specific requestId, run this NRQL query:

SELECT * FROM NrIntegrationError where requestId ='REQUEST_ID'

Programmatically retrieve NrIntegrationError events

To programmatically retrieve these errors:

  1. Ensure you have an Insights query API key (go to insights.newrelic.com > Manage data > API keys).
  2. Create an HTTP request as shown below:

    If your account hosts data in the EU data center, ensure you're using the proper API endpoints for EU region accounts.

    curl -H "Accept: application/json" -H "X-Query-Key:YOUR_API_KEY_HERE" "https://insights-api.newrelic.com/v1/accounts/YOUR_ACCONT_HERE/query?nrql=SELECT%20*%20FROM%20NrIntegrationError%20where%20newRelicFeature='Metrics'"

For more help

Recommendations for learning more:

  • Browse New Relic's Explorers Hub for community discussions about New Relic's APIs.
  • Use your preferred search engine to find other New Relic resources.