New Relic has resource limits in place to protect your experience, our systems, and our other customers. These limits range from the maximum number of characters you can have in a query, to API request rates, and more.
This page describes the limit metrics and NrIntegrationError events that enable you to view your limits, your current data usage and overall resource consumption as compared to those limits, and the impact of experiencing a limit event. We also provide a handful of queries that, when compiled into a dashboard, can give you consistent insight into your limits status.
Important
While NrIntegrationError events provide data on many limits types, resource limit metrics currently only cover request rate ingestion and API query rate limits.
What happens when you reach a limit
Our response to reaching a limit depends on a handful of factors: the type of limit that's reached, as well as the duration, frequency, and amount at which you exceed the limit. Exceeding a limit doesn't always mean you experience a limit event, such as dropped data, rejected traffic, or having your data turned off for the rest of the day. We sometimes allow a small buffer before enforcing a limit. That said, any resource consumed above 100% is at risk for limit impact at any time.
Many of our rate limits apply proportionally. That means if you're barely exceeding the limit, we will take less action than if you're exceeding by 200%.
Limit metrics are only visible if you're sending data in to a corresponding dataType or limitName API. For example, if you send in data via the Metric API, you'll see the Metric API resource metrics, but if you don't send any data in, you won't see APM resource metrics.
Tip
Impact metrics will be generated regardless of impact; if there's no impact, you'll see a 0.
An NrIntegrationError event is generated when you experience impact and is a good way to quickly see if you're experiencing any limit events. See View System Limits for more information.
Create a dashboard to view your limit status
Using three limit metrics together on a dashboard, you can quickly see detailed visuals of your Ingest Resource Request Per Minute limits, and with NrIntegrationError get a view into more limits.
Dashboard displaying limits status using a handful of queries.
We used the following queries to create this dashboard. To make a dashboard like this in New Relic, select Dashboards, and then Create a dashboard. Then, add a new chart for each query you want to regularly monitor. The three limits metrics included in these queries are described in a separate section, below.
From left to right, top to bottom:
FROM Metric select rate(sum(newrelic.resourceConsumption.currentValue), 1 minute) /latest(newrelic.resourceConsumption.limitValue) * 100 facet limitName where limitTimeInterval = '1 minute' timeseries limit max
SELECT max(`usage`) FROM (FROM Metric select rate(sum(newrelic.resourceConsumption.currentValue), 1 minute) / latest(newrelic.resourceConsumption.limitValue) * 100 as 'usage' facet limitName timeseries ) facet limitName limit max
FROM Metric select rate(sum(newrelic.resourceConsumption.currentValue), 1 minute) where limitName = 'APM Agent API transaction events requests per minute' TIMESERIES
FROM Metric select rate(sum(newrelic.resourceConsumption.currentValue), 1 minute) as 'usage', latest(newrelic.resourceConsumption.limitValue) as 'limit' where limitName = 'Trace API requests per minute' TIMESERIES
From Metric select rate(sum(newrelic.resourceConsumption.impact), 1 minute) facet dataType, impact, resource TIMESERIES 1 minute limit max
FROM NrIntegrationError select count(*) facet limitName TIMESERIES MAX since 1 day ago limit max
If you want to see limits for multiple accounts on one chart:
run this query from one of the accounts:
FROM Metric select rate(sum(newrelic.resourceConsumption.currentValue), 1 minute) / latest(newrelic.resourceConsumption.limitValue) * 100 facet limitName, consumingAccountId where limitTimeInterval = '1 minute' timeseries limit max
Click Add another query.
Select a different account.
Then run this query again:
FROM Metric select rate(sum(newrelic.resourceConsumption.currentValue), 1 minute) / latest(newrelic.resourceConsumption.limitValue) * 100 facet limitName, consumingAccountId where limitTimeInterval = '1 minute' timeseries limit max
Finally, save it.
FROM Metric, NrIntegrationError select rate(sum(newrelic.resourceConsumption.currentValue), 1 minute) as 'Per Minute Count',latest(newrelic.resourceConsumption.limitValue) as ' limit Value',(rate(sum(newrelic.resourceConsumption.currentValue), 1 minute)/latest(newrelic.resourceConsumption.limitValue)*100)as 'Percent Used', filter (count(*), where NrIntegrationError.limitValue is not null) as 'limit reached count' facet limitName limit 1000
Limit metrics
These metrics, used in the dashboard queries above, can hone in on a single limit or resource. Or, with the help of FACET limitName or resource provide a view across all your limits.
limitValue allows you to see the setting for a limit by limitName and understand more about what resource is linked to this limit. The following examples use the limit value metric in the query:
Example for Metric API requests per minute.
FROM Metric select latest(newrelic.resourceConsumption.limitValue) where limitName = 'Metric API requests per minute'
To show all limits, add FACET limitName and consider grouping by limitTimeInterval.
FROM Metric select latest(newrelic.resourceConsumption.limitValue) WHERE limitTimeInterval = '1 minute' FACET limitName limit max
currentValue shows you how much of a given resource you’re currently consuming. To get a better glimpse into how our systems are viewing your consumption, use a rate() function with the time period that aligns with the limitTimeInterval. Limit 200.
Example for Metric API request per minute:
FROM Metric select rate(sum(newrelic.resourceConsumption.currentValue),1 minute) where limitName = 'Metric API requests per minute'
To show all limits, add FACET limitName and consider grouping by limitTimeInterval.
FROM Metric select rate(sum(newrelic.resourceConsumption.currentValue),1 minute) where limitTimeInterval = '1 minute' FACET limitName limit max
impact lets you know for any given resource what impact limit events are having. Zeros mean you are not currently impacted.
The most granular we have is dataType. It is possible for multiple instances of limitName to impact a single type, such as Metric RPM and DPM. If we know, we will display limitName.
From Metric select rate(sum(newrelic.resourceConsumption.impact), 1 minute) facet dataType, resource, impact, limitName TIMESERIES limit max
Metric attributes
Attributes on newrelic.resourceConsumption.limitValue and newrelic.resourceConsumption.currentValue:
limitName: The Name of the limit for the metric data, for example RPM Metric API.
dataType: What kind of data the metric is tracking, for example Metric, Log, or APM.
Resource: What resource is being consumed, for example Requests or DPM.
limitTimeInterval: What time window this resource is evaluated for limiting.
consumingAccountId: The New Relic account where the resource is being consumed.
Attributes on newrelic.resourceConsumption.impact
dataType: The kind of data that is being impacted, for example Metric, Log, or APM.
Resource: What resource is being impacted, for example Request Rate.
Impact: A count of what is happening when resource has exceeded set limit, for example dropped requests.
consumingAccountId: The New Relic account where the resource is being consumed.
Set alerts on resource metrics
While building a dashboard to see all your limits is handy, being able to automate it is even better. You can set on your limit metrics to provide updates on limits changes.
Tip
Because we currently only have metrics on 1 minute time windows, setting TimeWindow = 1 minute, will cover them all. Eventually, we make more metrics available, you might want to set separate alerts for limits that are enforced by different time windows.
You can use the following NRQL queries to create alerts. Learn about creating alerts with NRQL queries here.
From Metric select (rate(sum(newrelic.resourceConsumption.currentValue), 1 minute)/latest(newrelic.resourceConsumption.limitValue))*100 facet limitName
From Metric select (rate(sum(newrelic.resourceConsumption.currentValue), 1 minute)/latest(newrelic.resourceConsumption.limitValue))*100 where limitName = 'my limit'
