New Relic supports PromQL, the Prometheus query language, using a PromQL-like query mode that automatically translates Prometheus query language (PromQL) syntax queries into the closest NRQL approximation. This resource describes the supported promQL features, which account for approximately 97% of queries across the 7 million top Grafana dashboard downloads, as well as differences to be aware of.
For general information about Prometheus queries and operators, see the Prometheus.io documentation.
New Relic supports the following aggregation, arithmetic, mathematical, and rate-like functions. As we continue to expand support for Prometheus and PromQL, this list will be updated.
- Aggregation operators and functions
- Arithmetic binary operators
- Logical operators
- Date/time functions
- Mathematical functions
- Rate-like functions
- Predictive functions
- Time-series selectors
New Relic offers support for PromQL time-series selectors including the following:
Note that New Relic only supports
offsetqueries if every vector in the query has the same offset value.
This section describes differences in behavior between Prometheus/PromQL and New Relic's PromQL-like querying behavior, and how to work with these differences for the impacted supported features. The information here is particularly relevant for people working with advanced queries and using PromQL mode in the New Relic chart builder.
Prometheus recommendations note that you should only use some functions, like delta(), on gauges, and only use others like rate() and increase() on counters, but queries in Prometheus still work most of the time even if they don’t follow those instructions.
However, because NRDB converts PromQL-style accumulating counters to
delta counters, New Relic's implementation is unforgiving of uses of these functions on the wrong data type and will produce different/incorrect answers.
For this reason, it's best to follow all Prometheus recommendations when working via New Relic, even if you don't follow these recommendations in Prometheus.
- In order to ensure the stability and performance of our system for all users, New Relic places some limits on what queries can be run. In all cases, we enforce a limit of 366 steps in range queries. We also default to only returning 100 timeseries from queries by default.
- If you want to see more (or fewer), you need to explicitly add a
topk()to your query. (Note that the
topk()implementation via New Relic is different from that of Prometheus.)
- New Relic limits the total memory a query can use. This means that requests for large numbers of time steps or large numbers of time series may be rejected, particularly if they are combined with an aggregation like unique
quantilewhich require significantly more memory to compute than simple arithmetic aggregations.
Range vector selectors (sliding windows and smoothing behavior)
At the time, New Relic does not support sliding window timeseries aggregations, only tumbling windows. As a result, the time duration on range vector selectors is ignored in most cases, and only the samples within a single step interval are included.
For functions like
delta, the result from a single step is multiplied by the appropriate factor to approximate the result over the longer time interval.