• Log inStart now

View and query your Prometheus data

To query and visualize the metrics collected for your Prometheus OpenMetrics or remote write integration with New Relic, you can use NRQL. You can also translate your PromQL-style queries to NRQL using either Grafana or the query builder.

All metrics for Docker and Kubernetes are stored in the Metric type.

Default attributes for the OpenMetrics integration

By default, the following attributes will be added to all metrics for Docker and Kubernetes integrations:

Default attributes
(all integrations)

Description

clusterName

The name of the cluster provided in the scraper configuration.

integrationName

The name of this integration (nri-prometheus).

integrationVersion

The version of the integration; for example, 0.2.0.

metricName

The name of the metric itself.

nrMetricType

The type of the New Relic Metric type; for example, Gauges.

promMetricType

The metric type of the Prometheus metric

scrapedEndpoint

The URL of the endpoint is being scraped.

img-integration-k8.png Kubernetes: If the scraper is running in Kubernetes, New Relic also adds the following attributes to all the metrics:

Additional Kubernetes attributes

Description

deploymentName

Name of the deployment, if scraping a pod.

label

The Kubernetes labels of the object being scraped, prefixed by "label".

namespaceName

Name of the namespace.

nodeName

Name of the node where the pod being scraped is running, if applicable.

podName

Name of the pod being scraped, if applicable.

serviceName

Name of the service being scraped, if applicable

Default attributes for the remote write integration

By default, the following attributes will be added to Prometheus remote write metrics:

Default attributes
(all integrations)

Description

prometheus_server

A user supplied label specified as a Prometheus remote write URL parameter. The value supplied should be unique as it is intended to differentiate between source Prometheus servers at query time. Unspecified by default.

newrelic.source

The name of the New Relic ingest point (prometheusAPI).

instrumentation.provider

prometheus

instrumentation.name

remote-write

instrumentation.source

A user supplied identifier for the source of the Prometheus data that matches the value of prometheus_server.

instrumentation.version

Used to identify the version of the remote write API; for example, 0.0.1.

NRQL query examples

When you build queries, be aware that there is no linking between the metrics, entities, and attributes. Use the following NRQL queries to find out which metrics are available and which attributes are present on these metrics:

Build the query

Using metric name and attributes, you can query your data. For more information about facets, time series, and time selection, see the NRQL documentation.

To build PromQL-style queries, see our documentation about [supported PromQL features]](/docs/integrations/prometheus-integrations/view-query-data/supported-promql-features).

View data in New Relic

When you query the data, you can view the results in the New Relic UI. You can also visualize the data as charts, histograms, etc.

To view the NRQL query results for your Prometheus integration's data: Go to one.newrelic.com > Query your data. For more information, see our query builder documentation.

Generate histograms and calculate percentiles

Using Prometheus remote write or version 2.0.0 and higher of the Prometheus OpenMetrics Integration (POMI), you can generate histograms and calculate percentiles from your data. For Prometheus histograms, a bucket <basename>_bucket{le="42"} will be sent as the metric <basename>_bucket, and the dimension will be {histogram.bucket.le="42"}.

NRQL has two functions that work on Prometheus histograms ingested via remote write or the Prometheus OpenMetrics Integration (starting with version 2.0.0): bucketPercentile(), and histogram(). The links include query examples.

Copyright © 2022 New Relic Inc.