Set up your Prometheus remote write integration

You can get Prometheus data flowing in New Relic with just a few simple steps. If you don't already have an account, you can sign up for New Relic right now.

This page covers basic setup for the remote write integration, as well as a few common troubleshooting topics. For information on integrating Prometheus servers in a high availability (HA) configuration, see the Prometheus High Availability resource.

Set up the integration

From the Prometheus remote write setup page, complete these steps.

  1. Enter a name for the Prometheus server you’re connecting to and your remote_write URL.

    The name you enter for the server will create an attribute on your data, and will be the name used to identify which Prometheus server you're connecting to.

  2. Add a new remote_write URL to your Prometheus YML file. A good place to add this information is under global_config in the file, at the same indentation level as the global section. We recommend to use the following syntax:
    remote_write: 
    - url: https://metric-api.newrelic.com/prometheus/v1/write?prometheus_server=YOUR_DATA_SOURCE_NAME  
      bearer_token:YOUR_LICENSE_KEY
    Alternatively, you could use this configuration:
    remote_write: 
    - url: https://metric-api.newrelic.com/prometheus/v1/write?X-License-Key=YOUR_LICENSE_KEY&prometheus_server=YOUR_DATA_SOURCE_NAME
    

    If you are completing the remote write integration using Kubernetes and Helm, you should instead add the remote write URL to your Helm values.yaml file. Replace remoteWrite: [] with two lines similar to the example below. Be sure to use your remote write URL and use indentation that matches the rest of the file:

    remoteWrite: 
    - url: https://metric-api.newrelic.com/prometheus/v1/write?prometheus_server=YOUR_DATA_SOURCE_NAME 
      bearerToken:YOUR_LICENSE_KEY
    
  3. Restart your Prometheus server.
  4. View your data.

Mapping of Prometheus metric types

The Prometheus remote write protocol does not include metric type information or other helpful metric metadata when sending metrics to New Relic. Because the remote write protocol doesn't include this information, New Relic infers the metric type based on Prometheus naming conventions. Metrics not following these naming conventions may not be mapped correctly.

New Relic maps Prometheus metrics types into New Relic metric types based on Prometheus metric naming conventions as follows:

  • metricName_bucket is stored as a New Relic count metric type
  • metricName_count is stored as a New Relic count metric type
  • metricName_total is stored as a New Relic count metric type
  • metricName_sum is stored as a New Relic summary metric type

Everything else is stored as a New Relic gauge metric type.

Customize remote write behavior

You may be interested in customizing the following parameters if you are writing to more than one account in New Relic or are connecting more than one Prometheus data source to the same account in New Relic.

X-License Key

Your account's license key is not an API key. The license key is used for authentication and to identify which account to write data into.

Details:

  • If you are configuring Prometheus to write into different New Relic accounts or sub-accounts you should use a different key on each remote write URL.
prometheus_server URL parameter

The prometheus_server parameter is a label or attribute used to add to stats that are written to NRDB. You should use this same label when configuring your Grafana data source to limit results to just those from a particular prometheus_server.

Error messages

If you receive an integration error message from New Relic or error messages in your Prometheus server logs after restarting your Prometheus server, there are several actions you can take to get data flowing properly.

For detailed information about common errors including missing or incorrect characters, bad request errors, request entity too large errors, rate limit errors and the associated fixes, see Remote write errors and error messages.

Remove the integration

To remove the integration, remove the configuration code snippet from your Prometheus YML file and restart the server.

When you remove the integration, this stops new data from flowing, but will not purge or remove any historical data.

For more help

If you need more help, check out these support and learning resources: