Add, rename, or copy Prometheus attributes

The New Relic Prometheus OpenMetrics integration provides controls to transform the Prometheus metrics for Docker before sending them to New Relic. After you define the transformations in the integration config file, they are performed for all endpoints.

Hierarchy

The nri-prometheus-latest.yaml manifest file includes the nri-prometheus-cfg config map showing an example configuration. The transformations are executed in the following order:

Ignore metrics.
Add or include attributes.
Rename attributes.
Copy attributes.

Important

Avoid sending Prometheus OpenMetrics integration data that is not relevant to your monitoring needs. Instead, use filters to ignore or include specific metrics. This will help you control the amount and types of data you send to New Relic. This will also help you avoid additional billing charges. For more information, see Ignore or include Prometheus metrics.

Example configuration

To use these options, set up the scraper container configuration file (config.yaml in the current directory):

bash

$docker run -d --restart unless-stopped \
>    --name nri-prometheus \
>    -e CLUSTER_NAME="YOUR_CLUSTER_NAME" \
>    -e LICENSE_KEY="YOUR_LICENSE_KEY" \
>    -v "$(pwd)/config.yaml:/config.yaml" \
>    newrelic/nri-prometheus:latest --configfile=/config.yaml

Here is an example configuration file containing all of these examples:

transformations:
  - description: "Transformation for MySQL exporter"
    add_attributes:
      - metric_prefix: "mysql_"
        attributes:
          owningTeam: "database-team"
    rename_attributes:
      - metric_prefix: "mysql_"
        attributes:
          table: "tableName"
          under_score: "CamelCase"
    copy_attributes:
      - from_metric: "mysql_version_info"
        to_metrics:
          - "mysql_"
        attributes:
          - "innodb_version"
          - "version"
    ignore_metrics:
      - prefixes:
          - "go_"
          - "process_"

Add attributes

This transformation allows you to include a set of statically defined attributes to a set of target metrics.

Configuration:

To include the owningTeam attribute to all metrics starting with mysql_:

add_attributes:
  - prefix: "mysql_"
    attributes: 
      owningTeam: "database-team"

To include the datacenter attribute to all the metrics:

add_attributes:
  - prefix: ""
    attributes: 
      datacenter: "europe"

Input:

mysql_info_schema_table_rows{schema="sys",table="host_summary"} 123 another_metric{table="first"} 800

Output:

mysql_info_schema_table_rows{schema="sys",table="host_summary","owningTeam":"database-team","datacenter":"europe"} 123 another_metric{table="first","datacenter":"europe"} 800

Rename attributes

Not all Prometheus endpoints have consistent naming. You can rename the attributes as needed.

Configuration:

To rename the table attribute to tableName for metrics that start with mysql_:

rename_attributes:
  - metric_prefix: "mysql_" 
    attributes:
      table: "tableName"

Input:

mysql_info_schema_table_rows{schema="sys",table="host_summary"} 123 another_metric{table="first"} 800

Output:

mysql_info_schema_table_rows{schema="sys",tableName="host_summary"} 123 another_metric{table="first"} 800

Copy attributes

Some Prometheus endpoints provide an _info or _static metric containing metadata about the service, such as the version. It can be helpful to have this attribute on all metrics for that service. This transformation allows you to copy attributes from a source metric to a set of target metrics.

Important

You can only copy attributes between metrics in the same endpoint.

Configuration:

To copy the innodb_version and version attributes from the mysql_version_info metric to all metrics starting with mysql_:

copy_attributes:
  - from_metric: "mysql_version_info"
    to_metrics:
      - "mysql_" 
    attributes: 
      - "innodb_version"
      - "version"

Input:

# HELP mysql_version_info MySQL version and distribution. 
mysql_version_info{innodb_version="5.7.14",version="5.7.14",version_comment="MySQL Community Server (GPL)"} 1

# HELP mysql_global_variables_slave_transaction_retries Generic gauge metric from SHOW GLOBAL VARIABLES. 
mysql_global_variables_slave_transaction_retries 10

Output:

mysql_global_variables_slave_transaction_retries{innodb_version="5.7.14",version="5.7.14"} 10

Add, rename, or copy Prometheus attributes

Hierarchy.css-21sua1{background:none;border:none;width:0;padding:0;}

Important

Example configuration

Add attributes

Rename attributes

Rename attributes

Copy attributes

Important

Copy attributes

Hierarchy