The New Relic Prometheus OpenMetrics integration provides controls to transform the Prometheus metrics for Docker and Kubernetes before sending them to New Relic. After you define the transformations in the integration config file, they are performed for all endpoints.
Hierarchy
The nri-prometheus-latest.yaml
manifest file includes the nri-prometheus-cfg
config map showing an example configuration. The transformations are executed in the following order:
- Ignore metrics.
- Add or include attributes.
- Rename attributes.
- Copy attributes.
Avoid sending Prometheus OpenMetrics integration data that is not relevant to your monitoring needs. Instead, use filters to ignore or include specific metrics. This will help you control the amount and types of data you send to New Relic. This will also help you avoid additional billing charges. For more information, see Ignore or include Prometheus metrics.
Example configuration
To use these options, set up the scraper container configuration file (config.yaml
in the current directory):
docker run -d --restart unless-stopped \ --name nri-prometheus \ -e CLUSTER_NAME="YOUR_CLUSTER_NAME" -e LICENSE_KEY="YOUR_LICENSE_KEY" \ -v "$(pwd)/config.yaml:/config.yaml" \ newrelic/nri-prometheus:latest --configfile=/config.yaml
Here is an example configuration file containing all of these examples:
transformations: - description: "Transformation for MySQL exporter" add_attributes: - metric_prefix: "mysql_" attributes: owningTeam: "database-team" rename_attributes: - metric_prefix: "mysql_" attributes: table: "tableName" under_score: "CamelCase" copy_attributes: - from_metric: "mysql_version_info" to_metrics: - "mysql_" attributes: - "innodb_version" - "version" ignore_metrics: - prefixes: - "go_" - "process_"
Add attributes
This transformation allows you to include a set of statically defined attributes to a set of target metrics.
- Add attributes
-
Configuration:
To include the
owningTeam
attribute to all metrics starting withmysql_
:add_attributes: - prefix: "mysql_" attributes: owningTeam: "database-team"
To include the
datacenter
attribute to all the metrics:add_attributes: - prefix: "" attributes: datacenter: "europe"
Input:
mysql_info_schema_table_rows{schema="sys",table="host_summary"} 123 another_metric{table="first"} 800
Output:
mysql_info_schema_table_rows{schema="sys",table="host_summary","owningTeam":"database-team","datacenter":"europe"} 123 another_metric{table="first","datacenter":"europe"} 800
Rename attributes
Not all Prometheus endpoints have consistent naming. You can rename the attributes as needed.
- Rename attributes
-
Configuration:
To rename the
table
attribute totableName
for metrics that start withmysql_
:rename_attributes: - metric_prefix: "mysql_" attributes: table: "tableName"
Input:
mysql_info_schema_table_rows{schema="sys",table="host_summary"} 123 another_metric{table="first"} 800
Output:
mysql_info_schema_table_rows{schema="sys",tableName="host_summary"} 123 another_metric{table="first"} 800
Copy attributes
Some Prometheus endpoints provide an _info
or _static
metric containing metadata about the service, such as the version. It can be helpful to have this attribute on all metrics for that service. This transformation allows you to copy attributes from a source metric to a set of target metrics.
You can only copy attributes between metrics in the same endpoint.
- Copy attributes
-
Configuration:
To copy the
innodb_version
andversion
attributes from themysql_version_info
metric to all metrics starting withmysql_
:copy_attributes: - from_metric: "mysql_version_info" to_metrics: - "mysql_" attributes: - "innodb_version" - "version"
Input:
# HELP mysql_version_info MySQL version and distribution. mysql_version_info{innodb_version="5.7.14",version="5.7.14",version_comment="MySQL Community Server (GPL)"} 1 # HELP mysql_global_variables_slave_transaction_retries Generic gauge metric from SHOW GLOBAL VARIABLES. mysql_global_variables_slave_transaction_retries 10
Output:
mysql_global_variables_slave_transaction_retries{innodb_version="5.7.14",version="5.7.14"} 10