Avoid sending Prometheus data that isn't relevant to your monitoring needs. Instead, use filters to ignore or include specific metrics, which help you control the amount of data you send to New Relic and avoid additional billing charges.
As explained in the configuration guide, some targets are scraped by default, and all the data that's exposed from those discovered targets is sent. So you may exceed New Relic's platform limits and increase your billing charges.
To reduce the amount of data being set, you can customize the prometheus agent configuration:
- By setting up which targets are being scraped.
- By adjusting the scrape interval.
- By applying metric filters.
- By setting up which prometheus integrations are enabled.
Also, see our troubleshooting tips.
The prometheus agent uses the
source_labels fields to scrape metrics from Prometheus integrations, which you can customize. You can check the default values in the chart GitHub repository.
source_labelsdefines the label names to be used to filter metrics from the corresponding resources.
app_valuesdefines the values of those label names which are filtered.
For instance, with the following values:
# (...)kubernetes:source_labels: ["app.kubernetes.io/name", "app.newrelic.io/name"]app_values: ["redis"]# (...)
All jobs with
integrations_filter.enabled: true would include only the metrics including the labels corresponding to
"app.newrelic.io/name", whose value is
You can extend the default values to include additional labels or application values. You can also take out the values whose services referred in the default application you aren't interested in.
If you have some kind of requirement which isn't currently supported by the configuration layer the Prometheus agent provides, you can set up additional scrape configuration in Prometheus format. To do so, use the
extra_ fields including valid Prometheus configuration which will not be parsed in any way.
extra_ fields should include raw Prometheus configuration.
Therefore, the metrics collected won't have by default the metadata such as
service, added by the agent for the static targets or kubernetes jobs. Use this configuration only as a workaround when the supported fields don't cover your needs. See more details in the helm-chart documentation.