You can collect metrics and logs from your infrastructure hosts with OpenTelemetry and leverage the same infrastructure experiences that are available for New Relic agents. Specific receivers and processors are required in the OTel collector to collect and report host telemetry.
If you're looking for help with other collector use cases, see the newrelic-opentelemetry-examples repository.
Step 1: Prerequisites
Make sure you've completed the following before going further:
- If you haven't already done so, sign up for a free New Relic account.
- Get the for the New Relic account to which you want to report data.
Step 2: Install the OpenTelemetry Collector
To do a basic installation for single hosts in the cloud or on-premises, see OpenTelemetry's instructions for up-to-date installation steps from the community. Instructions are available for the following:
- Linux: Debian systems
- Linux: Red Hat
- Windows
- Docker, Kubernetes, and other options
Your deployment experience might vary depending on which vendor-specific distributions you use. For example, installation via a package manager might be available for Linux hosts.
Important
To set up infrastructure monitoring, you need to install and configure components that are included in the collector-contrib
release. For example, the host receiver is required to collect basic host metrics such as CPU, memory, disk, and network stats and is only available in the OpenTelemetry Collector-contrib release.
Step 3: Configure host metrics and logs
This collector example is meant to serve as a starting point from which you can extend, customize, and validate configurations before using them in production.
The collector-contrib
release includes:
- The
hostmetrics
receiver that generates metrics about the system scraped from various sources. Deploy the collector as an agent when you use ahostmetrics
receiver. - The
filelog
receiver that tails and parses logs from files.
When using the hostmetrics
receiver as part of the collector configuration, New Relic will detect host metrics as part of a Host
entity. This means you will have the same experience as with the New Relic infrastructure agent. To make this work, you need to meet the following configuration requirements:
host.id
attribute is present in host metrics.service.name
andcontainer.id
attributes are not present in host metrics.
Learn more about available metrics and advanced configurations from the OpenTelemetry documentation in GitHub.
Here is a sample configuration YAML file for a Linux host. Be sure to do the following:
- Set the
$NEW_RELIC_OTLP_ENDPOINT
env var to the appropriate endpoint. - Set the
$NEW_RELIC_API_KEY
env var to you your . - Adjust the target log files in the filelog receiver section based on your requirements.
- Adjust the
memory_limiter
default values based on your environment requirements. - Several bits of config are useful for local demonstration, but likely not needed for production deployments. Review these sections and explanatory comments and remove as needed.
See Host Monitoring with OpenTelemetry Collector Setup for a working code example.
extensions: health_check:
receivers: otlp: protocols: grpc: http:
hostmetrics: # Mount the host file system when running in docker so we can monitor the host system, # not the docker container. For more info see: # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/hostmetricsreceiver#collecting-host-metrics-from-inside-a-container-linux-only # Delete for production deployments. root_path: ${HOST_METRICS_ROOT_PATH} collection_interval: 20s scrapers: cpu: metrics: system.cpu.utilization: enabled: true load: memory: metrics: system.memory.utilization: enabled: true disk: filesystem: metrics: system.filesystem.utilization: enabled: true # Reading /containers/services causes error running in docker. # Delete for production deployments. exclude_mount_points: mount_points: ["/containers/services"] match_type: strict network: paging: metrics: system.paging.utilization: enabled: true processes: process: metrics: process.cpu.utilization: enabled: true process.cpu.time: enabled: false # Mute various errors reading process metrics running locally in docker. # Delete for production deployments. mute_process_exe_error: true mute_process_user_error: true mute_process_io_error: true
filelog: include: - /var/log/alternatives.log - /var/log/cloud-init.log - /var/log/auth.log - /var/log/dpkg.log - /var/log/syslog - /var/log/messages - /var/log/secure - /var/log/yum.log
processors:
transform/truncate: trace_statements: - context: span statements: - truncate_all(attributes, 4095) - truncate_all(resource.attributes, 4095) log_statements: - context: log statements: - truncate_all(attributes, 4095) - truncate_all(resource.attributes, 4095)
memory_limiter: check_interval: 1s limit_mib: 1000 spike_limit_mib: 200
batch:
resourcedetection: detectors: ["env", "system"] system: hostname_sources: ["os"] resource_attributes: host.id: enabled: true
resourcedetection/cloud: detectors: ["gcp", "ec2", "azure"] timeout: 2s override: false
# host.id is required for NewRelic host entity synthesis and relationships, but is # not included by any resourcedetection detector when running with docker on macOS. # We include a fallback value for demonstration purposes. # Delete for production deployments. resource: attributes: - key: host.id value: localhost action: upsert
exporters: otlphttp: endpoint: $NEW_RELIC_OTLP_ENDPOINT headers: api-key: $NEW_RELIC_API_KEY
service: pipelines:
metrics/hostmetrics: receivers: [hostmetrics] processors: [memory_limiter, resourcedetection, resourcedetection/cloud, resource, batch] exporters: [otlphttp]
metrics: receivers: [otlp] processors: [memory_limiter, transform/truncate, resourcedetection, resourcedetection/cloud, resource, batch] exporters: [otlphttp]
traces: receivers: [otlp] processors: [memory_limiter, transform/truncate, resourcedetection, resourcedetection/cloud, resource, batch] exporters: [otlphttp]
logs: receivers: [otlp, filelog] processors: [memory_limiter, transform/truncate, resourcedetection, resourcedetection/cloud, resource, batch] exporters: [otlphttp]
extensions: [health_check]
Step 4: View your data
You can view your collector data in a variety of places in the New Relic UI.
Browse host data in infrastructure UI
By using the recommended configuration in the collector, you can view data through the standard features in the Infrastructure UI experience.
Query host metrics and logs
Once telemetry is successfully ingested in New Relic, they are available in metrics and events and query builder.
The following NRQL queries show examples to help you explore the metrics you received:
Listing number of metric updates ingested by metric name
SELECT count(*) FROM Metric WHERE metricName LIKE 'system.%' FACET metricName LIMIT maxQuerying specific metrics faceted by host
SELECT average(system.disk.operations) FROM Metric FACET host.name TIMESERIESListing dimensions available for a given metric
SELECT keyset() FROM Metric WHERE metricName = 'system.disk.operations'Querying number of log events per host
SELECT count(*) FROM Log FACET host.name TIMESERIES
Learn more about querying the metric data type.
What's next?
After you set up your collector, check out our best practices guide for tips to improve your use of OpenTelemetry and New Relic.