Collector for host monitoring

You can collect metrics and logs from your infrastructure hosts with OpenTelemetry and leverage the same infrastructure experiences that are available for New Relic agents. Specific receivers and processors are required in the OTel collector to collect and report host telemetry.

If you're looking for help with other collector use cases, see the newrelic-opentelemetry-examples repository.

Step 1: Prerequisites

Make sure you've completed the following before going further:

• If you haven't already done so, sign up for a free New Relic account.
• Get the license key for the New Relic account to which you want to report data.

Step 2: Install the OpenTelemetry Collector

To do a basic installation for single hosts in the cloud or on-premises, see OpenTelemetry's instructions for up-to-date installation steps from the community. Instructions are available for the following:

• Linux: Debian systems
• Linux: Red Hat
• Windows
• Docker, Kubernetes, and other options

Your deployment experience might vary depending on which vendor-specific distributions you use. For example, installation via a package manager might be available for Linux hosts.

Step 3: Configure host metrics and logs

This collector example is meant to serve as a starting point from which you can extend, customize, and validate configurations before using them in production.

The collector-contrib release includes:

• The hostreceiver that generates metrics about the system scraped from various sources. Deploy the collector as an agent when you use a hostreceiver.
• The filelogreceiver that tails and parses logs from files.

When using the host receiver as part of the collector configuration, New Relic automatically detects host metrics as part of a Host entity and will synthesize its golden metrics providing the same experience as with the New Relic infrastructure agent. The following are the configuration requirements to enable the Host entity experience in New Relic UI:

• host.id attribute is present in host metrics.
• service.name and container.id attributes are not present in host metrics.

Learn more about available metrics and advanced configurations from the OpenTelemetry documentation in GitHub.

Here is a sample configuration YAML file for a linux host. Be sure to do the following:

• Replace OTLP_ENDPOINT_HERE with the appropriate endpoint.
• Replace YOUR_KEY_HERE with your license key.
• Adjust the target log files in the filelog receiver section based on your requirements.
• Adjust the memory_limiter default values based on your environment requirements.

extensions:
  health_check:

receivers:
  hostmetrics:
    collection_interval: 20s
    scrapers:
      cpu:
        metrics:
          system.cpu.utilization:
            enabled: true
      load:
      memory:
        metrics:
          system.memory.utilization:
            enabled: true
      disk:
      filesystem:
        metrics:
          system.filesystem.utilization:
            enabled: true
      network:
      paging:
        metrics:
          system.paging.utilization:
            enabled: true
      processes:
      process:
        metrics:
          process.cpu.utilization:
            enabled: true
          process.cpu.time:
            enabled: false

  filelog:
    include:
      - /var/log/alternatives.log
      - /var/log/cloud-init.log
      - /var/log/auth.log
      - /var/log/dpkg.log
      - /var/log/syslog
      - /var/log/messages
      - /var/log/secure
      - /var/log/yum.log

processors:
  
  transform/truncate:
    trace_statements:
      - context: span
        statements:
          - truncate_all(attributes, 4095)
          - truncate_all(resource.attributes, 4095)
    log_statements:
      - context: log
        statements:
          - truncate_all(attributes, 4095)
          - truncate_all(resource.attributes, 4095)

  memory_limiter:
    check_interval: 1s
    limit_mib: 1000
    spike_limit_mib: 200
  
  batch:
  
  resourcedetection:
    detectors: [env, system]
    system:
      hostname_sources: ["os"]
      resource_attributes:
        host.id:
          enabled: true
  
  resourcedetection/cloud:
    detectors: ["gcp", "ec2", "azure"]
    timeout: 2s
    override: false

exporters:
  otlp:
    endpoint: OTLP_ENDPOINT_HERE
    headers:
      api-key: YOUR_KEY_HERE

service:
  pipelines:
    
    metrics:
      receivers: [hostmetrics]
      processors: [memory_limiter, resourcedetection, resourcedetection/cloud, batch]
      exporters: [logging, otlp]
    
    traces:
      receivers: [otlp]
      processors: [memory_limiter, transform/truncate, resourcedetection, resourcedetection/cloud, batch]
      exporters: [logging, otlp]
    
    logs:
      receivers: [otlp, filelog]
      processors: [memory_limiter, transform/truncate, resourcedetection, resourcedetection/cloud, batch]
      exporters: [logging, otlp]

  extensions: [health_check]

Step 4: View your data

You can view your collector data in a variety of places in the New Relic UI.

Browse host data in infrastructure UI

By using the recommended configuration in the collector, you can view data through the standard features in the Infrastructure UI experience.

Query host metrics and logs

Once telemetry is successfully ingested in New Relic, they are available in metrics and events and query builder.

The following NRQL queries show examples to help you explore the metrics you received:

• Listing number of metric updates ingested by metric name

  SELECT count(*) FROM Metric WHERE metricName LIKE 'system.%' FACET metricName LIMIT max

  Copy

• Querying specific metrics faceted by host

  SELECT average(system.disk.operations) FROM Metric FACET host.name TIMESERIES

  Copy

• Listing dimensions available for a given metric

  SELECT keyset() FROM Metric WHERE metricName = 'system.disk.operations'

  Copy

• Querying number of log events per host

  SELECT count(*) FROM Log FACET host.name TIMESERIES

  Copy

Learn more about querying the metric data type.

What's next?

After you set up your collector, check out our best practices guide for tips to improve your use of OpenTelemetry and New Relic.