Incorrect data reported

Problem

The agent is working, but the infrastructure monitoring UI shows unexpected data for some of the events, metrics or attributes collected from the infrastructure agent.

Troubleshooting

The infrastructure agent includes trace level logs that can be enabled on-demand to help troubleshooting complex scenarios and determine if the expected data is leaving the host as expected. This setting generates a lot of data very quickly, we recommend only enabling it temporarily for troubleshooting purposes.

1. Edit the newrelic-infra.yml configuration file and add required flags to enable logs. For example, to enable all trace logs on the default log file path:

   log:
       level: trace

   Copy

Infrastructure agent version 1.26.0 or older require a different logging configuration:

verbose: 1
  trace: 
    # v3.submission enables detailed logging for events, examples: SystemSample, NetworkSample, etc.
    - v3.submission
    # dm.submission enables detailed logging for integrations using Dimensional Metric format. 
    - dm.submission

2. Use your init system to restart the agent service:

   Restart the agent with SystemD

   Use SystemD commands with CentOS 7, Debian 8, RHEL 7, and Ubuntu 15.04 or higher:

   sudo systemctl restart newrelic-infra

   Copy

   Restart the agent with System V

   Use System V commands with Debian 7:

   sudo /etc/init.d/newrelic-infra restart

   Copy

   Restart the agent with Upstart

   Use Upstart commands with Amazon Linux, CentOS 6, RHEL 6, and Ubuntu 14.10, or lower:

   sudo initctl restart newrelic-infra

   Copy

   Restart the agent in Windows

   net stop newrelic-infra
   net start newrelic-infra

   Copy

3. Identify the new trace log lines to confirm the data being sent to the Telemetry Data Platform.

Log example when v3.submission is enabled:

time="2021-12-28T09:27:28Z" level=debug msg="Sending events to metrics-ingest." component=MetricsIngestSender key=... numEvents=3 postCount=1 timestamps="[2021-01-01 09:27:28 +0000 UTC]"
time="2021-12-28T09:27:28Z" level=debug msg="Preparing metrics post." component=MetricsIngestSender postCount=1
time="2021-12-28T09:27:28Z" level=trace msg="[{\"EntityID\":111,\"IsAgent\":true,\"Events\":[{\"eventType\":\"SystemSample\",\"timestamp\":1640683648,\"entityKey\":\"...\",\"cpuPercent\":0.2004008016032026, ...}]" feature=v3.submission
time="2021-12-28T09:27:29Z" level=debug msg="Metrics post succeeded." component=MetricsIngestSender postCount=1