Forward your logs using New Relic Infrastructure

BETA

If you already have a New Relic Infrastructure subscription, you can forward your logs to New Relic Logs using the Infrastructure agent, our lightweight data collector. This makes all of your logging data available in one centralized location, providing deeper visibility both into application and platform performance data.

Compatibility and requirements

The log forwarding feature is compatible with the following operating systems:

Operating system

Supported version

Amazon Linux

Amazon Linux 2

CentOS

Version 7 or higher

Debian

Version 9 ("Stretch") or higher

Red Hat Enterprise Linux (RHEL)

Version 7 or higher

SUSE Linux Enterprise Server (SLES)

Version 12

Ubuntu

Versions 16.04.x and 18.04.x (LTS versions)

Windows Server

2012, 2016, and 2019, and their service packs

To use the log forwarder of the Infrastructure agent, make sure you meet the following requirements:

The beta version of log forwarding does not support containerized agents.

Enable log forwarding using the Infrastructure agent

To enable log forwarding through the Infrastructure agent:

  1. Install the New Relic Infrastructure agent, version 1.11.1 or higher.
  2. Configure your log sources and other parameters in the logging.d directory.
  3. Test log forwarding via the Infrastructure agent.
  4. Generate some traffic and wait a few minutes, then check your account for data.

Install the Infrastructure agent

Starting from version 1.11.1, the New Relic Infrastructure agent can forward logs to New Relic Logs. To install and run the Infrastructure agent, use a package manager (Linux) or the MSI installer (Windows).

For detailed procedures, see Linux | Windows | Elastic Beanstalk | Ansible | Chef | Puppet.

Configure the Infrastructure agent

Configuration files describe which log sources are forwarded. You can add as many config files as you want.

To add a new configuration file for the log forwarding feature:

  1. Navigate to the logging forwarder configuration folder:

    • Linux: /etc/newrelic-infra/logging.d/

    • Windows: C:\Program Files\New Relic\newrelic-infra\logging.d\

  2. Create a configuration file (for example, logs.yml) with the following content:

    • Linux example:

      logs:
      - name: "test_log"
        file: /var/log/test.log
      
    • Windows example (note that you must use forward-slash / in the path):

      logs:
      - name: "test_log"
        file: "C:/ProgramData/New Relic/newrelic-infra/newrelic-infra.log"
      
  3. Save the file.

The Infrastructure agent automatically processes new configuration files without having to restart it.

Configuration parameters

The log forwarder configuration supports the following parameters:

Key

Type

Description

name decorator

Name of the log or logs.

file source

Path to the log file or files. The agent tracks changes on the log files in a way similar to tail -f shell.

Your file can point to a specific log file or multiple ones by using wildcards and patterns; for example, /logs/*.log.

folder source

Path to the logs directory. Allows to tail all the files from the directory.

systemd

source

(Linux)

Service name. Once the systemd input is activated, log messages are collected from the journald daemon in Linux environments.

This input type requires the agent to run in root mode.

Example: cupsd

max_line_kb setting

Maximum size of log entries/lines in KB. If log entries exceed the limit, they are skipped.

Default value is 128.

pattern filter

Allows filtering by matching specific records based on a regular expression.

You can set as many source parameters as you need per configuration file.

Test log forwarding

To test if log forwarding through the Infrastructure agent is working:

  1. Run the following command to append a test log message to your log file:

    echo "This is a test message." >> /PATH/TO/YOUR/LOG/FILE	
    

    For example:

    echo "This is a test message." >> /var/log/test.log
  2. Search New Relic Logs for test message.

View log data

If everything is configured correctly and data is being collected, you should see data in both of these places:

Troubleshoot log forwarding

Here are some troubleshooting tips:

No data appears

If no data appears after you enable New Relic Logs, follow standard Logs troubleshooting procedures.

Send the agent's logs to New Relic Logs

The Infrastructure agent can be configured to send its own logs to New Relic Logs. This can be useful for troubleshooting issues with log forwarding, the Infrastructure agent, or when contacting Support.

To forward the Infrastructure agent logs to New Relic Logs:

  1. Edit your newrelic-infra.yml file.

  2. Enable agent logging in troubleshooting mode by adding verbose: 3

    Recommendation: Enable agent logging in JSON format to log_format: json.

  3. Restart the agent so that the new settings can be loaded.

On systems that don't use systemd or where journald is not accessible, like Windows, verbose:3 causes the agent to write the logs on the disk. Revert to verbose:0 to prevent this.

Runtime error on Windows

The following error message may appear when enabling log forwarding on Windows:

The code execution cannot proceed because VCRUNTIME140.dll was not found.

To solve the issue, install the Microsoft Visual C++ Redistributable: x64 or x86.

What's next?

Now that you've enabled Logs and log forwarding through the Infrastructure agent, here are some next steps:

For more help

Recommendations for learning more: