• /
  • Log in

Forward your logs using the infrastructure agent

You can forward your logs to New Relic using our infrastructure monitoring agent. This makes all of your logging data available in one location and provides deeper visibility into both your application and your platform performance data.

Enable log forwarding using the infrastructure agent

To enable log forwarding through the infrastructure agent:

  1. If you haven't already, create a New Relic account. It's free, forever.
  2. Start by verifying the system requirements needed for configuring Logs.
  3. Ensure you have installed the infrastructure agent, version 1.11.4 or higher.
  4. Create a logging.yml file in the infrastructure agent's logging.d directory.
  5. Configure your log sources and other parameters.
  6. Generate some traffic and wait a few minutes, then check your account for data.
  7. Explore your log data in the Logs UI and benefit from the log attributes automatically inserted by the infrastructure agent.

System Requirements

To use the log forwarder of the infrastructure agent, make sure you meet the following requirements:

  • Infrastructure agent version 1.11.4 or higher
  • OpenSSL library 1.1.0 or higher is required by infra-agent starting from v1.16.4.

The log forwarding feature is compatible with the following operating systems:

Operating system

Supported version

Amazon Linux

Amazon Linux 2

CentOS

Version 7 or higher

Debian

Version 9 ("Stretch") or higher

Red Hat Enterprise Linux (RHEL)

Version 7 or higher

SUSE Linux Enterprise Server (SLES)

Version 12

Ubuntu

Versions 16.04.x, 18.04.x and 20.04.x (LTS versions)

Windows

Windows Server 2012, 2016, and 2019, and their service packs.

Windows 10

Caution

The log forwarding feature is not supported on containerized infrastructure agents.

Install the infrastructure agent

Starting with version 1.11.4, the infrastructure agent can forward logs to New Relic. To install and run the agent, use a package manager (Linux) or the MSI installer (Windows).

To use the following links, make sure you are logged to your New Relic account.

Amazon Linux
Amazon Linux
CentOS
CentOS
Debian
Debian
Red Hat
RHEL
SLES
SLES
Ubuntu
Ubuntu
amazonlinux.png
Windows

Tip

We are working to integrate log forwarder in the infrastructure agent for arm architecture, meanwhile check this post with the steps you can follow.

If you don't have a New Relic account yet, or prefer to follow the procedure manually, see our tutorial to install the package manager.

Configure the infrastructure agent

Configuration files describe which log sources are forwarded. The Infrastructure agent uses .yml files for configuring logging. You can add as many config files as you want.

To add a new configuration file for the log forwarding feature:

  1. Navigate to the logging forwarder configuration folder:

    • Linux: /etc/newrelic-infra/logging.d/
    • Windows: C:\Program Files\New Relic\newrelic-infra\logging.d\
  2. Create a logging.yml configuration file and add the parameters you need. The logging.d directory has various .yml.example files that can be used as a reference or starting point.

  3. The agent automatically processes new configuration files without having to restart the Infrastructure service. The only exception to this is when configuring a custom Fluent Bit configuration.

Log forwarding parameters

The infrastructure log forwarding .yml config supports the following parameters:

Name (required)

To start, make sure to define a name for the logs you want to forward:

Log source (required)

What you use for the log source will depend on where you want to forward your logs from. The available options are listed below:

Optional Configuration

The following are configuration paramaters that are not required but are still recommended.

Sample configuration file

Here is an example of a logging.d/ configuration file in YAML format. For more configuration examples, see the infrastructure agent repository.

View your log data

If everything is configured correctly and data is being collected, you should see data in both of these places:

  • New Relic Logs UI

  • New Relic tools for running NRQL queries; for example, you can execute a query like this:

    SELECT * FROM Log

Troubleshoot log forwarding

Important

Fluent Bit's tail plugin does not support network drives.

If no data appears after you enable log management, follow standard troubleshooting procedures.

No data appears when tailing a file

The log forwarding feature requires the agent to have permission to read the data sources. When running the infrastructure agent in privileged or non-privileged modes, make sure that the log files you want to forward (and any intermediary directory in its path) are readable by the user running nri-agent.

No data appears when capturing via a Syslog socket

The log forwarding feature requires that the agent has permission to read the data sources. When running the infrastructure agent in privileged or non-privileged modes:

  • If you're using Unix domain socket files, make sure that the nri-agent user can access these files (please refer to the previous section) and that they have read and write permissions (666) so that other users than nri-agent can write to them.
  • If you're using IP sockets, ensure that the port that you are using is not a system reserved one (like port 80, for example).

If no data appears after you enable log management, follow standard log management troubleshooting procedures.

No data appears using infrastructure agent proxy

As explained in the infrastructure agent configuration guidelines, the proxy parameter must use either HTTP or HTTPS and be in the form https://user:password@hostname:port. The agent can parse the parameter without the HTTP or HTTPS, but the log-forwarder cannot. You will see an error like the following in the agent verbose logs:

[ERROR] building HTTP transport: parse \"hostname:port\":
first path segment in URL cannot contain colon

To solve this problem, check your newrelic-infra.yml file, and ensure the proxy parameter adheres to this form.

Send the agent's logs to New Relic

You can configure the infrastructure agent to send its own logs to New Relic. This is useful for troubleshooting issues with log forwarding, the agent, or when contacting support.

To forward the infrastructure agent logs to New Relic:

  1. Edit your newrelic-infra.yml file.

  2. Enable agent logging in troubleshooting mode by adding verbose: 3

    Important

    On Windows and systems that don't use systemd or where journald is inaccessible, verbose:3 causes the agent to write the logs on the disk. Revert to verbose:0 to prevent this.

  3. (Recommended): Enable agent logging in JSON format to log_format: json.

  4. Restart the agent to load the new settings.

This configuration sets up the agent in troubleshooting mode, but the log forwarder (based on Fluent Bit) will continue in a non-verbose mode. Sometimes you can have issues with the log forwarder itself. For example, there may be problems accessing a specific channel when shipping Windows log events or when accessing a particular log file.

In these situations, you can also enable the verbose mode for the log forwarder:

  1. Set verbose to a value other than 0.
  2. Add the following configuration option: trace: ["log.fw"].

Caution

Check whether you are using the fluentbit option. When setting verbose: 3 and trace: ["log.fw"], ensure that you don't define any [OUTPUT] section pointing to stdout in an external Fluent Bit configuration file,

Fluent Bit does not start with the infra-agent

For Linux versions previous to 2016 you may need to update the OpenSSL library to 1.1.0 (or higher). To check if you have this problem:

  1. See if infra-agenthas started Fluent Bit by doing
ps -aux | grep fluent-bit
  1. If it isn't running go to /var/db/newrelic-infra/newrelic-integrations/logging and run
./fluent-bit -i systemd -o stdout
  1. If you get the following error, update OpenSSL to 1.1.0 or higher: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory

Runtime error on Windows

One of the following error messages may appear when enabling log forwarding on Windows:

The code execution cannot proceed because VCRUNTIME140.dll was not found.

OR

error="exit status 3221225781" process=log-forwarder

This is caused by a missing DLL.

To solve the issue, install the Microsoft Visual C++ Redistributable: x64 or x86.

What's next?

Now that you've enabled Logs, here are some potential next steps:

For more help

If you need more help, check out these support and learning resources:

Create issueEdit page
Copyright © 2021 New Relic Inc.