You can forward your logs to New Relic using our infrastructure monitoring agent. This makes all of your logging data available in one location and provides deeper visibility into both your application and your platform performance data.
To use log management and the rest of our observability platform, join the New Relic family! Sign up to create your free account in only a few seconds. Then ingest up to 100GB of data for free each month. Forever.
To enable log forwarding through the infrastructure agent:
- Start by verifying the system requirements needed for configuring Logs.
- Ensure you have installed the infrastructure agent, version 1.11.4 or higher.
- Create a
logging.ymlfile in the infrastructure agent's
- Configure your log sources and other parameters.
- Generate some traffic and wait a few minutes, then check your account for data.
- Explore your log data in the Logs UI and benefit from the log attributes automatically inserted by the infrastructure agent.
To use the log forwarder of the infrastructure agent, make sure you meet the following requirements:
- Infrastructure agent version 1.11.4 or higher
- OpenSSL library 1.1.0 or higher is required by
infra-agentstarting from v1.16.4.
The log forwarding feature is compatible with the following operating systems:
Amazon Linux 2
Version 7 or higher
Version 9 ("Stretch") or higher
Red Hat Enterprise Linux (RHEL)
Version 7 or higher
SUSE Linux Enterprise Server (SLES)
Versions 16.04.x, 18.04.x and 20.04.x (LTS versions)
Windows Server 2012, 2016, and 2019, and their service packs.
The log forwarding feature is not supported on containerized infrastructure agents.
To use the following links, make sure you are logged to your New Relic account.
If you don't have a New Relic account yet, or prefer to follow the procedure manually, see our tutorial to install the package manager.
Configuration files describe which log sources are forwarded. The Infrastructure agent uses
.yml files for configuring logging. You can add as many config files as you want.
To add a new configuration file for the log forwarding feature:
Navigate to the logging forwarder configuration folder:
C:\Program Files\New Relic\newrelic-infra\logging.d\
logging.ymlconfiguration file and add the parameters you need. The
logging.ddirectory has various
.yml.examplefiles that can be used as a reference or starting point.
The agent automatically processes new configuration files without having to restart the Infrastructure service. The only exception to this is when configuring a custom Fluent Bit configuration.
The infrastructure log forwarding
.yml config supports the following parameters:
To start, make sure to define a name for the logs you want to forward:
What you use for the log source will depend on where you want to forward your logs from. The available options are listed below:
The following are configuration paramaters that are not required but are still recommended.
Here is an example of a
logging.d/ configuration file in YAML format. For more configuration examples, see the infrastructure agent repository.
If everything is configured correctly and data is being collected, you should see data in both of these places:
New Relic Logs UI
New Relic tools for running NRQL queries; for example, you can execute a query like this:SELECT * FROM Log
Fluent Bit's tail plugin does not support network drives.
If no data appears after you enable log management, follow standard troubleshooting procedures.
The log forwarding feature requires the agent to have permission to read the data sources. When running the infrastructure agent in privileged or non-privileged modes, make sure that the log files you want to forward (and any intermediary directory in its path) are readable by the user running
The log forwarding feature requires that the agent has permission to read the data sources. When running the infrastructure agent in privileged or non-privileged modes:
- If you're using Unix domain socket files, make sure that the
nri-agentuser can access these files (please refer to the previous section) and that they have read and write permissions (
666) so that other users than
nri-agentcan write to them.
- If you're using IP sockets, ensure that the port that you are using is not a system reserved one (like port
80, for example).
If no data appears after you enable log management, follow standard log management troubleshooting procedures.
As explained in the infrastructure agent configuration guidelines, the
proxy parameter must use either HTTP or HTTPS and be in the form https://user:password@hostname:port. The agent can parse the parameter without the HTTP or HTTPS, but the log-forwarder cannot. You will see an error like the following in the agent verbose logs:
[ERROR] building HTTP transport: parse \"hostname:port\":first path segment in URL cannot contain colon
To solve this problem, check your
newrelic-infra.yml file, and ensure the
proxy parameter adheres to this form.
To forward the infrastructure agent logs to New Relic:
Enable agent logging in troubleshooting mode by adding
On Windows and systems that don't use
verbose:3causes the agent to write the logs on the disk. Revert to
verbose:0to prevent this.
(Recommended): Enable agent logging in JSON format to
Restart the agent to load the new settings.
This configuration sets up the agent in troubleshooting mode, but the log forwarder (based on Fluent Bit) will continue in a non-verbose mode. Sometimes you can have issues with the log forwarder itself. For example, there may be problems accessing a specific channel when shipping Windows log events or when accessing a particular log file.
In these situations, you can also enable the verbose mode for the log forwarder:
verboseto a value other than
- Add the following configuration option:
Check whether you are using the
fluentbit option. When setting
verbose: 3 and
trace: ["log.fw"], ensure that you don't define any
[OUTPUT] section pointing to
stdout in an external Fluent Bit configuration file,
For Linux versions previous to 2016 you may need to update the OpenSSL library to 1.1.0 (or higher). To check if you have this problem:
- See if
infra-agenthas started Fluent Bit by doing
ps -aux | grep fluent-bit
- If it isn't running go to
./fluent-bit -i systemd -o stdout
- If you get the following error, update OpenSSL to 1.1.0 or higher:
error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
One of the following error messages may appear when enabling log forwarding on Windows:
The code execution cannot proceed because VCRUNTIME140.dll was not found.
error="exit status 3221225781" process=log-forwarder
This is caused by a missing DLL.
Now that you've enabled Logs, here are some potential next steps: