You can forward your logs to New Relic using our infrastructure monitoring agent. This makes all of your logging data available in one location and provides deeper visibility into both your application and your platform performance data.
To use log management and the rest of our observability platform, join the New Relic family! Sign up to create your free account in only a few seconds. Then ingest up to 100GB of data for free each month. Forever.
Enable log forwarding using the infrastructure agent
To enable log forwarding through the infrastructure agent:
- Start by checking what you need.
- Install the infrastructure agent, version 1.11.4 or higher.
- Configure your log sources and other parameters in the
logging.d
directory. - Generate some traffic and wait a few minutes, then check your account for data.
- Explore your log data in the Logs UI and benefit from the log attributes automatically inserted by the infrastructure agent.
What you need
The log forwarding feature is compatible with the following operating systems:
Operating system | Supported version |
---|---|
Amazon Linux | Amazon Linux 2 |
CentOS |
Version 7 or higher |
Debian | Version 9 ("Stretch") or higher |
Red Hat Enterprise Linux (RHEL) | Version 7 or higher |
SUSE Linux Enterprise Server (SLES) | Version 12 |
Ubuntu | Versions 16.04.x and 18.04.x (LTS versions) |
Windows |
Windows Server 2012, 2016, and 2019, and their service packs. Windows 10 |
To use the log forwarder of the infrastructure agent, make sure you meet the following requirements:
-
Infrastructure agent version 1.11.4 or higher
- Windows: Install the Microsoft Visual C++ Redistributable: x64 or x86.
The log forwarding feature is not supported on containerized agents.
Install the infrastructure agent
Starting with version 1.11.4, the infrastructure agent can forward logs to New Relic. To install and run the agent, use a package manager (Linux) or the MSI installer (Windows).
To use the following links, make sure you are logged to your New Relic account.








If you don't have a New Relic account yet, or prefer to follow the procedure manually, see our tutorial to install the package manager.
Test log forwarding
To quickly check that the log forwarding feature works, follow these steps.
-
If you're running the infrastructure monitoring agent in privileged or non-privileged modes, make sure that the user executing
nri-agent
has read permissions for the data sources. -
Navigate to the logging forwarder configuration folder:
-
Linux:
/etc/newrelic-infra/logging.d/
-
Windows:
C:\Program Files\New Relic\newrelic-infra\logging.d\
-
-
Create a configuration file (for example,
logs.yml
) with this content:-
Linux example:
# Remember to only use spaces for indentation logs: - name: "test_log" file: /var/log/test.log
-
Windows example:
# Remember to only use spaces for indentation logs: - name: "test_log" file: 'C:\ProgramData\New Relic\newrelic-infra\newrelic-infra.log'
-
-
Run the following command to append a test log message to your log file:
echo "This is a test message." >> /PATH/TO/YOUR/LOG/FILE
For example:
echo "This is a test message." >> /var/log/test.log
- Search New Relic Logs for
test message
.
Configure the infrastructure agent
Configuration files describe which log sources are forwarded. You can add as many config files as you want, and set as many sources as you need per config file.
To add a new configuration file for the log forwarding feature:
-
Navigate to the logging forwarder configuration folder:
-
Linux:
/etc/newrelic-infra/logging.d/
-
Windows:
C:\Program Files\New Relic\newrelic-infra\logging.d\
-
-
Create a configuration file with the parameters you need. Use our sample config file as reference.
The agent automatically processes new configuration files without having to restart it.
Log forwarding parameters
The log forwarder config supports the following parameters (for examples, see the sample configuration).
name
Name of the log or logs.
attributes
List of custom attributes, as key-value pairs, that can be used to send additional data with the logs which you can then query. For example, you can enable built-in parsing rules by setting the logtype
attribute.
Example:
... - name: tcp-simple-test tcp: uri: tcp://0.0.0.0:1234 format: none separator: \t attributes: # You can add custom attributes to any source of logs tcpFormat: none logtype: nginx # See https://docs.newrelic.com/docs/logs/log-management/ui-data/new-relic-logs-parsing-built-rules-custom-parsing someOtherAttribute: associatedValue
file
Path to the log file or files. The agent tracks changes on the log files in a way similar to tail -f shell
.
Your file
can point to a specific log file or multiple ones by using wildcards applied to names and extensions; for example, /logs/*.log
Example:
logs: - name: file-with-attributes file: /var/log/test.log # Path to a single file or pattern pattern: Error # Regular expression to filter log entries
Wildcards can also be used in place of directories in a file path, which can be used to tail files located in different directories.
Example:
logs: - name: docker-logs file: /var/lib/docker/containers/*/*.log # Path to multiple folders and files pattern: redis # Regular expression to filter log entries
Use of wildcards may significantly increase the number of file descriptors the Fluent Bit process keeps open, which can interfere with log collection if the host's file descriptor limit is reached.
We recommend increasing the file descriptor limit on Linux hosts running Fluent Bit by adding the following to the host's /etc/security/limits.conf
file:
root soft nofile 65536
root hard nofile 65536
*soft nofile 65536
*hard nofile 65536
Once you add these changes, reboot the host to ensure your changes are applied.
systemd
(Linux only)
Service name. Once the systemd
input is activated, log messages are collected from the journald
daemon in Linux environments.
This input type requires the agent to run in root mode.
Example:
logs: - name: systemd-example systemd: cupsd
syslog
(Linux only)
Syslog data source.
Parameters:
uri:
Syslog socket. Format varies depending on the protocol:- TCP/UDP network sockets:
[tcp/udp]://LISTEN_ADDRESS:PORT
- Unix domain sockets:
unix_[tcp/udp]:// + /socket/path
- TCP/UDP network sockets:
parser:
Syslog parser. Default isrfc3164
. Userfc5424
if your messages include fractional seconds. Note:rfc3164
currently does not work on SuSE.unix_permissions:
default is0644
for domain sockets; this limits entries to processes running as root. You can use0666
to listen for non-root processes, at your own risk.
When running the agent in privileged mode, ports and sockets must be available or owned by nri-agent
, with 0666
file permissions, so that other processes can write logs to the sockets.
logs: # TCP network socket - name: syslog-tcp-test syslog: uri: tcp://0.0.0.0:5140 # Use the tcp://LISTEN_ADDRESS:PORT format parser: rfc5424 # Default syslog parser is rfc3164 # UDP network socket - name: syslog-udp-test syslog: uri: udp://0.0.0.0:6140 # Use the udp://LISTEN_ADDRESS:PORT format max_line_kb: 35 # Unix TCP domain socket - name: syslog-unix-tcp-test syslog: uri: unix_tcp:///var/unix-tcp-socket-test unix_permissions: 0666 # Default is 0644. Change at your own risk # Unix UDP domain socket - name: syslog-unix-udp-test syslog: uri: unix_udp:///var/unix-udp-socket-test parser: rfc5424
tcp
Logs retrieved over TCP connections.
Parameters:
uri:
TCP/IP socket to listen for incoming data. The URI format istcp://LISTEN_ADDRESS:PORT
format:
format of the data. It can bejson
ornone
.separator:
Ifformat: none
is used, you can define a separator string for splitting records (default:\n
).
logs: - name: tcp-simple-test tcp: uri: tcp://0.0.0.0:1234 # Use the tcp://LISTEN_ADDRESS:PORT format format: none # Raw text - this is default for 'tcp' separator: \t # String for separating raw text entries max_line_kb: 32 - name: tcp-json-test tcp: uri: tcp://0.0.0.0:2345 # Use the tcp://LISTEN_ADDRESS:PORT format format: json
max_line_kb
Maximum size of log entries/lines in KB. If log entries exceed the limit, they are skipped. Default is 128
.
winlog
Collect events from Windows log channels.
Parameters:
channel:
name of the channel logs will be collected from.collect-eventids:
a list of Windows Event IDs to be collected and forwarded to New Relic. Event ID ranges are supported.exclude-eventids:
a list of Windows Event IDs to be excluded from collection. Event ID ranges are supported.
All events are collected from the specified channel by default. Configure the collect-eventids
and exclude-eventids
sections to avoid sending unwanted logs to your New Relic account.
Add event IDs or ranges to collect-eventids
or exclude-eventids
to forward or drop specific events. exclude-eventids
takes precedence over collect-eventids
if the same event ID is present in both sections.
Example:
logs: - name: windows-security winlog: channel: Security collect-eventids: - 4624 - 4265 - 4700-4800 exclude-eventids: - 4735
pattern
Regular expression for filtering records. Only supported for the tail, systemd, syslog, and tcp (only with format none
) sources.
This field works in a way similar to grep -E
in Unix systems. For example, for a given file being captured, you can filter for records containing either WARN
or ERROR
using:
- name: only-records-with-warn-and-error file: /var/log/logFile.log pattern: WARN|ERROR
No filtering is applied by default.
fluentbit
External Fluent Bit configuration and parser files. If defined, they are merged with the existing configuration and parser files generated by the Infrastructure agent.
The infrastructure agent processes the configuration files located in the logging.d
directory and will generate a run-time Fluent Bit configuration file that contains the appropriate [INPUT]
, [FILTER]
and [OUTPUT]
sections. Optionally, it will also declare an @INCLUDE
in case you provided an external Fluent Bit configuration file via the fluentbit
option.
The runtime file does not define a [SERVICE]
section, leaving all default Fluent Bit configuration values. You can still override Fluent Bit's default settings by defining your own [SERVICE]
section in your external Fluent Bit configuration file and include it via the fluentbit
option.
Parameters:
config_file:
path to an existing Fluent Bit configuration file. Note that any overlapping source results in duplicate messages in New Relic Logs.
parsers_file:
path to an existing Fluent Bit parsers file. The following parser names are reserved: rfc3164
, rfc3164-local
and rfc5424
.
Sample configuration file
Here is an example of a logging.d/
configuration file in YAML format. For more configuration examples, see the infrastructure agent repository.
- logging.d/sample.yaml
-
# Remember to only use spaces for indentation logs: # Example of 'file' source - name: file-with-attributes file: /var/log/test.log # Path to a single file or pattern attributes: # You can use custom attributes to enrich your data logtype: nginx team: The A Team pattern: Error # Regular expression to filter log entries # Example of 'systemd' source (Linux only) - name: systemd-example systemd: cupsd # Examples of 'syslog' source, one per protocol # TCP network socket - name: syslog-tcp-test syslog: uri: tcp://0.0.0.0:5140 # Use the tcp://LISTEN_ADDRESS:PORT format parser: rfc5424 # Default syslog parser is rfc3164 # UDP network socket - name: syslog-udp-test syslog: uri: udp://0.0.0.0:6140 # Use the udp://LISTEN_ADDRESS:PORT format max_line_kb: 35 # Paths for Unix sockets are defined by combining protocol and path: # unix_udp:// + /path/socket - for example, unix_udp:///tmp/socket # Unix TCP domain socket - name: syslog-unix-tcp-test syslog: uri: unix_tcp:///var/unix-tcp-socket-test unix_permissions: 0666 # Default is 0644. Change at your own risk # Unix UDP domain socket - name: syslog-unix-udp-test syslog: uri: unix_udp:///var/unix-udp-socket-test parser: rfc5424 # Examples of 'tcp' source for formats 'none' and 'json' - name: tcp-simple-test tcp: uri: tcp://0.0.0.0:1234 # Use the tcp://LISTEN_ADDRESS:PORT format format: none # Raw text - this is default for 'tcp' separator: \t # String for separating raw text entries attributes: # You can add custom attributes to any source of logs tcpFormat: none someOtherAttribute: associatedValue max_line_kb: 32 - name: tcp-json-test tcp: uri: tcp://0.0.0.0:2345 # Use the tcp://LISTEN_ADDRESS:PORT format format: json attributes: tcpFormat: json yetAnotherAttribute: 12345 # Example of Fluent Bit configuration import - name: fluentbit-import fluentbit: config_file: /path/to/fluentbit.config parsers_file: /path/to/fluentbit/parsers.conf
View your log data
If everything is configured correctly and data is being collected, you should see data in both of these places:
-
New Relic tools for running NRQL queries; for example, you can execute a query like this:
SELECT * FROM Log
Log attributes automatically inserted by the infrastructure agent
The infrastructure agent automatically inserts log attributes for your convenience. Some of them are inserted for any log record, while other depend on the configuration parameters you used while setting up the Log Forwarder. Find a summary of these attributes in the following table:
Attribute name | Inserting conditions | Description |
---|---|---|
entity.guids |
Always inserted |
The infrastructure agent inserts the Entity GUID assigned by New Relic to identify the host where it's running. It is available in the Note: If the captured logs belong to an application instrumented using APM, the |
fb.input |
Always inserted | The underlying Fluent Bit input plugin type used to capture the logs. Currently, its values are tail , systemd , winlog , syslog , and tcp . |
filePath |
When using the file input type |
Absolute file path of the file being monitored. |
hostname |
Always inserted | The hostname of the machine/VM/container executing the infrastructure agent. |
plugin.type |
Always inserted | Indicates the utility used to capture the logs. In this case, it is the infrastructure agent itself, so this attribute always has the value nri-agent . |
Troubleshoot log forwarding
If no data appears after you enable log management, follow standard troubleshooting procedures.
No data appears when tailing a file
The log forwarding feature requires the agent to have permission to read the data sources. When running the infrastructure agent in privileged or non-privileged modes, make sure that the log files you want to forward (and any intermediary directory in its path) are readable by the user running nri-agent
.
- Example: checking file access under Linux
-
Let's check whether the file
/var/log/restrictedLogs/logFile.log
can be monitored by thenri-agent
user. In Linux, you can do a quick check with thenamei
command:sudo -u nri-agent namei -ml /var/log/restrictedLogs/logFile.log f: /var/log/restrictedLogs/logFile.log drwxr-xr-x root root / drwxr-xr-x root root var drwxrwxr-x root syslog log drwxr--r-- root root restrictedLogs logFile.log - No such file or directory
This command failed because the file is not visible to the
nri-agent
user. By inspecting the previous output, we can detect that therestrictedLogs
directory is missing the execution flag forothers"
.We can fix this by executing:
sudo chmod 755 /var/log/restrictedLogs
And then check for file access again:
# sudo -u nri-agent namei -ml /var/log/restrictedLogs/logFile.log f: /var/log/restrictedLogs/logFile.log drwxr-xr-x root root / drwxr-xr-x root root var drwxrwxr-x root syslog log drwxr-xr-x root root restrictedLogs -rw-r----- vagrant vagrant logFile.log
The file is now visible to the
nri-agent
user. You must ensure that the file is also readable by thenri-agent
user. To check this, use:# sudo -u nri-agent head /var/log/restrictedLogs/logFile.log head: cannot open '/var/log/restrictedLogs/logFile.log' for reading: Permission denied
In this example, the file is missing the read rights for the
others
group (users other thanvagrant
and thevagrant
user group). You could fix this by granting read permissions toothers
, but the application could change these permissions upon restart.To avoid this, a better approach is to add the
nri-agent
user to thevagrant
user group.
No data appears when capturing via a Syslog socket
The log forwarding feature requires that the agent has permission to read the data sources. When running the Infrastructure agent in privileged or non-privileged modes:
- If you're using Unix domain socket files, make sure that the
nri-agent
user can access these files (please refer to the previous section) and that they have read and write permissions (666
) so that other users thannri-agent
can write to them. - If you're using IP sockets, ensure that the port that you are using is not a system reserved one (like port
80
, for example).
If no data appears after you enable log management, follow standard log management troubleshooting procedures.
No data appears using infrastructure agent proxy
As explained in the infrastructure agent configuration guidelines, the proxy
parameter must use either HTTP or HTTPS and be in the form https://user:password@hostname:port. The agent can parse the parameter without the HTTP or HTTPS, but the log-forwarder cannot. You will see an error like the following in the agent verbose logs:
[ERROR] building HTTP transport: parse \"hostname:port\":
first path segment in URL cannot contain colon
To solve this problem, check your newrelic-infra.yml
file, and ensure the proxy
parameter adheres to this form.
Send the agent's logs to New Relic
The infrastructure agent can be configured to send its own logs to New Relic. This can be useful for troubleshooting issues with log forwarding, the agent, or when contacting support.
To forward the infrastructure agent logs to New Relic:
-
Edit your
newrelic-infra.yml
file. - Enable agent logging in troubleshooting mode by adding
verbose: 3
On Windows and systems that don't use
systemd
or wherejournald
is inaccessible,verbose:3
causes the agent to write the logs on the disk. Revert toverbose:0
to prevent this. - (Recommended): Enable agent logging in JSON format to
log_format: json
. -
Restart the agent so that the new settings can be loaded.
This configuration sets up the agent in troubleshooting mode, but the log forwarder (based on Fluent Bit) will continue in a non-verbose mode. Sometimes you can have issues with the log forwarder itself. For example, there may be problems accessing a specific channel when shipping Windows log events or when accessing a particular log file.
In these situations, you can also enable the verbose mode for the log forwarder:
- Set
verbose
to a value other than0
. - Add the following configuration option:
trace: ["log.fw"]
.
Check whether you are using the fluentbit
option. When setting verbose: 3
and trace: ["log.fw"]
, ensure that you don't define any [OUTPUT]
section pointing to stdout
in an external Fluent Bit configuration file,
Runtime error on Windows
One of the following error messages may appear when enabling log forwarding on Windows:
The code execution cannot proceed because VCRUNTIME140.dll was not found.
OR
error="exit status 3221225781" process=log-forwarder
This is caused by a missing DLL.
To solve the issue, install the Microsoft Visual C++ Redistributable: x64 or x86.
What's next?
Now that you've enabled Logs, here are some potential next steps:
- Explore your data using the Logs UI.
- Configure your agent to see contextual log data, such as distributed tracing, stack traces, application logs, and more.
- Query your data and create custom dashboards or alerts.