• English日本語한국어
  • Log inStart now

Fluentd plugin for log forwarding

If your log data is already being monitored by Fluentd, you can use our Fluentd integration to forward and enrich your log data in New Relic.

Forwarding your Fluentd logs to New Relic will give you enhanced capabilities to collect, process, explore, query, and alert on your log data.

Basic process

To enable log management capabilities:

  1. Make sure you have:
  • A New Relic
  • Fluentd 1.0 or higher
  1. Install the Fluentd plugin.
  2. Configure the Fluentd plugin.
  3. If applicable, configure EU endpoint
  4. Test the Fluentd plugin.
  5. Generate some traffic and wait a few minutes, then check your account for data.

Install the Fluentd plugin

To install the Fluentd plugin:

Configure the Fluentd plugin

Tip

If you're configuring Fluentd for the first time, you may find it helpful to review our collection of pre-built configuration files addressing common use cases.

To configure your Fluentd plugin:

  1. In your fluent.conf file, or td-agent.conf if using the td-agent, add the following block of data, replacing the placeholder text with your :
#Tail one or more log files
<source>
@type tail
<parse>
@type none
</parse>
path /path/to/file
tag example.service
</source>
#Add hostname and service_name to all events with "example.service" tag
<filter example.service>
@type record_transformer
<record>
service_name ${tag}
hostname "#{Socket.gethostname}"
</record>
</filter>
#Forward all events to New Relic
<match <DoNotTranslate>**>
@type newrelic
license_key YOUR_LICENSE_KEY
</match>
  1. Restart the Fluentd service to ensure your changes are applied.

Did this doc help with your installation?

Configure the Fluentd Plugin for EU accounts

By default the Fluentd plugin forwards logs to our US endpoint: https://log-api.newrelic.com/log/v1. If your New Relic organization is in our EU data center, you must manually set the base_uri property to the EU endpoint. For example:

#Tail one or more log files
<source>
@type tail
<parse>
@type none
</parse>
path /path/to/file
tag example.service
</source>
#Add hostname and service_name to all events with "example.service" tag
<filter example.service>
@type record_transformer
<record>
service_name ${tag}
hostname "#{Socket.gethostname}"
</record>
</filter>
#Forward all events to New Relic EU Endpoint
<match **</DoNotTranslate>>
@type newrelic
license_key YOUR_LICENSE_KEY
base_uri https://log-api.eu.newrelic.com/log/v1
</match>

Test the Fluentd plugin

To test if your Fluentd plugin is receiving input from a log file:

  1. Run the following command to append a test log message to your log file:
echo "test message" >> /PATH/TO/YOUR/LOG/FILE
  1. Search our logs UI for test message.

View log data

If everything is configured correctly and your data is being collected, you should see logs in both of these places:

SELECT * FROM Log

If no data appears after you enable our log management capabilities, follow our standard log troubleshooting procedures.

Tune up log Fluentd buffer

By default, the plugin sends logs to New Relic every five seconds. If you want to change this timing, add a <buffer> block to the configuration by following this example:

[...]
# Forward all events to New Relic EU Endpoint
<match <DoNotTranslate>**>
@type newrelic
license_key YOUR_LICENSE_KEY
<buffer time>
timekey 60s
</buffer>
</match>

For more information, see the Fluentd documentation about buffer configurations.

Configure UTF-16LE to UTF-8 transformation

In this example for Microsoft SQL Server error logs, use Fluentd to send UTF-16LE encoded logs to New Relic with the required UTF-8 encoding for ingest. You can also adopt this example to other encodings.

Tip

We also add an appropriate logtype for these logs.

#Tail one or more log files
<source>
@type tail
<parse>
@type none
</parse>
path "D:/sqlserver/error.log"
tag example.service
encoding UTF-8
from_encoding UTF-16LE
</source>
#Add hostname and service_name to all events with "example.service" tag
<filter example.service>
@type record_transformer
<record>
service_name ${tag}
hostname "#{Socket.gethostname}"
logtype MSSQL
</record>
</filter>
#Forward all events to New Relic
<match **</DoNotTranslate>>
@type newrelic
license_key YOUR_LICENSE_KEY
</match>

What's next?

Explore logging data across your platform with our logs UI.

Disable log forwarding

To disable log forwarding capabilities, follow standard procedures in Fluentd documentation. You do not need to do anything else in New Relic.

Copyright © 2024 New Relic Inc.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.