Fluentd plugin for log forwarding

New Relic offers a Fluentd output plugin to connect your Fluentd monitored log data to New Relic. Read on to learn how to enable this feature.

Compatibility and requirements

To forward logs to New Relic using Fluentd, ensure your configuration meets the following requirements:

• New Relic license key
• Fluentd 1.0 or higher

Enable Fluentd for New Relic log management

To enable log management with Fluentd:

1. Install the Fluentd plugin.
2. Configure the Fluentd plugin.
3. Optional: Configure EU Endpoint
4. Test the Fluentd plugin.
5. Generate some traffic and wait a few minutes, then check your account for data.

Install the Fluentd plugin

To install the Fluentd plugin:

fluent-gem install fluent-plugin-newrelic

td-agent-gem install fluent-plugin-newrelic

Configure the Fluentd plugin

To configure your Fluentd plugin:

1. In your fluent.conf file, or td-agent.conf if using the td-agent, add the following block of data, replacing the placeholder text with your New Relic license key:

#Tail one or more log files
<source>
  @type tail
  <parse>
    @type none
  </parse>
  path /path/to/file
  tag example.service
</source>

#Add hostname and service_name to all events with "example.service" tag
<filter example.service>
  @type record_transformer
  <record>
    service_name ${tag}
    hostname "#{Socket.gethostname}"
  </record>
</filter>

#Forward all events to New Relic
<match **>
  @type newrelic
  license_key YOUR_LICENSE_KEY
</match>

2. Restart the Fluentd service to ensure your changes are applied.

Configure the Fluentd Plugin for EU accounts

By default the Fluentd plugin forwards logs to New Relic's US endpoint: https://log-api.newrelic.com/log/v1. If your account is in New Relic's EU datacenter then you'll need to manually set the base_uri property to the EU endpoint. For example:

#Tail one or more log files
<source>
  @type tail
  <parse>
    @type none
  </parse>
  path /path/to/file
  tag example.service
</source>

#Add hostname and service_name to all events with "example.service" tag
<filter example.service>
  @type record_transformer
  <record>
    service_name ${tag}
    hostname "#{Socket.gethostname}"
  </record>
</filter>

#Forward all events to New Relic EU Endpoint
<match **>
  @type newrelic
  license_key YOUR_LICENSE_KEY
  base_uri https://log-api.eu.newrelic.com/log/v1
</match>

Test the Fluentd plugin

To test if your Fluentd plugin is receiving input from a log file:

1. Run the following command to append a test log message to your log file:

   echo "test message" >> /PATH/TO/YOUR/LOG/FILE

   Copy

2. Search New Relic Logs UI for test message.

View log data

If everything is configured correctly and your data is being collected, you should see data logs in both of these places:

• New Relic Logs UI
• New Relic tools for running NRQL queries. For example, you can execute a query like this:

SELECT * FROM Log

Tune-up log Fluentd buffer

By default, the plugin sends logs to New Relic One every 5 seconds. If you want to change this timing, add a <buffer> block to the configuration by following this example:

[...]

# Forward all events to New Relic EU Endpoint
<match **>
  @type newrelic
  license_key YOUR_LICENSE_KEY

  <buffer time>
    timekey 60s
  </buffer>
</match>

To read more about all the available configurations for buffers, check out the fluentd documentation about Buffer configurations.

What's next?

Now that you've enabled Logs, here are some potential next steps:

• Explore your data using the Logs UI.
• Configure your agent to see contextual log data, such as distributed tracing, stack traces, application logs, and more.
• Query your data and create custom dashboards or alerts.

If no data appears after you enable New Relic Logs, follow the troubleshooting procedures.