Fluentd plugin for Logs

New Relic offers a Fluentd output plugin to connect your Fluentd monitored log data to New Relic Logs. This document explains how to enable this feature.

Compatibility and requirements

To use New Relic Logs with Fluentd, ensure your configuration meets the following requirements:

Logs requires an active trial or paid subscription for any New Relic product.

Enable Fluentd for New Relic Logs

To enable New Relic Logs with Fluentd:

  1. Install the Fluentd plugin.
  2. Configure the Fluentd plugin.
  3. Test the Fluentd plugin.
  4. Optional: Configure additional plugin attributes.
  5. Generate some traffic and wait a few minutes, then check your account for data.

Install the Fluentd plugin

To install the Fluentd plugin:

Install using fluent-gem install

To install using fluent-gem install, enter the following command into your terminal or command line interface:

fluent-gem install fluent-plugin-newrelic
Install using td-agent-gem

To install using the td-agent plugin, enter the following command into your terminal or command line interface:

td-agent-gem install fluent-plugin-newrelic

Configure the Fluentd plugin

To configure your Fluentd plugin:

  1. In your fluent.conf file, or td-agent.conf if using the td-agent, add the following block of data, replacing the placeholder text with your New Relic license key or Insights API Insert key.

    Configure with the New Relic license key (recommended)

    Configure with the New Relic license key:

    <source>
      @type tail
      <parse>
        @type none
      </parse>
      path /var/log/*
      tag sample.tag
    </source>
    
    <filter sample.tag>
      @type record_transformer
      <record>
        service_name ${tag}
        hostname "#{Socket.gethostname}"
      </record>
    </filter>
    
    <match **>
      @type newrelic
      license_key <YOUR LICENSE KEY>
    </match>
    
    Configure with the New Relic Insights API Insert key

    Configure with the New Relic Insights API Insert key:

    <source>
      @type tail
      <parse>
        @type none
      </parse>
      path /var/log/*
      tag sample.tag
    </source>
    
    <filter sample.tag>
      @type record_transformer
      <record>
        service_name ${tag}
        hostname "#{Socket.gethostname}"
      </record>
    </filter>
    
    <match **>
      @type newrelic
      api_key <YOUR API INSERT KEY>
    </match>
    
  2. Restart your Fluentd instance.

Test the Fluentd plugin

To test if your Fluentd plugin is receiving input from a log file:

  1. Run the following command to append a test log message to your log file:

    echo "test message" >> /path/to/your/log/file
  2. Search New Relic Logs for test message.

Optional configuration

Once you have installed and configured the Fluentd plugin, you can use the following attributes to configure how the plugin sends data to New Relic:

Property Description Default value
base_uri The New Relic ingestion endpoint. US endpoint: https://log-api.newrelic.com/log/v1
EU endpoint: https://log-api.eu.newrelic.com/log/v1

For more information and examples, see Example Configurations for Fluentd.

View log data

If everything is configured correctly and your data is being reported, you should see data logs in the New Relic Logs UI or by going to Insights and querying:

SELECT * FROM Log

What's next?

Now that you've enabled Logs, here are some potential next steps:

If no data appears after you enable New Relic Logs, follow the troubleshooting procedures.

For more help

Recommendations for learning more: