Enable New Relic Logs for Fluent Bit

New Relic offers a Fluent Bit output plugin to connect your Fluent Bit monitored log data to New Relic Logs. This document explains how to enable this feature.

Compatibility and requirements

To use New Relic Logs with Fluent Bit, ensure your configuration meets the following requirements:

Enable Fluent Bit for New Relic Logs

To enable New Relic Logs with Fluent Bit:

  1. Install the Fluent Bit plugin.
  2. Configure the Fluent Bit plugin.
  3. Test the Fluent Bit plugin.
  4. Generate some traffic and wait a few minutes, then check your account for data.

Recommended Usage

We have published a container with the plugin installed. It serves as a base image to be used by our Kubernetes integration. We recommend you use this base image and layer your own custom configuration files.

Install the Fluent Bit plugin

To install the Fluent Bit plugin:

  1. Navigate to New Relic's Fluent Bit plugin repository on GitHub.
  2. From the repository page, clone or download the repository.
  3. Run the following command to build your plugin:

    cd newrelic-fluent-bit-output && make all

Configure the Fluent Bit plugin

Fluent Bit needs to know the location of the New Relic plugin and the New Relic license key (recommended) or Insights Insert key to output data to New Relic. To configure your Fluent Bit plugin:

Pay attention to white space when editing your config files. Be sure to use four spaces to indent and one space between keys and values.

  1. Locate or create a plugins.conf file in your plugins directory.
  2. In the plugins.conf file, add a reference to out_newrelic.so, adjacent to your fluent-bit.conf file:

    [PLUGINS]
    Path /path/to/newrelic-fluent-bit-output/out_newrelic.so
  3. In the fluent-bit.conf file, add the following line under the service block:

    [SERVICE]
    # This is the main configuration block for fluent bit.
    # Ensure the follow line exists somewhere in the SERVICE block
    Plugins_File plugins.conf
            
  4. At the bottom of the fluent-bit.conf file, add the following to set up the input and output filters. Replace the placeholder text with your New Relic license key (recommended) or Insights Insert key:

    [INPUT]
    Name tail
    Path /path/to/your/log/file
    
    [OUTPUT]
    Name newrelic
    Match *
    licenseKey <LICENSE KEY>
    
    # Optional
    maxBufferSize 256000
    maxRecords 1024
            
  5. Restart your Fluent Bit instance with the following command:

    fluent-bit -c /path/to/fluent-bit.conf

Test the Fluent Bit plugin

To test if your Fluent Bit plugin is receiving input from a log file:

  1. Run the following command to append a test log message to your log file:

    echo "test message" >> /path/to/your/log/file
  2. Search New Relic Logs for test message.

Optional: Configure plugin attributes

Plugin configuration

Once you have installed and configured the Fluent Bit plugin, you can use the following attributes to configure how the plugin sends data to New Relic:

Key Description
licenseKey

The New Relic license key. Use either licenseKey (recommended) or apiKey, not both.

Default: none

maxBufferSize

The maximum size the payloads sent, in bytes.

Default: 256000

maxRecords

The maximum number of records to send at a time.

Default: 1024

apiKey New Relic Insights Insert key. Use either licenseKey (recommended) or apiKey, not both.

View log data

If everything is configured correctly and your data is being reported, you should see data logs in the New Relic Logs UI or by going to Insights and querying:

SELECT * FROM Log

What's next?

Now that you've enabled Logs, here are some potential next steps:

For more help

Recommendations for learning more: