• English日本語한국어
  • Log inStart now

Fluent Bit plugin for log forwarding

If your log data is already being monitored by Fluent Bit, you can use our Fluent Bit output plugin to forward and enrich your log data in New Relic.

Forwarding your Fluent Bit logs to New Relic will give you enhanced capabilities to collect, process, explore, query, and alert on your log data.

Basic process

We have published a container with the plugin installed. It serves as a base image to be used by our Kubernetes integration. We recommend you use this base image and layer your own custom configuration files.

To forward your logs from Fluent Bit to New Relic:

  1. Make sure you have:
    • A New Relic
    • Fluent Bit 1.0 or higher (recommended), although v0.12 or higher is supported
    • Fluent Bit Windows install directions can be found here
    • Fluent Bit Linux install directions can be found here
  2. Install the Fluent Bit plugin.
  3. Configure the Fluent Bit plugin.
  4. Test the Fluent Bit plugin.
  5. Generate some traffic and wait a few minutes, then check your account for data.

Install the Fluent Bit plugin

To install the Fluent Bit plugin:

  1. Navigate to New Relic's Fluent Bit plugin repository on GitHub.
  2. From the repository page, clone or download the repository.
  3. Run the following command to build your plugin:
cd newrelic-fluent-bit-output && make all
  1. Store out_newrelic.so or out_newrelic_winXX.dll at a location that can be accessed by the fluent-bit daemon.

Tip

If you'd rather not compile the plugin yourself, you can download pre-compiled versions from our GitHub repository's releases page.

Configure the Fluent Bit plugin

Fluent Bit needs to know the location of the New Relic plugin and the New Relic to output data to New Relic. To configure your Fluent Bit plugin:

Important

Pay attention to white space when editing your config files. Be sure to use four spaces to indent and one space between keys and values.

  1. Locate or create a plugins.conf file in your plugins directory.

  2. In the plugins.conf file, add a reference to out_newrelic.so, adjacent to your fluent-bit.conf file:

    [PLUGINS]
    Path /PATH/TO/newrelic-fluent-bit-output/out_newrelic.so
  3. In the fluent-bit.conf file, add the following line under the service block:

    [SERVICE]
    # This is the main configuration block for fluent bit.
    # Ensure the follow line exists somewhere in the SERVICE block
    Plugins_File plugins.conf
  4. At the bottom of the fluent-bit.conf file, add the following to set up the input, filter, and output sections. Replace the placeholder text with your :

    [INPUT]
    Name tail
    Tag my.tag
    Path /PATH/TO/YOUR/LOG/FILE
    # If you have multiple sources, just add another [INPUT] section like this:
    # [INPUT]
    # Name tail
    # Tag my.other.tag
    # Path /PATH/TO/SOME/OTHER/LOG/FILE
    # Having multiple [FILTER] blocks allows you to control the flow of changes as they read top down.
    [FILTER]
    Name modify
    # Here we only match on one tag, my.tag, defined in the [INPUT] section earlier
    Match my.tag
    # Below, we're renaming the host.cpu attribute to CPU
    Rename host.cpu CPU
    [FILTER]
    Name record_modifier
    # Match on all tags, *, so all logs get decorated per the Record clauses below. Record adds attributes + their values to each record.
    Match *
    # Adding a logtype attribute ensures your logs will be automatically parsed by our built-in parsing rules
    Record logtype nginx
    # Add the server's hostname to all logs generated
    Record hostname ${HOSTNAME}
    [OUTPUT]
    Name newrelic
    Match *
    licenseKey YOUR_LICENSE_KEY
  5. Restart your Fluent Bit instance with the following command:

    fluent-bit -c /PATH/TO/fluent-bit.conf

Did this doc help with your installation?

Test the Fluent Bit plugin

To test if your Fluent Bit plugin is receiving input from a log file:

  1. Run the following command to append a test log message to your log file:

    echo "test message" >> /PATH/TO/YOUR/LOG/FILE
  2. Search our logs UI for test message.

For more options, see the Fluent Bit modify filter documentation and our documentation to forward your logs using the infrastructure agent.

Optional: Configure plugin attributes

Once you have installed and configured the Fluent Bit plugin, you can use the following attributes to configure how the plugin sends data to New Relic:

Key

Description

licenseKey

The . Use either licenseKey (recommended) or apiKey, not both. Default: none

maxBufferSize

The maximum size the payloads sent, in bytes. Default: 256000

maxRecords

The maximum number of records to send at a time. Default: 1024

apiKey

Deprecated. Takes a New Relic Insights insert key, but using the licenseKey attribute is preferred. Use either licenseKey or apiKey, not both.

endpoint

Defaults to https://log-api.newrelic.com/log/v1 - If using an EU key needs to be set to https://log-api.eu.newrelic.com/log/v1

View log data

If everything is configured correctly and your data is being collected, you should see log data in both of these places:

SELECT * FROM Log

If no data appears after you enable our log management capabilities, follow our standard log troubleshooting procedures.

What's next?

Explore logging data across your platform with our logs UI.

Disable log forwarding

To disable log forwarding capabilities, follow standard procedures in Fluent Bit documentation. You do not need to do anything else in New Relic.

Copyright © 2024 New Relic Inc.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.