AWS FireLens plugin for Logs

New Relic offers an AWS FireLens integration built on our Fluentbit output plugin to connect your FireLens monitored log data to New Relic Logs. This document explains how to enable this feature.

Compatibility and requirements

Logs requires an active trial or paid subscription for any New Relic product.

To use New Relic Logs with FireLens, ensure your configuration meets the following requirements:

Enable FireLens for New Relic Logs

To enable New Relic Logs with FireLens:

  1. Configure the FireLens log router container.
  2. Configure the Application container.
  3. Generate some traffic and wait a few minutes, then check your account for data.

Configure the FireLens log router container

New Relic Logs uses a Fluent Bit image to configure the FireLens Log Router container. This container handles all log routing from application plugins.

To enable FireLens with Logs, you need to add a sidecar container to your ECS task definition that will act as the Firelens log router. For help configuring ECS log routing, see Custom Log Routing, substituting the recommended images with the New Relic Fluentbit Output plugin image for your AWS region.

AWS Region Full Image Name
us-east-1 533243300146.dkr.ecr.us-east-1.amazonaws.com/newrelic/logging-firelens-fluentbit
us-east-2 533243300146.dkr.ecr.us-east-2.amazonaws.com/newrelic/logging-firelens-fluentbit
us-west-1 533243300146.dkr.ecr.us-west-1.amazonaws.com/newrelic/logging-firelens-fluentbit
us-west-2 533243300146.dkr.ecr.us-west-2.amazonaws.com/newrelic/logging-firelens-fluentbit
ca-central-1 533243300146.dkr.ecr.ca-central-1.amazonaws.com/newrelic/logging-firelens-fluentbit
eu-central-1 533243300146.dkr.ecr.eu-central-1.amazonaws.com/newrelic/logging-firelens-fluentbit
eu-west-1 533243300146.dkr.ecr.eu-west-1.amazonaws.com/newrelic/logging-firelens-fluentbit
eu-west-2 533243300146.dkr.ecr.eu-west-2.amazonaws.com/newrelic/logging-firelens-fluentbit
eu-west-3 533243300146.dkr.ecr.eu-west-3.amazonaws.com/newrelic/logging-firelens-fluentbit
eu-north-1 533243300146.dkr.ecr.eu-north-1.amazonaws.com/newrelic/logging-firelens-fluentbit

For example:

{
    "essential": true,
    // Image below is New Relic's fluentbit output plugin available on ECR
    "image": "533243300146.dkr.ecr.us-east-2.amazonaws.com/newrelic/logging-firelens-fluentbit",
    "name": "log_router",
    "firelensConfiguration": {
        "type": "fluentbit",
        "options": {
            "enable-ecs-log-metadata": "true"
        }
    }
}

Configure the Application Container

AWS Secrets Manager (recommended)

To prevent exposing your Insights Insert key in your task definition, we strongly recommend using the AWS Secrets Manager service. When adding the secret, use the Plaintext tab. Once you've added the secret to the Secrets Manager, you can then reference it using the logConfiguration block suggested below.

"logConfiguration": {
     "logDriver":"awsfirelens",
     "options": {
        "Name": "newrelic"
     },
     "secretOptions": [{
        "name": "apiKey",
        "valueFrom": "arn:aws:secretsmanager:region:aws_account_id:secret:secret_name-AbCdEf"
     }]
}

During configuration, outlined in FireLens Task Definitions, use the logConfiguration block suggested below, replacing INSIGHTS_INSERT_KEY with your New Relic Insights Insert key.

"logConfiguration": {
     "logDriver":"awsfirelens",
     "options": {
        "Name": "newrelic",
        "apiKey": "INSIGHTS_INSERT_KEY"
    }

Example configuration

Example Task Definition configuration

Example Task Definition for a basic nginx server:

{
    "family": "newrelic-firelens",
    "networkMode": "awsvpc",
    "requiresCompatibilities": [
        "FARGATE"
    ],
    "containerDefinitions": [
        // FireLens log router container
        {
            "essential": true,
            // Image below is New Relic's fluentbit output plugin available on ECR
            "image": "533243300146.dkr.ecr.us-east-2.amazonaws.com/newrelic/logging-firelens-fluentbit",
            "name": "log_router",
            "firelensConfiguration": {
                "type": "fluentbit",
                "options": {
                    "enable-ecs-log-metadata": "true"
                }
            }
         },
         // Application container
         {
            "essential": true,
            "name": "webserver",
            // Application image goes here
            "image": "nginx",
            "cpu": 512,
            "memoryReservation": 1024,
            "portMappings": [{
                "containerPort": 5000
            }],
            "environment": [{
                "name": "VERSION",
                "value": "V1"
            }],
            // New Relic Fluentbit Output configuration
             "logConfiguration": {
                 "logDriver":"awsfirelens",
                 "options": {
                    "Name": "newrelic"
                 },
                 "secretOptions": [{
                    "name": "apiKey",
                    "valueFrom": "arn:aws:secretsmanager:region:aws_account_id:secret:secret_name-AbCdEf"
                 }]
            }
        }
    ],
    // Use your own role here
    "executionRoleArn": "arn:aws:iam::XXXXXXXXXXXX:role/ecsTaskExecutionRole",
    "taskRoleArn": "arn:aws:iam::XXXXXXXXXXXX:role/ecsTaskExecutionRole",
    "cpu": "1 vcpu",
    "memory": "2 gb"
}

Sending logs to an EU New Relic account

If you want to send logs from Firelens to an EU account then you need to add an additional property to the "options" field of the "logConfiguration" object in your application containers.

"endpoint": "https://log-api.eu.newrelic.com/log/v1"

View log data

If everything is configured correctly and your data is being collected, you should see data logs in both of these places:

What's next?

Now that you've enabled Logs, here are some potential next steps:

If no data appears after you enable New Relic Logs, follow the troubleshooting procedures.

For more help

Recommendations for learning more: