Enable New Relic Logs for AWS FireLens

New Relic offers an AWS FireLens integration built on our Fluentbit output plugin to connect your FireLens monitored log data to New Relic Logs. This document explains how to enable this feature.

Compatibility and requirements

To use New Relic Logs with FireLens, ensure your configuration meets the following requirements:

Enable FireLens for New Relic Logs

To enable New Relic Logs with FireLens:

  1. Configure the FireLens log router container.
  2. Configure the Application container.
  3. Generate some traffic and wait a few minutes, then check your account for data.

Configure the FireLens log router container

New Relic Logs uses a Fluent Bit image to configure the FireLens Log Router container. This container handles all log routing from application plugins.

To enable FireLens with Logs, you need to add a sidecar container to your ECS task definition that will act as the Firelens log router. Follow the installation steps as indicated on the FireLens GitHub, substituting the recommended images with the New Relic Fluentbit Output plugin image for your AWS region.

AWS Region Full Image Name
us-east-1 533243300146.dkr.ecr.us-east-1.amazonaws.com/newrelic/logging-firelens-fluentbit
us-east-2 533243300146.dkr.ecr.us-east-2.amazonaws.com/newrelic/logging-firelens-fluentbit
us-west-1 533243300146.dkr.ecr.us-west-1.amazonaws.com/newrelic/logging-firelens-fluentbit
us-west-2 533243300146.dkr.ecr.us-west-2.amazonaws.com/newrelic/logging-firelens-fluentbit
ca-central-1 533243300146.dkr.ecr.ca-central-1.amazonaws.com/newrelic/logging-firelens-fluentbit
eu-central-1 533243300146.dkr.ecr.eu-central-1.amazonaws.com/newrelic/logging-firelens-fluentbit
eu-west-1 533243300146.dkr.ecr.eu-west-1.amazonaws.com/newrelic/logging-firelens-fluentbit
eu-west-2 533243300146.dkr.ecr.eu-west-2.amazonaws.com/newrelic/logging-firelens-fluentbit
eu-west-3 533243300146.dkr.ecr.eu-west-3.amazonaws.com/newrelic/logging-firelens-fluentbit
eu-north-1 533243300146.dkr.ecr.eu-north-1.amazonaws.com/newrelic/logging-firelens-fluentbit

For example:

{
    "essential": true,
    // Image below is New Relic's fluentbit output plugin available on ECR
    "image": "533243300146.dkr.ecr.us-east-2.amazonaws.com/newrelic/logging-firelens-fluentbit",
    "name": "log_router",
    "firelensConfiguration": {
        "type": "fluentbit",
        "options": {
            "enable-ecs-log-metadata": "true"
        }
    }
}

Configure the Application Container

AWS Secrets Manager (recommended)

To prevent exposing your Insight Insert key in your task definition we strongly recommend using the AWS Secrets Manager service. Once you've added the secret to the Secrets Manager ,you can then reference it using the logConfiguration block suggested below.

"logConfiguration": {
     "logDriver":"awsfirelens",
     "options": {
        "Name": "newrelic"
     },
     "secretOptions": [{
        "name": "apiKey",
        "valueFrom": "arn:aws:secretsmanager:region:aws_account_id:secret:secret_name-AbCdEf"
     }]
}

During configuration, outlined in FireLens Task Definitions, use the logConfiguration block suggested below, replacing INSIGHTS_INSERT_KEY with your New Relic Insights Insert key.

"logConfiguration": {
     "logDriver":"awsfirelens",
     "options": {
        "Name": "newrelic",
        "apiKey": "INSIGHTS_INSERT_KEY"
    }

Example configuration

Example Task Definition configuration

Example Task Definition for a basic nginx server:

{
    "family": "newrelic-firelens",
    "networkMode": "awsvpc",
    "requiresCompatibilities": [
        "FARGATE"
    ],
    "containerDefinitions": [
        // FireLens log router container
        {
            "essential": true,
            // Image below is New Relic's fluentbit output plugin available on ECR
            "image": "533243300146.dkr.ecr.us-east-2.amazonaws.com/newrelic/logging-firelens-fluentbit",
            "name": "log_router",
            "firelensConfiguration": {
                "type": "fluentbit",
                "options": {
                    "enable-ecs-log-metadata": "true"
                }
            }
         },
         // Application container
         {
            "essential": true,
            "name": "webserver",
            // Application image goes here
            "image": "nginx",
            "cpu": 512,
            "memoryReservation": 1024,
            "portMappings": [{
                "containerPort": 5000
            }],
            "environment": [{
                "name": "VERSION",
                "value": "V1"
            }],
            // New Relic Fluentbit Output configuration
             "logConfiguration": {
                 "logDriver":"awsfirelens",
                 "options": {
                    "Name": "newrelic",
                 },
                 "secretOptions": [{
                    "name": "apiKey",
                    "valueFrom": "arn:aws:secretsmanager:region:aws_account_id:secret:secret_name-AbCdEf"
                 }]
            }
        }
    ],
    // Use your own role here
    "executionRoleArn": "arn:aws:iam::XXXXXXXXXXXX:role/ecsTaskExecutionRole",
    "taskRoleArn": "arn:aws:iam::XXXXXXXXXXXX:role/ecsTaskExecutionRole",
    "cpu": "1 vcpu",
    "memory": "2 gb"
}

View log data

If everything is configured correctly and your data is being reported, you should see data logs in the New Relic Logs UI or by going to Insights and querying:

SELECT * FROM Log

What's next?

Now that you've enabled Logs, here are some potential next steps:

For more help

Recommendations for learning more: