• /
  • Log in
  • Free account

AWS FireLens plugin for log forwarding

If your log data is already being monitored by Amazon AWS FireLens, you can use our FireLens integration to forward and enrich your log data in New Relic. This integration is built on our Fluent Bit output plugin.

Forwarding your FireLens logs to New Relic will give you enhanced log management capabilities to collect, process, explore, query, and alert on your log data.

Basic process

To enable log management capabilities:

  1. Make sure you have:
  1. Configure the FireLens log router container to run as a sidecar. (A sidecar is a way to move part of a service's core responsibility into a containerized module that is deployed alongside the core application.)
  2. Configure the application container.
  3. Generate some traffic and wait a few minutes, then check your account for data.

Configure the FireLens log router container

New Relic uses a Fluent Bit image to configure the FireLens log router container. This container handles all log routing from application plugins.

To forward your logs from FireLens to New Relic:

  1. Add a sidecar container to your pre-existing ECS task definition that will act as the Firelens log router. For help configuring ECS log routing, see the Amazon ECS documentation for custom log routing.
  2. Substitute the recommended images with the New Relic Fluent Bit output plugin image for your AWS region.
  3. Set the memoryReservation attribute for this container.

AWS region

Full image name

ap-northeast-1

533243300146.dkr.ecr.ap-northeast-1.amazonaws.com/newrelic/logging-firelens-fluentbit

ap-northeast-2

533243300146.dkr.ecr.ap-northeast-2.amazonaws.com/newrelic/logging-firelens-fluentbit

ap-northeast-3

533243300146.dkr.ecr.ap-northeast-3.amazonaws.com/newrelic/logging-firelens-fluentbit

ap-south-1

533243300146.dkr.ecr.ap-south-1.amazonaws.com/newrelic/logging-firelens-fluentbit

ap-southeast-1

533243300146.dkr.ecr.ap-southeast-1.amazonaws.com/newrelic/logging-firelens-fluentbit

ap-southeast-2

533243300146.dkr.ecr.ap-southeast-2.amazonaws.com/newrelic/logging-firelens-fluentbit

ca-central-1

533243300146.dkr.ecr.ca-central-1.amazonaws.com/newrelic/logging-firelens-fluentbit

eu-central-1

533243300146.dkr.ecr.eu-central-1.amazonaws.com/newrelic/logging-firelens-fluentbit

eu-north-1

533243300146.dkr.ecr.eu-north-1.amazonaws.com/newrelic/logging-firelens-fluentbit

eu-west-1

533243300146.dkr.ecr.eu-west-1.amazonaws.com/newrelic/logging-firelens-fluentbit

eu-west-2

533243300146.dkr.ecr.eu-west-2.amazonaws.com/newrelic/logging-firelens-fluentbit

eu-west-3

533243300146.dkr.ecr.eu-west-3.amazonaws.com/newrelic/logging-firelens-fluentbit

sa-east-1

533243300146.dkr.ecr.sa-east-1.amazonaws.com/newrelic/logging-firelens-fluentbit

us-east-1

533243300146.dkr.ecr.us-east-1.amazonaws.com/newrelic/logging-firelens-fluentbit

us-east-2

533243300146.dkr.ecr.us-east-2.amazonaws.com/newrelic/logging-firelens-fluentbit

us-west-1

533243300146.dkr.ecr.us-west-1.amazonaws.com/newrelic/logging-firelens-fluentbit

us-west-2

533243300146.dkr.ecr.us-west-2.amazonaws.com/newrelic/logging-firelens-fluentbit

For example:

{
"essential": true,
// Image below is New Relic's fluentbit output plugin available on ECR
"image": "533243300146.dkr.ecr.us-east-2.amazonaws.com/newrelic/logging-firelens-fluentbit",
"name": "log_router",
"firelensConfiguration": {
"type": "fluentbit",
"options": {
"enable-ecs-log-metadata": "true"
}
}
}

Configure the application container

To prevent exposing your New Relic license key in your task definition, we strongly recommend using the Amazon AWS Secrets Manager service.

When adding the secret, use the Plaintext tab. Once you've added the secret to the Secrets Manager, you can then reference it using the logConfiguration block, replacing SECRET_NAME with the name of your AWS secret. For example:

"logConfiguration": {
     "logDriver":"awsfirelens",
     "options": {
        "Name": "newrelic"
     },
     "secretOptions": [{
        "name": "apiKey",
        "valueFrom": "arn:aws:secretsmanager:region:aws_account_id:secret:SECRET_NAME"
     }]
}

Plaintext key configuration

Use the logConfiguration block, replacing INSERT_API_KEY with your New Relic license key. For configuration details, see the FireLens task definitions on GitHub.

"logConfiguration": {
     "logDriver":"awsfirelens",
     "options": {
        "Name": "newrelic",
        "apiKey": "NEW_RELIC_LICENSE_KEY"
    }

Example configuration

Forward logs to New Relic EU account

To forward logs from Firelens to a New Relic EU account, add an additional property to the options field of the logConfiguration object in your application containers.

"endpoint": "https://log-api.eu.newrelic.com/log/v1"

View log data

If everything is configured correctly and your data is being collected, you should see data logs in both of these places:

SELECT * FROM Log

If no data appears after you enable our log management capabilities, follow our standard log troubleshooting procedures.

What's next?

Explore logging data across your platform with the New Relic One UI.

For more help

If you need more help, check out these support and learning resources:

Create issueEdit page
Copyright © 2021 New Relic Inc.