Forward AWS services logs to New Relic

The New Relic log management solution provides deeper visibility into application and infrastructure performance data, including events, errors, and traces. You can reduce Mean Time to Resolution (MTTR) and facilitate quick troubleshooting of production incidents. By forwarding AWS logs to New Relic, you can achieve comprehensive visibility into system performance, identify issues before they escalate, and make data-driven decisions efficiently. It helps enhance your team's operational efficiency while maintaining security and compliance.

Dica

Want to learn how to get started with New Relic logs management? See Get started with log management.

Instrumentation options

Select an AWS service to instrument

Before you begin

To set up AWS integration, ensure you have the following:

Importante

If you are already using Lambda for S3 or CloudWatch, refer the Guidelines for upgrading to the new Lambda.

Set up your AWS integration with log forwarding

Using a single CloudFormation template, you can easily set up your AWS integration and forward New Relic AWS services metrics, logs, or both.

  1. Log in to your New Relic account.
  2. Navigate to + Integrations & Agents.
  3. Search and select Integrate your AWS account from the All tab.
  4. Select a New Relic account and continue.
  5. Select the telemetry type you want to send to your New Relic account.
  6. Select Automate AWS with CloudFormation and continue. This is currently the only setup method available for log integration.

Configure logs forwarding

There are multiple ways to send logs from your AWS services to New Relic. Choose the path that best suits your needs. Both the integration instructions and the CloudFormation stack will adapt to your selections.

Select AWS sink(s)

Most AWS service logs are generated in S3 buckets or CloudWatch log groups. Select one or both AWS sinks. Select AWS sinks

Select log integration path(s)

Consider using the Amazon Kinesis Data Firehose for direct and high-volume data streaming, and the New Relic Lambda Function forwarder if you need high control over data transformation before forwarding logs. Select logs integration path

Set up triggers(s)

Setting up triggers ensures that new logs are sent to New Relic every time an AWS service logs to an S3 bucket or a CloudWatch log group. With a single CloudFormation stack, you can configure up to 40 triggers (S3 buckets and/or CloudWatch log groups).

  • For AWS service logs stored in S3 buckets, specify the S3 bucket name and Prefix (optional).
  • For AWS service logs stored in CloudWatch log groups, specify the Log Group name and Filter (optional).
Set up triggers

Select New Relic logs API endpoint

The system automatically sets the endpoint based on the region of your New Relic account.

  • For US account, logs are sent to https://log-api.newrelic.com/log/v1
  • For UE account, logs are sent to https://log-api.eu.newrelic.com/log/v1

Add custom attributes

Add custom attributes to organize your logs and make it easier to search, filter, analyze, and parse.

Launch CloudFormation template

  1. Click Launch your CloudFormation in AWS to navigate to your AWS Console. The CloudFormation stack includes the configured parameters based on your selections in New Relic One, so no additional editing is necessary unless you want to make changes.
  2. Copy and paste your license key into your AWS console.

We recommend you store your New Relic license key in AWS Secrets Manager by keeping the environment variable set to true (default value).

Deploy CloudFormation stack

After you customize the CloudFormation stack parameters to meet your requirements, click Create stack. Creating the stack's resources may take several minutes and may generate multiple stacks, depending on how you configure log and metric forwarding. With a single deployment command, you can create a new stack to forward logs from multiple AWS resources. If stack creation fails, refer to the troubleshooting procedure.

See your data

After the new stack is created, it forwards your collected logs to New Relic One. To see your data:

For logs forwarded via the New Relic Lambda forwarder:

SELECT * FROM Log WHERE instrumentation.provider = 'aws' AND instrumentation.name = 'lambda'

For logs forwarded via the Kinesis Data Firehose integration:

SELECT * FROM Log WHERE instrumentation.provider = 'aws' AND instrumentation.name = 'firehose'

If no data appears after you enable the log management capabilities, or the custom attributes are not appearing in the log, refer to the following troubleshooing procedures:

Importante

When configuring your CloudFormation integration, it's important to understand how triggers and filters are managed. For guidance on handling S3 triggers and CloudWatch filters, refer to Manage triggers and filters with CloudFormation.

What's next?

Explore logging data across your platform with our logs UI.