New Relic Logs offers AWS users a comprehensive, integrated log management solution that enhances both troubleshooting efficiency and system observability within AWS environments. New Relic's AWS Logs in Context seamlessly correlates logs from various AWS services with their metrics, unlocking the ability to swiftly identify and resolve issues out-of-the-box. AWS Logs in Context enables development and operations teams to minimize system downtime and ensure reliability. The New Relic platform's advanced search and filtering capabilities, along with real-time log analysis, provide deep insights into the performance and usage of AWS services, helping drive cost efficiencies. With New Relic Logs, AWS users gain powerful insights and streamlined processes, supporting more effective and informed management of their cloud resources.
How does AWS Logs in Context work?
AWS Logs in Context feature is designed to enhance log management by providing a seamless linkage between logs and their corresponding AWS resources (entities). Logs are enriched with metadata that is needed to link it back to its corresponding AWS resource. The added enrichment enables:
Automatic entity creation & association: every log reported from AWS services automatically gets associated with the specific AWS resource (or entity) that generated it. If the entity does not yet exist, then the system automatically creates it.
Seamless bidirectional navigation between entities and Logs: logs enriched with the required contextual information allows users to utilize the Entity Explorer to search for specific AWS resources they are interested in, and navigate quickly between logs and their generating entities.
The AWS Logs in Context feature works out-of-the-box for most AWS services. However, for some services, you will need to do some adaptations to your AWS resource configuration. The following sections describe whether any configuration customization is required for each supported AWS resource type.
AWS Logs in Context works out-of-the-box for ALB access logs and connection logs delivered to an S3 bucket.
AWS Logs in Context works out-of-the-box for ELB access logs delivered to an S3 bucket.
AWS Logs in Context currently supports CloudFront access logs delivered to an S3 bucket using the Amazon S3 (Legacy) delivery method and Web Application Firewall (WAF) logs delivered to a CloudWatch log group
When you configure your CloudFront standard logging destination, select the type "Amazon S3 (Legacy)" in the "Deliver to" option and set the log prefix option toCLOUDFRONT_ACCESS_LOGS:
CloudFront allows the collection of Web Application Firewall (WAF) logs via CloudWatch. Simply enable the Web Application Firewall at the "Security" tab of your CloudFront distribution and click on the "Enable AWS WAF logs" button. However, no matter what AWS region your CloudFront Distribution is deployed into, its WAF logs get collected in CloudWatch in the us-east-1 (N. Virginia) region in a log group named aws-waf-logs-CloudFrontDistribution-YOUR_DISTRIBUTION_NAME. In order to ship these logs you will need to also deploy our AWS log forwarding integration in the us-east-1 region and create a corresponding subscription filter from that log group.
AWS Logs in Context works out-of-the-box for Elastic Beanstalk EC2 Instance streamed logs and rotated logs delivered to CloudWatch and S3, respectively.
AWS Logs in Context works out-of-the-box for NLB access logs delivered to an S3 bucket.
AWS Logs in Context works out-of-the-box for RDS instance logs delivered to CloudWatch.
AWS Logs in Context supports S3 server access logs delivered to an S3 bucket.
When you configure your S3 Server Access logging configuration, you must set the "Destination" field so that the "Destination prefix" field becomes S3_ACCESS_LOGS/ (with final forward slash), and select the date-based partitioned log object key format, as in the following screenshot:
AWS Logs in Context works out-of-the-box for SQS management and data events (related to a particular SQS queue) delivered via CloudTrail to either CloudWatch or an S3 bucket.
AWS Logs in Context enrichment
The AWS Logs in Context feature automatically enriches your logs with attributes that allow correlating them with the entities created in New Relic. These attributes vary depending on the AWS service that generated the logs and are detailed in the following sections.
Attribute name
Description
aws.Arn
The Amazon Resource Name (ARN) of the AWS resource.
entity.guid
The unique identifier of the emitting entity.
entity.name
The name of the emitting entity, as displayed in the Entity Explorer or Catalog views.
entity.type
The entity type, as described in our entity definitions, such as AWSALB, AWSS3BUCKET, etc.
entityId
Only added if you collect logs together with metrics using the AWS API Polling integration. It allows us to correlate your log with the entity created by this integration.
aws.alb.loadBalancer
The name of the Application Load Balancer.
Attribute name
Description
aws.Arn
The Amazon Resource Name (ARN) of the AWS resource.
entity.guid
The unique identifier of the emitting entity.
entity.name
The name of the emitting entity, as displayed in the Entity Explorer or Catalog views.
entity.type
The entity type, as described in our entity definitions, such as AWSALB, AWSS3BUCKET, etc.
entityId
Only added if you collect logs together with metrics using the AWS API Polling integration. It allows us to correlate your log with the entity created by this integration.
aws.elb.loadBalancer
The name of the Classic Load Balancer.
Attribute name
Description
aws.Arn
The Amazon Resource Name (ARN) of the AWS resource.
entity.guid
The unique identifier of the emitting entity.
entity.name
The name of the emitting entity, as displayed in the Entity Explorer or Catalog views.
entity.type
The entity type, as described in our entity definitions, such as AWSALB, AWSS3BUCKET, etc.
entityId
Only added if you collect logs together with metrics using the AWS API Polling integration. It allows us to correlate your log with the entity created by this integration.
aws.cloudfront.DistributionId
The identifier of the Cloudfront distribution.
Attribute name
Description
aws.Arn
The Amazon Resource Name (ARN) of the AWS resource.
entity.guid
The unique identifier of the emitting entity.
entity.name
The name of the emitting entity, as displayed in the Entity Explorer or Catalog views.
entity.type
The entity type, as described in our entity definitions, such as AWSALB, AWSS3BUCKET, etc.
entityId
Only added if you collect logs together with metrics using the AWS API Polling integration. It allows us to correlate your log with the entity created by this integration.
aws.ec2.InstanceId
The identifier of the EC2 instance.
Logs sent to CloudWatch will be enriched with the following attributes:
Attribute name
Description
aws.Arn
The Amazon Resource Name (ARN) of the AWS resource.
entity.guid
The unique identifier of the emitting entity.
entity.name
The name of the emitting entity, as displayed in the Entity Explorer or Catalog views.
entity.type
The entity type, as described in our entity definitions, such as AWSALB, AWSS3BUCKET, etc.
entityId
Only added if you collect logs together with metrics using the AWS API Polling integration. It allows us to correlate your log with the entity created by this integration.
The name of the emitting entity, as displayed in the Entity Explorer or Catalog views.
entity.type
The entity type, as described in our entity definitions, such as AWSALB, AWSS3BUCKET, etc.
entityId
Only added if you collect logs together with metrics using the AWS API Polling integration. It allows us to correlate your log with the entity created by this integration.
Attribute name
Description
aws.Arn
The Amazon Resource Name (ARN) of the AWS resource.
entity.guid
The unique identifier of the emitting entity.
entity.name
The name of the emitting entity, as displayed in the Entity Explorer or Catalog views.
entity.type
The entity type, as described in our entity definitions, such as AWSALB, AWSS3BUCKET, etc.
entityId
Only added if you collect logs together with metrics using the AWS API Polling integration. It allows us to correlate your log with the entity created by this integration.
aws.nlb.loadBalancer
The name of the Network Load Balancer.
Attribute name
Description
aws.Arn
The Amazon Resource Name (ARN) of the AWS resource.
entity.guid
The unique identifier of the emitting entity.
entity.name
The name of the emitting entity, as displayed in the Entity Explorer or Catalog views.
entity.type
The entity type, as described in our entity definitions, such as AWSALB, AWSS3BUCKET, etc.
entityId
Only added if you collect logs together with metrics using the AWS API Polling integration. It allows us to correlate your log with the entity created by this integration.
aws.rds.DBInstanceIdentifier
The identifier (name) of the RDS database instance.
Attribute name
Description
aws.Arn
The Amazon Resource Name (ARN) of the AWS resource.
entity.guid
The unique identifier of the emitting entity.
entity.name
The name of the emitting entity, as displayed in the Entity Explorer or Catalog views.
entity.type
The entity type, as described in our entity definitions, such as AWSALB, AWSS3BUCKET, etc.
entityId
Only added if you collect logs together with metrics using the AWS API Polling integration. It allows us to correlate your log with the entity created by this integration.
aws.s3.BucketName
The name of the S3 bucket.
Attribute name
Description
aws.Arn
The Amazon Resource Name (ARN) of the AWS resource.
entity.guid
The unique identifier of the emitting entity.
entity.name
The name of the emitting entity, as displayed in the Entity Explorer or Catalog views.
entity.type
The entity type, as described in our entity definitions, such as AWSALB, AWSS3BUCKET, etc.
entityId
Only added if you collect logs together with metrics using the AWS API Polling integration. It allows us to correlate your log with the entity created by this integration.
aws.sqs.QueueName
The name of the SQS queue.
aws.sqs.queueUrl
The URL of the SQS queue.
Troubleshoot
The AWS Logs in Context feature will only work when using the default CloudWatch log group name or S3 bucket names set up by each AWS service. Despite some services allow customizing these values (for example, Lambda, doing so will result in logs not correctly being attached to an entity, and potentially incorrect entities being created. Unless indicated in this section, we strongly recommend using the default configuration values.
Did this doc help with your installation?
What's next?
After you set up AWS Logs in Context for your AWS resources, make the most of your logging data in the New Relic UI:
Explore the logging data across your platform with our Logs UI.
Search the entity associated with your AWS resource through the entity explorer and quickly navigate to its logs by selecting its "Logs" tab.
