Auto-discovery of AWS entities

preview

We're still working on this feature, but we'd love for you to try it out!

This feature is currently provided as part of a preview program pursuant to our pre-release policies. The public preview includes access to New Map experience and Transaction 360, bundled together. Having opted into the preview, you gain access to all three capabilities at no cost; opting out of the preview applies to all three as well.

Auto-Discovery is an Advanced Compute feature that automatically maps your AWS infrastructure, providing insights into the relationships between applications and their underlying resources. By enabling this feature, you gain a comprehensive view of your cloud environment, including current configurations and resource connections. Additionally, Auto-Discovery offers a reliability scorecard to assess your infrastructure against AWS best practices, aiding in effective management and optimization.

With Auto-Discovery, you can:

Automatically detect and visualize the relationships between AWS entities on Maps.
Keep the representation of your infrastructure topology updated as changes occur in your AWS environment.
View the current configuration of your AWS resources.
Assess the reliability of your AWS infrastructure via the AWS reliability Scorecards.

The Auto-Discovery feature is available only to Advanced Compute customers. To enroll, navigate to one.newrelic.com > Administration > Preview & Trials, and enroll for New experiences: Maps, Transaction 360, and Auto-Discovery.

Set up AWS integration with Auto-Discovery

If you are setting up AWS integration for the first time and want to enable Auto-Discovery, follow these steps:

Before you begin

To set up AWS integration, ensure you have the following:

A New Relic license key.
An AWS account with permissions to deploy new AWS resources and IAM roles.
A New Relic account with permissions for cloud integrations.

Select data type

Log in to your New Relic account and navigate to Infrastructure > AWS.
To integrate with AWS, select an account and click Set up integration.
Select the data type as Metrics, Logs, or both, and continue.
Select Automate AWS with CloudFormation (Recommended) and continue. This is currently the only setup method available for log integration.

Select integration type

If you have selected Metrics as data type, to set up integration, perform the following steps:

Select metric integration type:
- CloudWatch Metric Streams is the recommended method to create a real-time stream of metrics to New Relic.
- API Polling is a legacy solution to receive AWS data in New Relic.
Give a name to the selected integration.
CloudWatch Metric Streams doesn't support few AWS services mentioned on the screen. If you want to include those services, select Yes and continue to configure API polling for those.
Give a name for the API integration.
Select the services you want to include in the integration, and continue.

If you have selected Logs as data type, then there are multiple ways to send logs from your AWS services to New Relic. Select the path that best suits your needs. Both the integration instructions and the CloudFormation stack adapts to your selections.

Select AWS sink(s): Most AWS service logs are generated in S3 buckets or CloudWatch log groups. Select one or both AWS sinks.
Select log integration path(s): Amazon Kinesis Data Firehose is for direct and high-volume data streaming, and the New Relic Lambda Function is for high control over data transformation before forwarding logs.
Set up trigger(s): Setting up triggers ensures that new logs are sent to New Relic every time an AWS service logs to an S3 bucket or a CloudWatch log group. With a single CloudFormation stack, you can configure up to 40 triggers (S3 buckets and/or CloudWatch log groups).
- For AWS service logs stored in S3 buckets, specify the S3 bucket name and Prefix (optional).
- For AWS service logs stored in CloudWatch log groups, specify the Log Group name and Filter (optional).
Select New Relic logs API endpoint: The system automatically sets the endpoint based on the region of your New Relic account.
- For US account, logs are sent to https://log-api.newrelic.com/log/v1
- For UE account, logs are sent to https://log-api.eu.newrelic.com/log/v1
Add custom attributes: Add custom attributes to organize your logs and make it easier to search, filter, analyze, and parse.

Enable Auto-Discovery

Turn on the scan to enable Auto-Discovery of your AWS entities and continue. It enables the cloudformation:StartResourceScan permission in your AWS account to scan your AWS resources and discover the relationships between them.

Tip

By default, New Relic scans all regions enabled in your account once in a day. You can update the scan frequency and regions after the stack is created, by managing the Auto-Discovery settings in New Relic One.

Connect to AWS

Click Launch your CloudFormation in AWS to navigate to your AWS Console. The CloudFormation stack includes the configured parameters based on your selections in New Relic One, so no additional editing is necessary unless you want to make changes.
Copy and paste your license key into your AWS console.

We recommend you store your New Relic license key in AWS Secrets Manager by keeping the environment variable set to true (default value).

Deploy the CloudFormation stack

After you customize the CloudFormation stack parameters to meet your requirements, click Create stack. Creating the stack's resources may take several minutes and may generate multiple stacks, depending on how you configure log and metric forwarding. With a single deployment command, you can create a new stack to forward logs from multiple AWS resources. If stack creation fails, refer to the troubleshooting procedure.

Validate the AWS data on New Relic

After the new stack is created, go to New Relic > Infrastructure > AWS to view all integrated AWS services. If any New Relic service is connected to AWS entities, you can explore relationships from its map, including those between AWS entities, between APM entities, and between AWS and APM entities.

Set up Auto-Discovery for an existing AWS integration

If you have already set up an AWS integration and want to enable Auto-Discovery, follow these steps:

Before you begin

To update AWS integration, ensure you have the following:

An AWS account with permissions to deploy new AWS resources and IAM roles.
A New Relic account with permissions for cloud integrations.

Edit account

Log in to your New Relic account and navigate to Infrastructure > AWS.
To check the Auto-Discovery enablement status of your AWS accounts, click Manage Auto-Discovery.
From the account list, click Enable for the AWS account in which you want to configure it.

Update the CloudFormation stack

To update the CloudFormation stack, on the side panel, click Launch Cloudformation Template.
On the AWS console, acknowledge the CloudFormation stack and click Create stack.
After the stack is created, go back to the New Relic One.

Select region and frequency of scan

By default, New Relic scans all regions enabled in your account once in a day. You can customize it as follows on the side panel:

To run the scan for specific regions, select or remove regions as per your requirement.
To change the frequency of the scan, select Once a Day or Twice a Day, as per your requirement.

Enable Auto-Discovery

To enable Auto-Discovery, on the confirmation screen of New Relic One, click Save.

Caution

When Auto-Discovery is disabled, you experience a loss of access to entities it identifies and their configurations. AWS Reliability Scorecards provide inaccurate scores, and the infrastructure map no longer displays relationships discovered by Auto-Discovery.

If you want to disable Auto-Discovery, follow these steps:

Log in to your New Relic account and navigate to Infrastructure > AWS.
To check the Auto-Discovery enablement status of your AWS accounts, click Manage Auto-Discovery.
From the account list, find the account in which you want to disable Auto-Discovery.
To change the configuration, click Edit.
On the side panel, turn off the scan.
To disable Auto-Discovery, click Save.

preview

Set up AWS integration with Auto-Discovery

Before you begin

Select data type

Select integration type

For Metrics data type

For Logs data type

Enable Auto-Discovery

Tip

Connect to AWS

Deploy the CloudFormation stack

Validate the AWS data on New Relic

Set up Auto-Discovery for an existing AWS integration

Before you begin

Edit account

Update the CloudFormation stack

Select region and frequency of scan

Enable Auto-Discovery

View configuration of an AWS entity

Disable Auto-Discovery

Caution

Auto-discovery of AWS entities

preview

Set up AWS integration with Auto-Discovery .css-21sua1{background:none;border:none;width:0;padding:0;}

Before you begin

Select data type

Select integration type

For Logs data type

Enable Auto-Discovery

Tip

Connect to AWS

Deploy the CloudFormation stack

Validate the AWS data on New Relic

Set up Auto-Discovery for an existing AWS integration

Before you begin

Edit account

Update the CloudFormation stack

Select region and frequency of scan

Enable Auto-Discovery

View configuration of an AWS entity

Disable Auto-Discovery

Set up AWS integration with Auto-Discovery