• /
  • EnglishEspañol日本語한국어Português
  • Log inStart now

AWS integrations metrics

You can set up alert conditions or query your data in the New Relic platform when you've integrated with AWS. This doc describes how we store AWS dimensional metrics so you can create alerts and query your data.

If you haven't yet, we recommend installing or migrating to the AWS CloudWatch Metric Streams integration to ingest AWS services' metrics. If you're using API polling to collect your AWS data, you can review the metrics we collect for each integration in our API polling metrics section.

Queries, metric storage, and mapping

We've mapped metrics from individual cloud integrations to new mappings from AWS CloudWatch Metric Streams. You can use either metric naming conventions and your alerts, dashboards, and queries will continue to work. Keep in mind that:

  • All metrics coming from the metric stream have aws.MetricStreamArn and collector.name = 'cloudwatch-metric-streams' attributes.
  • Metrics coming from AWS CloudWatch are stored as dimensional metrics of type summary.

You can review more about how New Relic stores AWS metrics as dimensional metrics below:

  • Dimensional metric naming convention explains how New Relic updates the AWS namespace in our backend.
  • Metrics with multiple dimension combinations describes how to use query aggregations to parse metrics with multiple dimensions
  • Metrics with the by + dimension includes a table of all metrics that are suffixed with .by + dimensionName.

Create alert conditions

You can create NRQL alert conditions on metrics from a metric stream. Make sure your filter limits data to metrics from CloudWatch Metric Stream only. To do that, construct your queries like this:

SELECT sum(aws.s3.5xxErrors) FROM Metric WHERE collector.name = 'cloudwatch-metric-streams' FACET aws.accountId, aws.s3.BucketName

To make sure your alerts process the data correctly, configure advanced signal settings. Advance signal settings addresses how AWS CloudWatch receives metrics from services with a certain delay. For example, Amazon guarantees that 90% of EC2 metrics are available in CloudWatch within 7 minutes of them being generated. When you add streaming metrics from AWS to New Relic, however, it can add up to 1 minute additional delay as data needs to buffer in Amazon Firehose.

To configure the signal settings, go to Condition Settings and click on Advanced Signal Settings. Enter the following values:

  1. Aggregation window: We recommend setting it to 1 minute. If you're having issues with flapping alerts or alerts not triggering, consider increasing it to 2 minutes.
  2. Offset evaluation by: Depending on the service, CloudWatch may send metrics with a certain delay. The value is set in Windows. With a 1-minute aggregation window, setting the offset to 8 ensures the majority of the metrics are evaluated correctly. You may be able to use a lower offset if the delay introduced by AWS and Firehose is less.
  3. Fill data gaps with: Leave this void, or use Last known value if gaps in the data coming from AWS lead to false positives or negatives.

Tags collection

New Relic provides enhanced dimensions from metrics coming from AWS CloudWatch metric streams. Resource and custom tags are automatically pulled from most services and are used to decorate metrics with additional dimensions. Use metrics and events to see which tags are available on each AWS metric.

The following query shows an example of tags being collected and queried as dimensions in metrics:

SELECT average(`aws.rds.CPUUtilization`) FROM Metric FACET `tags.mycustomtag` SINCE 30 MINUTES AGO TIMESERIES

Not all metrics have their custom tags as dimensions. Only metrics linked to entities listed in the entity explorer have their custom tags associated. The AWS CloudWatch Metric Streams integration doesn't include tags as part of the stream message; hence, additional processing is required on the New Relic side.

Metadata collection

Like with custom tags, New Relic also pulls metadata information from relevant AWS services in order to decorate AWS CloudWatch metrics with enriched metadata collected from AWS Services APIs. This metadata is accessible in New Relic as additional dimensions on the metrics provided by AWS CloudWatch.

Important

This optional capability is complementary to the CloudWatch Metric Streams integration. Your service discovery and monitoring will be incomplete if you don't enable this optional capability.

The solution relies on AWS Config, which might incur in additional costs in your AWS account. AWS Config provides granular controls to determine which services and resources are recorded. New Relic will only ingest metadata from the available resources in your AWS account.

The following services and namespaces are supported:

  • ALB/NLB
  • API Gateway (excluding API v1)
  • DynamoDB
  • EBS
  • EC2
  • ECS
  • ELB
  • Lambda
  • RDS
  • S3

Infrastructure agent metrics and EC2 metadata decoration

As with the EC2 API polling integration, when the infrastructure agent is installed on a host and the EC2 namespace is active via AWS CloudWatch Metric Streams integration, then all the infrastructure agent events and metrics are decorated with additional metadata.

The following attributes will decorate infrastructure samples. Some of these may not be applicable on all environments: awsAvailabilityZone, ec2InstanceId, ec2PublicDnsName, ec2State, ec2EbsOptimized, ec2PublicIpAddress, ec2PrivateIpAddress, ec2VpcId, ec2AmiId, ec2PrivateDnsName, ec2KeyName, ec2SubnetId, ec2InstanceType, ec2Hypervisor, ec2Architecture, ec2RootDeviceType, ec2RootDeviceName, ec2VirtualizationType, ec2PlacementGroupName, ec2PlacementGroupTenancy.

Custom metrics and percentiles

The CloudWatch Metric Streams integration automatically ingests new metrics configured in the stream, including custom metrics and percentiles.

Custom metrics

To ingest CloudWatch custom metrics, your custom namespace needs to be visible in the CloudWatch Metric Strams configuration. Make sure it's not being filtered by inclusion or exclusion rules.

Percentiles

AWS CloudWatch allows you to define additional statistics, including percentiles.

Follow these steps to add percentiles to any metric available in the CloudWatch stream:

  1. On AWS, update the CloudWatch stream configuration (via API, CLI, or AWS Console) with the required percentiles in the StatisticConfiguration setting. For example, you can add p90, p95, and p99 percentiles to the ELB latency metric (aws.elb.Latency).

  2. After a few minutes, the new statistic should be made available in the stream and ingested by New Relic. Percentiles can be queried using this naming convention:

    From Metric select max(aws.elb.Latency.p99) where collector.name = 'cloudwatch-metric-streams' timeseries

Although AWS supports other statistics in the stream beyond percentiles, those aren't made available in the Open Telemetry export format (only JSON) and are currently not supported by New Relic.

Learn more about pricing, limitations, and advance configurations from the AWS documentation.

Manage your data

The New Relic UI provides a set of tools to keep track of the data being ingested in your account. Go to Manage your data in the settings menu to see all details. Metrics ingested from AWS Metric Streams integrations are considered in the Metric bucket.

Important

Metrics sent via AWS Metric Streams count against your Metric API limits for the New Relic account where data will be ingested.

If you need a more granular view of the data, use the bytecountestimate() function on Metric to estimate the data being ingested. These sample queries can help you understand your AWS ingest:

We recommend the following actions to control the data being ingested:

  • Make sure metric streams are enabled only on the AWS accounts and regions you want to monitor with New Relic.
  • Use the inclusion and exclusion filters in the CloudWatch Metric Streams in order to select which services or namespaces are being monitored by New Relic.
  • Consider using drop data rules to discard metrics based on custom filters. (For example, drop metrics by namespace and tag, tag value, or any other valid NRQL criteria.)

API Polling metrics

For a reference on available metrics from each one of the polling integrations and their names, check out our doc on the individual integrations.

The following noncomprehensive list displays the metrics collected by the AWS polling integrations and their dimensional metrics translations.

Copyright © 2024 New Relic Inc.
CareersTerms of ServiceDMCA PolicyPrivacy NoticeCookie PolicyUK Slavery Act

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.