Configure polling frequency and data collection for cloud integrations

New Relic Infrastructure cloud integrations get data from cloud providers APIs. Metrics are generally collected from monitoring APIs such as AWS CloudWatch, Azure Monitor, and GCP Stackdriver, while inventory metadata are collected from the specific services APIs.

If you require more or less data from your cloud integrations, or if you need to control the use of the cloud providers APIs to prevent reaching rate and throttling limits in your cloud account, you can change the configuration settings to modify the amount of data they report.

You might have business reasons why you would want to manage your polling frequency, such as:

  • Billing: If you need to manage your AWS CloudWatch bill, you may want to decrease the polling frequency. Before you do this, make sure that any alert conditions set for your cloud integrations are not affected by this reduction.
  • New services: If you are deploying a new service or configuration and you want to collect data more often, you may want to increase the polling frequency temporarily.

Changing the configuration settings for your integrations may impact alert conditions and chart trends.

Change polling frequency

The polling frequency configuration determines how often New Relic reports data from your cloud provider for each service. By default, the polling frequency is set to the maximum frequency that is available for each service.

Anyone with access to an Infrastructure account can view the data reported to that account. Only Owners, Admins, and Infrastructure add-on managers can manage the integrations.

To change the polling frequency for a cloud integration:

  1. Go to infrastructure.newrelic.com > Integrations.
  2. Select the tab that corresponds to your cloud service provider.
  3. Select Configure next to the integration.
  4. Use the dropdowns next to Data polling interval every to select how frequently you want New Relic to capture your cloud integration data.

Specify data to be fetched

You can specify which information you want captured for your cloud integration by enabling the collection of additional data and by applying multiple filters to each integration.

To change this settings for your cloud integration:

  1. Go to infrastructure.newrelic.com > Integrations.
  2. Select the tab that corresponds to your cloud service provider.
  3. Select Configure next to the integration.
  4. Under Data collections and filters, turn the toggles you want On.
  5. For filters, select or enter the values that you want included in your reported data.

For some cloud integrations, an additional number of calls to the cloud provider APIs are needed in order to collect data. For example, to fetch tags for AWS Elastic Map Reduce clusters, an additional call to the service API is required.

Therefore, in order to better control the amount of API calls that are sent to your cloud account for these integrations, New Relic allows you to specify if you need these data to be collected. There are different data collection toggles available, depending on the integration.

Toggle Description
Collect tags

Some integrations require additional API calls to the cloud provider to report tags. Tag collection is enabled by default.

Switch this to Off if you don't want the integration to collect your cloud resource tags and thus reduce the volume of API calls.

Collect extended inventory

Some integrations can collect extended inventory metadata about your cloud resources by making additional API calls to the cloud provider. The metadata that is included within extended inventory for each cloud integration is described in the integration documentation.

Extended inventory collection is enabled by default.

Switch this to Off if you don't want to monitor extended inventory and thus reduce the volume of API calls.

Collect shards data

Available for AWS Kinesis Streams integration.

By default, Infrastructure does not report shard metrics. Switch this to On if you want to monitor shard metrics in addition to data stream metrics.

Collect Lambda@Edge data

Available for AWS CloudFront integration.

By default, Infrastructure does not report Lambda@Edge data. Switch this to On if you're using Lambda@Edge in AWS CloudFront and want to get Lambda execution location metadata.

Collect NAT Gateway data and Collect VPN data

Available for AWS VPC integration.

By default, Infrastructure does not report NAT Gateway nor VPN metrics. Switch these to On if you want to monitor NAT Gateway and VPN metrics and inventory, in addition to other VPC related entities inventory.

Filters

When a filter is On, you specify the data that you want to be collected; for example, if the Limit to AWS region is On, the regions that you select will be the ones that data will be collected for. There are different filters available, depending on the integration.

Filter Description
Region Select the regions that include the resources that you want to monitor.
Queue prefixes

Available for AWS SQS integration.

Enter each name or prefix for the queues that you want to monitor. Filter values are case-sensitive.

Load balancer prefixes

Available for AWS ALB integration.

Enter each name or prefix for the application load balancers that you want to monitor. Filter values are case-sensitive.

Stage name prefixes

Available for AWS API Gateway integration.

Enter each name or prefix for the stages that you want to monitor. Filter values are case-sensitive.

Tag key Enter one tag key that is associated with the resources that you want to monitor. Filter values are case-sensitive, and you can use this filter in combination with tag value filter.
Tag value Enter one tag value that is associated with the resources that you want to monitor. Filter values are case-sensitive, and you can use this filter in combination with tag key.
Resource group

Select the resource groups that are associated with the resources that you want to monitor.

Potential impact on alerts and charts

If you change an integration's configuration, it can impact alert conditions and charts. Here are some things to consider:

If you change this configuration setting... It may have this impact...
Any configuration setting When you change the configuration settings, the data that New Relic displays in Infrastructure charts, on the inventory page, and in the events feed changes as well.
Any filters When you create alert conditions after you set filters, make sure that your alerts are not triggered by resources that you filtered out.
Filter for regions

If you filter for specific regions, it may lower the amount of data reported to New Relic, which could trigger an alert.

If you create an alert condition for a specific region and then filter that region out, the region would no longer report data and would never trigger the alert.

Polling frequency When you create an alert, make sure that you define the threshold for a time period that is longer than the polling frequency.
Tags and extended inventory If you turn on tags and/or extended inventory, Infrastructure makes more API calls to the cloud provider, which could increase your cloud provider API usage bill.

For more help

Recommendations for learning more: