Amazon Kinesis Data Streams monitoring integration

Important

Enable the AWS CloudWatch Metric Streams integration to monitor all CloudWatch metrics from your AWS services, including custom namespaces. Individual integrations are no longer our recommended option.

New Relic includes an integration for monitoring Amazon Kinesis Data Streams. This document explains how to activate the integration and describes the data that can be reported.

Features

Amazon Kinesis Data Streams is a platform for streaming data on AWS, making it easy to load and analyze data in real time. It also gives you the ability to build custom streaming data applications for specialized needs.

New Relic's Kinesis Data Streams integration gathers metric and configuration data on all of the streams associated with your account.

Data collected includes bytes put to and retrieved from the stream, records put and retrieved, time taken by operations, and other metrics. You can view your Kinesis Data Streams data in pre-built dashboards and create custom queries and charts in New Relic.

You also have an option for enabling shard data collection. AWS data records are sequentially processed in shards, which are in turn grouped in streams. The number of shards in a stream determines the total data read and write capacity of the service.

Activate integration

To enable this integration follow standard procedures to Connect AWS services to New Relic.

Polling and configuration

You can change the polling frequency and filter data using configuration options.

Default polling information for the Amazon Kinesis Data Streams integration:

New Relic polling interval: 15 minutes
Amazon CloudWatch data interval: 1 minute

Shard data: To collect shard data, enable Collect shards data in the integration's configuration options in the UI.

Important

If you enable collecting shard data, this will increase the number of API calls to your AWS CloudWatch service. There will be seven calls per shard per polling interval.

Find and use data

To find your integration data, go to one.newrelic.com > All capabilities > Infrastructure > AWS and select one of the Kinesis Data Streams integration links.

You can query and explore your data using the QueueSample event type. The provider value includes:

Stream metrics: KinesisStream
Shard metrics: KinesisStreamShard

For more on how to use your data, see Understand and use integration data.

Metric data

This integration collects the following metrics. For additional information, see Amazon's documentation for monitoring Amazon Kinesis Data Streams.

This integration collects the following metrics:

Name	Description
`getRecordsBytes`	The number of bytes retrieved from the Kinesis stream, measured over the specified time period. Minimum, Maximum, and Average statistics represent the bytes in a single `GetRecords` operation for the stream in the specified time period.
`getRecordsIteratorAgeMilliseconds`	The age of the last record in all `GetRecords` calls made against an Kinesis stream, measured over the specified time period, in milliseconds. Age is the difference between the current time and when the last record of the `GetRecords` call was written to the stream. The Minimum and Maximum statistics can be used to track the progress of Kinesis consumer applications. A value of zero indicates that the records being read are completely caught up with the stream.
`getRecordsLatency`	The time taken in milliseconds per `GetRecords` operation, measured over the specified time period.
`getRecordsRecords`	The number of records retrieved from the shard, measured over the specified time period. Minimum, Maximum, and Average statistics represent the records in a single `GetRecords` operation for the stream in the specified time period.
`incomingBytes`	The number of bytes successfully put to the Kinesis stream over the specified time period. This metric includes bytes from `PutRecord` and `PutRecords` operations. Minimum, Maximum, and Average statistics represent the bytes in a single put operation for the stream in the specified time period.
`incomingRecords`	The number of records successfully put to the Kinesis stream over the specified time period. This metric includes record counts from `PutRecord` and `PutRecords` operations. Minimum, Maximum, and Average statistics represent the records in a single put operation for the stream in the specified time period.
`putRecordBytes`	The number of bytes put to the Kinesis stream using the `PutRecord` operation over the specified time period.
`putRecordLatency`	The time taken in milliseconds per `PutRecord` operation, measured over the specified time period.
`putRecordSuccess`	The number of successful `PutRecord` operations per Kinesis stream, measured over the specified time period. Average reflects the percentage of successful writes to a stream.
`putRecordsBytes`	The number of bytes put to the Kinesis stream using the `PutRecords` operation over the specified time period.
`putRecordsLatency`	The time taken in milliseconds per `PutRecords` operation, measured over the specified time period.
`putRecordsRecords`	The number of successful records in a `PutRecords` operation per Kinesis stream, measured over the specified time period.
`putRecordsSuccess`	The number of `PutRecords` operations where at least one record succeeded, per Kinesis stream, measured over the specified time period.
`readProvisionedThroughputExceeded`	The number of GetRecords calls throttled for the stream over the specified time period. The most commonly used statistic for this metric is Average.
`writeProvisionedThroughputExceeded`	The number of records rejected due to throttling for the stream over the specified time period. This metric includes throttling from `PutRecord` and `PutRecords` operations. The most commonly used statistic for this metric is Average. When the Minimum statistic has a non-zero value, records were being throttled for the stream during the specified time period. When the Maximum statistic has a value of 0 (zero), no records were being throttled for the stream during the specified time period.

Amazon Kinesis Data Streams monitoring integration

Important

Features .css-21sua1{background:none;border:none;width:0;padding:0;}

Activate integration

Polling and configuration

Important

Find and use data

Metric data

Features