New Relic includes an integration for collecting your Amazon Kinesis Data Firehose data. This document explains how to activate this integration and describes the data that can be reported.
Amazon Kinesis Data Firehose provides a simple way to capture and load streaming data. It can capture, transform, and load streaming data into Amazon Kinesis Data Analytics, Amazon S3, Amazon Redshift, and Amazon Elasticsearch Service, enabling near real-time analytics.
New Relic's Kinesis Data Firehose integration reports data such as indexed records and bytes, counts of data copied to AWS services, the age and freshness of records, and other metric data and service metadata.
To enable this integration follow standard procedures to Connect AWS services to New Relic.
You can change the polling frequency and filter data using configuration options.
Default polling for the Amazon Kinesis Data Firehose integration:
- New Relic polling interval: 5 minutes
- Amazon CloudWatch data interval: 1 minute
To find your integration data, go to one.newrelic.com > Infrastructure > AWS and select one of the Kinesis Data Firehose integration links.
For more on how to use your data, see Understand and use integration data.
This integration collects the following metrics:
The number of bytes indexed to Amazon ES over the specified time period.
The number of records indexed to Amazon ES over the specified time period.
The sum of the successfully indexed records over the sum of records that were attempted.
The number of bytes copied to Amazon Redshift over the specified time period.
The number of records copied to Amazon Redshift over the specified time period.
The sum of successful Amazon Redshift COPY commands over the sum of all Amazon Redshift COPY commands.
The number of bytes delivered to Amazon S3 over the specified time period.
The age (from getting into Kinesis Firehose to now) in seconds of the oldest record in Kinesis Firehose. Any record older than this age has been delivered to the S3 bucket.
The number of records delivered to Amazon S3 over the specified time period.
The sum of successful Amazon S3 put commands over the sum of all Amazon S3 put commands.
The number of bytes ingested into the Kinesis Firehose stream over the specified time period.
The number of records ingested into the Kinesis Firehose stream over the specified time period.
The time taken in milliseconds per
The number of bytes put to the Kinesis Firehose delivery stream using
The time taken in milliseconds per
New Relic collects the following Kinesis Firehose inventory data. For more about inventory data, see Understand and use data.
The date and time that the delivery stream was created.
The destinations. Format is an array of DestinationDescription objects.
The date and time that the delivery stream was last updated.
The status of the delivery stream.
Each time the destination is updated for a delivery stream, the version ID is changed, and the current version ID is required when updating the destination. This is so that the service knows it is applying the changes to the correct version of the delivery stream.
If you need more help, check out these support and learning resources:
- Browse the Explorers Hub to get help from the community and join in discussions.
- Find answers on our sites and learn how to use our support portal.
- Run New Relic Diagnostics, our troubleshooting tool for Linux, Windows, and macOS.
- Review New Relic's data security and licenses documentation.