--- title: AWS Kinesis Data Analytics monitoring integration source: https://docs.newrelic.com/docs/infrastructure/amazon-integrations/aws-integrations-list/aws-kinesis-data-analytics-monitoring-integration --- > #### ⚠️ IMPORTANT > > Enable the [AWS CloudWatch Metric Streams integration](https://docs.newrelic.com/docs/infrastructure/amazon-integrations/aws-integrations-list/aws-metric-stream/) to monitor all CloudWatch metrics from your AWS services, including custom namespaces. Individual integrations are no longer our recommended option. Our [infrastructure monitoring integrations](https://docs.newrelic.com/docs/infrastructure/introduction-infra-monitoring) include an integration for reporting your AWS Kinesis Data Analytics data to our products. Read on to learn more about how to activate the integration and what data it collects. ## Activate integration [#activate] To enable this integration follow standard procedures to [Connect AWS services to New Relic](https://docs.newrelic.com/docs/infrastructure/infrastructure-integrations/getting-started/connect-aws-integrations-infrastructure). ## Configuration and polling [#polling] You can change the polling frequency and filter data using [configuration options](https://docs.newrelic.com/docs/integrations/new-relic-integrations/getting-started/configure-polling-frequency-data-collection-cloud-integrations). Default [polling](https://docs.newrelic.com/docs/infrastructure/amazon-integrations/aws-integrations-list/aws-polling-intervals-infrastructure-integrations) information for the AWS Kinesis Data Analytics integration: - New Relic polling interval: 5 minutes - Amazon CloudWatch data interval: 1 minute ## Find and use data [#find-data] To find your integration data in our infrastructure UI, go to **[one.newrelic.com > All capabilities](https://one.newrelic.com/all-capabilities) > Infrastructure > AWS** and select an integration. Data is attached to the following [event type](https://docs.newrelic.com/docs/data-apis/understand-data/new-relic-data-types/#event-data): | Entity | Event Type | Provider | | ----------------------- | -------------------------------------------------- | -------------------------------------------- | | Application | `AwsKinesisAnalyticsApplicationSample` | `AwsKinesisAnalyticsApplication` | | ApplicationKpu | `AwsKinesisAnalyticsApplicationSample` | `AwsKinesisAnalyticsApplication` | | Task | `AwsKinesisAnalyticsTaskSample` | `AwsKinesisAnalyticsTask` | | TaskOperator | `AwsKinesisAnalyticsTaskOperatorSample` | `AwsKinesisAnalyticsTaskOperator` | | TaskParallelism | `AwsKinesisAnalyticsTaskParallelismSample` | `AwsKinesisAnalyticsTaskParallelism` | | TaskOperatorParallelism | `AwsKinesisAnalyticsTaskOperatorParallelismSample` | `AwsKinesisAnalyticsTaskOperatorParallelism` | For more on how to use your data, see [Understand and use integration data](https://docs.newrelic.com/docs/infrastructure/integrations/find-use-infrastructure-integration-data). ## Metric data [#metrics] This integration collects AWS Kinesis Data Analytics data for Application, ApplicationKpu, Task, TaskOperator, TaskParallelism and TaskOperatorParallelism. ### Kinesis Data Analytics Application data | Metric | Unit | Description | | ----------------------------------------- | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `bytesConsumedRate` | BytesPerSecond | The average number of bytes consumed per second for a topic. | | `commitsFailed` | Count | The total number of offset commit failures to Kafka, if offset committing and checkpointing are enabled. | | `commitsSucceeded` | Count | The total number of successful offset commits to Kafka, if offset committing and checkpointing are enabled. | | `cpuUtilization` | Percent | Overall percentage of CPU utilization across task managers. For example, if there are five task managers, Kinesis Data Analytics publishes five samples of this metric per reporting interval. | | `currentInputWatermark` | Milliseconds | The last watermark this application/operator/task/thread has received. | | `currentOutputWatermark` | Milliseconds | The last watermark this application/operator/task/thread has emitted. | | `downtime` | Milliseconds | For jobs currently in a failing/recovering situation, the time elapsed during this outage. | | `fullRestarts` | Count | The total number of times this job has fully restarted since it was submitted. This metric does not measure fine-grained restarts. | | `heapMemoryUtilization` | Percent | Overall heap memory utilization across task managers. For example, if there are five task managers, Kinesis Data Analytics publishes five samples of this metric per reporting interval. | | `lastCheckpointDuration` | Milliseconds | The time it took to complete the last checkpoint. | | `lastCheckpointSize` | Bytes | The total size of the last checkpoint. | | `numRecordsInPerSecond` | CountPerSecond | The total number of records this operator or task has received per second. | | `numRecordsOut` | Count | The total number of records this operator or task has emitted. | | `numRecordsOutPerSecond` | CountPerSecond | The total number of records this operator or task has emitted per second. | | `numberOfFailedCheckpoints` | Count | The number of times checkpointing has failed. | | `numRecordsIn` | Count | The total number of records this operator or task has received. | | `oldGenerationGcCount` | Count | The total number of old garbage collection operations that have occurred across all task managers. | | `oldGenerationGcTime` | Milliseconds | The total time spent performing old garbage collection operations. | | `recordsLagMax` | Count | The maximum lag in terms of number of records for any partition in this window. | | `threadsCount` | Count | The total number of live threads used by the application. | | `uptime` | Milliseconds | The time that the job has been running without interruption. | | `bytes` | Bytes | The number of bytes read (per input stream) or written (per output stream). | | `inputProcessing.DroppedRecords` | Count | The number of records returned by a Lambda function that were marked with Dropped status. | | `inputProcessing.Duration` | Milliseconds | The time taken for each AWS Lambda function invocation performed by Kinesis Data Analytics. | | `inputProcessing.OkBytes` | Count | The sum of bytes of the records returned by a Lambda function that were marked with Ok status. | | `inputProcessing.OkRecords` | Count | The number of records returned by a Lambda function that were marked with Ok status. | | `inputProcessing.ProcessingFailedRecords` | Count | The number of records returned by a Lambda function that were marked with ProcessingFailed status. | | `inputProcessing.Success` | Count | The number of successful Lambda invocations by Kinesis Data Analytics. | | `lambdaDelivery.DeliveryFailedRecords` | Count | The number of records returned by a Lambda function that were marked with DeliveryFailed status. | | `lambdaDelivery.Duration` | Milliseconds | The time taken for each Lambda function invocation performed by Kinesis Data Analytics. | | `lambdaDelivery.OkRecords` | Count | The number of records returned by a Lambda function that were marked with Ok status. | | `millisBehindLatest` | Milliseconds | Indicates how far behind from the current time an application is reading from the streaming source. | | `records` | Count | The number of records read (per input stream) or written (per output stream). | | `success` | Count | 1 for each successful delivery attempt to the destination configured for your application; 0 for every failed delivery attempt. The average value of this metric indicates how many successful deliveries are performed. | | `bytesRequestedPerFetch` | Bytes | The bytes requested in a single call to getRecords. | | `committedOffsets` | Other | The last successfully committed offsets to Kafka, for each partition. A particular partition's metric can be specified by topic name and partition id. | | `currentOffsets` | Other | The consumer's current read offset, for each partition. A particular partition's metric can be specified by topic name and partition id. | ### Kinesis Data Analytics ApplicationKpu data | Metric | Unit | Description | | ------ | ----- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `KPUs` | Count | The number of Kinesis Processing Units (KPUs) currently in use. **Known issue**: as this metric has a longer lag than the rest, in some cases, it's not shown | ### Kinesis Data Analytics Task data | Metric | Unit | Description | | ------------------------ | -------------- | ----------------------------------------------------------------------------------------------------------- | | `commitsFailed` | Count | The total number of offset commit failures to Kafka, if offset committing and checkpointing are enabled. | | `commitsSucceeded` | Count | The total number of successful offset commits to Kafka, if offset committing and checkpointing are enabled. | | `currentInputWatermark` | Milliseconds | The last watermark this task has received. | | `currentOutputWatermark` | Milliseconds | The currently output watermark. | | `numLateRecordsDropped` | Count | The number of records this task has dropped due to arriving late. | | `numRecordsIn` | Count | The total number of records this task has received. | | `numRecordsInPerSecond` | CountPerSecond | The total number of records this task has received per second. | | `numRecordsOut` | Count | The total number of records this task has emitted. | | `numRecordsOutPerSecond` | CountPerSecond | The total number of records this task has emitted per second. | | `recordsLagMax` | Count | The maximum lag in terms of number of records for any partition in this window. | ### Kinesis Data Analytics TaskOperator data | Metric | Unit | Description | | ------------------------ | -------------- | ----------------------------------------------------------------------------------------------------------- | | `bytesConsumedRate` | BytesPerSecond | The average number of bytes consumed per second for a topic. | | `commitsFailed` | Count | The total number of offset commit failures to Kafka, if offset committing and checkpointing are enabled. | | `commitsSucceeded` | Count | The total number of successful offset commits to Kafka, if offset committing and checkpointing are enabled. | | `currentInputWatermark` | Milliseconds | The last watermark this task has received. | | `currentOutputWatermark` | Milliseconds | The last watermark this application/operator/task/thread has emitted. | | `numLateRecordsDropped` | Count | The number of records this operator or task has dropped due to arriving late. | | `numRecordsIn` | Count | The total number of records this task has received. | | `numRecordsInPerSecond` | CountPerSecond | The total number of records this task has received per second. | | `numRecordsOut` | Count | The total number of records this task has emitted. | | `numRecordsOutPerSecond` | CountPerSecond | The total number of records this task has emitted per second. | | `recordsLagMax` | Count | The maximum lag in terms of number of records for any partition in this window. | ### Kinesis Data Analytics TaskParallelism data | Metric | Unit | Description | | ------------------------ | -------------- | ---------------------------------------------------------------------- | | `currentInputWatermark` | Milliseconds | The last watermark this application/operator/task/thread has received. | | `numLateRecordsDropped` | Count | The number of records this task has dropped due to arriving late. | | `numRecordsIn` | Count | The total number of records this task has received. | | `numRecordsInPerSecond` | CountPerSecond | The total number of records this task has received per second. | | `numRecordsOut` | Count | The total number of records this task has emitted. | | `numRecordsOutPerSecond` | CountPerSecond | The total number of records this task has emitted per second. | ### Kinesis Data Analytics TaskOperatorParallelism data | Metric | Unit | Description | | ------------------------ | -------------- | ----------------------------------------------------------------------------------------------------------- | | `bytesConsumedRate` | BytesPerSecond | The average number of bytes consumed per second for a topic. | | `commitsFailed` | Count | The total number of offset commit failures to Kafka, if offset committing and checkpointing are enabled. | | `commitsSucceeded` | Count | The total number of successful offset commits to Kafka, if offset committing and checkpointing are enabled. | | `currentInputWatermark` | Milliseconds | The last watermark this application/operator/task/thread has received. | | `currentOutputWatermark` | Milliseconds | The last watermark this application/operator/task/thread has emitted. | | `numRecordsIn` | Count | The total number of records this operator or task has received. | | `numRecordsInPerSecond` | CountPerSecond | The total number of records this operator or task has received per second. | | `numRecordsOut` | Count | The total number of records this operator or task has emitted. | | `numRecordsOutPerSecond` | CountPerSecond | The total number of records this operator or task has emitted per second. | | `recordsLagMax` | Count | The maximum lag in terms of number of records for any partition in this window. |