New Relic's integrations include an integration for reporting your GCP Dataproc data to our products. Here we explain how to activate the integration and what data it collects.
Activate integration
To enable the integration follow standard procedures to connect your GCP service to New Relic.
Configuration and polling
You can change the polling frequency and filter data using configuration options.
Default polling information for the GCP Dataproc integration:
- New Relic polling interval: 5 minutes
Find and use data
To find your integration data, go to one.newrelic.com > All capabilities > Infrastructure > GCP and select an integration.
Data is attached to the following event type:
Entity | Event Type | Provider |
|---|---|---|
Cluster |
|
|
For more on how to use your data, see Understand and use integration data.
Metric data
This integration collects GCP Dataproc data for Cluster.
Dataproc Cluster data
Metric | Unit | Description |
|---|---|---|
| Count | Indicates the number of HDFS DataNodes that are running inside a cluster. |
| Gibibytes | Indicates capacity of HDFS system running on cluster in GB. |
| Percent | The percentage of HDFS storage currently used. |
| Count | Indicates the number of unhealthy blocks inside the cluster. |
| Seconds | The time jobs took to complete from the time the user submits a job to the time Dataproc reports it is completed. |
| Seconds | The time jobs have spent in a given state. |
| Count | Indicates the number of jobs that have failed on a cluster. |
| Count | Indicates the number of jobs that are running on a cluster. |
| Count | Indicates the number of jobs that have been submitted to a cluster. |
| Seconds | The time operations took to complete from the time the user submits a operation to the time Dataproc reports it is completed. |
| Seconds | The time operations have spent in a given state. |
| Count | Indicates the number of operations that have failed on a cluster. |
| Count | Indicates the number of operations that are running on a cluster. |
| Count | Indicates the number of operations that have been submitted to a cluster. |
| Percent | The percentage of YARN memory is allocated. |
| Count | Indicates the number of active YARN applications. |
| Count | Indicates the number of YARN containers. |
| Gibibytes | Indicates the YARN memory size in GB. |
| Count | Indicates the number of YARN NodeManagers running inside cluster. |
| Gibibytes | The current memory request, in GB, that is pending to be fulfilled by the scheduler. |
| Count | Indicates the number of virtual cores in YARN. |