• /
  • Log in

Google Cloud Composer monitoring integration

BETA FEATURE

This feature is currently in beta.

We offer a cloud integration for reporting your GCP Composer data to our platform. Here we explain how to activate the integration and what data it collects.

Activate the integration

To enable the integration follow standard procedures to connect your GCP service.

Configuration and polling

You can change the polling frequency and filter data using configuration options.

Default polling information for the GCP Composer integration:

  • New Relic polling interval: 5 minutes

Find and use data

To find your integration data, go to one.newrelic.com > Infrastructure > GCP and select the integration.

Data is attached to the following event types:

Entity

Event Type

Provider

Environment

GcpComposerEnvironmentSample

GcpComposerEnvironment

Workflow

GcpComposerWorkflowSample

GcpComposerWorkflow

For more on how to use your data, see Understand and use integration data.

Metric data

This integration collects GCP Composer data for Environment and Workflow.

Composer Environment data

Metric

Unit

Description

environment.api.Request

Count

Number of Composer API requests seen so far.

environment.api.RequestLatencies

Milliseconds

Distribution of Composer API call latencies.

environment.dag_processing.ParseError

Count

Number of errors raised during parsing DAG files.

environment.dag_processing.Processes

Count

Number of currently running DAG parsing processes.

environment.dag_processing.ProcessorTimeout

Count

Number of file processors terminated due to processing timeout.

environment.dag_processing.TotalParseTime

Seconds

Number of seconds taken to scan and import all DAG files once.

environment.DagbagSize

Count

The current dag bag size.

environment.database.cpu.ReservedCores

Count

Number of cores reserved for the database instance.

environment.database.cpu.UsageTime

Seconds

CPU usage time of the database instance.

environment.database.cpu.Utilization

Count

CPU utilization ratio (from 0.0 to 1.0) of the database instance.

environment.database.disk.BytesUsed

Bytes

Used disk space in bytes on the database instance.

environment.database.disk.Quota

Bytes

Maximum data disk size in bytes of the database instance.

environment.database.disk.Utilization

Count

Disk quota usage ratio (from 0.0 to 1.0) of the database instance.

environment.database.memory.BytesUsed

Bytes

Memory usage of the database instance in bytes.

environment.database.memory.Quota

Bytes

Maximum RAM size in bytes of the database instance.

environment.database.memory.Utilization

Count

Memory utilization ratio (from 0.0 to 1.0) of the database instance.

environment.executor.OpenSlots

Count

Number of open slots on executor.

environment.executor.RunningTasks

Count

Number of running tasks on executor.

environment.FinishedTaskInstance

Count

Overall task instances.

environment.NumCeleryWorkers

Count

Number of Celery workers.

environment.SchedulerHeartbeat

Count

Scheduler heartbeats.

environment.TaskQueueLength

Count

Number of tasks in queue.

environment.worker.PodEviction

Count

The number of Airflow worker pods evictions.

environment.ZombieTaskKilled

Count

Number of zombie tasks killed.

Composer Workflow data

Metric

Unit

Description

workflow.Run

Count

Number of workflow runs completed so far.

workflow.RunDuration

Seconds

Duration of workflow run completion.

workflow.task.Run

Count

Number of workflow tasks completed so far.

workflow.task.RunDuration

Seconds

Duration of task completion.

For more help

If you need more help, check out these support and learning resources:

Create issueEdit page
Copyright © 2021 New Relic Inc.