Google Cloud Pub/Sub monitoring integration

Access to this feature will depend on your subscription level. Requires Infrastructure Pro.

New Relic Infrastructure's integrations with the Google Cloud Platform (GCP) include an integration to report Google Cloud Pub/Sub data to New Relic products. This document explains how to activate the GCP Pub/Sub integration and describes the data that will be reported.

Features

As part of the Google Cloud stream analytics solution, the Cloud Pub/Sub service ingests and delivers event streams for quick data processing and analysis.

Activate integration

Anyone can view and explore the GCP data reporting to your New Relic Infrastructure account.

Only Owner, Admins, or Infrastructure add-on managers can manage the GCP integrations.

To enable the integration:

  1. Make sure you have installed the Infrastructure agent before you activate GCP integrations from your Infrastructure account.
  2. Follow standard procedures to connect your GCP service to New Relic Infrastructure.

Polling frequency

New Relic Infrastructure integrations query your GCP services according to a polling interval, which varies depending on the integration. The polling frequency for Google Cloud Pub/Sub is five minutes. The resolution is 1 data point every minute.

Find and use data

After activating the integration and waiting a few minutes (based on the polling frequency), data will appear in the New Relic UI. To find and use your data, including links to your dashboards and alert settings, go to infrastructure.newrelic.com > Integrations > Google Cloud Platform > (select an integration).

Metric data

Pub/Sub topics are named entities that represent feeds of messages, and subscriptions are named entities that represent message destinations on a particular topic.

To view metric data for your GCP Pub/Sub integration in Insights, create NRQL queries for GcpPubSubTopicSample and GcpPubSubSubscriptionSample events and their related attributes.

GcpPubSubTopicSample

Query GcpPubSubTopicSample events in Insights to view data for the following attributes:

Attribute Description
topic.ByteCost Cost of operations, measured in bytes. This is used to measure utilization for quotas.
topic.ConfigUpdates Cumulative count of configuration changes, grouped by operation type and result.
topic.MessageSizes Distribution of publish message sizes (in bytes).
topic.NumRetainedAckedMessagesByRegion Number of acknowledged messages retained in a topic, broken down by Cloud region.
topic.NumUnackedMessagesByRegion Number of unacknowledged messages in a topic, broken down by Cloud region.
topic.OldestRetainedAckedMessageAgeByRegion Age (in seconds) of the oldest acknowledged message retained in a topic, broken down by Cloud region.
topic.OldestUnackedMessageAgeByRegion Age (in seconds) of the oldest unacknowledged message in a topic, broken down by Cloud region.
topic.RetainedAckedBytesByRegion Total byte size of the acknowledged messages retained in a topic, broken down by Cloud region.
topic.SendMessageOperation Cumulative count of publish message operations.
topic.SendRequest Cumulative count of publish requests.
topic.UnackedBytesByRegion Total byte size of the unacknowledged messages in a topic, broken down by Cloud region.

GcpPubSubSubscriptionSample

Query GcpPubSubSubscriptionSample events in Insights to view data for the following attributes:

Attribute Description
subscription.BacklogBytes Total byte size of the unacknowledged messages (a.k.a. backlog messages) in a subscription.
subscription.ByteCost Cumulative cost of operations, measured in bytes. This is used to measure quota utilization.
subscription.ConfigUpdates Cumulative count of configuration changes for each subscription, grouped by operation type and result.
subscription.ModAckDeadlineMessageOperation Cumulative count of ModifyAckDeadline message operations, grouped by result.
subscription.ModAckDeadlineRequest Cumulative count of ModifyAckDeadline requests, grouped by result.
subscription.NumOutstandingMessages Number of messages delivered to a subscription's push endpoint, but not yet acknowledged.
subscription.NumRetainedAckedMessages Number of acknowledged messages retained in a subscription.
subscription.NumRetainedAckedMessagesByRegion Number of acknowledged messages retained in a subscription, broken down by Cloud region.
subscription.NumUnackedMessagesByRegion Number of unacknowledged messages in a subscription, broken down by Cloud region.
subscription.NumUndeliveredMessages Number of unacknowledged messages (a.k.a. backlog messages) in a subscription.
subscription.OldestRetainedAckedMessageAge Age (in seconds) of the oldest acknowledged message retained in a subscription.
subscription.OldestRetainedAckedMessageAgeByRegion Age (in seconds) of the oldest acknowledged message retained in a subscription, broken down by Cloud region.
subscription.OldestUnackedMessageAge Age (in seconds) of the oldest unacknowledged message (a.k.a. backlog message) in a subscription.
subscription.OldestUnackedMessageAgeByRegion Age (in seconds) of the oldest unacknowledged message in a subscription, broken down by Cloud region.
subscription.PullAckMessageOperation Cumulative count of acknowledge message operations, grouped by result.
subscription.PullAckRequest Cumulative count of acknowledge requests, grouped by result.
subscription.PullMessageOperation Cumulative count of pull message operations, grouped by result.
subscription.PullRequest Cumulative count of pull requests, grouped by result.
subscription.PushRequest Cumulative count of push attempts, grouped by result. Unlike pulls, the push server implementation does not batch user messages, so each request only contains one user message. The push server retries on errors, so a given user message can appear multiple times.
subscription.PushRequestLatencies Distribution of push request latencies (in microseconds), grouped by result.
subscription.RetainedAckedBytes Total byte size of the acknowledged messages retained in a subscription.
subscription.RetainedAckedBytesByRegion Total byte size of the acknowledged messages retained in a subscription, broken down by Cloud region.
subscription.StreamingPullAckMessageOperation Cumulative count of StreamingPull acknowledge message operations, grouped by result.
subscription.StreamingPullAckRequest Cumulative count of streaming pull requests with non-empty acknowledge ids, grouped by result.
subscription.StreamingPullMessageOperation Cumulative count of streaming pull message operations, grouped by result.
subscription.StreamingPullModAckDeadlineMessageOperation Cumulative count of StreamingPull ModifyAckDeadline operations, grouped by result.
subscription.StreamingPullModAckDeadlineRequest Cumulative count of streaming pull requests with non-empty ModifyAckDeadline fields, grouped by result.
subscription.StreamingPullResponse Cumulative count of streaming pull responses, grouped by result.
subscription.UnackedBytesByRegion Total byte size of the unacknowledged messages in a subscription, broken down by Cloud region.

Inventory data

To view inventory data for GCP Pub/Sub services, go to infrastructure.newrelic.com > Integrations > Inventory and search for or select the following:

gcp/pubsub/topic

Name Description

name

Topic name.

projectId

The GCP project Id.

tags*

Topic tags.

* Only available if the GCP project is linked to New Relic through a service account.

gcp/pubsub/subscription

Name Description

name

Subscription name.

projectId

The GCP project Id.

tags*

Topic tags.

ackDeadlineSeconds*

Time (in seconds) Pub/Sub waits for the subscriber to acknowledge receipt of a message before resending it.

secondsMessageRetention*

Time (in seconds) that unacknowledged messages are kept in the subscription's backlog, from the moment a message is published.

pushEndpoint*

A URL locating the endpoint to which messages should be pushed.

retainAckedMessages*

Indicates whether to retain acknowledged messages. If true, then messages are not expunged from the subscription's backlog, even if they are acknowledged, until they fall out of the `message_retention_duration` window.

* Only available if the GCP project is linked to New Relic through a service account.

For more help

Recommendations for learning more: