Google Cloud Pub/Sub monitoring integration

Access to this feature will depend on your subscription level. Requires Infrastructure Pro.

New Relic Infrastructure's integrations with the Google Cloud Platform (GCP) include an integration to report Google Cloud Pub/Sub data to New Relic products. This document explains how to activate the GCP Pub/Sub integration and describes the data that will be reported.

Features

As part of the Google Cloud stream analytics solution, the Cloud Pub/Sub service ingests and delivers event streams for quick data processing and analysis.

Activate integration

Anyone can view and explore the GCP data reporting to your New Relic Infrastructure account.

Only Owner, Admins, or Infrastructure add-on managers can manage the GCP integrations.

To enable the integration:

Make sure you have installed the Infrastructure agent before you activate GCP integrations from your Infrastructure account.
Follow standard procedures to connect your GCP service to New Relic Infrastructure.

Polling frequency

New Relic Infrastructure integrations query your GCP services according to a polling interval, which varies depending on the integration. The polling frequency for Google Cloud Pub/Sub is five minutes. The resolution is 1 data point every minute.

Find and use data

After activating the integration and waiting a few minutes (based on the polling frequency), data will appear in the New Relic UI. To find and use your data, including links to your dashboards and alert settings, go to infrastructure.newrelic.com > Integrations > Google Cloud Platform > (select an integration).

Metric data

Pub/Sub topics are named entities that represent feeds of messages, and subscriptions are named entities that represent message destinations on a particular topic.

To view metric data for your GCP Pub/Sub integration in Insights, create NRQL queries for GcpPubSubTopicSample and GcpPubSubSubscriptionSample events and their related attributes.

GcpPubSubTopicSample

Query GcpPubSubTopicSample events in Insights to view data for the following attributes:

Attribute	Description
`topic.ByteCost`	Cost of operations, measured in bytes. This is used to measure utilization for quotas.
`topic.ConfigUpdates`	Cumulative count of configuration changes, grouped by operation type and result.
`topic.MessageSizes`	Distribution of publish message sizes (in bytes).
`topic.NumRetainedAckedMessagesByRegion`	Number of acknowledged messages retained in a topic, broken down by Cloud region.
`topic.NumUnackedMessagesByRegion`	Number of unacknowledged messages in a topic, broken down by Cloud region.
`topic.OldestRetainedAckedMessageAgeByRegion`	Age (in seconds) of the oldest acknowledged message retained in a topic, broken down by Cloud region.
`topic.OldestUnackedMessageAgeByRegion`	Age (in seconds) of the oldest unacknowledged message in a topic, broken down by Cloud region.
`topic.RetainedAckedBytesByRegion`	Total byte size of the acknowledged messages retained in a topic, broken down by Cloud region.
`topic.SendMessageOperation`	Cumulative count of publish message operations.
`topic.SendRequest`	Cumulative count of publish requests.
`topic.UnackedBytesByRegion`	Total byte size of the unacknowledged messages in a topic, broken down by Cloud region.

GcpPubSubSubscriptionSample

Query GcpPubSubSubscriptionSample events in Insights to view data for the following attributes:

Attribute	Description
`subscription.BacklogBytes`	Total byte size of the unacknowledged messages (a.k.a. backlog messages) in a subscription.
`subscription.ByteCost`	Cumulative cost of operations, measured in bytes. This is used to measure quota utilization.
`subscription.ConfigUpdates`	Cumulative count of configuration changes for each subscription, grouped by operation type and result.
`subscription.ModAckDeadlineMessageOperation`	Cumulative count of ModifyAckDeadline message operations, grouped by result.
`subscription.ModAckDeadlineRequest`	Cumulative count of ModifyAckDeadline requests, grouped by result.
`subscription.NumOutstandingMessages`	Number of messages delivered to a subscription's push endpoint, but not yet acknowledged.
`subscription.NumRetainedAckedMessages`	Number of acknowledged messages retained in a subscription.
`subscription.NumRetainedAckedMessagesByRegion`	Number of acknowledged messages retained in a subscription, broken down by Cloud region.
`subscription.NumUnackedMessagesByRegion`	Number of unacknowledged messages in a subscription, broken down by Cloud region.
`subscription.NumUndeliveredMessages`	Number of unacknowledged messages (a.k.a. backlog messages) in a subscription.
`subscription.OldestRetainedAckedMessageAge`	Age (in seconds) of the oldest acknowledged message retained in a subscription.
`subscription.OldestRetainedAckedMessageAgeByRegion`	Age (in seconds) of the oldest acknowledged message retained in a subscription, broken down by Cloud region.
`subscription.OldestUnackedMessageAge`	Age (in seconds) of the oldest unacknowledged message (a.k.a. backlog message) in a subscription.
`subscription.OldestUnackedMessageAgeByRegion`	Age (in seconds) of the oldest unacknowledged message in a subscription, broken down by Cloud region.
`subscription.PullAckMessageOperation`	Cumulative count of acknowledge message operations, grouped by result.
`subscription.PullAckRequest`	Cumulative count of acknowledge requests, grouped by result.
`subscription.PullMessageOperation`	Cumulative count of pull message operations, grouped by result.
`subscription.PullRequest`	Cumulative count of pull requests, grouped by result.
`subscription.PushRequest`	Cumulative count of push attempts, grouped by result. Unlike pulls, the push server implementation does not batch user messages, so each request only contains one user message. The push server retries on errors, so a given user message can appear multiple times.
`subscription.PushRequestLatencies`	Distribution of push request latencies (in microseconds), grouped by result.
`subscription.RetainedAckedBytes`	Total byte size of the acknowledged messages retained in a subscription.
`subscription.RetainedAckedBytesByRegion`	Total byte size of the acknowledged messages retained in a subscription, broken down by Cloud region.
`subscription.StreamingPullAckMessageOperation`	Cumulative count of StreamingPull acknowledge message operations, grouped by result.
`subscription.StreamingPullAckRequest`	Cumulative count of streaming pull requests with non-empty acknowledge ids, grouped by result.
`subscription.StreamingPullMessageOperation`	Cumulative count of streaming pull message operations, grouped by result.
`subscription.StreamingPullModAckDeadlineMessageOperation`	Cumulative count of StreamingPull ModifyAckDeadline operations, grouped by result.
`subscription.StreamingPullModAckDeadlineRequest`	Cumulative count of streaming pull requests with non-empty ModifyAckDeadline fields, grouped by result.
`subscription.StreamingPullResponse`	Cumulative count of streaming pull responses, grouped by result.
`subscription.UnackedBytesByRegion`	Total byte size of the unacknowledged messages in a subscription, broken down by Cloud region.

Inventory data

To view inventory data for GCP Pub/Sub services, go to infrastructure.newrelic.com > Integrations > Inventory and search for or select the following:

gcp/pubsub/topic

Name	Description
`name`	Topic name.
`projectId`	The GCP project Id.
`tags*`	Topic tags.

* Only available if the GCP project is linked to New Relic through a service account.

gcp/pubsub/subscription

Name	Description
`name`	Subscription name.
`projectId`	The GCP project Id.
`tags*`	Topic tags.
`ackDeadlineSeconds*`	Time (in seconds) Pub/Sub waits for the subscriber to acknowledge receipt of a message before resending it.
`secondsMessageRetention*`	Time (in seconds) that unacknowledged messages are kept in the subscription's backlog, from the moment a message is published.
`pushEndpoint*`	A URL locating the endpoint to which messages should be pushed.
`retainAckedMessages*`	Indicates whether to retain acknowledged messages. If true, then messages are not expunged from the subscription's backlog, even if they are acknowledged, until they fall out of the `message_retention_duration` window.