Google BigQuery monitoring integration

Access to this feature will depend on your subscription level. Requires Infrastructure Pro.

New Relic Infrastructure's integrations with the Google Cloud Platform (GCP) include an integration to report Google BigQuery data to New Relic products. This document explains how to activate the Google BigQuery integration and describes the data that can be reported.

Features

Google BigQuery is a data warehouse with no infrastructure to manage. With BigQuery you can specify the data schema, or allow the service to discover it for you. You can explore your data using SQL queries.

Activate integration

Anyone can view and explore the Google BigQuery data reporting to your New Relic Infrastructure account.

Only Owner, Admins, or Infrastructure add-on managers can manage this type of integration.

To enable the integration follow standard procedures to connect your GCP service to New Relic Infrastructure.

Polling frequency

New Relic Infrastructure integrations query GCP services according to a polling interval that varies depending on the integration. The polling frequency for the Google BigQuery integration is five minutes. For information about the resolution, see metric data details.

Find and use data

After activating the integration and waiting a few minutes (based on the polling frequency), data will appear in the New Relic UI. To find and use your data, including links to your dashboards and alert settings, go to infrastructure.newrelic.com > GCP > (select an integration).

Metric data

To view metric data for your Google BigQuery integration in Insights, create NRQL queries for GcpBigQueryProjectSample, GcpBigQueryDatasetSample, and GcpBigQueryTableSample events and their attributes.

GcpBigQueryProjectSample

Query GcpBigQueryProjectSample events in Insights to view data for the following metrics. Resolution is one data point per minute with an 8-minute delay.

Attribute Description
query.Count In-flight queries.
query.ExecutionTimes Distribution of query execution times.
slots.Allocated Number of slots currently allocated for project.
slots.AllocatedForProject Number of slots currently allocated for query jobs in the project.
slots.AllocatedForProjectAndJobType Number of slots currently allocated for the project and job type.
slots.AllocatedForReservation Number of slots currently allocated for project in the reservation.
slots.TotalAllocatedForReservation Number of slots currently allocated across projects in the reservation.
slots.TotalAvailable Total number of slots available for the project.

GcpBigQueryDatasetSample

Query GcpBigQueryDatasetSample events in Insights to view data for the following metrics. Resolution is one data point every 30 minutes with a 3-hour delay.

Attribute Description
storage.StoredBytes Amount of stored bytes.
storage.Tables Number of tables.

GcpBigQueryTableSample

Query GcpBigQueryTableSample events in Insights to view data for the following metrics. Resolution is one data point per minute with a 6-hour delay.

Attribute Description
storage.UploadedBytes Uploaded bytes.
storage.UploadedBytesBilled Uploaded bytes billed. Currently only streaming API is billed.
storage.UploadedRowCount Uploaded rows.

Inventory data

To view inventory data for Google BigQuery services, go to infrastructure.newrelic.com > Integrations > Inventory and search for or select the following attributes.

Data indicated with an asterisk * are only fetched when the GCP project is linked to New Relic through a service account, and when extended inventory collection is on.

gcp/bigquery/project

Name Description

projectId

The ID of the project.

gcp/bigquery/dataset

Name Description

projectId

The ID of the project.

name

The region where the instance is.

zone*

The location of the dataset.

description*

Description of the dataset.

creationTime*

Timestamp when the dataset was created.

lastModified*

Last time the dataset or any of its tables was modified.

defaultTableLifeTime*

Default lifetime of all tables in the dataset in milliseconds.

friendlyName*

User-friendly name for the dataset.

selfLink*

URL to access the resource.

labels*

Labels of the dataset.

gcp/bigquery/table

Name Description

projectId

The ID of the project.

dataSetId

The ID of the dataset.

name

The ID of the table.

zone*

The location of the table.

creationTime*

Timestamp when the table was created.

type*

The table type.

description*

The description of the table.

expirationTime*

Timestamp when the table will expire.

friendlyName*

User-friendly name for the table.

selfLink*

URL to access the resource.

labels*

Labels of the table.

lastModified*

Time when the table was last modified.

Notes

As of November 20, 2018, for the metric storage/uploaded_bytes_billed, information is only available if data is uploaded to BigQuery via streaming.

For more help

Recommendations for learning more: