Create a baseline report

Install the data ingest governance baseline dashboard

This is the dashboard you'll use to create baselines and monitor your ingest data. The Overview tab provides a variety of charts to help keep an eye on your ingest, and there are multiple tabs that provide detailed views of specific telemetry types such as browser data, APM data, logs, and traces.

For generating the report you need to begin optimizing your telemetry, you need to use the Baseline Reports tab after you install the dashbaord using the steps below:

To install the dashboard:

1. Navigate to the data ingest governance quickstart.
2. Click Install this quickstart in the upper right portion of your browser window.
3. Select your primary or top-level account in the account switcher, if applicable.
4. Click Done.
5. When the quickstart finishes installing, open the Data ingest governance baseline dashboard.

Add ingest target indicators to your dashboard

Once you've set up the baseline dashboard, you should add target indicators to the dashboard to monitor. These indicators help you optimize your ingest data by providing target metrics for you to either exceed or stay below, depending on your needs. Some targets you may want to create are:

• An overall organizational target on daily rate or monthly ingest.
• Targets per data type to ensure the optimal breakdown (for example 1 TB per day for logs and 2 TB per day for metrics).
• Targets for specific sub-accounts or business units.

To add these target indicators to your dashboard, you can either create new charts entirely or modify charts that already exist in the dashboard.

To create a new chart:

1. In the baseline dashboard, select + Add widget.
2. Select Add a chart.
3. Enter the NRQL query that the chart will use.
4. Select Run.

For example, if you wanted your total organizational ingest to be less than 360 TB per month, you can add a threshold line chart by adding the static number 360000 to your SELECT statement using the following NRQL statement:

SELECT 360000, rate(sum(GigabytesIngested), 30 day) AS '30 Day Rate' FROM NrConsumption WHERE productLine='DataPlatform' since 30 days ago limit max compare with 1 month ago TIMESERIES 7 days

To modify an existing chart:

1. In the baseline dashboard, select ... for the chart you want to modify.
2. Select Edit.
3. Modify the NRQL query that the chart uses as needed.
4. Select Run.

For example, after creating the chart above, you may decide that you want to create a daily ingest rate target by dividing 360000 by 30 and using 12000 as the daily target number. To do that, you would update the Daily ingest rate (compare with 3 months prior) chart using the following query:

SELECT 12000, rate(sum(GigabytesIngested), 1 day) AS avgGbIngestTimeseries FROM NrConsumption WHERE productLine='DataPlatform' TIMESERIES AUTO since 9 months ago limit max COMPARE WITH 3 months ago

Generate your 30-day ingest report

Once you've installed the baseline dashboard and changed the data it reports to suit your needs, you can generate a tabular report of your ingest data after enough time has passed for you to gather the appropriate amount of data you. You can do this from the Baseline report, and the most common timespan is 30 days. To generate the report:

1. Open the previously installed data ingest governance baseline dashboard.
2. Select the Baseline report tab.
3. Select ... on the Last 30 days table and choose Export as CSV.
4. Choose whether to import the CSV into Google Sheets or another spreadsheet of your choice.

SELECT sum(GigabytesIngested) AS 'gb_ingest_30_day_sum', rate(sum(GigabytesIngested), 1 day) AS 'gb_ingest_daily_rate', derivative(GigabytesIngested, 90 day) as 'gb_ingest_90_day_derivative' FROM NrConsumption WHERE productLine='DataPlatform' since 30 days ago facet consumingAccountName, usageMetric  limit max