• /
  • ログイン
  • 無料アカウント

Understand and manage data ingest

When you report data to New Relic, we process what we receive and apply data dropping and transformation rules. Then we count the bytes needed to represent your data in a standard format, like JSON. If you're on our New Relic One pricing model, you're charged for the number of bytes written to our database that are above and beyond the free per-month amount.

If you're trying to estimate the cost of your data ingest, see Calculate data ingest.

Data ingestion UI

To learn how to easily analyze the data your account is ingesting, watch this short video (3:18 minutes).

The Data ingestion tab is located in the Data management UI. The Data ingestion UI shows your ingest rates for the time period specified by the time picker in the upper right.

The page shows your daily average GBs, and the total GBs for that time range. You can toggle between an overall ingest view and an account view to identify which of your accounts are sending the most data. The page also provides the current month-to-date, and the projected end-of-month total ingest rates. With this information, you can proactively manage your data ingest in various ways.

To see the underlying NRQL query that is used to generate the chart, click View query.

Gif shows data ingestion with source and account views

From the account dropdown, select Manage your data, and then select Data ingestion.

For how to get more details about ingested data, see Get ingest details.

Data ingestion sources

The data ingestion UI chart shows you a high level breakdown of your billable data usage. The table below explains those sources. In this table, "usage metric group" refers to the value of that source's usageMetric attribute value on the NrConsumption event.

Data sources

Description

Metrics

In the data ingestion chart, Metrics is a combination of two types of metrics: metric timeslice data and dimensional metrics.

Usage metric group: MetricsBytes.

Metric timeslice data averages to one-hour periods after eight days. After 90 days, the permanent metric data continues to be stored in one-hour periods. We currently store the raw metric data for 30 days.

You are only billed for the initial ingest volume. You are not billed for subsequent rollups.

APM

This includes APM events, like Transaction and TransactionError.

Usage metric group: ApmEventsBytes.

Infrastructure

Includes several categories of infrastructure monitoring events, described below.

Infrastructure host data.

Usage metric group:InfraHostBytes.

Information related to your servers and virtual machines coming from infrastructure agents, including storage and network data.

Infrastructure process data stored in ProcessSample.

Usage metric group: InfraProcessBytes.

Data related to each process running on the hosts running the infrastructure agent. This feature is turned off by default. For more information, see Process metrics.

Infrastructure integrations.

Usage metric group: InfraIntegrationBytes.

Performance data related to applications and services, typically managed by the customer, including data related to Docker containers, Windows services, Nagios checks, and cloud integrations such as managed services in AWS, Azure, and GCP.

Logging

Includes logs and any Log_<value> custom data partition created.

Usage metric group: LoggingBytes.

Log records are stored on the Log data type by default. Additional custom data partitions will create new data types, which are always prefixed with Log_ and are counted as part of the overall set of log data stored.

With LogExtendedRecord, log messages longer than 4KB are split into multiple events that, when needed, are stitched together to display the original message; this reduces the size of message data.

As of September 2021, log storage as blobs replaces LogExtendedRecord. With blob storage, NRDB can store up to 128,000 bytes for any attribute, not just messages. For more information, see our log blobs docs.

Default

Custom events.

Usage metric group: CustomEventsBytes

Mobile events

Mobile events, including the general Mobile event, MobileRequestError, MobileBreadcrumb, MobileSession, MobileHandledException, MobileCrash.

Usage metric group: MobileEventsBytes.

Tracing

Usage metric group: TracingBytes. This includes the Span data type and OpenTelemetry's SpanEvent. You are not charged for DistributedTraceSummary events.

Browser events

Browser events, including the namespaces of Browser, Browser:EventLog, Browser:JSErrors, and PcvPerf (PageView timing).

Usage metric group: BrowserEventsBytes.

Lambda

AWS Lambda events.

Usage metric group: ServerlessBytes.

Understand where data is coming from

You can inspect your data ingest to gain more information about your ingest health.

From the data ingestion UI page, you can analyze your usage in more detail. Spending some time understanding your ingested data and where it comes from and how it changes over time can be valuable. You'll know your ingest baselines, and you'll be able to more easily spot anomalies, like ingest spikes, and understand their source.

On the data ingestion chart, time is on the X axis and the bands representing data sources are located along the Y axis. Click on a data source band you want to inspect at the spot in the X axis that corresponds with the date you want to investigate.

screenshot of data facet selection

This image shows the data source band for June 15 right before it's clicked.

A modal opens with the account, data source, and facet selected. You can do a handful of things on this page:

  • Change the account, data source, or facet you want to drill down into.
  • Change the time range.
  • Review the results of the query in chart form. The chart displays the top 15 results for the facet query.
  • Open the NRQL query in the Query builder where you'll find additional facets that you can use.

For more about creating more detailed queries:

How ingested data is broken down

Some of the usage data in this UI can vary depending on your account. This information is intended to help you understand how we're working with your ingest data:

  • The chart on the Data ingestion page shows data usage for a little longer time frame than that covered by your retention settings for each data ingest source. If you choose a date outside of your designated retention period for an ingest source, you'll get the message that there's no chart data available. Select a more recent date to fix this problem.
  • If you inspect a data source for an account that has less than a terrabyte of data, we compute the ingest volume over a 24 hour period; otherwise, we compute it for a one hour period.
  • The ingest value provided on the main Data ingestion chart will be slightly different from that reflected during inspection. This is because our facet computation is an estimate.

Set alerts for data use

For how to set alerts that will notify you when you're reaching data ingest levels you don't want to cross, see Query and alert on usage data. For example, you might set an alert on logs, which can accumulate quickly in an active system.

Adjust your data ingest

Here are some ideas for managing your data:

Drop unwanted data

On ingest, we apply data dropping rules so you won't be charged for data that's not useful. Learn how to set additional data dropping rules yourself. For how to drop log data, see Drop log data.

Disable agents and integrations

If you have agents or integrations that you don't need at all, you can uninstall/delete those tools. For instructions, see the specific docs for that tool.

Adjust APM data ingest

Options for adjusting APM data include:

Adjust infrastructure data ingest

Options for adjusting infrastructure data include:

Adjust log data ingest

Options for adjusting log data ingest include:

問題を作成する
Copyright © 2022 New Relic Inc.