• EnglishEspañol日本語한국어Português
  • Log inStart now

NerdGraph tutorial: Loss of signal and gap filling

You can customize New Relic alerting loss of signal detection and gap filling using our NerdGraph API. For example, you can configure how long to wait before considering the signal lost, or what value should be used for filling gaps in the time series.

Loss of signal occurs when New Relic stops receiving data for a while; technically, we detect loss of signal after a significant amount of time has elapsed since data was last received in a time series. Loss of signal can be used to trigger or resolve an incident, which you can use to set up alerts.

Gap filling can help you solve issues caused by lost data points. When gaps are detected between valid data points, we automatically fill those gaps with replacement values, such as the last known values or a static value. Gap filling can prevent alerts from triggering or resolving when they shouldn't.

Tip

The system fills gaps in actively reported signals. This signal history is dropped after 2 hours of inactivity. For gap filling, data points received after this period of inactivity are treated as new signals.

To learn more about signal loss, gap filling, and how to request access to these features, see this Support Forum post.

In this guide we cover the following:

Customize your loss of signal detection

Loss of signal detection opens or closes incidents if no data is received after a certain amount of time. For example, if you set the duration of the expiration period to 60 seconds and an integration doesn't seem to send data for more than a minute, a loss of signal threshold would be triggered.

You can configure the duration of the signal loss and whether to open an incident or close it by using these three fields in NerdGraph:

  • expiration.expirationDuration: How long to wait, in seconds, after the last data point is received by our platform before considering the signal as lost. This is based on the time when data arrives at our platform and not on data timestamps. The default is to leave this null, and therefore this wouldn't enable Loss of Signal Detection.
  • expiration.openViolationOnExpiration: If true, a new incident is opened when a signal is lost. Default is false. To use this field, a duration must be specified.
  • expiration.closeViolationsOnExpiration: If true, open incidents related to the signal are closed on expiration. Default is false. To use this field, a duration must be specified.

You also have the option to skip opening an incident when the signal is expected to be lost. To do this, use this field in NerdGraph:

  • expiration.ignoreOnExpectedTermination: If true, an incident will not be opened when the signal is expected to be lost. To indicate a signal is expected to be lost, the tag termination: expected must be present on the entity. Default is false.

View loss of signal settings for an existing condition

Existing NRQL conditions may have their loss of signal settings already configured. To view the existing condition settings, select the fields under nrqlCondition > expiration:

{
actor {
account(id: YOUR_ACCOUNT_ID) {
alerts {
nrqlCondition(id: NRQL_CONDITION_ID) {
... on AlertsNrqlStaticCondition {
id
name
nrql {
query
}
expiration {
closeViolationsOnExpiration
expirationDuration
openViolationOnExpiration
ignoreOnExpectedTermination
}
}
}
}
}
}
}

You should see a result like this:

{
"data": {
"actor": {
"account": {
"alerts": {
"nrqlCondition": {
"expiration": {
"closeViolationsOnExpiration": false,
"expirationDuration": 300,
"openViolationOnExpiration": true,
"ignoreOnExpectedTermination": false
},
"id": "YOUR_ACCOUNT_ID",
"name": "Any less than - Extrapolation",
"nrql": {
"query": "SELECT average(value) FROM AlertsSmokeTestSignals WHERE wave_type IN ('min-max', 'single-gap') FACET wave_type"
}
}
}
}
}
},
...

Create a new condition with loss of signal settings

Let's say that you want to create a new create a NRQL static condition that triggers a loss of signal threshold after no data is received for two minutes. You would set expirationDuration to 120 seconds and set openViolationOnExpiration to true, like in the example below.

mutation {
alertsNrqlConditionStaticCreate(
accountId: YOUR_ACCOUNT_ID
policyId: YOUR_POLICY_ID
condition: {
name: "Low Host Count - Catastrophic"
enabled: true
nrql: {
query: "SELECT uniqueCount(host) FROM Transaction WHERE appName='my-app-name'"
}
signal {
aggregationWindow: 60
aggregationMethod: EVENT_FLOW
aggregationDelay: 120
}
terms: [{
threshold: 2
thresholdOccurrences: AT_LEAST_ONCE
thresholdDuration: 600
operator: BELOW
priority: CRITICAL
}]
valueFunction: SINGLE_VALUE
violationTimeLimitSeconds: 86400
expiration: {
expirationDuration: 120
openViolationOnExpiration: true
}
}
) {
id
name
}
}

Update the loss of signal settings of a condition

What if you want to update loss of signal parameters for an alert condition? The following mutation allows you to update a NRQL static condition with new expiration values.

mutation {
alertsNrqlConditionStaticUpdate(
accountId: YOUR_ACCOUNT_ID
id: YOUR_STATIC_CONDITION_ID
condition: {
expiration: {
closeViolationsOnExpiration: true
expirationDuration: 300
openViolationOnExpiration: false
ignoreOnExpectedTermination: true
}
}
) {
id
expiration {
closeViolationsOnExpiration
expirationDuration
openViolationOnExpiration
ignoreOneExpectedTermination
}
}
}

Customize gap filling

Gap filling replaces gap values in a time series with either the last value found or a static, arbitrary value of your choice. We fill gaps only after another data point has been received after the gaps in signal (after data reception has been restored).

You can configure both the type of filling and the value, if the type is set to static:

  • signal.fillOption: Type of replacement value for lost data points. Values can be:
    • NONE: Gap filling is disabled.
    • LAST_VALUE: The last value seen in the time series.
    • STATIC: An arbitrary value, defined in fillValue.
  • signal.fillValue: Value to use for replacing lost data points when fillOption is set to STATIC.

Important

Gap filling is also affected by expiration.expirationDuration. When a gap is longer than the expiration duration, the signal is considered expired and the gap will no longer be filled.

For example, here's how to create a static NRQL condition with gap filling configured:

mutation {
alertsNrqlConditionStaticCreate(
accountId: YOUR_ACCOUNT_ID
policyId: YOUR_POLICY_ID
condition: {
enabled: true
name: "Example Gap Filling Condition"
nrql: { query: "SELECT count(*) FROM Transaction" }
terms: {
operator: ABOVE
priority: CRITICAL
threshold: 1000
thresholdDuration: 300
thresholdOccurrences: ALL
}
valueFunction: SINGLE_VALUE
violationTimeLimitSeconds: 28800
signal: {
aggregationWindow: 60
aggregationMethod: EVENT_FLOW
aggregationDelay: 120
fillOption: STATIC
fillValue: 1
}
}
) {
id
}
}
Copyright © 2024 New Relic Inc.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.