• Log inStart now

Custom anomaly detection

Custom anomalies allow your team the most versatility when detecting unusual behavior in your system. Not only are they flexible and dynamic, custom anomalies provide your team with the capability to alert on any NQRL condition and to adjust and optimize your thresholds. Custom anomalies are built using the same advanced tuning settings as static alerting so you can ensure your team sees only incidents that are important to you.

Configure anomaly thresholds

After selecting 'anomaly' for threshold type, you'll need to set your threshold. This sets the criteria that you want to be notified on.

To do this you'll need to:

  • Set whether you want the condition to check for upper violations, lower violations, or both.
  • Set the behavior that will trigger an incident (for example: "deviating for more than five minutes")
  • Adjust how sensitive the condition is to fluctuations in the data source by adjusting the sliding bar to more violations or fewer violations. This bar calculates how many standard deviations away from the prediction you want to be notified.

When your data escapes the predicted "normal" behavior and meets the criteria you've chosen, you'll receive a notification.

Here are some tips for setting anomaly thresholds:

  • Set the anomaly direction to monitor violations that happen either above or below the anomaly.
  • Set the preview chart to either 2 days or 7 days of displayed data. This is not applicable for NRQL alert conditions.
  • Use the slider bar to adjust the Critical threshold sensitivity, represented in the preview chart by the light gray area around the anomaly. The tighter the band around the anomaly, the more sensitive it is and the more violations it will generate.
  • Optional: You can create a Warning threshold (the darker gray area around the anomaly).
  • For NRQL alerts, see the allowed types of NRQL queries.
  • If the alert condition applies to multiple apps, you can select a choice from the dropdown above the chart to use different metrics. (Not applicable for NRQL alert conditions.)

Faceted anomaly conditions

Once you've run a faceted NRQL query for an anomaly condition, you can scope your results to a single anomaly.

The single time series shows the anomaly, threshold band, and one or more violation regions, if there are any.

Anomaly direction: select upper or lower ranges

You can choose whether you want the condition to look for behavior that goes above the predicted value ("upper") or that goes below the predicted value ("lower"), or that goes either above or below. You choose these with the prediction direction selector.

Example use cases for this:

  • You might use the Upper setting for a data source like error rate, because you generally are only concerned if it goes up, and aren't concerned if it goes down.
  • You might use the Lower setting for a data source like throughput, because sudden upward fluctuations are quite common, but a large sudden downswing would indicate a problem.

Here are examples of how large fluctuations in your data would be treated under the different anomaly direction settings. The red areas represent violations.

Rules governing calculation of the predicted value

The algorithm for calculating the prediction is mathematically complex. Here are some of the major rules governing its predictive abilities:

  • Age of data On initial creation, the prediction is calculated using between 1 to 4 weeks of data, depending on data availability and prediction type. After its creation, the algorithm takes into account ongoing data fluctuations over a long time period, although greater weight is given to more recent data. For data that has only existed for a short time, the predicted value will likely fluctuate a good deal and not be very accurate. This is because there isn't enough data to determine its usual values and behavior. The more history the data has, the more accurate the prediction and thresholds will become.
  • Consistency of data For metric values that remain in a consistent range or that trend slowly and steadily, their more predictable behavior means that their thresholds will become tighter around the prediction. Data that is more varied and unpredictable will have looser (wider) thresholds.
  • Regular fluctuations For shorter-than-one-week cyclical fluctuations (such as weekly Wednesday 1pm deployments or nightly reports), the prediction algorithm looks for these cyclical fluctuations and attempts to adjust to them.

Preview chart: select 2 or 7 days

When setting thresholds, the preview chart has an option for displaying Since 2 days ago or Since 7 days ago. These selections are not the time period used to compute anomaly; they are only the time range used for a preview display. For more about the time range used to calculate the anomaly, see the algorithm rules. Not applicable for NRQL alert conditions.

Copyright © 2023 New Relic Inc.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.