Define thresholds that trigger an alert

As part of the alert policy workflow in New Relic Alerts, create an alert condition for your policy. Thresholds are the values and frequency for the alert condition that will trigger the alert when violated. For example:

  • Alert on web response time if any application has an average that is above the threshold of 5 seconds for at least the last 15 minutes.
  • Alert when error rate = 10% at least once in the last 60 minutes.
  • Alert on Component/summary-metric if any plugin has an average that is above 8 units for at least the last 10 minutes.

Because New Relic Synthetics alerts are based on monitor failure, you don't need to define numeric thresholds.

Five-minute threshold required

An alert violation must remain past the defined threshold for at least five minutes before New Relic Alerts creates an incident record. For example:

  • If a browser app crosses an Apdex Critical threshold of 90% for two minutes, returns to 96%, and then drops to 89% again for three minutes, New Relic Alert does not create an incident record or send a notification.
  • If a browser app crosses an Apdex Critical threshold of 90% for five consecutive minutes, New Relic Alerts will create an incident record and send a notification.

Add thresholds to an alert condition

Admins, Owner, or add-on managers: When you first create a condition for your alert policy, the Define thresholds section automatically appears on the selected policy's Alert conditions page. At a minimum, you must define a Critical (red) threshold for the selected targets.

Alerts v3: Thresholds for conditions
alerts.newrelic.com > Alert policies > (select a policy) > Alert conditions > (select a condition) > Define thresholds: Select the key metric's value and frequency for the alert condition, or define your own custom metric threshold values. A Critical (red) severity level is required.

To define the threshold for the alert condition:

  1. Go to alerts.newrelic.com > Alert policies > (select a policy) > Alert conditions > (select a condition) > Define thresholds.
  2. Select the key metric's value and frequency for the Critical (red) threshold, or define your own custom metric threshold values.
  3. Optional: Define a Warning (yellow) threshold before the situation reaches the Critical level.
  4. Optional: Change the part of the condition's name that defaults to the defined threshold.
  5. Optional: Include the URL with runbook instructions for handling the situation.
  6. Select Create condition.

After you save a new condition, the Condition name no longer appears in the Thresholds section. If you want to change the condition name later, edit it from the selected policy's Alert conditions page.

Select severity levels

New Relic Alerts allows you to define different levels of severity for thresholds:

Threshold levels Comments
Info (webhooks only) If you use webhooks for your alert notification channels, you can include general notifications (INFO) as the $SEVERITY value.

icon-alert-warning.png Warning (yellow)

Optional

No notification

Include a Warning threshold for an alert condition if you want to monitor when an event passes this value, but you do not want to receive an alert notification for it. In general, set the Warning threshold lower than the Critical threshold. This is because the Warning will trigger before a higher, Critical condition occurs.

For example, you may want to view Warning event details in the user interface when CPU usage passes 80% for the defined time period, but you want New Relic Alerts to send a Critical alert to the policy's selected notification channels when CPU usage passes 95%.

Exception: For Apdex conditions, set the Warning threshold higher than its corresponding Critical threshold. This is because the higher an Apdex score, the better. For example, you may want a Warning when an app's Apdex level is at 96%, but you want a Critical alert notification when the app's Apdex score falls below 92%.

icon-alert-critical.png Critical (red)

Required

Uses notification channel

Include a Critical threshold for an alert condition so that New Relic Alerts will notify the policy's selected notification channels when an event passes this level for the defined time period. You can also view and acknowledge (take ownership) of Critical event details from the user interface.

View an alert condition's thresholds

To view existing thresholds for an alert policy's conditions: Go to alerts.newrelic.com > Alert policies > (select a policy) > Alert conditions. The Alert conditions page shows a list of existing conditions, their products, and their thresholds.

The Alert policies index and the selected policy's Alert conditions page indicate how many conditions already exist for an alert policy. You can have a maximum of 100 conditions per policy.

Update an alert condition's thresholds

Owner, Admins or add-on managers

When you first add a condition to an alert policy, at a minimum, New Relic Alerts requires you to define the Critical (red) threshold. To update threshold information for an alert condition later:

  1. Go to alerts.newrelic.com > Alert policies > (select a policy) > Alert conditions > (select a condition).
  2. From the condition's Thresholds section, select the standard key metric values or your own custom metric values that will trigger the Critical (red) alert condition (required).
  3. To include a Warning (yellow) threshold, select Add a warning threshold, and select the values that will trigger the alert condition.

    OR:

    To update or delete (Remove) the Warning threshold, select the existing value.

  4. To include information or standard procedures for handling the incident that this alert condition triggers, select Add runbook URL.

    OR:

    To change or delete an existing runbook URL from the alert condition, select Remove.

  5. Select Save condition.

For more help

Additional documentation resources include:

Recommendations for learning more: