Alerts for infrastructure: Add, edit, or view host alert information

With New Relic's infrastructure monitoring, you can create alert conditions directly within the context of what you are currently monitoring with New Relic. For example, if you are monitoring a filter set and notice a problem, you do not need to recreate those criteria from New Relic Alerts.

Instead, you can immediately select your filter set and tailor the alert condition directly from the chart you are viewing. This helps you proactively manage and monitor the alerting system for your environment. Any alert violations will be created per entity within the filter set.

Create alert conditions for infrastructure

Alert conditions apply to alert policies. You can select an existing policy or create a new policy with email notifications from the Infrastructure monitoring UI. If you want to use other types of notification channels, create a new policy from within the Alerts UI.

The Infrastructure REST API has a limit of 3,700 alert conditions, including both active and disabled conditions. The API, whether used directly or via the UI, will reject all requests to add any additional alert conditions beyond the 3,700 alert condition limit.

To add an infrastructure alert condition to an alerts policy:

  1. Go to one.newrelic.com > Infrastructure, then select any of these Infrastructure monitoring pages: Hosts, Processes, Network, or Storage. Mouse over the chart you want to alert on, select the ellipses [ellipses icon] icon, and then select Create alert.

  2. Type a meaningful condition name.
  3. Select the Alert type, or refer to the examples to decide which type to select.
  4. Create individual filters, or copy all the filters from a filter set to identify the hosts that you want the alert condition to use.

    For more information about the rules behind filters, see Filter set logic.

  5. Define the Critical (required) and Warning (optional, if available) thresholds for triggering the alert notification.
  6. Optional: To create the condition criteria proactively but not receive alert notifications at this time, turn off the Enabled checkbox option.
  7. Select an existing policy for the new condition.

    OR

    Select the option to create a new policy and identify the email for alert notifications.

  8. Optional: Add a runbook url.

  9. Optional: Set Violation time limit for violations (this defaults to 24 hours).

  10. Select Create.

Create alert conditions for infrastructure integrations

To create an alert condition for your infrastructure integrations, use any of these options:

UI options Alert conditions for infrastructure integrations
From Alerts & AI

Go to one.newrelic.com > Alerts & AI > Alerts > Alert policies > New alert policy > Create new condition, then select Infrastructure as the product.

OR

Go to one.newrelic.com > Alerts & AI > Alerts > Alert policies > (select a policy), then select Add a condition.

From Infrastructure Go to one.newrelic.com > Infrastructure, then select any of these Infrastructure monitoring pages: Hosts, Processes, Network, or Storage. Mouse over the chart you want to alert on, select the ellipses [ellipses icon] icon, and then select Create alert.

Then continue with the procedures to add an alert condition to Infrastructure.

View host alert events

Anyone included in the policy's notification channels receive alert notifications directly. In addition, anyone with permissions for your New Relic account can view Infrastructure alert incidents and individual violations through the user interface.

  1. Go to one.newrelic.com > Infrastructure > Events.
  2. To change the hosts or time frame, use the search window, Filter set, or Time functions.
  3. From the Events list, select the alert violation.
  4. To view detailed information in Alerts about the selected violation, select the link.

Update or delete host alert information

To edit, disable (or re-enable), or delete host alert information:

  1. Go to one.newrelic.com > Infrastructure > Settings > Alerts.
  2. Optional: Use the search window or Select all checkbox to locate one or more alert conditions.
  3. Select any of the available functions to [pencil icon] edit, disable, enable, or [trash can icon] delete the selected conditions.

Use New Relic Alerts to monitor your entire infrastructure

New Relic Alerts provides a single, coordinated alerting tool across all of your New Relic products. This allows you to manage alert policies and conditions that focus on the metrics for entities that you care about the most, such as Docker containers, JVMs, and more.

Alert features Features in Infrastructure
Alert conditions
Information on alerts
Alert policies

View, add, change, disable, or delete:

  • For policies with a variety of notification channels: Use the Alerts UI.
  • For policies only needing email notifications: Go to one.newrelic.com > Infrastructure > Settings > Alerts > Create a new policy, and add one or more email addresses as needed.

Add host conditions to an existing policy: Use the Infrastructure UI.

Notification channels

To view, add, change, or delete available notification options:

  1. Go to one.newrelic.com > Infrastructure > Settings > Alerts.
  2. Optional: Search for the condition or policy name.
  3. From the list of conditions, select the policy link to view notification channel information in the Alerts UI.

Add a description

The use of the Description field is available for these alert condition types:

  • NRQL conditions: add a description using the NerdGraph API.
  • Infrastructure conditions: add a description using the UI or the REST API.

The text you place in an alert condition's Description field is passed downstream to associated violations and notifications. A description can be used for several purposes, including:

  • Capturing the reason for the alert condition.
  • Defining the signal being monitored.
  • Defining next steps.
  • Add metadata to downstream systems.

You can use template substitution to insert values from the attributes in the associated violation event. The template format is {{attributeName}}. For the attributes you can use when creating a description, see Violation event attributes.

One available attribute is the special {{tag.*}} attribute. This attribute prefix is used to access any of the tag values that are included with the target signal, or any of the entity tags that are associated with the target signal.

If there are entity tags associated with your violation, then they can be accessed using the entity tag name. An example of this would be {{tag.aws.awsRegion}}. When entity tags are available to use, you see them included with the violation, and displayed when you view the violations in an incident.

This field has a maximum character size of 4,000.

Add or edit a runbook URL

The alert condition creation process includes an option for setting a URL for runbook instructions. This lets you link to information or standard procedures for handling a violation. Before adding or updating the link, make sure you use a valid URL.

To add, update, or delete an alert condition's runbook URL:

  1. Select an alert condition, and make changes to the Runbook URL link.
  2. Save the condition.

In order to be saved, the URL must be a valid URL.

Violation time limit for violations

The violation time limit allows you to define a time period after which violations will be force-closed. By default, violation time limit is 24 hours.

To add, update, or remove an alert condition's violation time limit:

  1. Select an alert condition, and make changes to the violation time limit.
  2. To remove this limit, deselect the checkbox next to Close open violations after and select --.
  3. Save the condition.

Alert conditions that generate too-long NRQL queries

Alert conditions created for infrastructure rely on behind-the-scenes NRQL queries, and NRQL queries have a 4096-character limit. This means that if your condition generates a very complex NRQL query that filters on many elements (for example, including many hosts or many tags), it will exceed this limit and display an error message saying that the condition failed.

To solve this problem, reduce the number of elements you are using in your alert condition. For example:

Problem Solution
Hosts
  • If you entered a large number of hosts that caused the condition to fail, reduce the number of hosts.
  • Use substrings to target hosts. For example, instead of targeting prod-host-01, prod-host-02, and prod-host-03, just target all hosts with prod-host-0 in the name.
Entities
  • Edit your alert condition to target specific attributes that apply to the entities you're trying to target.
  • Create custom attributes for the entities you want to target, and use those attributes in your alert condition.

For more information, see Best practices for filtering in infrastructure alerts in New Relic's Explorers Hub.

For more help

If you need more help, check out these support and learning resources: