As part of the alert policy workflow in New Relic Alerts, use the Alert conditions page to identify what triggers an alert for the selected policy. An alert policy condition includes:
- Available New Relic product (APM, Browser, etc.) and type of condition (metric, external service, etc.)
- Entities (product targets) that the policy applies to, such as apps monitored by New Relic APM or New Relic Browser, mobile apps monitored by New Relic Mobile, etc.
- Thresholds for required Critical (red) and optional Warning (yellow) alert notifications, including what metric to alert on, what value it monitors, and what threshold must pass within the specified time frame
Policy conditions for New Relic products
An alert policy can have a single condition, up to a maximum of 250 conditions. Each condition applies to a specific New Relic product. The Alert policies index and the selected policy's Alert conditions page indicate how many conditions already exist for an alert policy.
- Condition example
An alert policy can apply to one or more New Relic products. For example, you can create a single alert policy with a condition for error rate thresholds for all apps monitored by New Relic APM, a second condition with a custom metric for selected apps monitored by New Relic APM, a third condition with a Apdex levels monitored by New Relic Browser, and additional conditions as needed.
Create an alert condition
You can add one or more conditions to an alert policy as part of the basic setup process, or you can save the policy name and add conditions later.
- For new policies, New Relic Alerts automatically shows a Create a condition button after you save the policy's name.
- For saved policies, you can define alert conditions anytime: From alerts.newrelic.com, select Alert policies > (selected policy) > Alert condition.
To create an alert condition:
- From the selected policy's Alert conditions page, select Create a condition (if the policy does not have any conditions) or Add a condition (if the policy already has existing conditions).
- From Categorize, select the New Relic product.
- Select the type of condition (metric, external service, etc.).
- Select Next, select targets.
- Continue with the procedures to select the targets this policy condition will apply to, then define the thresholds that trigger the alert condition.
- Optional: After you finish creating a condition, copy it and add it other alert policies in the selected New Relic account.
Alert conditions that provide fields for you to input numerical values accept decimal points up to the second decimal place (hundredths). For example,
0.01 is the smallest possible value.
Apdex and response time alert conditions
You can use New Relic to trigger alert notifications for response times. However, Apdex scores are almost always meaningful and provide a better reflection of application performance. For example, average response times can be skewed by outliers, while the Apdex score gives a more accurate assessment of acceptable response time rates that your users experience.
Types of conditions
Depending on which product you select (APM, Browser, etc.), the New Relic Alerts user interface shows what types of conditions (metric, external service, etc.) you can alert on.
Here are some additional special features.
- NRQL queries
- Baseline thresholds for conditions
Baseline alerting allows you to create alert conditions that dynamically adjust to changing data and trends, such as weekly or seasonal patterns. Baseline alerting is also useful for new systems whose behavior is unknown. This feature is available for New Relic APM, New Relic Browser apps using New Relic Alerts, and for NRQL alerts.
- Outlier detection (NRQL alert)
Outlier detection looks for groups in your data and for values that are outliers from those groups. Outlier detection is available for NRQL alerts.
- Violations for individual Java instances
New Relic Alerts allows you to set alert thresholds that trigger when they are violated by any of your Java app's instances.
By scoping alert thresholds to specific instances, New Relic Alerts can help you more quickly identify where potential problems are originating. This is useful, for example, to detect anomalies that are occurring only in a subset of your app's instances. These sorts of anomalies are easy to miss for apps that aggregate metrics across a large number of instances.
- JVM health metrics (Java apps)
For Java apps monitored by New Relic APM, New Relic Alerts allows you to set thresholds that trigger when the heap size or number of threads for a single JVM is out of the expected operating range.
New Relic calculates alerting threshold violations individually for each of the app's selected instances. When creating your alert condition, select JVM health metric as the type of condition for your Java app's alert policy, then select any of the available thresholds:
- Deadlocked threads
- Heap memory usage
- CPU utilization time
- Garbage collection CPU time
From the New Relic Alerts user interface you can also change the time when an incident automatically closes for a JVM health metric. Default is 24 hours.
- Web app response percentiles
New Relic Alerts includes the option to define a percentile as the threshold for your alert policy's condition when your web app's response time is above, below, or equal to this value. This is useful, for example, when Operations personnel want to alert on a percentile for an app server's overall web transaction response time rather than the average web response time.
To define the percentile threshold:
- Select Web transactions percentiles as the type of condition for your APM app's alert policy, then select a single app. (To alert on more than one app, create an individual Web transactions percentiles condition for each.)
- To define the thresholds that trigger the alert notification, type the Percentile nth response time value, then select its frequency (above, below, or equal to this value).
New Relic stores the transaction time in milliseconds, although the user interface shows the Critical and Warning values as seconds. If you want to define milliseconds, be sure to include the decimal point in your value.
- Dynamic targeting with labels for apps
By applying labels to applications, you can automatically link these entities to your alert policy condition. This makes it easy to manage all the applications within a dynamic environment. New Relic recommends using the agent's configuration file to best maintain entity labels.
A single label identifies all entities associated with that label (maximum 10,000 entities). Multiple labels only identify entities with the same corresponding labels.
To add, edit, or remove up to ten labels for a condition:
- Select APM > Application metric as the product type.
- When identifying entities, select the Labels tab. Search for a label by name, or select a label from the list of categories.
You can also create alert conditions directly within the context of what you are monitoring with New Relic Infrastructure.
- Infrastructure conditions
You can create alert conditions for your resources directly in New Relic Infrastructure.
For example, if you want to be notified when New Relic has stopped receiving data from an Infrastructure agent, use the host not reporting feature. Unlike the legacy availability monitoring feature for New Relic Servers, this allows you to dynamically alert on groups of hosts, configure the time window from 5 to 60 minutes, and take full advantage of New Relic Alerts.
Change an alert condition's name
If you want to change the condition name that New Relic Alerts automatically creates, make it short and descriptive (maximum 64 characters). Provide useful information for notification messages that have limited characters, such as email subject lines, online chat, etc.
- Use camel case or dotted decimal notation.
- Describe the essence of what is being violated.
To change an existing alert condition's freeform name:
- From alerts.newrelic.com, select Alert policies > (selected policy) > Alert conditions.
- Mouse over the alert condition's name, and select the Edit [pencil icon] icon.
- Type a meaningful name for the condition (maximum 64 characters), then save your changes.
You cannot change or rename the New Relic product and condition type associated with the alert condition's freeform name. Instead, you must delete the condition and create a new one with a different product and condition type.
Maintain alert policies and conditions
After you save the alert condition, the currently selected alert policy lists all alert conditions that apply to it. From here you can:
- Repeat the steps to add more conditions to the policy (maximum 250 conditions per policy).
- Continue the alert policy setup process by adding one or more notification channels to it.
- Change the condition's name, the entities it alerts on, or the thresholds for Critical (red) and Warning (yellow) alerts.
- Copy the condition and add it other alert policies in the selected New Relic account.
- Rename the alert policy.
- Disable or delete the alert policy or any of its conditions.
View existing conditions
The New Relic Alerts index lists alert policies in alphabetical order. To view or search for existing conditions in an alert policy:
- From alerts.newrelic.com, select Alert policies.
- Use the search box, sort any column, or scroll the list, then select the policy's name.
- Optional: From the Alert conditions page, use the search box to locate a specific condition.
- To view detailed information about entities associated with a condition, select its Targets link.
- To view information about the alert thresholds or runbook URL associated with a condition, select its Warning and Critical Thresholds link.
To view policy and condition information for a specific entity, select the Settings > Alert conditions page from the entity's New Relic product UI.