In New Relic Alerts, an alert condition describes a monitored data source and the behavior of that data source that will be considered a violation. This document will explain the types of conditions available, how to create a condition, and how to view existing conditions.
- Basic concepts and workflow
- Min/max limits (like maximum number of conditions per policy)
- Alerts API to list or edit alert settings
Types of alert conditions
Depending on which product you select (APM, Browser, etc.), the Alerts user interface shows what types of conditions (metric, external service, etc.) you can alert on.
Here are some additional special features:
- NRQL query conditions
- Baseline thresholds for conditions
Baseline alerting allows you to create alert conditions that dynamically adjust to changing data and trends, such as weekly or seasonal patterns. This feature is available for APM and Browser apps, as well as NRQL alerts.
- Outlier detection (NRQL alert)
Outlier detection looks for groups in your data and for values that are outliers from those groups. Outlier detection is available for NRQL alerts.
- Violations for individual Java instances
You can set alert thresholds that open a violation when they are breached by any of your Java app's instances' metrics.
By scoping alert thresholds to specific instances, you can more quickly identify where potential problems are originating. This is useful, for example, to detect anomalies that are occurring only in a subset of your app's instances. These sorts of anomalies are easy to miss for apps that aggregate metrics across a large number of instances.
- JVM health metrics (Java apps)
For Java apps monitored by APM, you can set thresholds that open a violation when the heap size or number of threads for a single JVM is out of the expected operating range.
We calculate alerting threshold violations individually for each of the app's selected instances. When creating your alert condition, select JVM health metric as the type of condition for your Java app's alert policy, then select any of the available thresholds:
- Deadlocked threads
- Heap memory usage
- CPU utilization time
- Garbage collection CPU time
Violations will automatically close when the inverse of the threshold is met, but by using the UI you can also change the time when a violation force-closes for a JVM health metric. Default is 24 hours.
- Web app response percentiles
We include the option to define a percentile as the threshold for your alert condition when your web app's response time is above, below, or equal to this value. This is useful, for example, when Operations personnel want to alert on a percentile for an app server's overall web transaction response time rather than the average web response time.
To define the percentile threshold:
- Select Web transactions percentiles as the type of condition for your APM app's alert condition, then select a single app. (To alert on more than one app, create an individual Web transactions percentiles condition for each.)
- To define the thresholds that open the alert violation, type the Percentile nth response time value, then select its frequency (above, below, or equal to this value).
We store the transaction time in milliseconds, although the user interface shows the Critical and Warning values as seconds. If you want to define milliseconds, be sure to include the decimal point in your value.
- Dynamic targeting with labels for apps
By applying labels to applications, you can automatically link these entities to your alert condition. This makes it easy to manage all the applications within a dynamic environment. We recommend using the agent's configuration file to best maintain entity labels.
A single label identifies all entities associated with that label (maximum 10,000 entities). Multiple labels only identify entities which share all the selected labels.
To add, edit, or remove up to ten labels for a condition:
- Select APM > Application metric as the product type.
- When identifying entities, select the Labels tab. Search for a label by name, or select a label from the list of categories.
You can also create alert conditions directly within the context of what you are monitoring with New Relic Infrastructure.
- Infrastructure conditions
You can create alert conditions for your resources directly in Infrastructure.
For example, if you want to be notified when we have stopped receiving data from an Infrastructure agent, use the host not reporting condition type. This allows you to dynamically alert on filtered groups of hosts and configure the time window from 5 to 60 minutes.
Create an alert condition
You can add one or more conditions to an alert policy as part of the basic setup process, or you can save the policy name and add conditions later.
- For new policies, we automatically show a Create a condition button after you save the policy's name.
- For saved policies, you can define alert conditions anytime: From alerts.newrelic.com, select Alert policies > (selected policy) > Alert condition.
To create an alert condition:
- From the selected policy's Alert conditions page, select Create a condition (if the policy does not have any conditions) or Add a condition (if the policy already has existing conditions).
- From Categorize, select the product.
- Select the type of condition (metric, external service, etc.).
- Select Next, select entities.
- Continue with the procedures to select the data sources this condition will apply to, then define the thresholds that cause the alert condition to open a violation.
- Optional: After you finish creating a condition, copy it and add it other alert policies in the selected account.
Alert conditions that provide fields for you to input numerical values accept decimal points up to the second decimal place (hundredths). For example,
0.01 is the smallest possible value.
Apdex and response time alert conditions
You can open violations and send alert notifications for response times. However, Apdex scores are almost always more meaningful and provide a better reflection of application performance. For example, average response times can be skewed by outliers, while the Apdex score gives a more accurate assessment of acceptable response time rates that your users experience.
Change an alert condition's name
If you want to change the default condition name, make it short and descriptive (maximum 64 characters). Provide useful information for notification messages that have limited characters, such as email subject lines, online chat, etc.
- Use camel case or dotted decimal notation.
- Describe the essence of what is being violated.
To change an existing alert condition's name:
- From alerts.newrelic.com, select Alert policies > (selected policy) > Alert conditions.
- Mouse over the alert condition's name, and select the Edit [pencil icon] icon.
- Type a meaningful name for the condition (maximum 64 characters), then save your changes.
You cannot edit the product and condition type associated with an alert condition. Instead, you must delete the condition and create a new one with a different product and condition type.
Maintain alert policies and conditions
After you save the alert condition, the currently selected alert policy lists all alert conditions that apply to it. From here you can:
- Repeat the steps to add more conditions to the policy (maximum 250 conditions per policy).
- Continue the alert policy setup process by adding one or more notification channels to it.
- Change the condition's name, the entities it's scoped to, or the critical (red) and warning (yellow) thresholds.
- Copy the condition and add it other alert policies in the selected New Relic account.
- Rename the alert policy.
- Disable any conditions in the policy, or delete the alert policy or any of its conditions.
View existing conditions
The alert policies index lists policies in alphabetical order. To view or search for existing conditions in an alert policy:
- From alerts.newrelic.com, select Alert policies.
- Use the search box, sort any column, or scroll the list, then select the policy's name.
- Optional: From the Alert conditions page, use the search box to locate a specific condition.
- To view detailed information about entities associated with a condition, select its Targets link.
- To view information about the alert thresholds or runbook URL associated with a condition, select its warning and critical thresholds link.
To view policy and condition information for a specific entity, select the Settings > Alert conditions page from the entity's New Relic product UI.