Scope alert thresholds to specific instances

You can set alert thresholds that trigger when they're breached by any of your Java app's instances. Scoping the condition to instances of your app is helpful for detecting anomalies that occur only in a subset of your app's instances.

These sorts of anomalies are easy to miss for apps that aggregate metrics across a large number of instances. By looking at each instance, you can more quickly identify where potential problems are originating.

Example

In this example, you set up a policy for a Java app with three instances. You want to open an incident when the condition's critical threshold for any instance's error rate is greater than .02% for at least five minutes.

During a five-minute time period the three instances have these error rates:

Create instance-based alert condition

To create a policy that triggers notifications for incidents by your app's individual instances:

1. Follow the basic workflow process to set up a policy.
2. When creating a condition (step 2), select APM.
3. Select Application metric as the type of condition.
4. To evaluate alert threshold incidents individually for each of the app's selected instances, select Scope to Java application instances.
5. Select Next, select entities, then identify one or more apps for this condition.
6. Optional: Change the time when alerts will force-close the incidents (default is 24 hours).
7. Use By condition or By condition and signal incident preference.
8. Continue the rest of the policy workflow process (step 3).

Use "By condition" incident preference

When setting the incident preference for policies containing instance-based conditions, we recommend that you select By condition instead of By condition and signal. Although apps are the selected entities for this condition, we evaluate each JVM as a separate entity.

If you set your Incident preference to By condition and signal, a separate incident will be opened for each JVM that breaches the critical threshold. When an app fails across multiple JVMs, this can lead to alert "fatigue" and frustration.

Use REST API for instance alerts

To create instance-based alert conditions with the New Relic REST API, include these items with your REST API call:

• Your API key.
• The numeric entity_id for the entity being monitored
• The condition_id (available from the API Explorer: Alerts Conditions > GET > List)
• The entity_type (set to "application")
• The condition_scope (set to "instance" for a Java application instance or "application" for a Java app)

Here is an example of the API request format and JSON response.

$
curl -X PUT 'https://api.newrelic.com/v2/alerts_entity_conditions/12345.json' \
>
     -H 'X-Api-Key:$API_KEY' -i \
>
     -H 'Content-Type: application/json' \
>
     -G -d 'entity_type=application&condition_id=234567'

{
  "condition": [
    {
      "id": 234567,
      "type": "apm_app_metric",
      "name": "demo",
      "enabled": true,
      "entities": [
        12345
      ],
      "metric": "apdex",
      "condition_scope": "instance",  <---<<< scope indicator
      "violation_close_timer": 24,
      "terms": [
        {
          "duration": "5",
          "operator": "above",
          "priority": "critical",
          "threshold": "1",
          "time_function": "all"
        }
      ]
    }
  ]
}