Incident Intelligence destination examples

Destinations are the data outputs to Applied Intelligence, where you can view your automatically correlated incidents. You can configure Incident Intelligence destinations to send data to PagerDuty or webhooks. This document gives examples of destination templates, webhook formats, and JSON schema.

Destination template examples

The following template example destinations are formatted in Jinja2.

Suggested VictorOps template example

{
	"monitoring_tool": {{ sources }},
	{% if state == 'closed' %}
		"message_type":  "OK",
	{% else %}
		"message_type":  {{ priority }},
	{% endif %}

	"custom_fields": {
		"issue_url": {{ url }},
		"description": {{ description }},
		"state": {{ state }},
		"is_correlated": {{ is_correlated }},
		"created_on": {{ created_on }},
		"modified_on": {{ modified_on }},
		"activated_on": {{ active_since }},
		"closed_on": {% if closed_on is defined %} {{ closed_on }} {% else %} None {% endif %},
		"related_incidents": [
			{% for incident in incidents %}
			{
				"id": {{ incident.id }},
				"events_count": {{ incident.events_count }},
				"labels": {{ incident.labels }},
				"title": {{ incident.title }},
				"description": {{ incident.description }},
				"state": {{ incident.state }},
				"sources": {{ incident.sources }},
				"modified_on": {{ incident.modified_on }},
				"opened_on": {{ incident.opened_on }},
				"closed_on": {{ incident.closed_on }}
			}
			{% if not loop.last %},{% endif %}
			{% endfor %}
		]
	},
	"state_message": {{ description }},
	"entity_id": {{ id }},
	"entity_display_name": {{ ui_name }},
	"vo_annotate.u.NRAI_Link": {{ url }}
}

Suggested OpsGenie template example

{
	"alias":  {{ id }},
	"message": {{ ui_name }},
	"source": '{{ sources }}',
	"priority": {{ priority }},
	"details": {
		"self_url": {{ url }},
		"state": {{ state }},
		"is_correlated": {{ is_correlated }},
		"created_on": {{ created_on }},
		"modified_on": {{ modified_on }},
		"activated_on": {{ active_since }},
		"closed_on": {% if closed_on is defined %} {{ closed_on }} {% else %} None {% endif %},
		"is_correlated": {{ is_correlated }}
	},
	"description": 'Incidents [
		{% for incident in incidents %}
		{
			"id": {{ incident.id }},
			"events_count": {{ incident.events_count }},
			"labels": {{ incident.labels }},
			"title": {{ incident.title }},
			"description": {{ incident.description }},
			"state": {{ incident.state }},
			"sources": {{ incident.sources }},
			"modified_on": {{ incident.modified_on }},
			"opened_on": {{ incident.opened_on }},
			"closed_on": {{ incident.closed_on }}
		}
		{% if not loop.last %},{% endif %}{% endfor %}
	]'
}

Suggested ServiceNow template example

{
"short_description": {{ ui_name }},
"description": 'Issue Id: {{ id }}, \n Description: {{ description }}, \n Sources: {{ sources }}, \n Priority: {{ priority }}, \n Details: { \n self_url: {{ url }}, \n state: {{ state }}, \n is_correlated: {{ is_correlated }}, \n created_on: {{ created_on }}, \n modified_on: {{ modified_on }}, \n activated_on: {{ active_since }}, \n closed_on: {% if closed_on is defined %} {{ closed_on }} {% else %} None {% endif %}, \n is_correlated: {{is_correlated}} }, \n\n incidents: {{ incidents }}, \n\n pathways: {{ pathways }}'
}

Suggested Slack template example

1. Go to Slack incoming WebHooks
2. Choose the right Slack workspace and click "Add to Slack"
3. Select the destination channel to receive the notifications to
4. Click “Add Incoming WebHooks Integration”
5. Copy the WebHook URL
6. In the next screen, click “Save settings” at the bottom.
7. In New Relic Applied Intelligence, under Incident Intelligence click Destinations
8. Add a WebHook
9. In the end point paste the WebHook url from slack.
10. In the custom payload, paste the following JSON:

{"blocks": [
        {"type": "section",
        "text": {"type": "mrkdwn", "text": "*MONITORING TOOL*:"},
        "fields":
        [{% for source in sources %}
            {"type": "plain_text", "text": {{ source }} }
           {% if not loop.last %},{% endif %}
           {% endfor %}
           ]
        }
        ,
        {"type": "divider"},
        {"type": "section",
        "text": {"type": "mrkdwn", "text": "*CUSTOM FIELDS*:"},
        "fields": [{"type": "mrkdwn", "text": "*entity_id*"},
                   {"type": "mrkdwn", "text": "*entity_display_name*"},
                   {"type": "plain_text", "text": {{ id }}  },
                   {"type": "plain_text", "text": {{ ui_name }} } ]},
        {"type": "section",
        "fields": [{"type": "mrkdwn", "text": "*issue_url*"},
                   {"type": "mrkdwn", "text": "*description*"},
                   {"type": "plain_text", "text": {{ url }}  },
                   {"type": "plain_text", "text": {{ description }} } ]},
        {"type": "section",
        "fields": [{"type": "mrkdwn", "text": "*state*"}, 
                   {"type": "mrkdwn", "text": "*is_correlated*"},
                   {"type": "plain_text", "text": {{ state  }} },
                   {"type": "plain_text", "text": "{{ is_correlated }}" }]},
        {"type": "section",
        "fields": [{"type": "mrkdwn", "text": "*created_on*"},
                   {"type": "mrkdwn", "text": "*modified_on*"},
                   {"type": "plain_text", "text": "{{ created_on }}" },
                   {"type": "plain_text", "text": "{{ modified_on }}" } ]},
        {"type": "section",
        "fields": [{"type": "mrkdwn", "text": "*activated_on*"},
                   {"type": "mrkdwn", "text": "*closed_on*"},
                   {"type": "plain_text", "text": "{{ active_since }}" },
                   {"type": "plain_text", "text": "{% if closed_on is defined %} {{ closed_on }} {% else %} None {% endif %}" } ]},
        {"type": "divider"},
        {"type": "section",
        "text": {"type": "mrkdwn", "text": "*RELATED_INCIDENTS*:"} },
        {% for incident in incidents %}
            {"type": "section",
            "fields": [
                       {"type": "mrkdwn", "text": "*id*"},
                       {"type": "mrkdwn", "text": "*events count*"},
                       {"type": "plain_text", "text": {{ incident.id }}  },
                       {"type": "plain_text", "text": "{{ incident.events_count}}" } ]},
            {"type": "section",
            "fields": [
                       {"type": "mrkdwn", "text": "*title*"},
                       {"type": "mrkdwn", "text": "*description*"},
                       {"type": "plain_text", "text": {{ incident.title}}  },
                       {"type": "plain_text", "text": {{ incident.description}} }] },
            {"type": "section",
            "fields": [
                       {"type": "mrkdwn", "text": "*state*"},
                       {"type": "mrkdwn", "text": "*modified on*"},
                       {"type": "plain_text", "text": {{ incident.state}}  },
                       {"type": "plain_text", "text": {{ incident.modified_on}} },
                       {"type": "mrkdwn", "text": "*opened on*"},
                       {"type": "mrkdwn", "text": "*closed on*"},
                       {"type": "plain_text", "text": {{ incident.opened_on}}  },
                       {"type": "plain_text", "text": {% if incident.closed_on is defined and incident.closed_on %} {{ incident.closed_on }} {% else %} "None" {% endif %} } ]},
            {"type": "section",
            "text": {"type": "mrkdwn", "text": "*LABELS*:"} ,
            "fields": [{% for name, value in incident['labels'].items() %}
                {% set bold_name = '*' + name + '*' %}
                {"type": "mrkdwn", "text": {{ bold_name }} },
                {"type": "plain_text", "text": {{ value }}  }
                {% if not loop.last %},{% endif %}
               {% endfor %} ]},
            {"type": "section",
            "text": {"type": "mrkdwn", "text": "*SOURCES*:"} ,
            "fields": [{% for source in incident['sources'] %}
                {"type": "plain_text", "text": {{ source }}  }
                {% if not loop.last %},{% endif %}
               {% endfor %} ]},
            {"type": "divider"}
               {% if not loop.last %},{% endif %}
               {% endfor %}
    ]
}

Webhook and JSON format examples

Applied Intelligence will send the event body in JSON format via HTTPS POST. The system expects the endpoint to return a successful HTTP code (2xx). If you use webhooks to configure Incident Intelligence destinations, use these examples of the webhook body format and JSON schema.

Webhook format and JSON example

Webhook format:

Name	Description
url Href	A link to the UI where the issue can be seen.
id String	Unique identifier for the issue.
title String	The issue title.
description String	The description of the issue.
priority Enum	The issue priority. Can be Critical, High, Medium, or Low.
state Enum	The issue status. Can be Active, Closed, or Acknowledged.
is_correlated Boolean	True if the issue is based on correlated data.
created_on String	The date and time the issue was created (in ISO format).
modified_on String	The date and time the issue was modified (in ISO format).
active_since String	The date and time the issue was activated (in ISO format).
closed_on String	The date and time the issue was closed (in ISO format).
sources List(String)	List of the different sources that were used to send the events into Applied Intelligence (for example: PagerDuty).
pathways List(Pathways)	List of pathways that are associated with the issue. Each pathway contains an id and name: [{“id”: “pathway id”, “name”: “pathway name”}].
pathways[].id String	The pathway ID.
pathways[].name String	The pathway name.
incidents List(Incident)	List of incidents that are attached to the issue. The list contains only the latest 100 incidents.
incidents[].id String	The incident ID.
incidents[].events_count Integer	The number of events used to create the incident.
incidents[].title String	The incident title.
incidents[].description String	The incident description.
incidents[].labels Dictionary (String)	A string to string mapping of the incident labels. Labels represent the unique entities that are used to describe the incident.
incidents[].priority Enum	The incident priority. Can be Critical, High, Medium, or Low.
incidents[].sources List(String)	The incident source.
incidents[].state Enum(open, closed)	The incident state.
incidents[].opened_on String	The date and time the incident was opened (in ISO format).
incidents[].closed_on String	The date and time the incident was closed (in ISO format).
ui_name String	Issue title.
accumulations['alert/signal'] String	Issue analysis summary golden signal/s (if applicable).
accumulations['alert/components'] String	Issue analysis summary golden components (if applicable).

JSON schema example

Applied Intelligence will send the event body in JSON format via HTTPS POST. The system expects the endpoint to return a successful HTTP code (2xx).

JSON schema:

{
 "$schema": "http://json-schema.org/draft-04/schema#",
 "definitions": {
   "Labels": {
     "type": "object",
     "properties": {},
     "additionalProperties": {
       "type": "string",
       "maxLength": 10000
     }
   },
   "LabelsSet": {
     "type": "array",
     "properties": {},
     "additionalProperties": {
       "type": "array",
       "items": {
         "$ref": "#/definitions/Labels"
       },
       "uniqueItems": true
     },
   },
   "Annotations": {
     "type": "object",
     "title": "Additional contextual information about the Entity",
     "description": "Fields like title, description and priority can be described as annotations since\nthey represent some additional information about the Incident.\nFor example, the rule that creates the entity can have some logic that will create a title annotation field.\nThe title will change according to the latest event received by the entity and as more\nevent will keep coming the field may change, all this as a decision by the rule.\nA different rule may decide not to populate the title at all.\nSince a title can also be produced from the labels themselves.",
     "properties": {
       "title": {
         "type": "string",
         "maxLength": 10000
       },
       "description": {
         "type": "string",
         "maxLength": 10000
       }
     },
     "patternProperties": {
       ".*": {
         "type": "string",
         "maxLength": 10000
       }
     },
     "additionalProperties": {
       "type": "string",
       "maxLength": 10000
     }
   },
   "Accumulations": {
     "type": "object",
     "title": "Accumulated data about the Incident",
     "description": "For example an entity with the label: host_name: foo will include a accumulated list of the\nrelevant process names that are relevant to the entity. Accumulations will be derived from the events",
     "properties": {},
     "additionalProperties": {
       "type": "array",
       "items": {
         "type": "string",
         "maxLength": 10000
       }
     }
   },
   "IssueState": {
     "type": "string",
     "enum": [
       "created",
       "active",
       "acknowledged",
       "closed"
     ],
     "properties": {},
     "additionalProperties": false
   },
   "Priority": {
     "type": "string",
     "enum": [
       "critical",
       "high",
       "medium",
       "low"
     ],
     "properties": {},
     "additionalProperties": false
   },
   "IssueClosingReason": {
     "type": "string",
     "enum": [
       "nacked",
       "resolved",
       "system"
     ],
     "properties": {},
     "additionalProperties": false
   },
   "IssueAction": {
     "type": "string",
     "enum": [
       "ack",
       "snooze",
       "nack",
       "end_snooze",
       "expire_snooze",
       "star",
       "un_snooze",
       "unstar",
       "archive",
       "unarchive",
       "resolve",
       "reopen",
       "open",
       "feedback",
       "feedback_file",
       "open_ticket",
       "merge_issues",
       "active",
       "create",
       "close",
       "incident_updated",
       "confidence_changed",
       "diagnoses_changed",
       "rule_properties_changed",
       "flapping",
       "notification",
       "incident_events_limit_reached",
       "follow",
       "unfollow",
       "priority_updated",
       "comment",
       "incident_appended",
       "diagnosis_feedback",
       "flapping_delay_close"
     ],
     "properties": {},
     "additionalProperties": false
   },
   "StringKeyValue": {
     "type": "object",
     "properties": {},
     "additionalProperties": {
       "type": "string"
     }
   },
   "JavaOptionalUuid": {
     "type": "string",
     "properties": {},
     "additionalProperties": false,
     "format": "string-uuid"
   }
 },
 "type": "object",
 "properties": {
   "id": {
     "type": "string",
     "format": "string-uuid"
   },
   "customer_id": {
     "type": "integer"
   },
   "labels_set": {
     "$ref": "#/definitions/LabelsSet"
   },
   "annotations": {
     "$ref": "#/definitions/Annotations"
   },
   "user_annotations": {
     "$ref": "#/definitions/Annotations"
   },
   "user_annotations_version": {
     "type": "integer"
   },
   "accumulations": {
     "$ref": "#/definitions/Accumulations"
   },
   "state": {
     "$ref": "#/definitions/IssueState"
   },
   "previous_state": {
     "$ref": "#/definitions/IssueState"
   },
   "priority": {
     "$ref": "#/definitions/Priority"
   },
   "closing_reason": {
     "oneOf": [
       {
         "$ref": "#/definitions/IssueClosingReason"
       },
       {
         "type": "null"
       }
     ]
   },
   "modified_on": {
     "type": "integer"
   },
   "created_on": {
     "type": "integer"
   },
   "closed_on": {
     "oneOf": [
       {
         "type": "integer"
       },
       {
         "type": "null"
       }
     ]
   },
   "acknowledged_on": {
     "oneOf": [
       {
         "type": "integer"
       },
       {
         "type": "null"
       }
     ]
   },
   "active_since": {
     "oneOf": [
       {
         "type": "integer"
       },
       {
         "type": "null"
       }
     ]
   },
   "assigned_to": {
     "oneOf": [
       {
         "type": "array",
         "items": {
           "type": "string",
           "minLength": 1
         },
         "uniqueItems": true
       },
       {
         "type": "null"
       }
     ]
   },
   "created_by": {
     "oneOf": [
       {
         "type": "array",
         "items": {
           "type": "string",
           "minLength": 1
         },
         "uniqueItems": true
       },
       {
         "type": "null"
       }
     ]
   },
   "progress_report": {
     "type": "array",
     "default": [],
     "items": {
       "type": "object",
       "properties": {
         "origin_id": {
           "oneOf": [
             {
               "type": "string",
               "minLength": 1
             },
             {
               "type": "null"
             }
           ]
         },
         "timestamp": {
           "type": "integer"
         },
         "action_timestamp": {
           "oneOf": [
             {
               "type": "integer"
             },
             {
               "type": "null"
             }
           ]
         },
         "origin_type": {
           "type": "string",
           "enum": [
             "user",
             "system"
           ]
         },
         "event_type": {
           "$ref": "#/definitions/IssueAction"
         },
         "issue_id": {
           "oneOf": [
             {
               "type": "string",
               "minLength": 1,
               "format": "string-uuid"
             },
             {
               "type": "null"
             }
           ]
         },
         "attributes": {
           "oneOf": [
             {
               "$ref": "#/definitions/StringKeyValue"
             },
             {
               "type": "null"
             }
           ]
         },
         "team_id": {
           "$ref": "#/definitions/JavaOptionalUuid"
         }
       },
       "required": [
         "timestamp",
         "origin_type",
         "event_type"
       ],
       "additionalProperties": false
     },
     "uniqueItems": true
   },
   "rule_id": {
     "oneOf": [
       {
         "type": "string",
         "minLength": 1,
         "maxLength": 255
       },
       {
         "type": "null"
       }
     ]
   },
   "incident_ids": {
     "oneOf": [
       {
         "type": "array",
         "items": {
           "type": "string",
           "format": "string-uuid"
         },
         "uniqueItems": true
       },
       {
         "type": "null"
       }
     ]
   },
   "confidence": {
     "type": "number",
     "default": 1.0,
     "minimum": 0.0,
     "maximum": 1.0
   }
 },
 "required": [
   "id",
   "customer_id",
   "labels_set",
   "annotations",
   "priority",
   "modified_on",
   "created_on",
   "confidence"
 ],
 "additionalProperties": true
}
		

Jinja2 Default Payload

Applied Intelligence uses a templating framework called Jinja2 in the Webhook interface.

Here is a default Jinja2 payload to use:

{
"id": {{ id }},
"url": {{ url }},
"ui_name": {{ ui_name }},
"description": {{ description }},
"priority": {{ priority }},
"state": {{ state }},
"is_correlated": {{ is_correlated }},
"created_on": {{ created_on }},
"modified_on": {{ modified_on }},
"active_since": {{ active_since }},
"closed_on": {% if closed_on is defined %} {{ closed_on }} {% else %} None {% endif %},
"sources": {{ sources }},
"incidents": {{ incidents }},
"pathways": {{ pathways }},
}

Jinja2 Useful Syntax

Below are a few useful Jinja2 commands to help you format your output.

Casting a value to integer

Example:

“severity”: {{ priority | int }}

If clause to check if an attribute’s value is set

Example:

"golden_signals": {% if accumulations['alert/signal'] is defined %} {{ 
accumulations['alert/signal'] }} {% else %} None {% endif %}

For more help