Destinations are the data outputs to Applied Intelligence, where you can view your automatically correlated incidents. You can configure Incident Intelligence destinations to send data to PagerDuty or webhooks. This document gives examples of destination templates, webhook formats, and JSON schema.
Destination template examples
The following template example destinations are formatted in Jinja2.
- Suggested VictorOps template example
-
{ "monitoring_tool": {{ sources }}, {% if state == 'closed' %} "message_type": "OK", {% else %} "message_type": {{ priority }}, {% endif %} "custom_fields": { "issue_url": {{ url }}, "description": {{ description }}, "state": {{ state }}, "is_correlated": {{ is_correlated }}, "created_on": {{ created_on }}, "modified_on": {{ modified_on }}, "activated_on": {{ active_since }}, "closed_on": {% if closed_on is defined %} {{ closed_on }} {% else %} None {% endif %}, "related_incidents": [ {% for incident in incidents %} { "id": {{ incident.id }}, "events_count": {{ incident.events_count }}, "labels": {{ incident.labels }}, "title": {{ incident.title }}, "description": {{ incident.description }}, "state": {{ incident.state }}, "sources": {{ incident.sources }}, "modified_on": {{ incident.modified_on }}, "opened_on": {{ incident.opened_on }}, "closed_on": {{ incident.closed_on }} } {% if not loop.last %},{% endif %} {% endfor %} ] }, "state_message": {{ description }}, "entity_id": {{ id }}, "entity_display_name": {{ ui_name }}, "vo_annotate.u.NRAI_Link": {{ url }} }
- Suggested OpsGenie template example
-
{ "alias": {{ id }}, "message": {{ ui_name }}, "source": '{{ sources }}', "priority": {{ priority }}, "details": { "self_url": {{ url }}, "state": {{ state }}, "is_correlated": {{ is_correlated }}, "created_on": {{ created_on }}, "modified_on": {{ modified_on }}, "activated_on": {{ active_since }}, "closed_on": {% if closed_on is defined %} {{ closed_on }} {% else %} None {% endif %}, "is_correlated": {{ is_correlated }} }, "description": 'Incidents [ {% for incident in incidents %} { "id": {{ incident.id }}, "events_count": {{ incident.events_count }}, "labels": {{ incident.labels }}, "title": {{ incident.title }}, "description": {{ incident.description }}, "state": {{ incident.state }}, "sources": {{ incident.sources }}, "modified_on": {{ incident.modified_on }}, "opened_on": {{ incident.opened_on }}, "closed_on": {{ incident.closed_on }} } {% if not loop.last %},{% endif %}{% endfor %} ]' }
- Suggested ServiceNow template example
-
{ "short_description": {{ ui_name }}, "description": 'Issue Id: {{ id }}, \n Description: {{ description }}, \n Sources: {{ sources }}, \n Priority: {{ priority }}, \n Details: { \n self_url: {{ url }}, \n state: {{ state }}, \n is_correlated: {{ is_correlated }}, \n created_on: {{ created_on }}, \n modified_on: {{ modified_on }}, \n activated_on: {{ active_since }}, \n closed_on: {% if closed_on is defined %} {{ closed_on }} {% else %} None {% endif %}, \n is_correlated: {{is_correlated}} }, \n\n incidents: {{ incidents }}, \n\n pathways: {{ pathways }}' }
- Suggested Slack template example
-
- Go to Slack incoming WebHooks
- Choose the right Slack workspace and click "Add to Slack"
- Select the destination channel to receive the notifications to
- Click “Add Incoming WebHooks Integration”
- Copy the WebHook URL
- In the next screen, click “Save settings” at the bottom.
- In New Relic Applied Intelligence, under Incident Intelligence click Destinations
- Add a WebHook
- In the end point paste the WebHook url from slack.
- In the custom payload, paste the following JSON:
{"blocks": [
{"type": "section",
"text": {"type": "mrkdwn", "text": "*MONITORING TOOL*:"},
"fields":
[{% for source in sources %}
{"type": "plain_text", "text": {{ source }} }
{% if not loop.last %},{% endif %}
{% endfor %}
]
}
,
{"type": "divider"},
{"type": "section",
"text": {"type": "mrkdwn", "text": "*CUSTOM FIELDS*:"},
"fields": [{"type": "mrkdwn", "text": "*entity_id*"},
{"type": "mrkdwn", "text": "*entity_display_name*"},
{"type": "plain_text", "text": {{ id }} },
{"type": "plain_text", "text": {{ ui_name }} } ]},
{"type": "section",
"fields": [{"type": "mrkdwn", "text": "*issue_url*"},
{"type": "mrkdwn", "text": "*description*"},
{"type": "plain_text", "text": {{ url }} },
{"type": "plain_text", "text": {{ description }} } ]},
{"type": "section",
"fields": [{"type": "mrkdwn", "text": "*state*"},
{"type": "mrkdwn", "text": "*is_correlated*"},
{"type": "plain_text", "text": {{ state }} },
{"type": "plain_text", "text": "{{ is_correlated }}" }]},
{"type": "section",
"fields": [{"type": "mrkdwn", "text": "*created_on*"},
{"type": "mrkdwn", "text": "*modified_on*"},
{"type": "plain_text", "text": "{{ created_on }}" },
{"type": "plain_text", "text": "{{ modified_on }}" } ]},
{"type": "section",
"fields": [{"type": "mrkdwn", "text": "*activated_on*"},
{"type": "mrkdwn", "text": "*closed_on*"},
{"type": "plain_text", "text": "{{ active_since }}" },
{"type": "plain_text", "text": "{% if closed_on is defined %} {{ closed_on }} {% else %} None {% endif %}" } ]},
{"type": "divider"},
{"type": "section",
"text": {"type": "mrkdwn", "text": "*RELATED_INCIDENTS*:"} },
{% for incident in incidents %}
{"type": "section",
"fields": [
{"type": "mrkdwn", "text": "*id*"},
{"type": "mrkdwn", "text": "*events count*"},
{"type": "plain_text", "text": {{ incident.id }} },
{"type": "plain_text", "text": "{{ incident.events_count}}" } ]},
{"type": "section",
"fields": [
{"type": "mrkdwn", "text": "*title*"},
{"type": "mrkdwn", "text": "*description*"},
{"type": "plain_text", "text": {{ incident.title}} },
{"type": "plain_text", "text": {{ incident.description}} }] },
{"type": "section",
"fields": [
{"type": "mrkdwn", "text": "*state*"},
{"type": "mrkdwn", "text": "*modified on*"},
{"type": "plain_text", "text": {{ incident.state}} },
{"type": "plain_text", "text": {{ incident.modified_on}} },
{"type": "mrkdwn", "text": "*opened on*"},
{"type": "mrkdwn", "text": "*closed on*"},
{"type": "plain_text", "text": {{ incident.opened_on}} },
{"type": "plain_text", "text": {% if incident.closed_on is defined and incident.closed_on %} {{ incident.closed_on }} {% else %} "None" {% endif %} } ]},
{"type": "section",
"text": {"type": "mrkdwn", "text": "*LABELS*:"} ,
"fields": [{% for name, value in incident['labels'].items() %}
{% set bold_name = '*' + name + '*' %}
{"type": "mrkdwn", "text": {{ bold_name }} },
{"type": "plain_text", "text": {{ value }} }
{% if not loop.last %},{% endif %}
{% endfor %} ]},
{"type": "section",
"text": {"type": "mrkdwn", "text": "*SOURCES*:"} ,
"fields": [{% for source in incident['sources'] %}
{"type": "plain_text", "text": {{ source }} }
{% if not loop.last %},{% endif %}
{% endfor %} ]},
{"type": "divider"}
{% if not loop.last %},{% endif %}
{% endfor %}
]
}
Webhook and JSON format examples
Applied Intelligence will send the event body in JSON format via HTTPS POST. The system expects the endpoint to return a successful HTTP code (2xx)
. If you use webhooks to configure Incident Intelligence destinations, use these examples of the webhook body format and JSON schema.
- Webhook format and JSON example
- Webhook format:
Name Description url
Href
A link to the UI where the issue can be seen.
id
String
Unique identifier for the issue.
title
String
The issue title. description
String
The description of the issue. priority
Enum
The issue priority. Can be Critical, High, Medium, or Low. state
Enum
The issue status. Can be Active, Closed, or Acknowledged. is_correlated
Boolean
True if the issue is based on correlated data. created_on
String
The date and time the issue was created (in ISO format). modified_on
String
The date and time the issue was modified (in ISO format). active_since
String
The date and time the issue was activated (in ISO format). closed_on
String
The date and time the issue was closed (in ISO format). sources
List(String)
List of the different sources that were used to send the events into Applied Intelligence (for example: PagerDuty).
pathways
List(Pathways)
List of pathways that are associated with the issue. Each pathway contains an id and name:
[{“id”: “pathway id”, “name”: “pathway name”}]
.pathways[].id
String
The pathway ID. pathways[].name
String
The pathway name. incidents
List(Incident)
List of incidents that are attached to the issue. The list contains only the latest 100 incidents.
incidents[].id
String
The incident ID. incidents[].events_count
Integer
The number of events used to create the incident.
incidents[].title
String
The incident title. incidents[].description
String
The incident description. incidents[].labels
Dictionary (String)
A string to string mapping of the incident labels. Labels represent the unique entities that are used to describe the incident.
incidents[].priority
Enum
The incident priority. Can be Critical, High, Medium, or Low. incidents[].sources
List(String)
The incident source. incidents[].state
Enum(open, closed)
The incident state. incidents[].opened_on
String
The date and time the incident was opened (in ISO format). incidents[].closed_on
String
The date and time the incident was closed (in ISO format). ui_name
String
Issue title. accumulations['alert/signal']
String
Issue analysis summary golden signal/s (if applicable). accumulations['alert/components']
String
Issue analysis summary golden components (if applicable). - JSON schema example
-
Applied Intelligence will send the event body in JSON format via HTTPS POST. The system expects the endpoint to return a successful HTTP code (
2xx
).JSON schema:
{ "$schema": "http://json-schema.org/draft-04/schema#", "definitions": { "Labels": { "type": "object", "properties": {}, "additionalProperties": { "type": "string", "maxLength": 10000 } }, "LabelsSet": { "type": "array", "properties": {}, "additionalProperties": { "type": "array", "items": { "$ref": "#/definitions/Labels" }, "uniqueItems": true }, }, "Annotations": { "type": "object", "title": "Additional contextual information about the Entity", "description": "Fields like title, description and priority can be described as annotations since\nthey represent some additional information about the Incident.\nFor example, the rule that creates the entity can have some logic that will create a title annotation field.\nThe title will change according to the latest event received by the entity and as more\nevent will keep coming the field may change, all this as a decision by the rule.\nA different rule may decide not to populate the title at all.\nSince a title can also be produced from the labels themselves.", "properties": { "title": { "type": "string", "maxLength": 10000 }, "description": { "type": "string", "maxLength": 10000 } }, "patternProperties": { ".*": { "type": "string", "maxLength": 10000 } }, "additionalProperties": { "type": "string", "maxLength": 10000 } }, "Accumulations": { "type": "object", "title": "Accumulated data about the Incident", "description": "For example an entity with the label: host_name: foo will include a accumulated list of the\nrelevant process names that are relevant to the entity. Accumulations will be derived from the events", "properties": {}, "additionalProperties": { "type": "array", "items": { "type": "string", "maxLength": 10000 } } }, "IssueState": { "type": "string", "enum": [ "created", "active", "acknowledged", "closed" ], "properties": {}, "additionalProperties": false }, "Priority": { "type": "string", "enum": [ "critical", "high", "medium", "low" ], "properties": {}, "additionalProperties": false }, "IssueClosingReason": { "type": "string", "enum": [ "nacked", "resolved", "system" ], "properties": {}, "additionalProperties": false }, "IssueAction": { "type": "string", "enum": [ "ack", "snooze", "nack", "end_snooze", "expire_snooze", "star", "un_snooze", "unstar", "archive", "unarchive", "resolve", "reopen", "open", "feedback", "feedback_file", "open_ticket", "merge_issues", "active", "create", "close", "incident_updated", "confidence_changed", "diagnoses_changed", "rule_properties_changed", "flapping", "notification", "incident_events_limit_reached", "follow", "unfollow", "priority_updated", "comment", "incident_appended", "diagnosis_feedback", "flapping_delay_close" ], "properties": {}, "additionalProperties": false }, "StringKeyValue": { "type": "object", "properties": {}, "additionalProperties": { "type": "string" } }, "JavaOptionalUuid": { "type": "string", "properties": {}, "additionalProperties": false, "format": "string-uuid" } }, "type": "object", "properties": { "id": { "type": "string", "format": "string-uuid" }, "customer_id": { "type": "integer" }, "labels_set": { "$ref": "#/definitions/LabelsSet" }, "annotations": { "$ref": "#/definitions/Annotations" }, "user_annotations": { "$ref": "#/definitions/Annotations" }, "user_annotations_version": { "type": "integer" }, "accumulations": { "$ref": "#/definitions/Accumulations" }, "state": { "$ref": "#/definitions/IssueState" }, "previous_state": { "$ref": "#/definitions/IssueState" }, "priority": { "$ref": "#/definitions/Priority" }, "closing_reason": { "oneOf": [ { "$ref": "#/definitions/IssueClosingReason" }, { "type": "null" } ] }, "modified_on": { "type": "integer" }, "created_on": { "type": "integer" }, "closed_on": { "oneOf": [ { "type": "integer" }, { "type": "null" } ] }, "acknowledged_on": { "oneOf": [ { "type": "integer" }, { "type": "null" } ] }, "active_since": { "oneOf": [ { "type": "integer" }, { "type": "null" } ] }, "assigned_to": { "oneOf": [ { "type": "array", "items": { "type": "string", "minLength": 1 }, "uniqueItems": true }, { "type": "null" } ] }, "created_by": { "oneOf": [ { "type": "array", "items": { "type": "string", "minLength": 1 }, "uniqueItems": true }, { "type": "null" } ] }, "progress_report": { "type": "array", "default": [], "items": { "type": "object", "properties": { "origin_id": { "oneOf": [ { "type": "string", "minLength": 1 }, { "type": "null" } ] }, "timestamp": { "type": "integer" }, "action_timestamp": { "oneOf": [ { "type": "integer" }, { "type": "null" } ] }, "origin_type": { "type": "string", "enum": [ "user", "system" ] }, "event_type": { "$ref": "#/definitions/IssueAction" }, "issue_id": { "oneOf": [ { "type": "string", "minLength": 1, "format": "string-uuid" }, { "type": "null" } ] }, "attributes": { "oneOf": [ { "$ref": "#/definitions/StringKeyValue" }, { "type": "null" } ] }, "team_id": { "$ref": "#/definitions/JavaOptionalUuid" } }, "required": [ "timestamp", "origin_type", "event_type" ], "additionalProperties": false }, "uniqueItems": true }, "rule_id": { "oneOf": [ { "type": "string", "minLength": 1, "maxLength": 255 }, { "type": "null" } ] }, "incident_ids": { "oneOf": [ { "type": "array", "items": { "type": "string", "format": "string-uuid" }, "uniqueItems": true }, { "type": "null" } ] }, "confidence": { "type": "number", "default": 1.0, "minimum": 0.0, "maximum": 1.0 } }, "required": [ "id", "customer_id", "labels_set", "annotations", "priority", "modified_on", "created_on", "confidence" ], "additionalProperties": true }
- Jinja2 Default Payload
-
Applied Intelligence uses a templating framework called Jinja2 in the Webhook interface.
Here is a default Jinja2 payload to use:
{ "id": {{ id }}, "url": {{ url }}, "ui_name": {{ ui_name }}, "description": {{ description }}, "priority": {{ priority }}, "state": {{ state }}, "is_correlated": {{ is_correlated }}, "created_on": {{ created_on }}, "modified_on": {{ modified_on }}, "active_since": {{ active_since }}, "closed_on": {% if closed_on is defined %} {{ closed_on }} {% else %} None {% endif %}, "sources": {{ sources }}, "incidents": {{ incidents }}, "pathways": {{ pathways }}, }
- Jinja2 Useful Syntax
-
Below are a few useful Jinja2 commands to help you format your output.
Casting a value to integer
Example:
“severity”: {{ priority | int }}
If clause to check if an attribute’s value is set
Example:
"golden_signals": {% if accumulations['alert/signal'] is defined %} {{ accumulations['alert/signal'] }} {% else %} None {% endif %}