• /
  • Log in
  • Free account

Incident workflows

BETA

Incident workflows (or "workflows") is a flexible notification system that works within Applied Intelligence to enhance and enrich alert notifications. Incident workflows use a friendly user interface to build rich notification message templates and customize how, when, and where they’re sent.

Tip

To use incident workflows and Applied Intelligence, as well as the rest of our observability platform, join the New Relic family! Sign up to create your free account in only a few seconds. Then ingest up to 100GB of data for free each month. Forever.

Why it matters

Incident workflows sends alert events to many destinations in any format, enriched with additional context.

The destinations include various notification platforms, like webhooks and Jira. Before being sent to a destination, a notification is enriched with extra information, including from the New Relic database (NRDB). Not only that, workflows creates a two-way connection between your notification platforms and New Relic. All of your alert policies can be grouped into a single incident workflow.

Tip

Incident workflows are currently in BETA. If you haven't signed up yet, go to one.newrelic.com > Alerts & AI and click Incident Workflows Request access.

How it works

Workflows are a collection of triggers and actions for alerts incidents. A workflow is a set of instructions that defines enriched notification templates, chooses triggers, and connects to multiple notification platforms:

  • A notification template maps the workflows fields to your notification platform.
  • Triggers are alerts policies that start workflow processes.
  • Actions are operations that execute when workflows trigger. Actions include enriching a notification with extra information or configuring your notification platform to define the notification message format.

Each workflow has a single account and uses that account’s conditions for its triggers.

When notifications are sent

Workflows use the same incidents you defined in your alert policies.

In alert policies, incident preferences define how incidents and notifications are created when conditions in the policy are violated.

Create or edit a workflow

From one.newrelic.com, click Alerts & AI. In the left pane, click Incident workflows to open the workflows dashboard.

Use the workflows dashboard to create, edit, enable, or disable your workflows. Here’s a basic overview:

  1. Add and name the workflow.
  2. Select triggers.
  3. Enrich your notifications with NRQL.
  4. Create notification actions.
  5. Test your workflow.
  6. Activate your workflow.

Step 1. Add and name a workflow

When you add a workflow, give it a unique, descriptive name that will help you remember its purpose later.

Step 2. Select your triggers

An alerts condition defines a violation. Violations are workflow triggers. These violations are either warning or critical thresholds defined in the Alert policy.

Tip

Each workflow must have at least one trigger. If you select more than one, only one of them needs to be true to trigger the actions that follow.

Step 3. Optional: Add NRQL query data to enrich your notifications

After you select triggers, enrich your notification data with NRQL queries.

The workflows enrich tool is similar to the query builder.

The NRQL query's results are included in the notification message.

If you create queries for your workflows, they run in real time.

You can save any valid (error free) query into your workflow, even if they return an empty result. You can also query with violation-specific variables; for example {{entity.id}}.

Screen capture showing where to add a query to enrich a notification.

An example of a NRQL query you might use to enrich your workflow.

NRQL query enrichment examples

Here are some examples of how to set up enrichment queries.

Step 4. Create notification actions

You can use the monitoring platforms ServiceNow, Atlassian Jira Cloud, and AWS EventBridge with workflows. You can also configure a webhook to other destinations.

Every workflow requires at least one notification action. To create an action:

  1. Configure the notifier
  2. Build the message
  3. Test the notifier

Configure the notifier

To configure your notifier, choose a notification platform, and then configure its destination fields.

  1. Click Add an action, and then under Notify your channels, click Add.
  2. Choose your notification platform.
  3. Complete the destination fields based on your notification platform.

To acknowledge and close an alerts incident with Jira or ServiceNow, check Allow two-way integration. For example configurations, see Jira and ServiceNow.

Configure your notification message

The notification message is delivered to your notification platform. A notification message includes one or more key-value fields.

Tip

When using Jira and ServiceNow, you must map a workflows field name value to its corresponding field in your notification platform.

A field’s value can be any combination of the following:

  • Free text
  • Output enriched with NRQL queries results
  • Output enriched with Custom variables

In each notifier, the payload is populated with your query results and some custom variables. You can remove those and add other custom variables, using the # sign.
Type # to see a list of available outputs and custom variables.

You can also write the custom variables manually:

  • {{enricher_name.result}} is replaced with its result.
  • {{variable_name}} is replaced with the variable’s value.

Choose the variable name from a list of common fields or create a custom string.

Screenshot showing the fields for building a notification message.

ServiceNow notification message example

Optional: Test the notifier

Send a notification message to verify that you have a good connection to your platform.

A workflow test creates a ticket with placeholders for the final values. You get live results from the NRQL queries used to enrich the notifications, but N/A when referring to violations-related data. This is expected behavior.

Updated variables are rendered on your notification platform when a real violation triggers the workflow. These values are part of the violation event. If you use variables as part of the notification message, the variables value uses the variable name (for example, entity.name) as a placeholder.

If the notifier test fails, check the following:

  • Make sure to use the proper domain, username, and password/token for your platform.
  • Confirm your destination is live/awake.
  • Make sure to use different names for your notifiers.

Notifier configuration examples

Here are some examples of specific notifier configurations.

Step 5. Optional: Test your workflow

When you test the workflow, it runs NRQL queries to enrich your notifications and send test notification messages to your notification platforms.

If your workflow test fails, check the following:

  • There's a unique name for your workflow and notifiers.
  • The queries are valid (without error messages in the NRQL query).
  • The notifiers passed their test.

Step 6. Activate your workflow

Make sure your workflow has at least one trigger and one notification channel, then save and activate the workflow.

For more help

If you need more help, check out these support and learning resources:

Create issueEdit page
Copyright © 2021 New Relic Inc.