Workflows

With workflows you control when and where you want to receive notifications about issues, tunnel the right information to the relevant person or team, and enrich your issue's notifications with additional New Relic data.

What is an issue?

Issues are groups of incidents that describe the underlying problem of your symptoms. When a new incident is created, incident intelligence opens an issue and evaluates other open issues for correlations. For more information see Use Incident Intelligence.

How are workflows triggered?

A workflow is triggered by issue events for issues that match the workflow filter and their matching notification triggers. The workflow is configured with recommended triggers by default, but you can control when to trigger a notification for each destination by clicking on the 3 dots and then choosing Notify when. The Activated notification trigger must also be enabled.

The following events trigger the workflow and send notifications to the relevant destination end-points:

Required permissions

Using workflows require specific permissions.

• To access destinations: View permissions for Applied intelligence > Destinations or Alerts.
• To access workflows: View permissions for Applied intelligence > Workflows.
• To create workflows: Modify permissions for Applied intelligence > Workflows and Applied intelligence > Channels.
• To modify workflows: Modify permissions for Applied intelligence > Workflows and all permissions for Applied intelligence > Channels and applied_intelligence.create.workflow.
• To delete workflows: Delete permissions for Applied intelligence > Workflows and Applied intelligence > Channels.

Add a workflow

The workflows feature is located under the Alerts menu.

1. Go to one.newrelic.com > All capabilities > Alerts > Enrich and Notify > Workflow > Add a workflow.

2. Name your workflow. This field is mandatory and needs to be unique.

3. With workflows, filter your issues and send them to the relevant destination. You can build a query to filter the right issues.

   Tip

   To set up separate notifications on WARNING incidents, filter by issues with a priority equal to HIGH.

4. Build a query by selecting an attribute, operator, and value/s to narrow down the right issues you want to be passed on.

   Tip

   The workflows filter auto populates with data from past issues as well as some static values from the condition metadata (such as condition name). If you can't find the value you're looking for, type it in and select Enter new. A warning that the filter doesn't match any past issue sometimes occurs.

5. Recommended: filter issues by team tag so all teams can be notified when their entities are included in an issue. Steps can be found in the demo below:

   For some tips on using tags, see this support forum post on workflow patterns.

6. Optional: Enrich your data.

   How to enrich data

   Currently, you can only send JSON or numeric outputs to Webhook and Jira destinations. Additionally, you can send JSON to AWS EventBridge by using the workflow variables pointing to the Enrich data result. Also, you can only send images to other destinations like Slack, Pagerduty, ServiceNow (App), and email. To get better results in Webhook and Jira, use a query that has a single-value output, such as count, min, or max.

   Details on using enrichments:

   • Enrichments can give additional context on alert notifications by adding NRQL query results to them

   • The workflows enrich tool is similar to the query builder.

   • The NRQL query's results are included in the notification message.

   • If you create queries for your workflows, they run in real time and the query results are the value of the enricher.

   • You can save any valid query into your workflow, even if they return an empty result, but you must run the query before saving the enrichment. You can also query with issue-specific variables; for example {{entitiesData.ids}}

   • For enrichment examples, see Workflow data enrichment examples.

   • Note: The query name needs to be unique because you'll use the name to decide how to include the result in the payload of the notification that will be sent by the workflow.

     Important

     At this stage, visual representation of enrichments are sent to Slack and email. For ServiceNow and JIRA destinations, we recommend to limit to NRDB queries with single value results. For example: count, min, or max.

     The maximum number of enrichments per workflow is 2.

7. Notify: Choose one or more destinations and add an optional message.

   Tip

   In any destination channel, start typing and a variable menu will open up. You'll see the names of the variables, which will be replaced with the variable's values at runtime. To use the enrichers' results, use their name.

   Notifier

   • To save and activate a workflow, you need to include a notifier.
   • Setting up a notifier requires having a notification destination configured.
   • You can either set it up through the Destination menu item, or through the Create new destination item in the destination selection drop down in each destination channel screen.
   • A notification destination includes the credentials required to send a notification to a third party platform and an option to customize the message sent by mapping fields from our issues to fields in the destination platform.
   • To learn how to set up destinations and configure message templates, check the documentation on notifications.
   • (Optional) You can mark which notification you want to get for each notifier. Otherwise, notifications will be sent to each notifier by default.

8. Click update message once completing the notifier requirements.

9. Optional: Test your workflow. We'll use existing data from your account to send a sample notification with your new configuration. The test only works if there are existing issues that match the filter. If there isn't any data, you'll see this message: We don't see any issues matching your filter. That doesn't mean it won't work.

10. Click Update workflow to complete the workflow.

Issue notifications log

The issue notification log provides users a detailed record to keep track of all notifications generated by workflows. This information troubleshoots issues, monitors workflow performance, and ensures that notifications are being sent to the correct destinations. As a result, users can quickly identify any issues that may arise and take corrective action as needed.

To access the issue notification log, go to one.newrelic.com > Alerts > Workflows, then click Issue notification log.

Navigating the issue notification log

The issue notification log displays the following columns:

1. Time: This column displays the time the workflow notification was sent.
2. Status: This column displays the status of the notification.
   • Sent: The notification was sent successfully from New Relic.
   • Failed: The notification was not sent successfully from New Relic.
   • Bounced: The receiving email server cannot or will not deliver the email. This usually happens because the email address is invalid or the email inbox is full. Bounced email addresses are dropped in future mailings.
   • Dropped: Email client did not attempt to send the email because the email address has already bounced, because the address has unsubscribed from our emails, or because the address has marked our emails as spam.
   • Deferred: The email didn't go through on the first attempt, due to temporary issues like connectivity. There will be another attempt, so a deferred notification can become successful or bounced later on.
3. Destination: This column displays the destination type receiving the workflow notification (for example, Slack or email).
4. Operation: The type of notification that was sent (e.g., Notify, Close, Comment, Ack, Test).
5. Issue Name: This column displays the name of the issue that triggered the workflow notification.
6. Trigger Event: This column displays the issue event that triggered the workflow notification. Clicking on the event will provide more information on how workflows are triggered.
7. Workflow: This column displays the workflow that triggered the notification.
8. Error Details: This column displays additional details if the notification failed.
9. Notification Result: This column displays a link to evidence in the third party (for example, a Slack thread or JIRA ticket) if the notification was sent successfully.
10. Payload: This is available for notifications sent to webhook destinations and displays the JSON payload of the notification that was sent.

NrAiNotification

You can query all information from the issue notifications using the NrAiNotification event.

Use this query to see all notifications sent from workflows faceted by destination types:

FROM NrAiNotification
SELECT count(*)
FACET destinationType

View additional details

Users can view additional details by clicking on any row in the issue notifications log. This will display the error details (if applicable) and a link to the notification in the third-party.