Muting rules: Suppress notifications

New Relic Alerts sends out timely notifications when your system is having problems. Sometimes there are notifications you know you don't need to see. You can use muting rules to stop being bombarded by messages you don't want.

Once you've spotted the common elements in your unwanted notifications, you'll be able to define muting rules that specifically target those elements, while letting other notifications through.

Even when a notification is muted, New Relic still gathers data on those violations. The muting rules don't interfere with the Alerts process and are applied at the point right before a notification is sent.

(You can read more about why we added muting rules to Alerts in this blog post.)

Manage muting rules

You can create, enable, disable, and manage muting rules in New Relic One at one.newrelic.com > Applied intelligence > Muting rules.

You can also do all of these things with our powerful GraphQL API, NerdGraph.

Create a muting rule

Before creating muting rules, you'll need to create Alerts policies and conditions that generate violation notifications.

muting_rules_UI.png
one.newrelic.com > Applied intelligence > Muting rules: You can create complex muting rules to target a small or large set of unwanted notifications.

A muting rule condition is the set of individual expressions made up of attributes, operators, and values that define which violations to target for muting.

Go here for a list of Violation event attributes that you can use in your muting rule.

Go here for a list of Sub-condition operators you can use to fine-tune your rules. A value is any text string that can be compared against one of your violation attributes, such as an Alerts policy ID or a condition name.

Manage muting rules with NerdGraph

In NerdGraph, you can use the following queries and mutations with your muting rules. In the NerdGraph GraphiQL tool, you can see the schema in more detail.

  • actor.account.alerts.mutingRule: Fetch a muting rule by id.
  • actor.account.alerts.mutingRules: Fetch a list of muting rules for an account.
  • alertsMutingRuleCreate: Create a muting rule for an account.
  • alertsMutingRuleUpdate: Update a muting rule by id and account id.
  • alertsMutingRuleDelete: Delete a muting rule by id and account id.

We've given you some sample queries and mutations examples in this document.

A muting rule has the following fields and components:

Muting rule Fields and components
id The unique identifier for the muting rule.
name (Required)

A text field for the user-friendly name of the muting rule. This is used when listing or referencing a rule. We don't require that the name be unique, but it is recommended.

description

This is an optional text field describing the muting rule. It can be a useful way to provide more context for your muting rule. This data is only used for management display purposes.

accountId

The muting rule's account ID. A muting rule will only affect violations that occur in a single account. To mute violations across multiple accounts, you'll need to create a muting rule for each account separately.

createdAt

The timestamp when the muting rule was created (UTC).

createdBy

The user ID of the person who created the muting rule.

updatedAt

The timestamp when the muting rule was last modified (UTC).

updatedBy

The user ID of the person who last modified the muting rule.

enabled

Enable or disable the muting rule (Boolean). Muting rules must be manually enabled and disabled.

condition

The set of individual expressions that define which violations to target. A muting rule condition has:

  • operator: The boolean operator AND or OR that defines how to combine the set of conditions.
  • conditions: The set of individual expressions (sub-conditions) that target attributes within a violation. These are evaluated together based on the operator. You can have a maximum of 20 sub-conditions for a single muting rule.

A sub-condition has:

  • attribute: A single attribute within a violation. Go here for a list of Violation event attributes.
  • operator: The comparison function used to compare the selected violation attribute against values in the condition. Go here for a list of sub-condition operators.
  • values: An array of string values to compare against selected violation attributes. When the muting rules evaluate a condition, if necessary, values will be coerced from strings. You can use a maximum of 500 values when using an operator that supports comparison against multiple values, such as IN.

Permissions and availability

The muting rules functionality is available to all New Relic accounts that have access to the more advanced Alerts product features, such as baseline conditions. This covers most paid New Relic accounts. Individual users and roles within the account can be also configured for limited access.

To set permissions for individual users and roles to view, modify, or delete muting rules:

  1. Go to rpm.newrelic.com > (account dropdown) > Account settings > Users and roles > Roles.
  2. Use the Lifecycle overrides capability in the Alerts permissions setting.
Alert incident lifecycle: Muting rule permissions
rpm.newrelic.com > (account dropdown) > Account settings > Users and roles > Roles: In the Alerts > Lifecycle overrides section, set permissions to add, modify, or delete muting rules.

How muting rules work

Muting rules are applied at the end of the default alert lifecycle in order to suppress, or mute, notifications. They don't disable existing alert policies or conditions. For example, you can mute notifications during known system disruptions, such as maintenance windows and deployments. System disruption violations will still be identified, even though the notifications for those violations are muted.

A muting rule uses a set of conditions that match against attributes in an alert violation event. The muting rules tell us how to:

  1. Identify individual violations after they're created, but before an incident is opened.
  2. Override their default condition to indicate that they should be "muted."

Currently, muting a violation means that the normal Alerts incident lifecycle will be maintained, except that an incident that only contains muted violations will not send out any notifications.

Muting rules override specific violations. They don't disable existing alert policies or conditions. This allows you to mute violations from specific entities that may be covered by a policy or condition that covers a large number of entities. This also keeps you from having to over-mute your monitoring when you are performing maintenance on a subset of your system.

Muting behavior

The following table describes how the Alerts incident lifecycle is affected by muted violations:

IF AND THEN
Event: Incident opens
An incident opens because of a violation that is not muted An “Open incident” notification will be sent (default).
An incident opens due to a violation that is muted An “Open incident” notification will not be sent (muted).
Event: Violation added to an open incident
A new muted violation is added to an already open incident No notification action is triggered (default).
A new unmuted violation is added to an open incident An “Open incident” notification has not been sent An “Open incident” notification will be sent.
A new unmuted violation is added to an open incident An “Open incident” notification has already been sent No notification action is triggered (default).
Event: Incident closed
An Incident is closed An “Open incident” notification has not been sent A “Close incident” notification will not be sent.
An Incident is closed An “Open incident” notification has been sent A “Close incident” notification will be sent.
Event: Incident acknowledged
An Incident is acknowledged An “Open incident” notification has not been sent An “Incident acknowledged” notification will not be sent.
An Incident is acknowledged An “Open incident” notification has been sent An “Incident acknowledged” notification will be sent.

View muted violations and incidents

When viewing an open or closed incident in the New Relic Alerts UI, violations and incidents are marked as Muted:

Alerts > Incidents > Open Incidents > Incident ####

Alert incident lifecycle: Muting rule violations
Here is an example of an incident with a critical violation that has been muted.

Violations and incidents are marked as Muted in the State column when viewing lists of incidents, violations, or events:

Alerts > Incidents > Open incidents

Alert incident lifecycle: Muting rule incidents
Here is an example of an open incident that triggered the muting rule.

Alerts > Events > Violations

Alert incident lifecycle: Muting rule critical violations
Here is an example of a list of alert violations. The State column shows which violations have been muted.

Alerts > Events > All events

Alert incident lifecycle: Muting rule events
Here is an example of a list of alert events. The State column shows which events have been muted.

Use attributes for muting

For an explanation of what attributes (metadata) is available for choosing what violations are muted, see Violation event attributes.

Mute faceted results using tag.

To mute results of faceted queries, use the tag.FACETED_ATTRIBUTE attribute, where FACETED_ATTRIBUTE represents the attributed you've run a NRQL FACET query on. For example: if your NRQL alert condition includes FACET host in its query, you can target that FACET attribute using tag.host.

NRQL condition queries can accept multiple facet attributes. If you want to be able to filter from attributes in your events or metric time series that have been aggregated, you must add those attributes to your NRQL query FACET clause; for example: FACET host, region, cluster.

For an example of using tag., see Create muting rule.

Sub-condition operators

The following are the logical operators available to use to compare attributes when you're constructing muting rules. For examples of these being used, see Examples.

  • EQUALS: Where the supplied value equals the violation attribute value.
  • NOT_EQUALS: Where the supplied value does not equal the violation attribute value.
  • IN: Where the violation attribute value is present in a list of supplied values (up to 500).
  • NOT_IN: Where the violation attribute value is not present in a list of supplied values (up to 500).
  • CONTAINS: Where the supplied value string is present in the violation attribute value.
  • NOT_CONTAINS: Where the supplied value string is not present in the violation attribute value.
  • ANY: Caution: A condition with this operator will mute all violations on the account.
  • IS_BLANK: Where the violation attribute value is blank. Null, empty string, etc.
  • IS_NOT_BLANK: Where the violation attribute value is not blank. Null, empty string, etc.
  • STARTS_WITH: Where the violation attribute value begins with the supplied value string.
  • NOT_STARTS_WITH: Where the violation attribute value does not begin with the supplied value string.
  • ENDS_WITH: Where the violation attribute value ends with the supplied value string.
  • NOT_ENDS_WITH: Where the violation attribute value does not end with the supplied value string.

Muting examples

For more information about making requests to NerdGraph, see the NerdGraph documentation, including GraphQL tutorials.

Fetch muting rules for account

To fetch the list of muting rules (with name and condition) for an account:

{
  actor {
    account(id: YOUR_ACCOUNT_ID) {
      alerts {
        mutingRules {
          name
          condition {
            operator
            conditions {
              attribute
              operator
              values
            }
          }
        }
      }
    }
  }
}
Create muting rule

To create a muting rule and return the muting rule's id for muting violations from NRQL conditions where host facet is host-1:

mutation {
  alertsMutingRuleCreate(accountId: YOUR_ACCOUNT_ID, rule: {
    name: "host-1 mute rule",
    description: "Mute host-1 violations",
    enabled: true,
    condition: {
      operator: AND,
      conditions: [{
        attribute: "tag.host",
        operator: EQUALS,
        values: ["host-1"]
      }]
    }
  }) {
    id
  }
}
Create muting rule for specific hosts or products

To create a muting rule and return the muting rule's id for muting violations from Infrastructure conditions where the target host is host-1 or the violation was from the Synthetics or Mobile products:

mutation {
  alertsMutingRuleCreate(accountId: YOUR_ACCOUNT_ID, rule: {
    name: "host-1 and Synthetics/Mobile mute rule",
    description: "Mute host-1 and Synthetics/Mobile violations",
    enabled: true,
    condition: {
      operator: OR,
      conditions: [
        {
          attribute: "targetName",
          operator: EQUALS,
          values: ["host-1"]
        },
        {
          attribute: "product",
          operator: IN,
          values: ["SYNTHETICS", "MOBILE"]
        }
      ]
    }
  }) {
    id
  }
}
Rename and disable muting rule

To change the name and disable a muting rule, and return its new name:

mutation {
  alertsMutingRuleUpdate(accountId: YOUR_ACCOUNT_ID, id: MUTING_RULE_ID, rule: {
    name: "updated name",
    enabled: false
  }) {
    name
  }
}
Delete muting rule

To delete a muting rule:

mutation {
 alertsMutingRuleDelete(accountId: YOUR_ACCOUNT_ID, id: MUTING_RULE_ID) {
    id
  }
}

For more help

Recommendations for learning more: