Alerts lets you create customized alerting solutions for monitoring your system. This document contains:
Introduction to important concepts
To use alerts well, it will help you to understand the terms we use:
A policy is a group of one or more alert conditions. You must create a policy before you can add conditions to it.
A policy has two settings that apply to all of its conditions: incident preference and notification channels (explained more below).
A condition includes: a) a monitored data source and b) thresholds that define the behavior that's considered a violation.
For example, a specific condition might be described in this way: "If the response time for any page load in my app goes above 8 seconds and lasts for more than 5 minutes, that's a violation."
A threshold is part of a condition; it defines the behavior that's considered a violation. When you create a condition, there's a required critical-level threshold. Optionally, you can set a secondary warning-level threshold.
A violation occurs when the value of a data source crosses a condition's threshold. This leads to the creation of a violation event, which is used to pass important information downstream.
A violation doesn't directly generate a notification; a violation may lead to an incident, which in turn can generate notifications.
Incidents are what generate notifications. At the policy level, the incident preference determines how violations are handled and combined to generate incidents.
For example, you may want to have every single violation generate an incident (many notifications) or you may want to have only a single incident open at a time across an entire alert policy (minimal notifications). Setting the incident preference gives you power over how notifications are created and helps prevent notification fatigue.
At the policy level, you choose what team members are notified when an incident occurs and how they're notified. We offer several notification channels, including webhooks, Slack rooms, email, etc. You can include charts about the incident to provide context, and share them with your team's notification.
For in-depth definitions of these and other terms, see the glossary.
Now that you understand some basic concepts and terms, let's look at a typical process for creating a policy and an associated condition:
- Create a policy. When you create a policy:
Create a condition that will be attached to that policy. Steps involved in creating a condition include:
- Optional: Add more conditions to that same policy.
To learn more about using alerts: