--- title: Configure correlation logic with decisions source: https://docs.newrelic.com/docs/alerts/organize-alerts/change-applied-intelligence-correlation-logic-decisions --- With alerts' correlation logic, related issues are grouped together to reduce distracting and redundant alerts. As events come into your system they are eligible for our correlation logic. Eligible issues are evaluated based on time, alert context, and relationship data. If multiple issues are related then our correlation logic will funnel the related alert events into a single, comprehensive [issue](https://docs.newrelic.com/docs/alerts-applied-intelligence/overview/#concepts-terms). We call this correlation logic **decisions**. We have built-in decisions but you can also create and customize your own on the decisions page. To find the decisions page go to **[one.newrelic.com > All capabilities](https://one.newrelic.com/all-capabilities) > Alerts > Decisions**. The more you configure your decisions to best suit your needs, the better New Relic can correlate your alert events, reduce noise, and provide increased context for on-call teams. /\* \*\*\[one.newrelic.com > All capabilities](https://one.newrelic.com/all-capabilities) > Alerts > Alert event intelligence > Decisions\*\*: Our UI shows how each decision correlates alert events. \*/ ## What is correlation and how does it work? [#what-is-correlaton] Your most recent and active alert events are available for our correlation logic. For example, let's say your system has received two alerts saying a synthetic monitor is failing in Australia and London. These two alerts will have created their own unique alert events. Those alert events will generate their own unique issues based on your teams existing [alert event creation policy](https://docs.newrelic.com/docs/alerts/organize-alerts/specify-when-alerts-create-events#preference-options). The correlation logic of New Relic will then test those alert events against each other to find similarities. In this case, it's the same monitor that is failing across multiple locations, so New Relic will merge both alert events into a single issue that contains each relevant event. When we correlate events, we check every pair of combinations against each other and combine as many as possible. For example: - Our algorithm correlates alert event A and B (call it "AB"). - Our algorithm correlates alert event B and C (call it "BC"). - Because B is present in both issues, the algorithm then correlates all three alert events together into one issue. ## Configure correlation policy [#configure-correlation] To enable correlation on [alert](https://docs.newrelic.com/docs/alerts-applied-intelligence/overview/#concepts-terms)-based issues, you'll need to connect to correlation for the respective [alert policy](https://docs.newrelic.com/docs/alerts-applied-intelligence/new-relic-alerts/alert-policies/create-edit-or-find-alert-policy/#alert-policy-name). ![A screenshot of how to enable correlation for an alert policy.](https://docs.newrelic.com/images/alerts_screenshot-full_decision-policy-.webp "Decision - enable correlation for alert policy") Check the box **Correlate and suppress noise** to enable correlation for the alert policy. ## Decision types [#decision-types] Decisions determine how alert event intelligence correlates issues together. The correlation logic of New Relic is available to your team in three different decision types: - **Global decision**: A broad set of default decisions are automatically enabled when you start using alerts. - **Suggested decision**: New Relic's correlation engine constantly evaluates your event data to suggest decisions that capture correlation patterns to reduce noise. You can preview simulation results of a suggested decision and choose to activate. - **Custom decision**: Your team can customize decisions based on your use case to enhance correlation effectiveness. The decision UI of New Relic gives you flexibility to configure all dimensions in a decision. ## Review your active decisions [#decisions] To review your teams existing decisions: 1. Go to **[one.newrelic.com](https://one.newrelic.com/all-capabilities)> Alerts > alert event intelligence > Decisions**. 2. Review the list of active decisions. To see the rule logic that creates correlations between your issues, click the decision. 3. To see examples of alert events the decision correlated, click the **Recent correlations** tab. 4. You have the option to enable or disable these global decisions. ## Configure sources [#configure-sources] Before configuring your decisions, it's important to determine the sources you would like to correlate. Sources are your data inputs. You can get data from any of the following sources: **Alerts** By enabling alert event intelligence for your alerts policies, you can get context and correlations from what you're monitoring. To get data from alerts: 1. From **[one.newrelic.com](https://one.newrelic.com/all-capabilities)**, click **Alerts**. 2. On the left under **Alerts**, click **Sources**, and then click **Alerts**. 3. Select the policies you want to connect to alerts, and click **Connect**. You can add additional alerts policies or remove policies you've already connected in **Sources > Alerts**. > #### 💡 TIP > > Adding alerts as a source will not affect your current configuration or notifications. **Aporia (MLOps)** By integrating alert event intelligence with your Aporia machine-learning models, you can monitor your machine learning model performance. To configure our Aporia integration, see our [docs](https://docs.newrelic.com/docs/integrations/mlops-integrations/aporia-mlops-integration/). **Superwise (MLOps)** By integrating alert event intelligence with your Superwise machine-learning models, you can monitor your machine learning model performance. To configure our Superwise integration, see our [docs](https://docs.newrelic.com/docs/alerts-applied-intelligence/mlops/integrations/superwise-mlops-integration/). **REST API** Alert event intelligence supports a dedicated REST API interface that lets you integrate with additional systems. The interface allows instrumentation of your code or other monitoring solutions to report any kind of metric or event. - A metric can be a raw data point such as CPU, memory, disk utilization, or business KPI. - An event can be a monitoring alert, deployment event, alert event, exceptions, or any other change in state that you want to describe. You can also send any type of data to alert event intelligence straight from your own systems or applications. The REST API supports secure token-based authentication and accepts JSON content as input. For more information on authentication and the full API reference, see [REST API for New Relic alerts](https://docs.newrelic.com/docs/rest-api-new-relic-ai). ### Global decisions [#global-decisions] Global decisions are automatically enabled when your team starts using alerts. They require no configuration and are immediately available for your team. Global decisions cover a variety of correlation scenarios. The table below provides descriptions for all of the global decisions that are automatically enabled. | Decision name | Description | | ------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Same New Relic Target Name (NRQL) | Correlation is activated when the entity name with an exceeded threshold and NRQL query are the same. Relevant events from the same [NRQL alert condition](https://docs.newrelic.com/docs/alerts-applied-intelligence/new-relic-alerts/alert-conditions/create-nrql-alert-conditions) will be identified. This decision helps relate issues that have the same transaction query latency deviation for example. | | Same New Relic Target Name (Non-NRQL) | Correlation is activated because the New Relic non-NRQL alert thresholds are the same. Does not apply to REST source. Non-NRQL entity refers to [entity](https://docs.newrelic.com/docs/new-relic-solutions/new-relic-one/core-concepts/what-entity-new-relic/), typically APPLICATION, HOST types, see [New Relic GitHub repo on entity synthesis](https://github.com/newrelic/entity-definitions#entity-definitions). With this decision, relevant issues from the same entity will be identified. For example, host high memory issue and host not-reporting issue could be highly possible due to the same cause. | | Same New Relic Target ID | Correlation is activated because the New Relic non-NRQL alert thresholds are the same. Does not apply to REST source. Use entity ID to uniquely identify an entity instance, learn more about [entity.guid](https://docs.newrelic.com/docs/new-relic-solutions/new-relic-one/core-concepts/what-entity-new-relic#reserved-attributes). | | Same New Relic Condition | Correlation is activated because the New Relic [condition IDs](https://docs.newrelic.com/docs/new-relic-solutions/get-started/glossary/#condition_id) are the same. For example, cpu usage increase with related services will trigger alert events from the same cpu usage condition, and thus be identified. This logic is valuable beyond [alert policy issue creation preference option](https://docs.newrelic.com/docs/alerts/organize-alerts/specify-when-alerts-create-events#preference-options) for one issue per condition, due to condition-level granularity and flexibity in defining correlation time window. | | Same New Relic Condition and Deep Link Url | Correlation is activated because the New Relic [condition IDs](https://docs.newrelic.com/docs/new-relic-solutions/get-started/glossary/#condition_id) and deep link url are the same. Deep link url provides time series and time range information in addition to [alert condition](https://docs.newrelic.com/docs/alerts/create-alert/create-alert-condition/alert-conditions). Correlating these issues make it easier for you to look at related alert events in the alert event response flow with time-scoped metrics, and perform deep analysis. Deep link url can be automatically generated if alert events are triggered by New Relic alert conditions, while for REST source [deepLinkUrl](https://docs.newrelic.com/docs/data-apis/ingest-apis/event-api/alert-event-rest-api/#api-specs) should be user defined. | | Same New Relic Condition and Title | Correlation is activated because the New Relic [condition names and titles](https://docs.newrelic.com/docs/alerts/create-alert/condition-details/alert-event-attributes/#attributes) are the same. This is a refined option by comparing titles in addition to conditions to reveal tighter relevance with the same alert message. | | Same k8s Deployment | Correlation logic is activated because the kubernetes deployments are the same. Many alert events are from single deployment changes. This decision is to reduce issues from the same troublesome Kubernetes entity deployment. | | Same Application Name, Policy and Id | Correlation logic is activated because custom application name, policy and custom ID are the same. We correlate issues with these elements to reduce application issues, particularly cater to custom tag users. Learn more about [tags](https://docs.newrelic.com/docs/new-relic-solutions/new-relic-one/core-concepts/use-tags-help-organize-find-your-data/). Custom tag ID could be defined by condition family ID or other ID values used as a key to identify connections between data. | | Similar Alert Message | Correlation is activated because alert events have similar titles, and are from the same entity. This is to reduce issues from the same entity that are caused by similar [alert conditions](https://docs.newrelic.com/docs/alerts-applied-intelligence/new-relic-alerts/alert-conditions/create-alert-conditions/). | | Same Secure Credential, Public Location and Type | Correlation is activated because the secure credential, public location and custom type are the same respectively. This is to correlate issues from the same geo location/region with the same security credentials that are normally triggered by a single root cause (for example, synthetics monitor failure), and could highly probable be addressed with the same solution. [Add tags](https://docs.newrelic.com/docs/new-relic-solutions/new-relic-one/core-concepts/use-tags-help-organize-find-your-data/#add-tags) to benefit from this decision. | | Similar Issue Structure | Correlation is activated because both alert events have similar attributes structure and data contents. This is a simpler version of clustering, it adopts advanced similarity algorithms in matrix computation to reduce highly related issues. | | Topologically Dependent | Correlation is activated because alert events are generated from instances that have dependent relationships. Learn more about [topology correlation out-of-the-box](#topology-requirements). | ### Use suggested decisions [#suggested-decisions] The data from your selected sources is continuously inspected for patterns to help reduce noise. Once patterns have been observed in your data, our correlation logic will suggest unique decisions that would allow these types of events to correlate in the future. To get started, click **Suggested decisions** tab on the topic of **Decisions** UI page. You can see the logic behind the suggested decision, and the estimated correlation rate by clicking each suggested decision. ![A screenshot of a suggested decision block](https://docs.newrelic.com/images/alerts_screenshot-full_suggested-decisions.webp "Suggested decision block") **[one.newrelic.com > All capabilities](https://one.newrelic.com/all-capabilities) > Alerts > Decisions**: Some example statistics from the decisions UI. To enable a suggested decision, click **Add to your decisions**. Once activated, the decision will appear in your teams main decision table. All suggested decisions will show the creator as New Relic AI (this refers to New Relic alerts). If the suggested decision isn't relevant to your needs, click **Dismiss**. ## Create custom decisions [#customize] You can reduce noise and improve correlation by building your own custom decisions. To start building a decision, go to **[one.newrelic.com > All capabilities](https://one.newrelic.com/all-capabilities) > Alerts > Decisions**, then click **Create new decision**. There are two versions of the decision builder: - Basic decision builder (in preview) - Advanced decision builder For more on how to use these decision builders, keep reading. ### Decision elements [#decision-elements] A decision is composed of these elements: - Correlate by attributes: Correlate all alert events by similarities or differences in their attributes. - Filter by specific values: Narrow down the alert events to those with specific values. - Filter by related entities: Select the kinds of shared connections or dependencies you want us to look for. - Correlation time range: Sets the maximum allowed time difference between the creation times of two alert events for them to be considered for correlation. Once the connections between alert events is set up, our [algorithm](#what-is-correlaton) groups correlated alert events into a single issue. ### Basic decision builder [#basic-decision-builder] **This feature is currently in preview and available for only some customers.** If you don't have access, see the instructions for the [advanced decision builder](#advanced-decision-builder). Here's a short video (3:25 minutes) showing how to use the basic decision builder: [Video](https://fast.wistia.net/embed/iframe/xmbcv8rhuu) The basic decision builder covers the majority of use cases and focuses on "correlate by attribute," where you can specify filter conditions for correlation matches. You can also apply the same filter logic for specific values to both alert events being correlated. For example, you can correlate alert events if the entity name is `host 1` for both. To create your own custom decision using the basic decision builder complete the following steps. Keep in mind that steps 1, 2, and 3 are optional on their own, but at least one of the three must be defined in order to create a decision. #### Step 1: Correlate by attributes [#basic-correlate-attributes] Choose an attribute from the dropdown menu. The `equal` operator, the most popular option, is preselected, or you can choose another [operator](#operators). The second attribute usually matches the first, so it's autopopulated. You can keep the autopopulated option or choose another operator. Once you're done, a [simulation](#simulations) runs automatically. You can repeat these steps to add up to eight logic filters. /\* \*/ #### Step 2: Filter by specific values [#basic-filter-values] 1. To open the `Filter by specific values` section and see additional filters, click **See more options**. 2. Choose an attribute. 3. The `equal` operator is preselected, or you can select another [operator](#operators). 4. Select expected values for the chosen attributes, with multiple selections supported. When complete, the [simulation](#simulations) will run automatically. You can repeat these steps to add up to eight logic filters. /\* \*/ #### Step 3: Filter by related entities [#basic-filter-related-entities] Click **Filter by related entities** and choose the entity classes. When your data is collected by [New Relic agents](https://docs.newrelic.com/docs/new-relic-solutions/new-relic-one/install-configure/compatibility-requirements-new-relic-agents-products/), you get automatic topology correlation. [Learn more about our default topology correlation](#topology-requirements). You can also set up [topology settings using our NerdGraph API](https://docs.newrelic.com/docs/apis/nerdgraph/examples/topology-nerdgraph-tutorial). This allows any topology-related decision to be matched with your topology data. [Learn more about setting up topology correlation](#topology). /\* \*/ #### Step 4: Set correlation time range [#basic-set-time-range] This sets the maximum allowed time difference between the creation times of two alert events for them to be considered for correlation. alert events within this range will be assessed based on specified rules, while those outside the range won't be correlated. The time range is set to 20 minutes by default. You can adjust it between 1-120 minutes. /\* \*/ #### Step 5: Testing your decision using a simulation [#basic-test-with-simulation] After adding filter logic, the system automatically runs a [simulation](#simulations) using the past 7 days of alert event data to help you validate the decision before applying it. You can also manually trigger the simulation by clicking **Simulate**, which you may want to do if something is changed in the decision. /\* \*/ #### Step 6: Name and save your decision [#basic-name-save-decision] To access the name and description panel, click **Create decision**. The system generates a name based on your decision. Customize the name and description as desired. /\* \*/ ### Advanced decision builder [#advanced-decision-builder] The advanced decision builder allows for more complex decision creation by applying different logic filters to the two alert events being correlated. For example, you can correlate alert events if one has entity name `host 1` and the other has entity name `host 2`. There are also more advanced settings besides being only able to configure the time window. To use the advanced decision builder: 1. Go to **[one.newrelic.com > All capabilities](https://one.newrelic.com/all-capabilities) > Alerts > Decisions**. 2. Click **Create new decision**, and then click **Use advanced builder**. For details on the available options, keep reading. Important terms: - Logic filter: Logic condition defined with an [operator](#operators) on an [attribute](https://docs.newrelic.com/docs/alerts/create-alert/condition-details/alert-event-attributes/#attributes). - Segment: A group of alert events that satisfy a combination of logic filters. To create your own custom decision complete the following steps. Keep in mind that steps 1, 2, and 3 are optional on their own, but at least one of the three must be defined in order to create a decision. #### Step 1: Filter your data [#filter-data] Correlation occurs between any two alert events. If no filters are defined then all incoming alert events will be considered by the decision. The more you configure your decisions to suit your needs, the better we can correlate your alert events, reduce noise, and provide increased context for on-call teams. Your team can define your filters for the first segment of alert events, and the second segment of alert events. Filter [operators](#operators) range from substring matching to [regex matching](#regex) to help you target the alert events you want and exclude those you don't. /\* \*/ #### Step 2: Correlate by attributes [#correlate-context] Once you've filtered your data, define the logic used when comparing the alert events' context. You can correlate events based on the following methods: - Attribute value comparisons with standard operators - Attribute value similarity using [similarity algorithms](#algorithms) - Attribute value [regex with capture groups](#regex) - Entire alert event comparisons using similarity or clustering algorithms /\* \*/ #### Step 3: Correlate by related entities [#topology-correlation] For automatic topology correlation, make sure your telemetry data is collected by [New Relic agents](https://docs.newrelic.com/docs/new-relic-solutions/new-relic-one/install-configure/compatibility-requirements-new-relic-agents-products/). Learn more about [topology correlation out-of-the-box](#topology-requirements). You can also set up [topology settings using our NerdGraph API](https://docs.newrelic.com/docs/apis/nerdgraph/examples/topology-nerdgraph-tutorial). This allows any topology-related decision to be matched with your topology data. [Learn more about setting up topology correlation](#topology). /\* \*/ #### Step 4: Give it a name [#name-your-decision] After you configure your decision logic, give it a recognizable name, and description. > #### 💡 TIP > > Minimize security concerns by ensuring you don't add sensitive or personal information to these open text fields. This is used in notifications and other areas of the UI to indicate which decision caused a pair of alert events to be correlated together. If you don't want to update default advanced settings in the next step, click **Create decision** to finish the creation. /\* \*/ #### Step 5: Use advanced settings [#advanced-settings] Use the advanced settings area to further customize how your decision behaves when correlating events. Each setting has a default value so customization is optional. - **Time window**: Sets the maximum time between two alert events created time for them to be eligible for correlation. - **Issue priority**: Overrides the default priority setting (`inherit priority`) to add higher or lower priority if the alert events are correlated. - **Frequency**: Modifies the minimum number of alert events that need to meet the decision logic for the decision to trigger. - **Similarity**: If you're using `similar to` operators in your decision logic, you can choose from a list of algorithms and set its sensitivity. This will apply to all `similar to` operators in your decision. /\* \*/ ## Logic operators [#operators] Decision provides a set of operators to help you flexibly define how an alert event's attribute value evaluates in a logic filter. The basic ones are **equals**, **contains**, **starts with**, **ends with**, **exists**, and their negate operators accordingly. For example, **does not equal**. There is a similarity operator **is similar to**, the underlying [similarity algorithm](#algorithms) can be specified for this operator. By default, it uses Levenshtein Distance. The **contains (regex)** operator allows define [regex](#regex) condition. Powerful to match arbitrary data values. ### Similarity algorithms [#algorithms] Here are technical details on the similarity algorithms we use: **Levenshtein distance** This measure is useful for comparing short strings with static schema and fixed length, like host names. Levenshtein distance is also known as edit distance. | Details | Description | | ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | How it works | The Levenshtein distance between two strings is the minimum number of single-character edits to get from one string to the other. Allowed edit operations are deletion, insertion, and substitution. The default similarity threshold for alert decisions is an edit distance of 3. You can change this in the **Advanced mode** of the decision builder. | | When to use it | This measure is most useful for comparing relatively short strings with static schema and fixed length. Common applications include spell checkers, computational biology, and speech recognition. | | Examples | `number/bumble: 3 (number → bumber → bumblr → bumble)` `trying/lying: 2 (trying → rying → lying)` `strong/through: 4 (strong → htrong → throng → throug → through)` | | Potential drawbacks | The levenshtein distance algorithm is not normalized by default to take into account string lengths. | **Fuzzy score** This metric is useful for comparing same-length strings where the same prefix would be a good indicator of correlation. | Details | Description | | ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | How it works | The fuzzy score algorithm works by allocating "points" for character matches between strings: - One point for each matching character - Two bonus points for subsequent matches The higher the fuzzy score, the greater the similarity between two strings. | | When to use it | Fuzzy score is most useful for strings that have the same and relatively short prefixes (ideally fewer than five characters). A minimum guaranteed score would be `(length(expected prefix) * 3) - 2`. | | Examples | Example: `Decisions / dcsions` `d: 1` `c: 1` `i 1` `s: 2` `o: 1` `n: 1` `si: 2` `io: 2` `on: 2` `ns: 2` `= 15 points` | | Potential drawbacks | If the first character of the first string can't be found in the second string, no points are awarded. | **Fuzzy wuzzy ratio** This metric is useful for comparing strings of similar length. | Details | Description | | ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | How it works | The **fuzzy wuzzy** family of similarity measures was [developed by SeatGeek](https://chairnerd.seatgeek.com/fuzzywuzzy-fuzzy-string-matching-in-python/) to help find tickets for the same event that have different labels across multiple platforms. The fuzzy wuzzy ratio for two strings is expressed as a percentage, where a higher number indicates a more similar string. It's based on the [SequenceMatcher algorithm](https://docs.python.org/3/library/difflib.html) in Python's difflib. | | When to use it | Fuzzy wuzzy ratio is effective for very short strings (such as hostname) or very long strings (such as event description), especially in comparing strings of similar length. | | Potential drawbacks | This algorithm is too sensitive to be used effectively for 3-10 word strings. One of the other modifications to fuzzy wuzzy (see below) may be a better choice. | **Fuzzy wuzzy partial ratio** This metric is useful for comparing strings of different length. This modification to the fuzzy wuzzy algorithm helps address the effective length limitation. | Details | Description | | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | How it works | With fuzzy wuzzy partial ratio, the shorter string is compared to each substring of the same length within the longer string. The score of the "best matching" substring is used to determine the fuzzy wuzzy partial ratio. | | When to use it | Fuzzy wuzzy partial ratio is especially effective for the types of comparisons the basic fuzzy wuzzy algorithm fails at: 3-10 word strings where some significant substrings are likely to be overlapping. | | Examples | For example, between the following strings: `DevOps and SRE teams` `DevOps` `DevOps` (the shorter string, length = 6) would be compared to each substring with length 6 within `DevOps and SRE teams`. Since one of those substrings (`DevOps`) is a perfect match, the fuzzy wuzzy partial ratio for these two strings will be high. | | Potential drawbacks | Where fuzzy wuzzy may be too conservative, fuzzy wuzzy partial match may be more liberal than expected with correlations. You can adjust the threshold in the decision builder according to your needs. | **Fuzzy wuzzy token set ratio** This metric is useful for comparing strings where the information may not be in the same order, and of possible different lengths. It works best for sentences such as messages, descriptions, etc. | Details | Description | | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | How it works | The token set ratio algorithm follows a few steps to compare strings: - Tokenize each string (for example, “DevOps and SRE teams” to "DevOps" "and" "SRE" "teams"; "SRE team and DevOps engineers" to "SRE" "teams" "and" "DevOps" "engineers") - Combine intersecting tokens into a new string, leaving the remaining tokens (for example intersecting: "DevOps", "and", "SRE"; remainder1: "teams"; remainder2: "team", "engineers") - Alphabetize each token group (eg. “and, DevOps, SRE”, “teams”, engineers, team”) - Compare the following pairs of strings: - Intersection group - Intersection group + remainder1 - Intersection group + remainder2 The comparison from these pairs ("best matches") is the fuzzy wuzzy token set ratio. | | When to use it | This metric is helpful in cases where similar strings may have overlapping words but different construction; for example, event descriptions for different issues with the same resource. | | Potential drawbacks | Where fuzzy wuzzy may be too conservative, fuzzy wuzzy token set match may be more liberal than expected with correlations. You can adjust the threshold in the decision builder according to your needs. | **Jaro-winkler distance** This metric is useful for short strings where identical prefixes are a strong indication of correlation. | Details | Description | | -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | How it works | This metric uses a scale of 0-1 to indicate the similarity between two strings, where 0 is no similarity (0 matching characters between strings) and 1 is an exact match. Jaro-Winkler similarity takes into account: - `matching`: two characters that are the same and in similar positions in the strings. - `transpositions`: matching characters that are in different sequence order in the strings. - `prefix scale`: the Jaro-Winkler distance is adjusted favorably if strings match from the beginning (a prefix is up to 4 characters). | | When to use it | This metric is fairly tolerant of transpositions, but transpositions further apart in the string are less useful. A generally safe number to use for Jaro-Winkler similarity in moderate to long strings is 0.9; you could use `{~}0.85` in cases where more leniency is okay (for example, if you have other, more specific logic in the decision). | **Cosine distance** This measure is most commonly used to compare large blocks of text (for example, alert event descriptions) and provides an easy visualization of similarity. | Details | Description | | ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | How it works | For each text block you're comparing, a vector is calculated to represent the count of each unique word in the block. The cosine distance of the resulting vectors is their dot product divided by the product of their magnitudes. | | When to use it | This measure is most useful to compare long blocks of text, specifically when the comparison is meant to consider the text as a whole, and not differences or misspellings in individual words. | | Examples | ```` It is not length of life, but depth of life. Depth of life does not depend on length. ``` Here are the word counts for these sentences: `it 1 0` `is 0 1` `not 1 1` `length 1 1` `of 2 1` `life 2 1` `but 1 0` `depth 1 1` `does 0 1` `depend 0 1` `on 0 1` And here are those counts represented as a vector: ``` [1, 0, 1, 1, 2, 2, 1, 1, 0, 0, 0] [0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1] ``` The cosine distance of these vectors is about 0.9 (1 is the highest similarity). ```` | | Potential drawbacks | Cosine distance is less useful for situations where small character differences in words are insignificant. Also, cosine distance ignores word order in the text blocks. | For more information on cosine distance implementation, see the [detailed walkthrough at blog.christianperone.com](http://blog.christianperone.com/2013/09/machine-learning-cosine-similarity-for-vector-space-models-part-iii/). **Hamming distance** This measure is useful for shorter text with static schema, but it works only for same-length strings. | Details | Description | | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | When to use it | Hamming distance requires the compared strings to be of equal length. This is a useful similarity metric for situations where the difference between two strings may be due to typos, or where you want to compare two attributes with known lengths. For example: ```` Low Disk Space in application myapp in data center us01 ``` If you wanted to be tolerant to data center changes, the hamming distance should be set to 4. An average use case for Hamming distance would be around 2-3. ```` | | Examples | A simpler version of "edit distance" metrics like Levenshtein distance, the Hamming distance between two strings is the number of characters in the string that don't match (in the same position). For example, in the strings below, the Hamming distance is 2: ```` flowers / florets ``` ```` | | Potential drawbacks | In the example above, if the application name changes instead of the data center, a correlation would also be created. As the distance grows, the usefulness of Hamming Distance plummets. For this reason, for anything remotely more complicated than being tolerant to 1-2 character substitutions (or if the string lengths will not match), use a different similarity measurement. | **Jaccard distance** This measure is useful for comparing large blocks of text, like descriptions or entire alert events. | Details | Description | | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | How it works | The distance, denoted as a percentage (0 being completely similar; 1 being totally dissimilar) is calculated with the following formula: ```` 1 - [(# of characters in both sets) / (# of characters in either set) * 100] ``` In other words, the Jaccard distance is the number of shared characters divided by the total number of characters (shared and un-shared). A Jaccard distance of 0.1 means that 10% or fewer characters between two alert events are different. ```` | | When to use it | Jaccard distance is very easy to interpret and especially useful in cases with large data sets. For example, in comparing the similarity between two entire alert events (as opposed to one attribute). | | Potential drawbacks | It's less effective for small data sets or situations with missing data. Also, different permutations of the character set don't affect Jaccard distance, so take care to prevent false positives. | ### Regex operators [#regex] When [building a decision](#customize), available operators include: - `contains (regex)`: used in [Step 1: Filter your data](#customize). - `regular expression match`: used in [Step 2: Contextual correlation](#customize). The decision builder follows the standards outlined in [these documents for regular expressions](https://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html). **Regex in Step 1** In order for your regex to test as true, the entire attribute value (the data you're evaluating) must be matched by the regular expression provided. Captured groups can be used but are not explicitly evaluated. For instance, if the attribute value is `foobarbaz`, these examples would meet the criteria and test as true: - `foo.*` - `^.*baz` - `\w+` **Regex in Step 2** In order for your regex to test as true, the entire attribute values for alert event 1 and alert event 2 must be included in the match. Also, each captured group (expressions in `( )` parentheses) must exist in both values (alert event 1 and alert event 2 attributes), and have the same value: - The number of captured groups must be equal for both alert event attributes. - Each group must be equal to the corresponding group between attribute values: the value of the first captured group in the alert event 1 attribute value is equal to the value of the first captured group in the alert event 2 attribute. For instance, if attribute value 1 is `abc-123-xyz` and attribute value 2 is `abc-777-xyz`, then `(\w+)-(?:\w+)-(\w+)` would meet the criteria: - The whole value is matched by the expression. - The first and third captured groups have the same respective values. - The second group is not captured using `?:`, which allows the whole value to match but isn’t used in the capture group comparison. **About flags** No flags are enabled by default. Some useful flags to include in regular expressions in the decision builder are: - CASE_INSENSITIVE: (?i) - MULTILINE: (?m) - DOTALL: (?s) See [Oracle's field detail documentation](https://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html#field.detail) for more notes on the function and implementation of each of these flags. ## Correlation assistant [#assistant] You can use the correlation assistant to more quickly analyze [alert events](https://docs.newrelic.com/docs/alerts/alerts-ai-overview-page#alert-events), create decision logic, and test the logic with a simulation. To use the correlation assistant: 1. Go to **[one.newrelic.com > All capabilities](https://one.newrelic.com/all-capabilities) > Alerts > Issues & activity > Alert events** tab. 2. Check the boxes of alert events you'd like to correlate. Then, at the bottom of the alert event list, click **Correlate alert events**. 3. For best results for correlating alert events, select common attributes with a low frequency percentage. [Learn more about using frequency](#frequency-tips). 4. Click **Simulate** to see the likely effect of your new decision on the last week of your data. 5. Click on examples of correlation pairs to determine which correlations to use. 6. If you like what's been simulated, click **Next**, and then name and describe your decision. 7. If the simulation result shows too many potential alert events, you may want to choose a different set of attributes and alert events for your decision, and run another simulation. [Learn more about simulation](#simulations). **Attribute analysis** Two types of attribute analysis appear in the UI: - **Common attributes:** This analysis simply highlights attributes and values that are the exact same between all selected alert events. - **Similar attributes:** Similarity analysis uses the Levenshtein algorithm with a distance of 3 to find attributes whose values would be the same if 3 or fewer character changes are performed. Numerical values as well as single character values are filtered out of the results. Similar attributes require two alert events to be selected, similarity analysis is not performed when 3, or more alert events are selected. To create the best decisions, we recommend choosing common attributes that have a lower frequency in your alert events. Here are tips for understanding how choosing low or high frequency attributes affects your decisions: - **Low frequency:** As an example, an attribute with a 0% in the frequency column is likely a unique identifier or an attribute that only recently reported in your data in the last month. Choosing low frequency attributes may correlate few events. - **High frequency:** On the other end, an attribute with 100% frequency would be one that is present on all your data. Choosing these attributes would correlate all of your events together. By default, the attributes are sorted by least frequency. Click an attribute's frequency percentage to get information about the distribution of values we've seen reported for that attribute in the last month. ### Simulation vs real-time correlation [#simulation-vs-decisions] It's important to understand the difference between simulation and real-time correlation in decisions: - **Simulation**: Simulation correlation involves analyzing two separate alert events to understand their relationship under simulated conditions. These alert events can originate from either the same underlying issue or from different issues. The focus is on determining potential causative factors or shared characteristics between individual alert events. Simulation helps you test and validate your correlation logic against historical data before applying it in real-time. - **Real-time correlation (decisions)**: In contrast, real-time correlation targets distinct issues, with each issue potentially encompassing multiple alert events. The aim is to detect and connect patterns across these multiple alert events to identify underlying issues for more efficient correlation. Real-time correlation leverages live data streams, allowing for prompt identification and response to emerging problems. ### Using simulation [#simulations] Simulation tests your correlation logic by analyzing two separate alert events from the last week of your data, showing you how many correlations would have happened. This allows you to validate your decision logic before it's applied to real-time correlation of issues. Here's a breakdown of the decision preview information displayed when you simulate: - **Potential correlation rate:** The percentage of tested alert events this decision would have affected. - **Total created alert events:** The number of alert events tested by this decision. - **Total estimated correlated alert events:** The estimated number of alert events this decision would have correlated. - **Alert event examples:** A list of alert event pairs that the decision would have correlated, including the rule's attributes and values, as well as other popular attributes in each pair. Click on alert events to view details. Run the simulation with different attributes as many times as you need until you see results you like. When you're ready, follow the UI prompts to save your decision. ## Topology correlation [#topology] For New Relic alerts, topology is a representation of your service map: how the services and resources in your infrastructure relate to one another. For decisions users, a [default topology decision](#global-decisions) is added and enabled in your account. You also have the option to [create custom decisions](#customize). Our topology correlation finds relationships between alert event sources to determine if [alert events](https://docs.newrelic.com/docs/alerts/alerts-ai-overview-page#alert-events) and thus their respective issues should correlate. Topology correlation is designed to improve the quality of your correlations and the speed at which they're found. ### Requirements [#topology-requirements] For automatic topology correlation (without the need to explicitly set up topology graph), make sure your telemetry data is collected by [New Relic agents](https://docs.newrelic.com/docs/new-relic-solutions/new-relic-one/install-configure/compatibility-requirements-new-relic-agents-products/). The more types of New Relic agents are installed in your services and environment, the more opportunities for topology decisions to correlate your alert events. ### How does topology correlation work? [#topology-explained] ![A screenshot of New Relic topology explained](https://docs.newrelic.com/images/alerts_diagram_topology-4.webp "topology-4.png") In this service map, the hosts and apps are the vertices, and the lines showing their relationships are the edges. To set up your topology in addition to the [entities and relationships](https://docs.newrelic.com/docs/new-relic-solutions/new-relic-one/core-concepts/what-entity-new-relic/) collected by [New Relic agents](https://docs.newrelic.com/docs/new-relic-solutions/new-relic-one/install-configure/compatibility-requirements-new-relic-agents-products/), use our [NerdGraph API](#create-topology-graph). Customized topology correlation relies on two main concepts: - **Vertex:** A vertex represents a monitored entity. It's the source from which your alert events are coming from, or describing a problematic symptom about. A vertex has attributes (key/value pairs) configured for it, like entity GUIDs or other IDs, which allow it be associated with incoming alert events. - **Edges:** An edge is a connection between two vertices. Edges describe the relationship between vertices. It may help to understand how topology is used to correlate alert events: 1. First, New Relic gathers all relevant alert events. This includes alert events where [decision logic steps 1 and 2](#customize) are true and that are also within the defined time window in advanced settings. ![A screenshot of New Relic topology explained](https://docs.newrelic.com/images/alerts_diagram_topology-1.webp "topology-1.png") 2. Next, we attempt to associate each alert event to a vertex in your [topology graph](#create-topology-graph), using a vertex's defining attributes and the available attributes on the alert event. ![A screenshot of New Relic topology explained](https://docs.newrelic.com/images/alerts_diagram_topology-2.webp "topology-2.png") An example of the steps for associating alert events with the information in the topology graph. 3. Then, the pairs of vertices which were associated with alert events are tested using the "topologically dependent" operator to determine if these vertices are connected to each other. ![A screenshot of New Relic topology explained](https://docs.newrelic.com/images/alerts_diagram_topology-3.webp "topology-3.png") This operator checks to see if there is any path in the graph that connect the two vertices within five hops. The alert events are then correlated and the issues are merged together. ### Add attributes to alert events [#add-attributes] Alert events are connected to vertices using a vertex's defining attributes. (In the example topology under [Topology explained](#topology-explained), each vertex has a defining attribute "CID" with a unique value.) Next, New Relic's alerts system finds a vertex that matches the attribute. If the defining attribute you'd like to use on your vertices isn't already on your alert events, use either of these options to add it: **Tag your entities in New Relic** By [tagging your entities](https://docs.newrelic.com/docs/new-relic-one/use-new-relic-one/core-concepts/use-tags-help-organize-find-your-data), those tags will enrich the alert events generated by alerts. For example, if you've tagged your entities with `CID` and their corresponding unique values, then you can have defining attributes on your vertex as follows: `'newrelic/tags/CID' : CID_VALUE` **Tag your entities in New Relic** Creating [NRQL alert conditions](https://docs.newrelic.com/docs/alerts-applied-intelligence/new-relic-alerts/alert-conditions/create-nrql-alert-conditions) with one or more [facets](https://docs.newrelic.com/docs/alerts-applied-intelligence/new-relic-alerts/alert-conditions/create-nrql-alert-conditions#syntax) defined will group your data by attribute. Also, alert events emitted will be enriched with those attributes and values. For alert events, faceted attributes follow the same format: `newrelic/tags/ATTRIBUTE_NAME` ### Create or view topology [#create-topology-graph] To set up your topology or view existing topology, see the [NerdGraph topology tutorial](https://docs.newrelic.com/docs/apis/nerdgraph/examples/topology-nerdgraph-tutorial).