• EnglishEspañol日本語한국어Português
  • Log inStart now

Remove sensitive data with drop filters

You have two options for dropping sensitive AI data before you send it to New Relic. This doc guides you through these two methods so you can have better control over the kinds of data the agent collects.

Disable ai.monitoring.record_content.enabled

When you disable ai_monitoring.record_content.enabled, event data containing end user prompts and AI responses won’t be sent to NRDB. You can read more about agent configurations at our AI monitoring configuration doc.

Create drop filters

Caution

Use caution when deciding to drop data. The data you drop is not recoverable. Before using this feature, review your data compliance responsibilities.

A single drop filter targets a specified attribute within one event type, but sensitive information from a single AI interaction is stored in multiple events. To drop information before it enters NRDB, you need six separate drop filters.

  1. Go to one.newrelic.com > All capabilities > AI monitoring > Drop filters, then click Create drop filter.
  2. Create a filter name. Because one kind of data requires at least six drop filters, we recommend a naming convention that helps you track the events you're dropping data from.
  3. Paste in the base NRQL query:
    SELECT <COLUMN> FROM <EVENT_TYPE> WHERE <COLUMN> RLIKE <REGEX>
  4. Referring to the table, update the <COLUMN> and <EVENT_TYPE> placeholders to match the attributes and events. For example:
    SELECT content FROM LlmChatCompletionMessage WHERE content RLIKE <REGEX>
  5. Add the regex that corresponds to the kind of information you want to drop. For example, if you're targeting IPv4 addresses, the completed query should read:
    SELECT content FROM LlmChatCompletionMessage WHERE content RLIKE r'.*^([0-9]{1,3})\.([0-9]{1,3})\.([0-9]{1,3})\.([0-9]{1,3})$.*'
  6. Repeat the above steps to create drop filters for the remaining events and column pairs.

How drop filters work

A drop filter evaluates data forwarded by the agent within the data ingest pipeline. A drop filter contains three parts:

  • Events: A stored record from an interaction within your system.
  • Attributes, or columns: A key-value pair attached to data objects.
  • Regex: A string of characters and operators that corresponds to kinds of information.

Events and attributes

In a typical AI interaction, a prompt or request undergoes certain processes (like embedding) that are recorded as discrete events. For example, let's say a customer requests a street address on file. The model processes the prompt, which pulls additional context through various services and databases. Your AI assistant then returns with a response that contains the requested information.

A complete set of drop filters for a piece of sensitive information should include queries for the six events provided in the drop filter table. For each event, you need to create separate filters for attributes when there are more than one. Your drop filters correspond to event and column pairs in a given row. A few things to remember:

  • The column content appears in the LlmChatCompletionMessage event and not in the LlmEmbedding event.
  • The column messages only appears in LlmFeedbackMessage but not in LlmTool.
  • The exception to this rule is the attribute input, which appears in both LlmEmbedding and LlmTool.

Regex

Since the agent's default behavior is to capture all parts of event data before sending it to New Relic, you need to direct the ingest pipeline to match sensitive information with regex. By targeting an attribute with regex, you can still capture the event itself without storing sensitive information in our databases.

Refer to the regex below to start building your first queries:

Your data compliance responsibilities

New Relic can't guarantee that this functionality completely resolves your data disclosure concerns, nor can we provide support for building out your NRQL queries. We recommend that you:

  • Review your drop filters and confirm they're accurately identifying and discarding the data you want dropped.
  • Check that your drop filters are still dropping sensitive information after you've created them.

While drop filters help ensure that personal information about your end users aren't stored in NRDB, creating the rules themselves imply the kinds of data you maintain, including the format of your data or systems. This is important when determining control permissions for certain users in your org, as certain permissions let users view and edit all information in the rules you create.

What's next?

Now that you've secured your customer's data, you can explore AI monitoring:

Copyright © 2024 New Relic Inc.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.