Set up RAG integration

With New Relic, you can enhance New Relic AI agents with Retrieval Augmented Generation (RAG) by associating your documentation, runbooks, incident retros, and even source code with your services. This process gives New Relic AI better insight into issues with your system. The tutorial outlines how to obtain your organization ID, create a RAG tool, and add your documents to the platform using the Blob API.

After adding your documents, you will create a relationship to associate them with the RAG tool. You can then verify your configuration by querying the relationships between the RAG documents and the RAG tool. The final step is to query the RAG tool itself to retrieve relevant, indexed information.

To learn more about the knowledge connector, refer to New Relic AI Knowledge connector.

Important

Before performing the following steps, ensure that you have "Org Product Admin" permissions.

To start indexing your content and benefit from the knowledge connector with New Relic AI, follow these mentioned steps:

Task 1: Create your RAG tool

Organization ID uniquely identifies your account and ensures that any RAG tool you create, documents you upload, or relationships you establish are associated with your organization in New Relic.

Organization ID is required to perform mutations and queries in NerdGraph when setting up and managing RAG tools and documents. Run the following query and keep the organization ID handy for later steps.

Sample query

{
  actor {
    organization {
      id
    }
  }
}

A RAG tool acts as a specialized container for organizing related documents and resources in New Relic. A clear name and accurate description for your RAG tools helps the LLM select the right tool for each prompt, ensuring relevant and context-aware responses.

To create a new RAG tool in your New Relic account, run the following entityManagementCreateRagTool mutation:

Make sure to replace ${ORGANIZATION_ID} with your actual organization ID obtained in the previous step.
If successful, you'll receive an id for your RAG tool.

Input parameters

Parameter Name	Data Type	Is it Required?	Description
`ragToolEntity`	Object	Yes	The input object that contains the configuration for the new RAG tool.
`description`	String	Yes	A clear and accurate description of the RAG tool's purpose.
`name`	String	Yes	The unique name for your RAG tool.
`scope`	Object	Yes	An object defining the context in which the tool will be created.
`scope.id`	String	Yes	The unique ID of your organization (`${ORGANIZATION_ID}`).
`scope.type`	String	Yes	The type of the scope, which must be `ORGANIZATION`.

Sample mutation

mutation {
  entityManagementCreateRagTool(
    ragToolEntity: {
      description: "Runbooks for resolving incidents with APIs",
      name: "API Runbooks",
      scope: {id: `${ORGANIZATION_ID}`, type: ORGANIZATION}
    }
  ) {
    entity {
      id
    }
  }
}

You should save the id returned as you'll need it in later steps to link documents to the tool, verify relationships, and query the tool for relevant information in New Relic.

Task 2

You have two options for providing context to your RAG tool. You can manually upload static files or set up an automated connector for living documentation.

Option A: Index your documents

If you have static documents such as PDFs, Word files, or local CSVs that are not hosted in a cloud knowledge base, use the Document Knowledge Connector. This method utilizes the Blob API to upload individual files directly to New Relic. Use this option for:

One-time context:Uploading specific runbooks or architectural diagrams that rarely change.
Local data: Indexing proprietary or internal files that live on your local machine rather than a wiki.
Agentic testing: Quickly providing a specific set of documents to an AI agent for a focused workflow.

Important

All indexed documents are visible to all users within your organization. Make sure the documents you index comply with your internal policies, and do not upload sensitive or private data.

The Blob API and its purpose

The Blob API is a New Relic service designed for uploading files, such as documentation and runbooks, to your account. NerdGraph is optimized for structured data queries and mutations and not for efficient transfer of files, so the Blob API is required for uploading documents.

Authentication requirements

You need a valid New Relic API key with permissions to upload documents. To get the API key for uploading a document to New Relic using the Blob API:

Log in to your New Relic account.
Create and manage your API keys from the API keys UI page.
Click Create a key, and fill the required details (or use an existing one with the required permissions).
Click Create a key and copy the generated key (it'll look like NRAK-XXXXXXXXXX).

Here's an example of how to upload a document using a curl bash command:

Input parameters

Parameter Name	Data Type	Is it Required?	Description
`Api-Key`	String	Yes	Your New Relic API key for authentication.
`NewRelic-Entity`	JSON Object	Yes	Metadata about the document, such as its name.
`Content-Type`	String	Yes	The format of the file being uploaded (e.g., `application/json`).
`payload` (`@incidents.json`)	File	Yes	The document file you are uploading, specified by its file path.

Sample query

bash

$curl -X POST https://blob-api.service.newrelic.com/v1/e/organizations/$ORGANIZATION_ID/RagDocuments \
>     -H 'Api-Key: NRAK-XXXXXXXXXX' \
>     -H 'NewRelic-Entity: {"name": "Runbooks for API service" }' \
>     -H 'Content-Type: application/json' \
>     -d @incidents.json

Sample response

Response	Data Type	Description
`entityGuid`	String	The unique identifier for the uploaded RAG document.
`blobVersionEntity`	Object	Represents the version of the uploaded blob.

{
  "entityGuid": "MTIyODU0NTN8TkdFUHxSQUdfRE9DVU1FTlR8MDE5NGUyOTgtYmQzMS03NzA4LWI3NzItYzQ4MTZlYjNhYThk",
  "blobVersionEntity": null
}

Next steps

After uploading your document, it is indexed and becomes available for New Relic AI to search and retrieve. You must save the entityGuid from the response to create a relationship with your RAG tool or to query the document in NerdGraph.

After a document is uploaded via the Blob API, running this query confirms that the upload was successful and that the document has been properly registered as a RAG document entity with its own unique identifier and properties.

Input parameters

Parameter Name	Data Type	Is it Required?	Description
`id`	String	Yes	The unique GUID of the RAG document to be retrieved.

Sample query

In the query below, replace the ${RAG_DOCUMENT_GUID} placeholder with the entityGuid you received in the previous step.

{
  actor {
    entityManagement {
      entity(
        id: `${RAG_DOCUMENT_GUID}`
      ) {
        ... on EntityManagementRagDocumentEntity {
          id
          name
          blob {
            url
          }
          type
        }
      }
    }
  }
}

This query will return the following details about your RAG document:

id: The unique ID of the RAG document.
name: The name of the RAG document.
blob { url }: The URL to access the uploaded document.
type: The type of the entity, which in this case is EntityManagementRagDocumentEntity.

Now that you've created a RAG tool, uploaded your document, and verified that the upload was successful, the next step is to associate the RAG tool and the RAG document thereby making your document searchable and usable by New Relic AI. To do this, run the entityManagementCreateRelationship mutation:

Replace ${RAG_DOCUMENT_GUID} with the entityGuid from the response of the document upload via the Blob API.
Replace ${RAG_TOOL_GUID} with the id from the response of the RAG tool creation mutation.

Input parameters

Parameter Name	Data Type	Is it Required?	Description
`relationship`	Object	Yes	The input object that contains the details for the relationship.
`source`	Object	Yes	The source entity of the relationship, which is the RAG document.
`source.scope`	String	Yes	The scope of the source entity, which must be `ORGANIZATION`.
`source.id`	String	Yes	The unique GUID of the RAG document (`${RAG_DOCUMENT_GUID}`).
`target`	Object	Yes	The target entity of the relationship, which is the RAG tool.
`target.scope`	String	Yes	The scope of the target entity, which must be `ORGANIZATION`.
`target.id`	String	Yes	The unique GUID of the RAG tool (`${RAG_TOOL_GUID}`).
`type`	String	Yes	The type of the relationship, which must be `"INDEXED_FOR"`.

Sample mutation

mutation {
  entityManagementCreateRelationship(
    relationship: {
      source: {
        scope: ORGANIZATION,
        id: `${RAG_DOCUMENT_GUID}`
      },
      target: {
        scope: ORGANIZATION,
        id: `${RAG_TOOL_GUID}`
      },
      type: "INDEXED_FOR"
    }
  ) {
    relationship {
      type
      target {
        id
        type
      }
      source {
        id
        type
      }
    }
  }
}

Option B: Index your Confluence documents

If your organization uses Confluence for documentation, you can index your Confluence documents into New Relic without needing to use the Blob API. This option allows you to connect your Confluence instance and select specific documents or spaces to be indexed and associated with your RAG tool. Use this option to ensure New Relic AI always has the latest version of your Confluence pages.

The connector requires an Atlassian API token to fetch your pages. Securely store this in the New Relic secrets manager.

Input parameters

Parameter Name	Data Type	Required?	Description
description	String	No	A brief summary of what the secret is used for.
key	String	Yes	The unique name used to reference this secret (for example, `CONFLUENCE_API_TOKEN`).
namespace	String	Yes	Must be set to rag-datafetching for the RAG indexer service to access it.
value	String	Yes	The actual Confluence API token value or password.
scope	Object	Yes	Defines the organizational context; the type must be ORGANIZATION.

Sample mutation

mutation createSecretKey {
  secretsManagementCreateSecret(
    description: "Confluence API token for RAG indexing"
    key: "CONFLUENCE_API_TOKEN"
    namespace: "rag-datafetching"
    scope: { id: "YOUR_ORGANIZATION_ID", type: ORGANIZATION }
    value: "YOUR_CONFLUENCE_API_TOKEN"
  ) {
    key
  }
}

Define the connection to your Confluence instance. This acts as the bridge between New Relic and Atlassian.

Input parameters

Parameter Name	Data Type	Required?	Description
name	String	Yes	A descriptive name for the integration (for example, "Engineering Confluence").
url	String	Yes	Your base Confluence URL (for example, https://example.atlassian.net/wiki).
confluenceUserId	String	Yes	The email address or ID associated with your Atlassian API token.
secretKey	String	Yes	The key name you defined in the previous step (`CONFLUENCE_API_TOKEN`).
scope	Object	Yes	Defines the context for the integration. The type must be set to ORGANIZATION.
tags	Object	No	Optional metadata used to categorize the integration by department or team.

Sample mutation

mutation {
  entityManagementCreateConfluenceIntegration(
    confluenceIntegration: {
      name: "Engineering Confluence Integration"
      url: "https://your-company.atlassian.net/wiki"
      confluenceUserId: "your-confluence-user-id"
      secretKey: "CONFLUENCE_API_TOKEN" # From Step 2
      scope: { id: "YOUR_ORGANIZATION_ID", type: ORGANIZATION }
      tags: { key: "department", values: "engineering" }
    }
  ) {
    entity {
      id
    }
  }
}

Create a RAG settings entity to define what content to fetch and how it should be chunked.

Input parameters

Parameter Name	Data Type	Required?	Description
confluenceIntegrationId	String	Yes	The unique ID of the Confluence integration created in the previous step.
confluenceQuery	String	Yes	A CQL (Confluence Query Language) filter (for example, space = 'ENG')..
intervalSeconds	Int	Yes	The number of seconds between indexing runs (for example, 3600 for every hour).
chunkSize	Int	Yes	The maximum number of characters in each text segment (for example, 4000).
chunkOverlap	Int	Yes	The number of characters to overlap between text segments (for example, 100) to maintain context.
textSplitterType	Enum	Yes	The type of text splitter to use (for example, `MARKDOWN_TEXT_SPLITTER`).
markdownTextSplitterOptions	Object	No	Configuration for markdown splitting, such as which header levels (#, ##) to split on.
tokenTextSplitterOptions	Object	No	Configuration for token splitting, including the encoding model (for example, `O200K_BASE`).
tags	Object	No	Optional metadata to categorize settings (for example, key: `knowledge_category`, values: `documentation`).

Text splitter types

Use these types to define the textSplitterType parameter:

TOKEN_TEXT_SPLITTER: Accurate token-based splitting.
MARKDOWN_TEXT_SPLITTER: Split by markdown headers.
CHARACTER_TEXT_SPLITTER: Split by character count.
JSON_TEXT_SPLITTER: Split JSON recursively.

If using TOKEN_TEXT_SPLITTER, define the encoding in tokenTextSplitterOptions:

O200K_BASE: For GPT-4o and newer models (recommended).
CL100K_BASE: For GPT-4 and GPT-3.5.

Sample mutation

mutation {
  entityManagementCreateConfluenceRagSettings(
    confluenceRagSettingsEntity: {
      name: "Engineering Docs RAG Settings"
      scope: { id: "YOUR_ORGANIZATION_ID", type: ORGANIZATION }
      confluenceIntegrationId: "INTEGRATION_ID_FROM_STEP_3"
      confluenceQuery: "space = ENG AND type = page"
      intervalSeconds: 3600 # Re-index every hour
      # Chunking Configuration
      chunkSize: 4000
      chunkOverlap: 100
      textSplitterType: MARKDOWN_TEXT_SPLITTER
      # Optional: Markdown-specific options
      markdownTextSplitterOptions: {
        headersToSplitOn: "#,##,###"
        returnEachLine: false
      }
      # Optional: Token splitter options (if using TOKEN_TEXT_SPLITTER)
      # tokenTextSplitterOptions: {
      #   encodingName: O200K_BASE
      # }

      # Optional: Character splitter options (if using CHARACTER_TEXT_SPLITTER)
      # characterTextSplitterOptions: {
      #   separator: "\n\n"
      #   isSeparatorRegex: false
      # }

      tags: { key: "knowledge_category", values: "documentation" }
    }
  ) {
    entity {
      id
    }
  }
}

Linking your RAG settings to your RAG Tool via an APPLY_TO relationship activates the automated indexing.

Input parameters

Parameter Name	Data Type	Required?	Description
source.id	String	Yes	The ID of the RAG settings created in the previous step.
target.id	String	Yes	The ID of the RAG tool created in Task 1.
type	String	Yes	Must be set to `APPLY_TO` for settings-to-tool connections.
tags	String	Yes	Optional metadata used to label the relationship (for example, key: "`auto_created`", values: "`true`")

Sample mutation

mutation {
  entityManagementCreateRelationship(
    relationship: {
      source: { id: "RAG_SETTINGS_ID_FROM_STEP_4", scope: ORGANIZATION }
      target: { id: "RAG_TOOL_ID_FROM_STEP_1", scope: ORGANIZATION }
      type: "APPLY_TO"
      tags: { key: "auto_created", values: "true" }
    }
  ) {
    relationship {
      type
    }
  }
}

Outcome: Automatic indexing triggered

Indexing will now happen automatically!

When the APPLY_TO relationship is created between your RAG settings and your RAG tool, the New Relic RAG indexer service initiates the following background process:

The service uses your confluenceQuery to search your Confluence instance for matching content.
It retrieves all matching pages and transforms them into a processable format.
The content is split into smaller segments based on your chunkSize, chunkOverlap, and textSplitterType configuration.
The service generates dense and sparse embeddings for each chunk and indexes them in the vector database (Pinecone).
The connector will re-index your content periodically based on the intervalSeconds you defined to ensure the AI has access to the most up-to-date documentation.

Task 3: Retrieve relevant information

After establishing a relationship (either by linking a specific RAG document via INDEXED_FOR or applying a Confluence configuration via APPLY_TO), you can verify the association and query the tool. This ensures your internal knowledge is properly indexed and available for New Relic AI to surface context-aware answers.

Replace ${RAG_DOCUMENT_ID} with the entityGuid of your uploaded document.

Input parameters

Parameter Name	Data Type	Is it Required?	Description
`relationships`	Query	Yes	The query to retrieve relationships between entities.
`filter`	Object	No	An object used to filter the relationships based on attributes.
`filter.sourceId`	Object	No	An object to filter by the source entity's unique identifier.
`filter.sourceId.eq`	String	No	The unique GUID of the RAG document to match.

Sample query

{
  actor {
    entityManagement {
      relationships(
        filter: {sourceId: {eq: `${RAG_DOCUMENT_ID}`}}
      ) {
        items {
          type
          target {
            id
            type
          }
        }
      }
    }
  }
}

After you've set up your RAG tool and indexed documents, you can query the RAG tool to retrieve relevant information based on your prompt. This allows New Relic AI to surface context-aware answers using your organization's documentation.

Input parameters

Parameter Name	Data Type	Is it Required?	Description
`prompt`	String	Yes	The natural language query you want the RAG tool to process.
`toolId`	String	Yes	The unique GUID of the RAG tool to be queried.

Sample query

{
  actor {
    machineLearning {
      ragQueryData(
        prompt: "tell me about the incident", 
        toolId: `${RAG_TOOL_GUID}`
      ) {
        blobId
        chunk
        documentId
        score
        toolId
      }
    }
  }
}

The response will include chunked matches from your indexed documents, which you can use directly or summarize with New Relic AI.

Important

Task 1: Create your RAG tool .css-21sua1{background:none;border:none;width:0;padding:0;}

Create a RAG tool

Task 2

Option A: Index your documents

Upload a document via the Blob API

View the RAG document entity represented in NerdGraph

Create a relationship between the RAG tool and the RAG document

Option B: Index your Confluence documents

Store your Confluence API token

Create the Confluence integration

Configure indexing logic (RAG settings)

Create relationship

Outcome: Automatic indexing triggered

Task 3: Retrieve relevant information

Query to see relationships between RAG documents and RAG tools

Query the RAG tool

Task 1: Create your RAG tool