• /
  • EnglishEspañol日本語한국어Português
  • Log inStart now

Introduction to AI monitoring

When people talk about artificial intelligence, they can mean different things. At New Relic, when we say AI, we mean the layer of your environment that uses a large language model (LLM) to generate a response when it receives an end user prompt. AI monitoring is an APM solution that gives you end-to-end visibility into your AI-powered app.

With AI monitoring, you can measure the performance of the engine powering your AI app, so that you can ensure your users have the best possible experience. To get started, all you need is to install one of our APM agents and enable AI monitoring.

A screenshot that shows the trace waterfall page for an individual AI response

Go to one.newrelic.com > AI monitoring > AI responses, then select a response row.

How does AI monitoring work?

To get started with AI monitoring, you'll install one of our APM agents to instrument your app. Instrumentation means that your app can be measured, which lets the agent capture data about app behaviors. Once instrumented, you need to enable AI monitoring on the configuration level.

Enabling AI monitoring allows the agent to recognize AI metadata associated with AI events. When your AI receives a prompt and returns a response, the agent can recognize data generated from external LLMs and vector stores, and parse out information about tokens usage.

Improve AI performance with AI monitoring

A screenshot of the AI responses page

To overview your AI-powered app's performance: Go to one.newrelic.com > AI monitoring > AI responses.

AI monitoring can help you answer critical questions about AI app performance: are your end users waiting too long for a response? Is there a recent spike in token usage? Are there patterns of negative user feedback around certain topics? With AI monitoring, you can see data specific to the AI-layer:

Get started with AI monitoring

Ready to get started? Make sure to confirm that you can instrument your AI library or framework. You may need to update the agent if you've already instrumented your app.

When you're ready, use our doc to manually install AI monitoring. This doc directs you to the relevant procedures for installing an APM agent, then walks you through configuring the agent for AI monitoring.

Copyright © 2024 New Relic Inc.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.