Monitor AI agents and tools

Enabling AI monitoring lets you observe the performance of your AI agents end-to-end. When your application runs a multi-step agentic workflow, New Relic captures the full execution, every agent invocation, every tool call, every handoff, so you can understand what happened, where it slowed down, and where something went wrong.

AI agent monitoring supports popular agentic frameworks including LangGraph, Strands, and AutoGen.

No additional configuration is required beyond enabling AI monitoring on your existing APM agent. Once enabled, New Relic automatically detects and instruments agents and tools within these frameworks.

Why monitor AI agents?

Modern AI agents are complex. A single user request may spawn multiple sub-agents, call external tools, query vector stores, and chain LLM calls together. When something goes wrong - a slow response, an unexpected error, a runaway token count - the cause is rarely obvious from the outside.

AI agent monitoring helps you unbox the black box:

• Trace every step of an agentic workflow from the initial trigger through each agent and tool invocation to the final response.
• Pinpoint latency across individual agents and tools so you know exactly where time is being spent.
• Catch errors early by surfacing which agent or tool in the chain caused a failure.
• Track token consumption at the agent level to understand cost drivers in complex workflows.

Agent performance overview

Once your agentic application reports data, you can monitor agent performance from the AI responses page. Agent-related metrics are surfaced alongside your existing AI data so you can see the full picture in one place.

Key performance indicators available for agents include:

• Latency: End-to-end response time for each agent invocation, helping you identify slow agents in a multi-step pipeline.
• Error rate: The frequency at which a given agent or tool returns an error, so you can triage failures quickly.
• Token usage: Token counts broken down at the agent level, giving you granular insight into cost allocation across complex workflows.

Each line in the Responses table represents a trace. Clicking on a table line will lead you to dig into the trace, where you can see the waterfall and the entity map.

AI agents in the entity map

New Relic adds AI agents and AI tools as first-class entities in the entity map. This means you can see your agentic application as a visual graph:

• Agents appear as distinct nodes, showing how they relate to the services, models, and tools they interact with.
• AI tools appear as connected nodes, illustrating which agents invoke which tools and in what order.
• Relationships between agents - such as one agent delegating work to a sub-agent - are rendered as directed edges, making orchestration patterns easy to understand at a glance.

Agent trace waterfall

The most detailed view of agentic execution is the trace waterfall. AI agents and AI tools are fully integrated into the existing trace waterfall view, giving you a unified, span-by-span breakdown of every step in a workflow.

Each row in the waterfall represents a single span. This includes agent and tools invocations.

Troubleshooting with the waterfall

The trace waterfall is your primary tool for troubleshooting issues in complex agentic pipelines:

• Errors are highlighted in red so you can immediately spot which agent or tool in the chain caused a failure, without manually tracing through logs.
• Long-running spans are visually obvious from the waterfall width, letting you identify which step is responsible for end-to-end latency.
• Span details are accessible by selecting any row. The details panel shows the span's attributes, error messages, and the inputs and outputs for that agent or tool call.

What's next?

Now that you understand how to monitor AI agents, explore these related topics:

• Learn how to view AI response data for your AI-powered applications
• Discover how to customize AI monitoring to fit your specific needs
• Find out about compatibility requirements for AI monitoring