• EnglishEspañol日本語한국어Português
  • 로그인지금 시작하기

Monitor OpenAI’s GPT Application Usage in New Relic

March 15, 2023

Monitor OpenAI applications running GPT-3, GPT-3.5, and beyond with just two lines of code using New Relic to track the cost, usage, and performance of your models in real-time.

If you're already using, or plan to use, OpenAI's GPT-3, GPT-3.5, or GPT-4 at scale, it's important to monitor metrics like average request time, total requests, and total cost. Doing so can help you ensure that OpenAI services like ChatGPT are working as expected, especially when those services are required for important functions like customer service and support.

Companies around the world are embracing GPT to power help desk tickets and live chat logs, develop content and images, and accelerate semantic searches. New Relic monitoring for OpenAI is fast, easy to use, and unlocks real-time metrics that can help engineering teams optimize usage, reduce costs, and achieve better results.

New Relic integrates with OpenAI, GPT-3, and beyond to provide the following benefits:

New Relic's OpenAI integration providing a pre-built dashboard for monitoring performance, cost, and usage of GPT applications.

  • Get started for free: Our out-of-the-box GPT monitoring solution is the first of its kind, and included at no additional cost to all New Relic full platform users.
  • Easy installation: With just two lines of code, users can import the monitor module from the nr_openai_monitor library and automatically generate a dashboard that displays a variety of key GPT performance metrics.
  • Monitor cost: Usage of OpenAI’s Davinci model costs $0.12 per 1000 tokens, which can add up quickly and make it difficult to operate at scale. New Relic provides engineering teams with real-time cost tracking of their GPT usage.
  • Optimize performance: New Relic gives engineering teams insight into the average response time and other key performance metrics around GPT requests, allowing engineers to optimize usage and ensure the best possible response times.
  • Analyze prompts and responses: New Relic provides valuable information about the usage, speed, and effectiveness of GPT to help engineering teams achieve better results from their ML models.

How to get started

With just two lines of code, simply import the monitor module from the nr_openai_monitor library and automatically generate a dashboard that displays a variety of key GPT-3 and GPT-3.5 performance metrics, such as cost, requests, average response time, and average tokens per request.

To get started, simply install the OpenAI Observability quickstart from New Relic I/O or visit our library repo for further instructions on how to integrate New Relic with your OpenAI apps and deploy the custom dashboard.

If you don’t have a New Relic account already, sign up for a free New Relic account today. Your free account includes 100 GB/month of free data ingest, one free full-access user, and unlimited free basic users.

Copyright © 2024 New Relic Inc.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.