OpenAI integration

With our OpenAI integration, you have the ability to seamlessly monitor all OpenAI queries. By adding just two lines of code, your team can gain access to key performance metrics such as cost, response time, rate limits, error rates, chat history, and sample inputs and outputs. Our OpenAI integration provides real-time metrics to help businesses optimize usage, reduce costs, and achieve better results.

This guide will show you how to integrate your OpenAI applications so you can start monitoring your data.

Install OpenAI with pip

After you've completed your OpenAI import, run:

bash

$pip install nr-openai-observability

Set your environment variable

Get your and set it as environment variable: NEW_RELIC_LICENSE_KEY.

For more details and instructions, check out our REST API documentation.

bash

$export NEW_RELIC_LICENSE_KEY=<license key>

import os
os.environ["NEW_RELIC_LICENSE_KEY"] = "<license key>"

Your NEW_RELIC_LICENSE_KEY can also be sent as a parameter at the monitor.initialization() call.

Modify your code

Add the following two lines to your code:

from nr_openai_observability import monitor
monitor.initialization()

Add your dashboard

To add the dashboard to your New Relic account:

import os

import openai
from nr_openai_observability import monitor

monitor.initialization()

openai.api_key = os.getenv("OPENAI_API_KEY")
openai.Completion.create(
    model="text-davinci-003",
    prompt="What is Observability?",
    max_tokens=20,
    temperature=0 
)

Install OpenAI

Use npm and run:

bash

$npm install @newrelic/openai-observability openai

Modify your code

Call monitorOpenAI with your configured OpenAI client.

Here is some example code:

import { Configuration, OpenAIApi } from 'openai';
import { monitorOpenAI } from '@newrelic/openai-observability';

const configuration = new Configuration({
  apiKey: 'OPENAI_API_KEY',
});

const openAIApi = new OpenAIApi(configuration);

monitorOpenAI(openAIApi, {
  applicationName: 'MyApp',
  newRelicApiKey: 'NEW_RELIC_LICENSE_KEY',
});

const response = await openAIApi.createChatCompletion({
  model: 'gpt-3.5-turbo',
  messages: [{ role: 'user', content: 'What is Observability?' }],
});

console.log(response.data.choices[0].message);

Make sure that you add module type to package.json:

"type": "module"

Initialization options

export interface MonitorOpenAIOptions {
  /**
   * Your application name in New Relic, must provided.
   */
  applicationName: string;
  /**
   * API key with insert access used to authenticate the request.
   * For more information on creating keys, please see:
   * https://docs.newrelic.com/docs/insights/insights-data-sources/custom-data/introduction-event-api#register
   */
  newRelicApiKey?: string;
  /**
   * Optional host override for event endpoint.
   */
  host?: string;
  /**
   * Optional port override for trace endpoint.
   */
  port?: number;
}

EU account users

If you're using an EU account, you should set your EVENT_CLIENT_HOST as follows:

bash

$export EVENT_CLIENT_HOST="insights-collector.eu01.nr-data.net"

import os
os.environ["EVENT_CLIENT_HOST"] = "insights-collector.eu01.nr-data.net"

Find your data

After you've integrated OpenAI, follow these steps to find observability data:

From one.newrelic.com.
Click on Model Performance.
Search under Applications.

See how it works

If you're struggling with these instructions, we encourage you to watch this demonstration video.

Install for Python applications

Install OpenAI with pip

Set your environment variable

Modify your code

Add your dashboard

Install with Node.js

Install OpenAI

Modify your code

Initialization options

EU account users

Find your data

See how it works