SetLlmTokenCountingCallback (.NET agent API)

Syntax

NewRelic.Api.Agent.NewRelic.SetLlmTokenCountingCallback(Func<string, string, int> callback);

Provide a callback method that calculates the token count.

Requirements

This API call is compatible with:

• Agent versions >= 10.23.0
• All app types

Description

Set a callback method that the agent will use to determine the token count for a LLM event. In High Security Mode or when content recording is disabled, this method will be called to determine the token count for the LLM event.

The callback method must accept a two arguments of type string, and return an integer. The first string argument is the LLM model name, and the second string argument is the input to the LLM. The callback method should return the token count for the LLM event. Values of 0 or less will be ignored.

Parameters

Example

Func<string, string, int> llmTokenCountingCallback = (modelName, modelInput) => {

  int tokenCount = 0;
  // split the input string by spaces and count the tokens
  if (!string.IsNullOrEmpty(modelInput))
  {
    tokenCount = modelInput.Split(' ').Length;
  }

    return tokenCount;
};

NewRelic.Api.Agent.NewRelic.SetLlmTokenCountingCallback(llmTokenCountingCallback);