Insert custom events via the Insights API

The New Relic Insights API lets you store custom events in Insights; these events appear in Insights as a new event type. Then, you can use NRQL or the Insights event explorer to sample, query, and chart this new data.

You can also:

  • Use the Insights Data explorer to ensure your events are reporting correctly and to generate queries.
  • Record custom events directly from supported agents by using the appropriate agent API for regular events; for example, Transactions (inserted via New Relic APM agents) and PageViews (inserted via New Relic Browser agents).
  • Monitor your custom events by creating alert conditions for basic Insights NRQL queries that return a number.

You must have access to the Insights UI in order to register Insert keys or to insert custom events via the Insights API.

To see all available New Relic APIs, see Intro to APIs.

Insert an event

Insights only supports event insertion via POST. To insert an event into your New Relic Insights account:

  1. Register an Insights API Insert key.
  2. Generate JSON for the event by instrumenting your application, querying an API, or some other method.
  3. Submit the JSON to the Insights HTTPS endpoint using cURL in a POST request.

This method will insert the events directly into your Insights account. You can query those events as a new event type. You can also use custom attributes to decorate your default Transaction and PageView events with additional attributes.

Insights limits the size, rate, and characters allowed in custom events. Like other Insights events, custom events cannot be updated or deleted after they are created.

Register an API key

Owner, Admins, or add-on managers

The Insert key allows you to insert events into Insights. If you have a paid Insights subscription, you can submit multiple event types under a single insert key. To help ensure security, use different insert keys for different applications or different data sources.

To add an insert key:

  1. Go to insights.newrelic.com > Manage data > API keys.
  2. From API key management, select the symbol next to the Insert keys heading.
  3. Enter a short description for the key.
  4. Select Save your notes.

The Insert key page lists the cURL command necessary to insert data for that key.

For security reasons, insert keys cannot be altered or read using the API. To change or read an insert key, use the New Relic UI.

Instrument your application

You can insert custom events from your APM-instrumented app directly from a New Relic agent. You can also insert events from end-user browsers with the Insights JavaScript API.

Format the JSON

Here is an example of a typical JSON data set to send to Insights. This call sends two Purchase type events as a JSON array. You can insert multiple events in a single HTTP call using a JSON array.

[
  {
    "eventType":"Purchase",
    "account":3,
    "amount":259.54
  },
  {
    "eventType":"Purchase",
    "account":5,
    "amount":12309,
    "product":"Item"
  }
]

When generating the JSON, make sure your attributes are properly formatted.

Attributes JSON format guidelines
eventType Required. This is the Insights name for the event.
Float values Use the format "label":value.
String values Use the format "label":"value".
Data types Insights only accepts key-value pairs, not map/object or array values. Supported data types for this API are: strings and numbers (integers or floats). For more information, see Data requirements.
Digits in strings For performance-related reasons, Insights does not cast values submitted to it. For example, Insights treats 123 as a number and "123" as a string.
Dates For attributes that contain date information, use an unformatted Unix timestamp in the Insights Manage data > Data formatter. You can define the date attribute either in seconds or in milliseconds, both relative to the Unix epoch.

Specify different timestamp

Unless otherwise specified, the timestamp for a submitted event is the time it was submitted to New Relic. To specify a different time for the event, use the timestamp attribute. The timestamp must be within 24 hours of the time the event is submitted to New Relic.

The timestamp attribute takes an unformatted Unix timestamp. You can define timestamps either in seconds or in milliseconds, both relative to the Unix epoch.

Submit the event

Data is submitted to Insights in JSON format using a simple HTTP request. The Insert key page automatically generates a sample cURL query for you to use as a template:

cat example_events.json | curl -d @- -X
  POST -H "Content-Type: application/json" -H "X-Insert-Key: YOUR_KEY_HERE"
  https://insights-collector.newrelic.com/v1/accounts/YOUR_ACCOUNT_ID/events

When generating the HTTP request, make sure it is properly formatted. If any errors occur, all events in the request are rejected to prevent partial inserts. An error response includes the source of the error to aid debugging.

  • The X-Insert-Key must contain the correct Insert Key.
  • The Content-Type must be application/json.
  • Use POST only (PUT and GET requests are not accepted).

HTTP/1.1 Persistent connections are supported. This is helpful to manage client-side performance under heavy event loads.

Submission errors

Refer to these submission errors for troubleshooting.

Error definitions
Error message Additional info
400 Missing API key No account ID path token supplied
400 Unparseable API key Invalid insert key or the account does not have access to Insights
400 Invalid content type Invalid Content Type. Must be application/JSON. The ICEI accepts any content type except multipart/related and assumes it can be parsed to JSON.
400 JSON parsing error Error parsing JSON payload at event index X. (X reflects the index within the JSON body at which the error occurred.)
400 Missing event type Missing required field event. Type at event index X. (X reflects the index within the JSON body at which the error occurred.)
400 Invalid event data Missing required field event. Type at event index X. (X reflects the index within the JSON body at which the error occurred.)
400 Missing or invalid content length Unable to process empty request.
403 Invalid insert key. Register a valid insert key.
408 Request timed out. Request took too long to process.
413 Content too large Request is too large to process. Refer to the limits and restricted characters to troubleshoot.
429 Too many requests Unable to accept request at this time

Limits and restricted characters

Insights limits the size and rate of custom events:

  • Attributes: 255 maximum per event
  • String attributes: 4KB maximum length
  • Batch total count: 1000 events per call
  • Batch total size: 1MB maximum per call
  • API calls exceeding 10 seconds will timeout
  • Payloads exceeding 100KB may see increased response times

Use only the following characters when inserting a custom event:

  • accountId: Must be an integer.
  • appId: Must be an integer.
  • eventType: Can be a combination of alphanumeric characters, _ underscores, and : colons.
  • timestamp: Must be a 64-bit integer within +/- 1 day (24 hours) of the current time on the server.

Payload sizes

The Insights API for inserting custom events is designed for high-frequency event payload processing. However, the size of JSON payloads does affect response times. While the API supports payloads up to 1MB in size, smaller payloads may be better for your use case:

  • If you need to publish data at high speed, or if the process requires each upload to process in the smallest amount of time possible, it is better to send small payloads and send them more frequently.

  • If you send data infrequently, and do not need the upload to complete rapidly, it may be easier to build large payloads and wait for them to process.

The API does not reject payloads for size (unless they exceed the maximum limits), but payloads will be queued and processed in order that they are received.

NRQL syntax terms as attribute names

Before creating custom attributes, review New Relic's list of reserved terms used by NRQL and Insights. Otherwise unexpected results may occur.

For more help

Recommendations for learning more: