Introduction to the Log API

New Relic offers a Log API that lets you send log data directly to New Relic Logs via an HTTP endpoint. This document explains how to enable this feature.

Compatibility and requirements

To use New Relic Logs with the HTTP integration, ensure your configuration meets the following requirements:

HTTP setup

To send log data to your New Relic account:

  1. Get your New Relic license key (recommended) or register an Insert API key.
  2. Generate the JSON message using the required headers and body fields.
  3. Submit the JSON message to the HTTP endpoint in a POST request.
  4. Generate some traffic and wait a few minutes, then check your account for data.

JSON headers

When creating your JSON headers, follow these format guidelines:

Header Supported values

Content-Type

Required

  • application/json
  • json
  • application/gzip
  • gzip

Exactly one of:

X-Insert-Key

or

X-License-Key

Required

Use either your New Relic Insert API key (with X-Insert-Key) or license key (with X-License-Key).

Gzipped JSON formatting is accepted. If sending compressed JSON, please include the application/gzip header.

JSON body

You can send your JSON message using either a simplified or detailed set of attributes:

Simplified JSON body message

When using the simplified format to create your JSON message, send a single JSON object with the following:

Field Value type Format Required Notes
"timestamp" Number Either milliseconds or seconds since epoch No If the field is not specific as millisecond or seconds since epoch, the message will be timestamped using the ingest time.
"message" String any string No This is the main log message field that is searched by default.
any other field String any string No These will become attributes of the log message.
Detailed JSON body message

When using the detailed format to create your JSON body, the body must be a JSON array of blocks, each block containing an array of logs messages and an optional common block.

Field Value type Format Required Notes
"common" Object See common. No Any attributes that are common to all log messages.
"logs" Array See logs. Yes

JSON message attributes

Common block attributes

This is a block containing attributes that will be common to all log entries in logs:

Field Value type Format Required Notes
"timestamp" Number Milliseconds or seconds since epoch No Message timestamp default to ingest time.
"message" String (any string) No This is the main log message field that is searched by default.
"attributes" Object JSON No This sub-object contains all other attributes of the message.
Logs block attributes

This is an array containing log entries with the following format:

Field Value type Format Required Notes
"timestamp" Number Milliseconds or seconds since epoch No Message timestamp default to ingest time.
"attributes" Object JSON No This sub-object contains all other attributes of the message.
"message" String (any string) Yes This is the main log message field that is searched by default.
"log" String (any string) No We will rewrite this string as the field message on ingest.
"LOG" String (any string) No We will rewrite this string as the field message on ingest.
"MESSAGE" String (any string) No We will rewrite this string as the field message on ingest.

JSON message attribute parsing

This will attempt to parse any message attribute as JSON. If the message attribute is JSON, it will be parsed and the resultant JSON attributes will be added to the event. If the message attribute is not JSON, it is left as is.

For example, the event:

{   
    "timestamp": 1562767499238,   
    "message": "{\"service-name\": \"login-service\", \"user\": {\"id\": 123, \"name\": \"alice\"}}"  
}

Will be treated as:

{  
  "timestamp": 1562767499238,   
  "message": "{\"service-name\": \"my-service\", \"user\": {\"id\": 123, \"name\": \"alice\"}}",    
  "service-name": "login-service",  
  "user": { 
    "id": 123,  
    "name": "alice" 
  } 
}

Log JSON example

Attributes may be scalar JSON types like string and number, but may also be compound (or nested) objects. Compound attributes will have their leaf attributes stored with flattened names.

For instance, a compound user attribute in a log entry's attributes:

"attributes": {
    "action": "login",
    "user": {
        "id": 123,
        "name": "alice"
    }
}

will result in the following attributes being stored with the log event:

Attribute Value
"action" "login"
"user.id" 123
"user.name" "alice"

Log POST example

Log POST message example:

POST /log/v1 HTTP/1.1
Host: log-api.newrelic.com
Content-Type: application/json
X-Insert-Key: 
Accept: */*
Content-Length: 319
[{
   "common": [{
     "attributes": {
       "service": "login-service",
       "hostname": "login.example.com"
     }
   }]
   "logs": [{
     "timestamp": 1550086450124,
     "message": "User 'xyz' logged in"
   },{
     "timestamp": 1550086451000,
     "message": "User 'xyz' logged out",
     "attributes": {
       "auditId": 123
     }
   }]
}]

The above POST message would result in the following log messages being stored in New Relic logging:

Example of stored common block attributes:

Attribute Value
"timestamp" 1550086450124
"message" "User 'xyz' logged in"
"service" "login-service"
"hostname" "login.example.com"

Example of stored logs block attributes example:

Attribute Value
"timestamp" 1550086450124
"message" "User 'xyz' logged out"
"auditId" 123
"service" "login-service"
"hostname" "login.example.com"

HTTP endpoint

Once configured, your JSON data can be sent to the following endpoint in a POST request:

US endpoint:

https://log-api.newrelic.com/log/v1

EU endpoint:

https://log-api.eu.newrelic.com/log/v1

Here is an example of a JSON POST request:

   POST /log/v1 HTTP/1.1
   Host: log-api.newrelic.com
   Content-Type: application/json
   X-Insert-Key: <YOUR_INSIGHTS_INSERT_KEY>
   Accept: */*
   Content-Length: 133
   {
     "timestamp": 1550086450124,
     "message": "User 'xyz' logged in",
     "service": "login-service",
     "hostname": "login.example.com"
   }

View log data

If everything is configured correctly and your data is being reported, you should see data logs in the New Relic Logs UI or by going to Insights and querying:

SELECT * FROM Log

What's next?

Now that you've enabled Logs, here are some potential next steps:

For more help

Recommendations for learning more: