• /
  • Log in

On-host integration executable file: JSON specifications

When using our Integrations SDK for infrastructure monitoring to build a custom on-host integration, the integration will consist of at least three files: an executable file and at least one configuration file. The executable file generates JSON data that is consumed by the infrastructure monitoring agent and sent to New Relic. We refer to the JSON object as the SDK integration protocol.

Executable file requirements

The executable can be any file that runs from a command-line interface; for example:

  • A shell script
  • A scripting language script
  • A compiled binary

The only requirement of your executable file is that it exports JSON data, in a single line format, that meets the specifications in this document.

Recommendation: Use Go to create integrations; it's the language we use to create on-host integrations and the integration building tools. However, you can create an integration in any language.

File placement

The executable file goes in this directory:

  • Linux:

    /var/db/newrelic-infra/custom-integrations
  • Windows:

    C:\Program Files\New Relic\newrelic-infra\newrelic-integrations

Integration protocol v4: Example JSON output

The following section explains the new JSON schema (integration protocol v4). The SDK v4 only supports this new protocol version. These are the most important changes:

  • A new integration object at the top level.
  • The entity and metrics objects have been modified.

See the v3 to v4 migration guide for more information.

{
"protocol_version":"4", # protocol version number
"integration":{ # this data will be added to all metrics and events as attributes,
# and also sent as inventory
"name":"integration name",
"version":"integration version"
},
"data":[ # List of objects containing entities, metrics, events and inventory
{
"entity":{ # this object is optional. If it's not provided, then the Entity will get
# the same entity ID as the agent that executes the integration.
"name":"redis:192.168.100.200:1234", # unique entity name per customer account
"type":"RedisInstance", # entity's category
"displayName":"my redis instance", # human readable name
"metadata":{} # can hold general metadata or tags. Both are key-value pairs that will
# be also added as attributes to all metrics and events
},
"metrics":[ # list of metrics using the dimensional metric format
{
"name":"redis.metric1",
"type":"count", # gauge, count, summary, cumulative-count, rate or cumulative-rate
"value":93,
"attributes":{} # set of key-value pairs that define the dimensions of the metric
}
],
"inventory":{...}, # Inventory remains the same
"events":[...] # Events remain the same
}
]
}

Integration protocol v3: Example JSON output

The JSON includes:

  • A header, with basic integration data (name, version)
  • A data list, which includes one or more entities reporting data (metric, inventory, and/or event data)

This diagram shows this structure:

New Relic Integrations SDK data structure diagram

Here is an example JSON output (formatted with line breaks for readability). Definitions and specifications follow this example:

{
"name": "my.company.integration",
"protocol_version": "3",
"integration_version": "x.y.z",
"data": [
{
"entity": {
"name": "my_garage",
"type": "building",
"id_attributes": [
{
"key": "environment",
"value": "production"
},
{
"key": "node",
"value": "master"
}
]
},
"metrics": [
{
"temperature": 25.3,
"humidity": 0.45,
"displayName": "my_garage",
"entityName": "building:my_garage",
"event_type": "BuildingStatus"
}
],
"inventory": {
"out_door": {
"status": "open"
}
},
"events": []
},
{
"entity": {
"name": "my_family_car",
"type": "car",
"id_attributes": [
{
"key": "environment",
"value": "production"
},
{
"key": "node",
"value": "master"
}
]
},
"metrics": [
{
"speed": 95,
"fuel": 768,
"displayName": "my_family_car",
"entityName": "car:my_family_car",
"event_type": "VehicleStatus"
}
],
"inventory": {
"motor": {
"brand": "renault",
"cc": 1800
}
},
"events": [
{
"category": "gear",
"summary": "gear has been changed"
}
],
"add_hostname": true
}
]
}

JSON: General specifications

Here are general specifications for the JSON output:

JSON: Header

Here's an example of the first part of an on-host integration's JSON output:

"name":"com.myorg.nginx",
"protocol_version":"3",
"integration_version":"1.0.0",
"data": [ {entities}...]

A minimal payload would be a JSON object with only the header fields. Recommendation: If there is no data to collect, use the program return code and log messages written to stderr.

JSON header fields

Description

name

Required. Must be identical to the name field in the configuration file.

Recommendation: Use reverse domain names to generate unique integration names.

protocol_version

Required. The version number of the exchange protocol between the integration and the agent that the integration executable is using.

  • The current version is 3. This protocol requires Infrastructure agent 1.2.25 or higher.

  • Protocol 2 requires Infrastructure agent 1.0.859 or higher.

  • Protocol 1 is compatible with all agents.

    For more information, see SDK changes.

integration_version

Optional. The integration version. Used to track the integration version running on each host.

An integration can have more than one executable. Therefore this is not simply the executable's version.

data

Required for reporting data. A list containing the data reported from one or more entities.

JSON: Entities

Inside the data list of the JSON output are one or more entities. The entity entry fields include:

Entity JSON fields

Description

entity

Required. Entity data or properties.

metrics

Optional. Entity related metric list.

inventory

Optional. Entity related inventory items.

events

Optional. Entity related event list.

add_hostname

Optional. Boolean. If true, the entity metrics will be decorated with the hostname.

Inside the data list of the JSON output are one or more entities and their data. The entity entry has two fields:

Entity data JSON fields

Description

name

Required. The identifier/name of the entity.

Recommendation: Use reverse domain names to generate unique integration names.

type

Required. The kind of entity. It will be used by the Infrastructure agent as a namespace to compose a unique identifier in conjunction with the name.

id_attributes

Optional. A list of key-value attributes that provide uniqueness to an entity. They are attached to the name in the form of key=value to ease readability, provide extra information, and improve entity name uniqueness.

Identifier attributes are useful when the entity name is not enough to work as a unique identifier, or when it doesn't provide enough meaningful information. For example:

[
{
"key": "service",
"value": "mysql"
},
{
"key": "role",
"value": "master"
}, ...
]

Loopback address replacement on entity names

As of Infrastructure agent version 1.2.25 or higher, protocol v3 improves remote entities uniqueness by adding local address replacement on entity names at agent level.

When several remote entities have their name based on an endpoint (either ip or hostname), and this name contains loopback addresses, there are two problems:

  • This localhost value does not provide valuable info without more context.
  • The name could collide with other service being named with a local address.

This happens when:

  • Endpoints names are like localhost:port.
  • Ports tend to be the same for a given service; for example, 3306 for Mysql.

On incoming protocol v3 data, the Infrastructure agent replaces loopback addresses on the entity name (and key) with the first available item of the following list:

  1. Cloud provider instance ID, retrieved by the agent if applicable
  2. Display name, set via the display_name agent config option
  3. Hostname, as retrieved by the agent

For example, if an integration using protocol v3 returns an entity with the name localhost:3306, and the agent is running on bare metal (doesn’t have cloud provider instance ID), the display_name has not been set, and the hostname is prod-mysql-01, then the agent will replace the localhost and produce the entity name prod-mysql-01:3306.

The Infrastructure agent enables loopback address replacement automatically for v3 integration protocol. You can also enable this for v2 via the agent configuration flag replace_v2_loopback_entity_names. In this case all the integrations being run by the agent using v2 will have their names replaced whenever they carry a local address.

JSON: Metric, inventory, and event data

Data values follow the executable file header. You can record three data types:

Important

From the perspective of New Relic Dashboards, the infrastructure metrics and events are both classified as event data types. To find both metrics and events, use the Insights Event data explorer, not the Metric data explorer.

For more help

If you need more help, check out these support and learning resources:

Create issueEdit page
Copyright © 2021 New Relic Inc.