Use the Plugin API

For an even better experience than plugins, go to:

  • newrelic.com/integrations: Integrate the on-host and cloud systems you already use with New Relic, so you can filter and analyze data, create dashboards, and set alerts within a single platform.
  • developer.newrelic.com: Use developer tools to collect data from any source, automate workflows, build apps, and use our APIs.

This is a guide for plugin developers to get started with writing an agent in any language that can work directly with the Plugin API for Plugin Central.

Before you begin

Using a development language other than Ruby, .NET, or Java for a plugin agent means you do not have an SDK to work with, but you do have some benefits.

  • You can use any language you want, as long as it supports sending JSON through HTTP POST. This allows for better integration with your systems.
  • For the same reason, it is the best option for SaaS-based plugin agents.

However, if you are not using the Plugins SDK for Java. .NET, or Ruby, you have some additional setup and planning to do in developing a plugin agent. This includes:

  • Error tracking on POST calls
  • A method for tracking and aggregating data when a POST fails
  • Your own support plans if a New Relic SDK for your language or development tools is not available

Any publicly available plugins in the Plugin Central should come bundled with their source code if the executable code is not plain text. This allows you to both try out plugins and to review the code.

Recommendation: Before authoring a plugin, install some existing plugins using the Java SDK, .NET SDK, or Ruby SDK to see how they are written.

Metric data POST

Metric data is sent as an HTTP POST of JSON data using this URI:

https://platform-api.newrelic.com/platform/v1/metrics

The MIME-type for the POST is application/json.

The Plugins feature is designed to receive a continuous stream of metrics at a certain maximum speed, and to present this information on useful charts. The recommended frequency for sending data to Plugins is to send 60 seconds worth of data once a minute. Agents sending data more frequently than twice a minute on average may be subject to enforced limits on the number of metrics being saved.

The following are recommended soft limits. Requests smaller than this will work; requests larger than this are subject to rejection or automatic data aggregation. As a hard cap, the total size of the POST payload should be no larger than 1MB (10^6 bytes).

If the metric is "expensive" to calculate and does not change quickly, consider writing your plugin agent so that it skips some polling cycles to retrieve data and then sends the last value. This produces better results for your plugin users' dashboards.

Type Limit Description
Components 500 Number of distinct components currently tracked. Please note this is a per POST limit only. More than 500 components are able to report to an account simultaneously.
Metrics per component 10,000 Total number of unique metrics per component. Take precautions to ensure metric names are not generated too dynamically. Even if the number of metrics being sent in each individual post is small, over time they may add up to a large number of unique metrics. When the number of metrics for a given component exceeds this limit, the server may start aggregating metrics together by globbing segments of the metric name with an asterisk (*).
Metrics per post 20,000 Number of metrics sent per post. A post may send data for multiple components in a single request as long as the total number of metrics in the request does not exceed this limit.
Frequency of post 2 per minute Frequency of update. Agents are expected to send data no more frequently than 1 per minute.

Data aggregation

The SDKs manage data aggregation in the event of a failed POST. If you are not using an SDK, you need to manage this yourself.

  • Include all five metric values in a POST: min, max, total, count, and sum or squares. (Exception: This may not be necessary for monotonic metrics where short term variation is not an issue.)
  • Recompute these values for the accumulating metric data as required by what is being measured, incrementing the duration accordingly, until a successful POST is sent.

Compression

Data can be sent in the following encoding formats:

  • identity
  • deflate
  • gzip

If data is sent compressed, make sure the Content-Encoding header specifies the type of encoding.

Examples

Here are some examples for developing plugins.

Example JSON

This is an example of the JSON that would be used to POST data to Plugins. The JSON data is a hash with two required keys at the top level:

  • agent: A hash describing the agent that is reporting metrics data to Plugins. A POST can contain information for only one agent. Host and version are required.
  • components: An array of components, each consisting of a hash of attributes for the individual component. Multiple components can be sent with a single. Each component has its own name, GUID, duration, and metrics.
{
  "agent": {
    "host" : "db.internal.your_company.com",
    "pid" : 1234,
    "version" : "1.0.0"
  },
  "components": [
    {
      "name": "Primary MySQL Database",
      "guid": "com.your_company_name.plugin_name",
      "duration" : 60,
      "metrics" : {
        "Component/ProductionDatabase[Queries/Second]": 100,
        "Component/AnalyticsDatabase[Queries/Second]": {
          "min" : 2,
          "max" : 10,
          "total": 12,
          "count" : 2,
          "sum_of_squares" : 104
        }
      }
    }
  ]
}
Pseudo-code template

This is a pseudo-code example that works with the Plugin API. It can be used as a template for developing plugin agents.

Initialization:

// globals

string platform_api_uri = "https://platform-api.newrelic.com/platform/v1/metrics"
int poll_cycle = 60 // time in seconds
string version = "1.0.0" // major_version.minor_version.patch_level
string agent_host = get_host_name_where_this_process_is_running()
string agent_pid = get_process_id_of_this_process()
time last_poll_time // initialize if necessary

initialize once
  create agent_hash with:
    agent_host
    agent_pid
    version
  for each newrelic_account do // just handling one account? then "for each" is unnecessary complexity
    for each monitored_component do
      create component_hash with:
        string guid = "com.your_company.component_name_in_snake_case"
        string name ="Human Readable Component Name"
        int duration = 0 // this will get updated each poll_cycle
        hash metrics_hash // this will be updated by populate_component_metrics_hash()
    end
  end
end

Loop:

every poll_cycle seconds do
  for each newrelic_account do // just handling one account? then "for each" is unnecessary complexity
    clear hash_to_send
    add agent_hash to hash_to_send
    for each component do
      populate_component_metrics_hash()
      this component.metrics_hash("duration") = time.now() - last_poll_time in seconds
      add component.metrics_hash to hash_to_send
    end
    json_to_send = serialize_to_json(hash_to_send)
    connection = open http_connection(platform_api_uri)
    add header("X-License-Key",this newrelic_account.license_key) to connection
    add header("Content-Type","application/json") to connection
    add header("Accept","application/json") to connection
    set http_verb to "POST" for connection
    response = send(json_to_send) to connection
    case response.code
      when response_code = 200
        clear component.metrics_hash
        last_poll_time = time.now()
      when response_code = 400
        // your request was malformed
        // consider reporting a "supportability" metric which counts the number of 400 responses you get
        // for example "Component/Supportability/http_error_codes/400"
        // you can use this on a "Supportability" Dashboard that helps diagnose your agent
      when response_code = 403
        // forbidden probably due to a bad license key
        // log error and shutdown the agent
      when response_code = 404
        // invalid URL
        // you should never get this error for https://platform-api.newrelic.com/platform/v1/metrics
      when response_code = 405
        // invalid method
        // HTTP verb should be "POST"
      when response_code = 413
        // POST body too large
        // try splitting at component boundaries
        // split along metric name spaces
        // fail gracefully - consider reporting a supportability metric (see 400)
      when response_code = 500
        // error on New Relic's collector
        // could be due to malformed data or system trouble
        // fail gracefully - consider reporting a supportability metric (see 400)
      when response_code = 503 or 504
        // New Relic collector busy 
        //- this happens by design from time-to-time
        // keep collecting metrics
        // do NOT reset last_poll_time
        // log error if the problem persists for several minutes
    end case
  end
end

Metric population:

function populate_component_metrics_hash()
  // collect metrics from monitored component at any interval
  // if this is the first time collecting metrics, set last_poll_time to
  // time.now - metric duration, the time duration for which these metrics
  // were collected
  //
  // if you collect 2 or more metrics from the monitored component before data
  // is reported to Plugins either because your metric collection interval is
  // faster than poll_cycle or because your agent was unable to report metrics to
  // Plugins (for example a 503 http response), aggregate your data by storing:
  // total_value, max, min, count, sum_of_squares for each metric
  //
  // if the interval is longer than poll_cycle, retain the metrics and
  // report them each poll_cycle until they are updated
end

API responses and error codes

Depending on whether you are using the Plugin API or an agent SDK for plugins, the HTTP responses and logging techniques may be different. For example, responses for the Plugin API are uncompressed JSON. Successful posts return this JSON:

{"status":"ok"}

The API does not support Accept-Encoding.

Debugging logs

To debug information, use either of these options:

  public static Logger getLogger();

OR

  public static void Logger setLogger(Logger logger) {
  LOGGER = logger;
  }
Error codes

If an error occurs, an appropriate status code is returned. The JSON returned is the hash key error with a detailed description of the error that occurred. For example:

{"error":"Failed to create agent with parameters=[...]"}
     {"error":"Missing metric data"}
     {"error":"Unable to parse body: Unexpected token RIGHT BRACE(}) at position 228."}
Code Name Description
400 Bad request The request or headers are in the wrong format, or the URL is incorrect, or the GUID does not meet the validation requirements.
403 Unauthorized Authentication error (no license key header, or invalid license key).
404 Not found Invalid URL.
405 Method not allowed Returned if the method is an invalid or unexpected type (GET/POST/PUT/etc.).
413 Request entity too large Too many metrics were sent in one request, or too many components (instances) were specified in one request, or other single-request limits were reached.
500 Internal server error Unexpected server error.
502 Bad gateway All 50X errors mean there is a transient problem in the server completing requests, and no data has been retained. Clients are expected to resend the data after waiting one minute. The data should be aggregated appropriately, combining multiple timeslice data values for the same metric into a single aggregate timeslice data value.
503 Service unavailable See 502 description.
504 Gateway timeout See 502 description.

For more help

If you need more help, check out these support and learning resources: