Work directly with the Plugin API

This is a guide for developers to get started with writing an agent in any language that can work directly with the API for New Relic Plugins.

Before you begin

Using a development language other than Ruby or Java for a plugin agent means you do not have an SDK to work with, but you do have some distinct benefits.

  • You can use any language you want, as long as it supports sending JSON through HTTP POST. This allows for better integration with your systems.
  • For the same reason, it is the best option for SaaS-based plugin agents.

However, if you are not using the New Relic Plugins SDK for Java. .NET, or Ruby, you have some additional setup and planning to do in developing a plugin agent. This includes:

  • Error tracking on POST calls to New Relic Plugins.
  • A method for tracking and aggregating data when a POST fails.
  • Your own support plans: If New Relic Plugins has not developed an SDK for your language or development tools, less support is available.

New Relic recommends that any publicly available plugins in the Plugin Central directory for New Relic Plugins should come bundled with their source code if the executable code is not plain text. This allows you to both try out plugins and to review the code.

Recommendation: Before authoring a plugin, install some existing plugins using the Java SDK, .NET SDK, or Ruby SDK for New Relic Plugins to see how they are written.

Metric data POST

Metric data is sent as an HTTP POST of JSON data using this URI:

https://platform-api.newrelic.com/platform/v1/metrics

The MIME-type for the POST is application/json.

The New Relic Plugins product is optimized to post the data at a frequency of once every 60 seconds.

Data aggregation

The SDKs manage data aggregation in the event of a failed POST. If you are not using an SDK, you need to manage this yourself. Here is the recommended approach:

  • Include all five metric values in a POST: min, max, total, count, and sum or squares. (Exception: This may not be necessary for monotonic metrics where short term variation is not an issue.)
  • Recompute these values for the accumulating metric data as required by what is being measured, incrementing the duration accordingly, until a successful POST is sent.

Example JSON

This is an example of the JSON that would be used to POST data to New Relic Plugins. The JSON data is a hash with two required keys at the top level:

  • agent: A hash describing the agent that is reporting metrics data to New Relic Plugins. A POST can contain information for only one agent. Host and version are required.
  • components: An array of components, each consisting of a hash of attributes for the individual component. Multiple components can be sent with a single. Each component has its own name, GUID, duration, and metrics.
{
  "agent": {
    "host" : "db.internal.your_company.com",
    "pid" : 1234,
    "version" : "1.0.0"
  },
  "components": [
    {
      "name": "Primary MySQL Database",
      "guid": "com.your_company_name.plugin_name",
      "duration" : 60,
      "metrics" : {
        "Component/ProductionDatabase[Queries/Second]": 100,
        "Component/AnalyticsDatabase[Queries/Second]": {
          "min" : 2,
          "max" : 10,
          "total": 12,
          "count" : 2,
          "sum_of_squares" : 104
        }
      }
    }
  ]
}

Pseudo-code template

This is a pseudo-code example that works with the Plugin API. It can be used as a template for developing plugin agents.

Initialization

// globals

string platform_api_uri = "https://platform-api.newrelic.com/platform/v1/metrics"
int poll_cycle = 60 // time in seconds
string version = "1.0.0" // major_version.minor_version.patch_level
string agent_host = get_host_name_where_this_process_is_running()
string agent_pid = get_process_id_of_this_process()
time last_poll_time // initialize if necessary

initialize once
  create agent_hash with:
    agent_host
    agent_pid
    version
  for each newrelic_account do // just handling one account? then "for each" is unnecessary complexity
    for each monitored_component do
      create component_hash with:
        string guid = "com.your_company.component_name_in_snake_case"
        string name ="Human Readable Component Name"
        int duration = 0 // this will get updated each poll_cycle
        hash metrics_hash // this will be updated by populate_component_metrics_hash()
    end
  end
end

Loop

every poll_cycle seconds do
  for each newrelic_account do // just handling one account? then "for each" is unnecessary complexity
    clear hash_to_send
    add agent_hash to hash_to_send
    for each component do
      populate_component_metrics_hash()
      this component.metrics_hash("duration") = time.now() - last_poll_time in seconds
      add component.metrics_hash to hash_to_send
    end
    json_to_send = serialize_to_json(hash_to_send)
    connection = open http_connection(platform_api_uri)
    add header("X-License-Key",this newrelic_account.license_key) to connection
    add header("Content-Type","application/json") to connection
    add header("Accept","application/json") to connection
    set http_verb to "POST" for connection
    response = send(json_to_send) to connection
    case response.code
      when response_code = 200
        clear component.metrics_hash
        last_poll_time = time.now()
      when response_code = 400
        // your request was malformed
        // consider reporting a "supportability" metric which counts the number of 400 responses you get
        // for example "Component/Supportability/http_error_codes/400"
        // you can use this on a "Supportability" Dashboard that helps diagnose your agent
      when response_code = 403
        // forbidden probably due to a bad license key
        // log error and shutdown the agent
      when response_code = 404
        // invalid URL
        // you should never get this error for https://platform-api.newrelic.com/platform/v1/metrics
      when response_code = 405
        // invalid method
        // HTTP verb should be "POST"
      when response_code = 413
        // POST body too large
        // try splitting at component boundaries
        // split along metric name spaces
        // fail gracefully - consider reporting a supportability metric (see 400)
      when response_code = 500
        // error on New Relic's collector
        // could be due to malformed data or system trouble
        // fail gracefully - consider reporting a supportability metric (see 400)
      when response_code = 503 or 504
        // New Relic collector busy 
        //- this happens by design from time-to-time
        // keep collecting metrics
        // do NOT reset last_poll_time
        // log error if the problem persists for several minutes
    end case
  end
end

Metric population

function populate_component_metrics_hash()
  // collect metrics from monitored component at any interval
  // if this is the first time collecting metrics, set last_poll_time to
  // time.now - metric duration, the time duration for which these metrics
  // were collected
  //
  // if you collect 2 or more metrics from the monitored component before data
  // is reported to New Relic Plugins either because your metric collection interval is
  // faster than poll_cycle or because your agent was unable to report metrics to
  // New Relic Plugins (for example a 503 http response), aggregate your data by storing:
  // total_value, max, min, count, sum_of_squares for each metric
  //
  // if the invterval is longer than poll_cycle, retain the metrics and
  // report them each poll_cycle until they are updated
end

For more help

Additional documentation resources include:

Recommendations for learning more: