StatsD monitoring integration (version 1)

Our StatsD integration lets you send data of your choosing to the New Relic platform, where it can be analyzed, used to create queries and custom charts, and used as a basis for alert conditions.

The diagram shows the basics of how the StatsD integration sends data to New Relic.

New Relic StatsD integration diagram
Your app or components send data to StatsD. The New Relic backend takes the aggregated StatsD metric data and sends it to the infrastructure agent using socket communication. Finally, the agent sends the data to the New Relic collector.

Read on to install the integration, and to see what data we collect.

Requirements

Before installing the integration, make sure that you meet the following requirements:

Install and activate

Note that we have a newer StatsD integration.

To activate the New Relic StatsD integration:

  1. Ensure that you have the most up-to-date version of the infrastructure agent.
  2. Edit the infrastructure agent config file to enable the HTTP endpoint by adding:

    http_server_enabled: true		
    http_server_host: 127.0.0.1 #(default host)
    http_server_port: 8001 #(default port)
    

    This is the default URL and port; you can edit them based on your preferences.

  3. Restart the agent.
  4. Install the New Relic .js backend in the StatsD backends directory. Example:

    cd /path/to/statsd
    npm install @newrelic/statsd-infra-backend
    
  5. Configure StatsD to run the New Relic StatsD backend by adding @newrelic/statsd-infra-backend to the StatsD config.js list of backends, like so:

    backends: ["@newrelic/statsd-infra-backend"]
    

    Recommendation: Before doing further configuration, finish this installation process and then verify that integration is working.

  6. Restart StatsD.

Additional notes:

Verify that the integration is working

Before you configure the StatsD config file, verify that the integration is capable of sending data:

  1. Use the example StatsD config file in place of your StatsD configuration file.
  2. Tell the config file to send data with this terminal command:

    echo "app1.production.localhost.sample_metric:2000|c" | nc -w 1 -u 127.0.0.1 8125
    
  3. Wait a few minutes, then run the following New Relic Insights query to confirm the data was reported:

    Select * from MyorgApplicationSample
    

Configure the integration

After installation and verification, you can further configure the StatsD config file. The config file accepts the following parameters. For more information, see the example config files.

Parameter Description
port The port where the infrastructure agent is listening for StatsD data, as configured in /etc/newrelic-infra.yml.
host The IP address of the parent host containing the New Relic infrastructure agent.
rules

An array of rules to filter metrics sent by StatsD to the New Relic StatsD backend. Using rules allows you to be specific about which metrics to record and how they should be sent to New Relic. Each rule can include one or more of these parameters:

The eventType parameter is required in order to query the metrics in Insights.

matchExpression

Report only metrics matching a JavaScript regular expression.

metricSchema

The schema to be matched to the regular expression. This allows you to use parts of the metric name received by StatsD to define the metric name in New Relic and its associated metadata.

The elements defined in the schema, which are inherited from the metricExpression, can be used in other parameters. For example, in the config file example, {hostname} is used to represent one of the parameters. If you used {hostname} in another parameter, it would substitute the detected value, in this case localhost.

eventType (naming data)

The name of the event type sample where all metrics matching this rule will be stored. New Relic strongly recommends following the convention below when naming your event types:

  • Use camel case.
  • Use the name of your organization as a prefix. This is to ensure the event type you use is unique. More detail on this below.
  • Use a name that clearly identifies what data it contains.

Examples: MyorgNginxStatsdSample, MyorgApplicationStatsdSample.

The organizational prefix is important to avoid collision with New Relic's default event types. For example, if you named your event type NginxSample, you would be mixing your data with the standard Nginx data generated by the out-of-the-box New Relic integration. This would cause confusion to your users and create some unexpected results when running queries.

For more information, see Naming tips.

labels

A hash containing a list of key-value pairs that will be applied to metrics matching this the rule.

Labels will be stored in a namespace named label in New Relic; for example: label.environment = "production".

You can use variables defined in the metricSchema as values for your labels.

Find and use data

To use your integration data in Infrastructure, go to infrastructure.newrelic.com > Third-party services and select one of the StatsD integration links.

In New Relic Insights, StatsD integration data is attached to the MyorgApplicationSample event type (assuming you've followed the event naming conventions).

For more on how to find and use your data, see Understand integration data.

Tips for naming your data

Metrics are sent and stored in New Relic in the form of samples. This is a list of key-value pairs that include metric data and metadata. Each sample is stored as an event in New Relic's event database.

You are responsible for creating and naming the StatsD data reported to New Relic. For this reason, it's recommended you follow the event naming conventions to ensure you have a consistent naming scheme.

It's also recommended you use the same naming scheme for similar metrics across different applications. For example: suppose you have an application named app1 and app2 that reports the following metrics:

App1.net.requests
App1.net.average_response_time
App1.system.memory_used
App2.net.requests
App2.net.average_response_time
app2.system.memory_used

By giving each app a different name but the same metrics, you could generate a sample for each similar to:

application: "app1"
net.requests: 234
net.average_reponse_time: 23
application: "app2"
net.requests: 234
net.average_reponse_time: 23

Example StatsD configuration files

The first example StatsD configuration file in this section can be used for doing an initial test of the integration after installation. The second config file is more complete and complex, and should give you an idea of how your own StatsD config file will look after configuration.

Simple example config file

This example config file is used for install verification.

{
  debug: false,
  port: 8125,
  backends: [ "@newrelic/statsd-infra-backend" ],
  newrelic: {
    port: 8001,
    host: "localhost",
    rules: [
            {
        matchExpression: "app1.production.localhost.sample_metric",
        metricSchema: "{app}.{environment}.{hostname}.{metricName}",
        eventType: "MyorgApplicationSample",
        labels: {
          role: "test",
          environment: "{environment}"
        }
      },
    ]
  }
}
Complete example config file

This example should give you an idea of how your own StatsD config file will look after configuration.

{
  debug: false,
  graphitePort: 2003,
  graphiteHost: "localhost",
  port: 8125,
  backends: [ "@newrelic/statsd-infra-backend" ],
  newrelic: {
    port: 8001,
    host: "localhost",
    rules: [
      // statsd metric keys:
      //  - myapp.production.redis.redisA.replication.connected_slaves
      //  - myapp.staging.redis.redisA.replication.connected_slaves
      // schema substitution:
      //  - {app: "myapp", environment: "production", service: "redis", serviceName: "redisA", metricName: "replication.connected_slaves"}
      //  - {app: "myapp", environment: "staging", service: "redis", serviceName: "redisA", metricName: "replication.connected_slaves"}
      {
        matchExpression: "myapp.*redis.*",
        metricSchema: "{app}.{environment}.{service}.{serviceName}.{metricName}",
        eventType: "MyOrgRedisStatsdSample",
        labels: {
          role: "cache",
          environment: "{environment}"
        }
      },
      // statsd metric keys:
      //  - mywebsite.staging.myhostname.connections.accepted
      // schema substitution:
      //  - {app: "mywebsite", environment: "staging", hostname: "myhostname", notimportant: "connections", metricName: "accepted"}
      {
        matchExpression: "mywebsite.*",
        metricSchema: "{app}.{environment}.{hostname}.{notimportant}.{metricName}",
        eventType: "MyOrgNginxStatsdSample",
        labels: {
          role: "webserver",
          environment: "{environment}"
        }
      },
    ]
  }
}

For more help

If you need more help, check out these support and learning resources: