Build your own integration: New Relic Flex

New Relic provides integrations and quickstarts for many popular services and frameworks. If you're already using New Relic and want to report data from a service for which we don't have an integration, you can create your own integration by following these options:

With , you can use our lightweight Flex tool (recommended) or build a complete on-host integration using our Integrations SDK.
For telemetry (metrics, traces) monitoring solutions, use our Telemetry SDKs.
Build a custom New Relic app that uses your own JavaScript UI functionality.

What is New Relic Flex?

Flex is an application-agnostic, all-in-one New Relic integration that allows you to collect events and metrics from a wide range of services. It's bundled with our infrastructure agent. You can instrument any application that exposes metrics over a standard protocol (HTTP, file, shell) in a standard format (for example, JSON or plain text): you create a YAML configuration file, start the infrastructure agent, and your data is reported to New Relic.

After collecting and cleaning up the data, you can then query Flex data in New Relic, create custom charts for it, and use that data in your dashboards.

Check the compatibility and requirements

Make sure your system meets these requirements:

Sign up for a free account if you haven't already. It's free!
The New Relic account is compatible with these operating systems and platforms:
- Kubernetes
- Linux
- macOS
- Windows
See our requirements for the infrastructure agent doc to make sure your system and any on-host integrations you configure meet the requirements.
Flex comes bundled with our infrastructure agent version 1.10.7 or higher running on Linux, Windows, or Kubernetes.

See the identify outdated agent versions from the UI document to check your version or the update the infrastructure agent document if you need to update it.

Installation

Follow these steps to install New Relic Flex:

Install the infrastructure agent

Starting from New Relic infrastructure agent version 1.10.7, Flex comes bundled with the agent. To install the infrastructure agent, see:

Install the infrastructure agent for Linux
Install the infrastructure agent for Windows
Tip
The agent must run in the root/administrator mode. You can start, stop, and restart the infrastructure agent from the command line.

Check that Flex is up and running

Follow these steps:

Navigate to the integrations folder of the Infrastructure agent:
- For Linux: /etc/newrelic-infra/integrations.d
- For Windows: C:\Program Files\New Relic\newrelic-infra\integrations.d\
Create the integration configuration file. For example, integrations.yml, if it doesn't exist.
Add the Flex configuration to the file:
```
integrations:
    - name: nri-flex
      config:
        name: just-testing
```
If you already have an integrations section in the file, add nri-flex to it.
After a few minutes, go to one.newrelic.com > All capabilities > Query your data and run this query:
```
FROM flexStatusSample
SELECT *
LIMIT 1
```
The query should give a table similar to this:
Go to one.newrelic.com > All capabilities > Query your data, add your query in the query builder, and click Run.
Tip
If you don't get anything, make sure that your YAML configuration file is well indented and that indentation levels don't use tabs instead of spaces. You can use a YAML validator, such as YAML Lint

Your first Flex integration

This example shows how to collect disk metrics from file systems not natively supported by New Relic using the df command in Linux.

The goal is to process the output of the df command, showing the file system and 1-byte blocks, while excluding file systems already supported by the agent. If unsupported file systems are not mounted, remove the -x arguments.

$ df -PT -B1 -x tmpfs -x xfs -x vxfs -x btrfs -x ext -x ext2 -x ext3 -x ext4
Filesystem     Type         1-blocks         Used    Available Capacity Mounted on
devtmpfs       devtmpfs    246296576            0    246296576       0% /dev
go_src         vboxsf   499963170816 361339486208 138623684608      73% /go/src

You need to convert the above tabular text output into a set of equivalent JSON samples with the following format. Notice that the agent decorates each sample with extra fields:

{
  "event": {
    "event_type": "FileSystemSample",
    "fs": "go_src",
    "fsType": "vboxsf",
    "capacityBytes": 499963170816,
    "usedBytes": 361345331200,
    "availableBytes": 138617839616,
    "usedPerc": 73,
    "mountedOn": "/go/src"
  }
}

First, you need to tell Flex how to perform the above table text to JSON transformation by specifying the following:

Name of the metric: FileSystem
Which command to run: df -PT -B1 ...
How to split the output table from df
How to assign the values to given metric names

This is achieved by placing the content below in the YAML configuration file:

integrations:
    - name: nri-flex
      config:
        name: linuxFileSystemIntegration
        apis:
          - name: FileSystem
            commands:
              - run: 'df -PT -B1 -x tmpfs -x xfs -x vxfs -x btrfs -x ext -x ext2 -x ext3 -x ext4'
                split: horizontal
                split_by: \s+
                row_start: 1
                set_header: [fs,fsType,capacityBytes,usedBytes,availableBytes,usedPerc,mountedOn]
            perc_to_decimal: true

apis is an array of entries for each sample. Each entry sets a name for the sample, as well as the commands and procedures to get and process the sample. The first entry in the example is named FileSystem, which is used to name the FileSystemSample event.
commands specifies how to get the information from CLI applications:
- run: 'df -PT -B1... specifies the command to run.
- split: horizontal states that each output line may return a metric.
- split_by explains how to split each line in different fields. In this case, we use the \s+ regular expression, which tells Flex that any sequence of one or more white spaces is a separator.
- row_start specifies that data starts right after the first row (which is 0).
- set_header specifies, in order, a matching name for each value of the aforementioned array.
- perc_to_decimal: true indicates to convert any percentage string into a decimal value, removing the trailing % symbol.

Once you've created the Flex configuration, the infrastructure agent autodetects the new configuration and begins collecting data. To check that your new integration is working, run this query:

FROM FileSystemSample 
SELECT mountedOn, fs, usedBytes, capacityBytes, usedBytes

The query should give a table similar to this:

Go to one.newrelic.com > All capabilities > Query your data, add your query in the query builder, and click Run.

How to add more Flex integrations

You can add more Flex integrations by adding the configuration in the ìntegrations.d file. Stand-alone Flex configurations start with the name of the integration and you can test them by invoking Flex from the command line:

bash

$sudo /var/db/newrelic-infra/newrelic-integrations/bin/nri-flex --verbose --pretty --config_file ./myconfig.yml

For example, if you want to add this integration:

name: linuxOpenFD
apis:
  - name: linuxOpenFD
    commands:
      - run: cat /proc/sys/fs/file-nr | awk '{print $1-$2,$3}'
        split: horizontal
        set_header: [openFD,maxFD]
        regex_match: true
        split_by: (\d+)\s+(.*)

You should open the ìntegrations.d file and add it like this:

integrations:
  - name: nri-flex
    config:
      name: linuxFileSystemIntegration
      apis:
        - name: FileSystem
          commands:
            - run: 'df -PT -B1 -x tmpfs -x xfs -x vxfs -x btrfs -x ext -x ext2 -x ext3 -x ext4'
              split: horizontal
              split_by: \s+
              row_start: 1
              set_header: [fs,fsType,capacityBytes,usedBytes,availableBytes,usedPerc,mountedOn]
          perc_to_decimal: true
          
        - name: linuxOpenFD
          commands:
            - run: cat /proc/sys/fs/file-nr | awk '{print $1-$2,$3}'
              split: horizontal
              set_header: [openFD,maxFD]
              regex_match: true
              split_by: (\d+)\s+(.*)

If you need to add multiple Flex configuration to the ìntegrations.d file, follow this pattern:

integrations:
  - name: nri-flex
    config:
      name: flexName_1
      # Flex config goes here
  - name: nri-flex
    config:
      name: flexName_2
      # Flex config goes here
  - name: nri-flex
    config:
      name: flexName_3
      # Flex config goes here

To minimize indentation issues, you can link to stand-alone Flex configuration files using the config_template_path directive:

integrations:
  - name: nri-flex
    config_template_path: /path/to/flex/integration.yml

You can find a lot of examples of custom integrations in the Flex repository.

Flex and Kubernetes

There are 3 container images you can use, depending on how you want to configure Flex in Kubernetes:

To run Flex only to monitor services running in Kubernetes, use the newrelic/infrastructure container image. This image only contains the infrastructure agent, and the Docker and Flex integrations. With this option, you will can't perform service discovery or use other New Relic integrations.
To run Flex alongside other New Relic integrations, use the newrelic/infrastructure-bundle container image. This adds all the other New Relic integrations.
If you also want to monitor your Kubernetes cluster, use the newrelic/infrastructure-k8s container image. This image adds all the integrations, including the Kubernetes integration.

Important

If you're running services in Kubernetes, we recommend you use the official container images from New Relic. See Introduction to the Kubernetes integration for more information.

Configure Flex in Kubernetes

After installing the Kubernetes integration, you'll have the infrastructure agent running in your cluster as well as these 2 configMap:

nri-default-integration-cfg: This is a configMap used to enable the New Relic Kubernetes integration. You can remove it if you don't to use this integration. If you've installed Kubernetes with the Helm command, the value integrations_config needs to be populated. See the New Relic's Helm charts repository for more information.
nri-integration-cfg-example: This is a configMap used to enable Flex and other New Relic integration.

To enable Flex, create a data section in the configMap, and add the infrastructure agent integration configuration under this new section:

apiVersion: v1
kind: ConfigMap
metadata:
  name: nri-integration-cfg-example
  namespace: default
data:
  nri-flex.yml: |
    integrations:
      - name: nri-flex
        config:
          name: example
          apis:
            - event_type: ExampleSample
              url: https://my-host:8443/admin/metrics.json

Sample configurations

Looking for different samples? Here are some example configurations to help you start with various data sources:

Databases:
Third-party APIs:
- JSON API
- Twilio health status
Command line utilities:

Troubleshooting

If you encounter an issue with the Flex configuration, you can follow these basic troubleshooting steps:

Test the configuration without the infrastructure agent: You can manually test a configuration file to ensure the output meets your expectations by running a command like this. Remember to replace <FILE_NAME> with the name of your config file:
bash
```
$# Linux default path: /opt/newrelic-infra/newrelic-integrations/bin/ 
$./nri-flex -verbose -pretty -config_path /etc/newrelic-infra/integrations.d/<FILE_NAME>
$
$# Windows default path: C:\Program Files\New Relic\newrelic-infra\newrelic-integrations
$.\nri-flex.exe -verbose -pretty -config_path "C:\Program Files\New Relic\newrelic-infra\integrations.d\<FILE_NAME>"
```
This will give you an output showing debug logging and JSON payload that will be integrated with the infrastucture agent. Make sure Flex is obtaining and formatting your telemetry as expected before continue with the rest of the troubleshooting steps. Learn more about testing Flex configurations from the GitHub repository.
Test with the infrastructure agent in dry-run mode: Use the dry-run flag in the infrastructure agent to test your Flex configuration. Verify the output contains the telemetry you expect to report to New Relic.
Debug the integration with the infrastructure agent: Ensure the agent is reporting the telemetry data as expected by enabling debug logs in the infrastructure agent.

Tip

You can fetch the standalone binary from here. See the README file to know more about Flex