• Log inStart now

Monitor services running on Kubernetes

With New Relic's Kubernetes integration you can monitor both Kubernetes and the services running on it, such as Cassandra, Redis, MySQL, and other supported services.

Get started

Our Kubernetes integration comes bundled with some of our on-host integrations (like Cassandra, MySQL, and Apache). This lets you get data for those supported services by adding a section to the Kubernetes integration's configuration, which lives as a ConfigMap inside a manifest.

For an example of how to monitor Redis running on a Kubernetes PHP Guestbook, see this tutorial.

What you need

To monitor services running on Kubernetes, you only need a Kubernetes cluster running the Kubernetes integration, version 1.16.0 or higher (install | check version | update).

We support the following services running on Kubernetes:

Enable monitoring of services using the Helm Chart

Monitoring services in Kubernetes works by leveraging our standard On-Host integrations and an autodiscovery mechanism to point them to pods with a specified selector.

To enable our Kubernetes integration to monitor one or more services:

Get the base configuration

You can get an example configuration file for our integrations that can run in K8s from the following links:

Please take into account that most of these examples will need to be configured for your particular environment, mainly to input the credentials required to authenticate against the particular service. You can read more about how to configure each integration in detail from the integration-specific pages, linked in the dropdown above.

Add the configuration to your values-newrelic.yaml

Tip

This format applies to the Kubernetes integration v3. For the older v2 version, check the section below.

A working configuration snippet should be a YAML document with the following structure:

# Top level name can be arbitrary, akin to a config file name
redis-sampleapp.yaml:
# Discovery section will define which pods will be monitored.
discovery:
command:
# nri-discovery-kubernetes is a tool that connects to the Kubelet to fetch local pods
# without putting stress in the API Server. It accepts the following arguments:
# --namespaces: Comma separated namespaces to limit discovery on
# --tls: Use tls for connecting to the kubelet
# --port: Port used to connect to the kubelet. Defaults to 10255.
exec: /var/db/newrelic-infra/nri-discovery-kubernetes --tls --port 10250
# Monitor pods which have a `app=sampleapp` label
match:
label.app: sampleapp
# Integrations section contains the integration configs.
# ${placeholders} will be replaced with the specified attributes for each pod that is discovered
integrations:
- name: nri-redis # Integration name, should not be changed
env:
# Using the discovered pod IP as the hostname address
HOSTNAME: ${discovery.ip}
PORT: 6379
# Other integration options go here

This snippet must be added to the integrations section, under newrelic-infrastructure in your values-newrelic.yaml file. For example:

global:
licenseKey: _YOUR_NEW_RELIC_LICENSE_KEY_
cluster: _K8S_CLUSTER_NAME_
# Other settings...
newrelic-infrastructure:
# verboseLog: true
integrations:
redis-sampleapp.yaml:
discovery:
command:
# --namespaces: Comma separated list of namespaces to discover pods on
# --port: Port used to connect to the kubelet. Default is 10255
# --tls: Use secure (TLS) connection
exec: /var/db/newrelic-infra/nri-discovery-kubernetes --tls --port 10250
match:
label.app: sampleapp
integrations:
- name: nri-redis
env:
HOSTNAME: ${discovery.ip}
PORT: 6379

Tip

Notice we are specifying --tls --port 10250. Earlier versions of the integration might have worked without this, as of version 3 of the Kubernetes integration it is mandatory to specify them. The reason for this change is that the integration now connects to the Kubelet using the nodeIP rather than localhost, with the former requiring TLS while the latter did not.

Integrations targeting other pods should have their own section next to redis-sampleapp.yaml.

Integrations are standalone binaries, and are executed by the infrastructure agent that is included in the newrelic-nrk8s-kubelet-xxxxx pods. Config files are deployed to all pods of the nrk8s-kubelet DaemonSet, but discovery ensures that each pod will only target service pods that are scheduled in the same node as that particular nrk8s-kubelet pod. If an instance of the nrk8s-kubelet DaemonSet does not have any pods matching the specified labels, the integration will not be executed by that instance.

(Legacy v2) Add the configuration to values.yaml

Verify the integration is working

Go to one.newrelic.com > Infrastructure, select Third party services, and then select the service's dashboard. You should see data being reported.

If you do not see the data there, the integration might be missing some parameter that it requires to run, or might not be able to reach the target service. You can get the logs of the integration by running:

bash
$
kubectl logs -n newrelic newrelic-nrk8s-kubelet-xxxxx agent

Make sure to select the particular pod of the nrk8s-kubelet DaemonSet that is scheduled next to the pod that the integration should be targeting. You can check which instance is running on which node by running the following command:

bash
$
kubectl get pods -n newrelic -o wide -l app.kubernetes.io/component=kubelet

Additional notes about enabling services

  • Enabling multiple services may use more resources than what is set in the resource limits of the Kubernetes integration config file. If this becomes an issue, raise the limit in the resources section.
  • The Kubernetes integration does not automatically update. For best results, regularly update.

Learn more

More resources for learning about configuration:

Enable monitoring of services using manifests

Tip

We strongly encourage you to configure integrations through the values-newrelic.yaml file and our Helm Chart, as explained in the section above. Configuring service monitoring on top of the manifest installation is substantially more difficult and offers no advantages whatsoever.

For each service you wish to monitor, you must add a configuration file for that integration to our Kubernetes integration's configuration. This document will cover these subjects:

How the service-specific YAML config works

Our Kubernetes integration's configuration follows the ConfigMap format. Using a ConfigMap allows us to decouple the configuration for the integrations from the Kubernetes image. The other benefit is that a ConfigMap can be updated automatically without reloading the running container.

Because the infrastructure agent uses YAML to configure its associated integrations, ConfigMaps are a good choice for storing YAML. (For more information on config file format, see the Integration config file format.)

The Kubernetes integration image comes with an auto-discovery feature that simplifies the configuration of multiple instances of services using a single configuration file. For example, if you have several NGINX instances running, creating an NGINX integration configuration file for every instance would be hard to implement and hard to update. With our auto-discovery option, you can discover and monitor all your NGINX instances with a single configuration file.

Each integration has its own specific configuration YAML. Our NGINX integration default config file looks like this:

discovery:
command:
# Use the following optional arguments:
# --namespaces: Comma separated list of namespaces to discover pods on
# --port: Port used to connect to the kubelet. Default is 10255
# --tls: Use secure (TLS) connection
# Custom Example:
# exec: /var/db/newrelic-infra/nri-discovery-kubernetes --namespaces namespace1,namespace2 --port 10250 --tls
# Default
exec: /var/db/newrelic-infra/nri-discovery-kubernetes --tls --port 10250
match:
label.app: nginx
integrations:
- name: nri-nginx
env:
STATUS_URL: http://${discovery.ip}/status
STATUS_MODULE: discover
METRICS: 1

The above config enables the following:

  • Runs nri-discovery-kubernetes to query the data for the node we are currently on.
  • Parses the data that comes back and looks for any Kubernetes pod that has a Kubernetes container with an app= label with value nginx.
  • For any matches, it attempts to run the NGINX integration. The status URL is built from:
    • The pod's IP address
    • The status page is pulled from the label on K8s pod called status_url

This automatic discovery works the same as the container auto-discovery used by the infrastructure agent. For more advanced options, see Container auto-discovery.

Add a service YAML to the Kubernetes integration config

In order for the snippet above to be recognized by the integration, it must be placed inside the designated ConfigMap. If you are using our Kubernetes integration v3, a ConfigMap should already be generated, with a name ending in -integrations-cfg. Locate the config map and add the modified snippet to it, so it ends up looking like this:

---
apiVersion: v1
kind: ConfigMap
metadata:
name: newrelic-infrastructure-integrations-cfg
namespace: newrelic
data:
nginx-config.yml: |
discovery:
command:
# Use the following optional arguments:
# --namespaces: Comma separated list of namespaces to discover pods on
# --port: Port used to connect to the kubelet. Default is 10255
# --tls: Use secure (TLS) connection
# Custom Example:
# exec: /var/db/newrelic-infra/nri-discovery-kubernetes --namespaces namespace1,namespace2 --port 10250 --tls
# Default
exec: /var/db/newrelic-infra/nri-discovery-kubernetes --tls --port 10250
match:
label.app: nginx
integrations:
- name: nri-nginx
env:
STATUS_URL: http://${discovery.ip}/status
STATUS_MODULE: discover
METRICS: 1

If you are using our Kubernetes integration v3, this ConfigMap will already be mounted in the required container.

Important

For the Kubernetes Integration version 2, you will need to add an entry for this ConfigMap in the volumes and volumeMounts section of the DaemonSet's spec, to ensure all the files in the ConfigMap are mounted in /etc/newrelic-infra/integrations.d/.

Please note that the same ConfigMap can hold multiple config files, which is recommended to keep modifications to the manifests at a minimum.

---
apiVersion: v1
kind: ConfigMap
metadata:
name: newrelic-infrastructure-integrations-cfg
namespace: newrelic
data:
nginx-config.yml: |
discovery: # ...
integrations:
- name: nri-nginx
# ...
redis-config.yml: |
discovery: # ...
integrations:
- name: nri-redis
# ...
Copyright © 2022 New Relic Inc.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.