---
title: Monitor services running on Kubernetes
source: https://docs.newrelic.com/docs/kubernetes-pixie/kubernetes-integration/advanced-configuration/monitor-services/monitor-services-running-kubernetes
---

If you have services that run on Kubernetes and are [supported by our applicable integrations](#requirements), you can enable those integrations via our Kubernetes integration.

## Get started [#overview]

Our Kubernetes integration comes bundled with some of our on-host integrations. This lets you get data for those services by adding a section to the Kubernetes integration's configuration, which lives as a `ConfigMap` inside a manifest.

For an example of how to monitor Redis running on a Kubernetes PHP Guestbook, see [this tutorial](https://docs.newrelic.com/docs/monitor-service-running-kubernetes-tutorial).

## Requirements [#requirements]

To monitor services running on Kubernetes, you need:

-   A [Kubernetes cluster running a supported version](https://docs.newrelic.com/docs/kubernetes-pixie/kubernetes-integration/get-started/kubernetes-integration-compatibility-requirements).
-   A Kubernetes cluster running the Kubernetes integration ([install](https://docs.newrelic.com/docs/integrations/kubernetes-integration/installation/kubernetes-installation-configuration) \| [check version](https://docs.newrelic.com/docs/integrations/kubernetes-integration/installation/kubernetes-installation-configuration#update) \| [update](https://docs.newrelic.com/docs/integrations/kubernetes-integration/installation/kubernetes-installation-configuration#update)).
-   A supported service running on Kubernetes that meets our requirements. Supported services are:
    -   [Apache](/install/apache)
    -   [Cassandra](/install/cassandra)
    -   [Couchbase](https://docs.newrelic.com/docs/integrations/host-integrations/host-integrations-list/couchbase-monitoring-integration)
    -   [Elasticsearch](/install/elastic)
    -   [HAProxy](/install/haproxy/)
    -   [HashiCorp Consul](https://docs.newrelic.com/docs/integrations/host-integrations/host-integrations-list/hashicorp-consul-monitoring-integration)
    -   [JMX](/install/jmx)
    -   [Kafka](/install/kafka)
    -   [Memcached](/install/memcached)
    -   [MongoDB](/install/mongodb)
    -   [MySQL](/install/mysql)
    -   [NGINX](/install/nginx)
    -   [PostgreSQL](/install/postgresql)
    -   [RabbitMQ](/install/rabbitmq)
    -   [Redis](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/redis/redis-integration)

For this install method, our RabbitMQ and Apache integrations do not report inventory data.

## Enable monitoring of services using the Helm Chart [#enable]

Monitoring services in Kubernetes works by leveraging our [infrastructure agent and on-host integrations](https://docs.newrelic.com/docs/infrastructure/host-integrations/installation/install-infrastructure-host-integrations) and an autodiscovery mechanism to point them to pods with a specified selector.

See [this example of configuration for version 2](https://docs.newrelic.com/docs/kubernetes-pixie/kubernetes-integration/advanced-configuration/k8s-version2/overview/#enable-services).

To enable our Kubernetes integration to monitor one or more services:

### Get the base configuration [#integration-config]

You can get an example configuration file for our integrations that can run in K8s from the following links:

**Get the config YAML for the integration**

For the services you want to monitor, follow the links to GitHub to get the YAML snippets you'll need for the next step:

-   [Apache](https://github.com/newrelic/nri-apache/blob/master/apache-config.yml.k8s_sample)
-   [Cassandra](https://github.com/newrelic/nri-cassandra/blob/master/cassandra-config.yml.k8s_sample)
-   [Couchbase](https://github.com/newrelic/nri-couchbase/blob/master/couchbase-config.yml.k8s_sample)
-   [Elasticsearch](https://github.com/newrelic/nri-elasticsearch/blob/master/elasticsearch-config.yml.k8s_sample)
-   [HAProxy](https://github.com/newrelic/nri-haproxy/blob/master/haproxy-config.yml.k8s_sample)
-   [HashiCorp Consul](https://github.com/newrelic/nri-consul/blob/master/consul-config.yml.k8s_sample)
-   [JMX](https://github.com/newrelic/nri-jmx/blob/master/jmx-config.yml.k8s_sample)
-   [Kafka](https://github.com/newrelic/nri-kafka/blob/master/kafka-config.yml.k8s_sample)
-   [Memcached](https://github.com/newrelic/nri-memcached/blob/master/memcached-config.yml.k8s_sample)
-   [MongoDB](https://github.com/newrelic/nri-mongodb/blob/master/mongodb-config.yml.k8s_sample)
-   [MySQL](https://github.com/newrelic/nri-mysql/blob/master/mysql-config.yml.k8s_sample)
-   [NGINX](https://github.com/newrelic/nri-nginx/blob/master/nginx-config.yml.k8s_sample)
-   [PostgreSQL](https://github.com/newrelic/nri-postgresql/blob/master/postgresql-config.yml.k8s_sample)
-   [RabbitMQ](https://github.com/newrelic/nri-rabbitmq/blob/master/rabbitmq-config.yml.k8s_sample)
-   [Redis](https://github.com/newrelic/nri-redis/blob/master/redis-config.yml.k8s_sample)

Please take into account that most of these examples will need to be configured for your particular environment, mainly to input the credentials required to authenticate against the particular service. You can read more about how to configure each integration in detail from the integration-specific pages, linked in the [dropdown above](#requirements).

### Add the configuration to your `values-newrelic.yaml` [#configure-values]

> #### 💡 TIP
>
> This format applies to the [Kubernetes integration v3](https://docs.newrelic.com/docs/kubernetes-pixie/kubernetes-integration/advanced-configuration/k8s-version2/changes-since-v3). For the older v2 version, see [Monitor services running on Kubernetes](https://docs.newrelic.com/docs/kubernetes-pixie/kubernetes-integration/advanced-configuration/k8s-version2/overview/#monitor-services).

A working configuration snippet should be a YAML document with the following structure:

```yaml
# Top level name can be arbitrary, akin to a config file name
redis-sampleapp.yaml:
  # Discovery section will define which pods will be monitored.
  discovery:
    command:
      # nri-discovery-kubernetes is a tool that connects to the Kubelet to fetch local pods
      # without putting stress in the API Server. It accepts the following arguments:
      # --namespaces: Comma separated namespaces to limit discovery on
      # --tls: Use tls for connecting to the kubelet
      # --port: Port used to connect to the kubelet. Defaults to 10255.
      exec: /var/db/newrelic-infra/nri-discovery-kubernetes --tls --port 10250
      # Monitor pods which have a `app=sampleapp` label
      match:
        label.app: sampleapp

  # Integrations section contains the integration configs.
  # ${placeholders} will be replaced with the specified attributes for each pod that is discovered
  integrations:
    - name: nri-redis  # Integration name, should not be changed
      env:
        # Using the discovered pod IP as the hostname address
        HOSTNAME: ${discovery.ip}
        PORT: 6379
        # Other integration options go here
```

This snippet must be added to the `integrations` section, under `newrelic-infrastructure` in your `values-newrelic.yaml` file. For example:

```yaml
global:
  licenseKey: _YOUR_NEW_RELIC_LICENSE_KEY_
  cluster: _K8S_CLUSTER_NAME_

# Other settings...

newrelic-infrastructure:
  # verboseLog: true
  integrations:
    redis-sampleapp.yaml:
      discovery:
        command:
          # --namespaces: Comma separated list of namespaces to discover pods on
          # --port: Port used to connect to the kubelet. Default is 10255
          # --tls: Use secure (TLS) connection
          exec: /var/db/newrelic-infra/nri-discovery-kubernetes --tls --port 10250
          match:
            label.app: sampleapp

      integrations:
        - name: nri-redis
          env:
            HOSTNAME: ${discovery.ip}
            PORT: 6379
```

> #### 💡 TIP
>
> Notice we are specifying `--tls --port 10250`. Earlier versions of the integration might have worked without this, as of [version 3](https://docs.newrelic.com/docs/kubernetes-pixie/kubernetes-integration/advanced-configuration/k8s-version2/changes-since-v3) of the Kubernetes integration it is mandatory to specify them. The reason for this change is that the integration now connects to the Kubelet using the nodeIP rather than `localhost`, with the former requiring TLS while the latter did not.

Integrations targeting other pods should have their own section next to `redis-sampleapp.yaml`.

Integrations are standalone binaries and are executed by the infrastructure agent that's included in the `newrelic-nrk8s-kubelet-xxxxx` pods. Config files are deployed to all pods of the `nrk8s-kubelet` DaemonSet, but discovery ensures that each pod will only target service pods that're scheduled in the same node as that particular `nrk8s-kubelet` pod. If an instance of the `nrk8s-kubelet` DaemonSet does not have any pods matching the specified labels, the integration will not be executed by that `###` instance.

#### Verify the integration is working

Go to **[one.newrelic.com > All capabilities](https://one.newrelic.com/all-capabilities) > Infrastructure**, select **Third party services**, and then select the service's dashboard. You should see data being reported.

If you do not see the data there, the integration might be missing some parameter that it requires to run, or might not be able to reach the target service. You can get the logs of the integration by running:

```shell
kubectl logs -n newrelic newrelic-nrk8s-kubelet-xxxxx agent
```

Make sure to select the particular pod of the `nrk8s-kubelet` DaemonSet that is scheduled next to the pod that the integration should be targeting. You can check which instance is running on which node by running the following command:

```shell
kubectl get pods -n newrelic -o wide -l app.kubernetes.io/component=kubelet
```

### Additional notes about enabling services [#additional-notes]

-   Enabling multiple services may use more resources than what is set in the resource limits of the Kubernetes integration config file. If this becomes an issue, raise the limit in the `resources` section.
-   The Kubernetes integration does not automatically update. For best results, regularly [update](/install/kubernetes#update).

### Learn more [#update-kear-more]

More resources for learning about configuration:

-   Learn [technical details about how configuration works](#config-details).
-   Learn how to [configure monitoring of multiple services with the same config file](#add-service).
-   See a [step-by-step tutorial showing how to monitor a Redis service on Kubernetes](https://docs.newrelic.com/docs/monitor-service-running-kubernetes-tutorial).

## Enable monitoring of services using manifests [#config-details]

> #### 💡 TIP
>
> We strongly encourage you to configure integrations through the `values-newrelic.yaml` file and our Helm Chart, as explained in the [section above](#enable). Configuring service monitoring on top of the manifest installation is substantially more difficult and offers no advantages whatsoever.

For each service you wish to monitor, you must add a configuration file for that integration to our Kubernetes integration's configuration. This document will cover these subjects:

-   [How the service-specific configuration YAML snippet works](#integration-config)
-   [Adding the service-specific YAML in the Kubernetes integration's config file](#configmap)
-   [Adding multiple services to the Kubernetes integration's config file](#add-service)

### How the service-specific YAML config works [#integration-config]

Our Kubernetes integration's configuration follows the [`ConfigMap`](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/) format. Using a `ConfigMap` allows us to decouple the configuration for the integrations from the Kubernetes image. The other benefit is that a `ConfigMap` can be updated automatically without reloading the running container.

Because the infrastructure agent uses YAML to configure its associated integrations, `ConfigMaps` are a good choice for storing YAML. (For more information on config file format, see the [Integration config file format](https://docs.newrelic.com/docs/integrations/integrations-sdk/file-specifications/integration-configuration-file-specifications).)

The Kubernetes integration image comes with an [auto-discovery feature](https://github.com/newrelic/nri-discovery-kubernetes) that simplifies the configuration of multiple instances of services using a single configuration file. For example, if you have several NGINX instances running, creating an [NGINX integration configuration file](https://docs.newrelic.com/docs/integrations/host-integrations/host-integrations-list/nginx-monitoring-integration#config) for every instance would be hard to implement and hard to update. With our auto-discovery option, you can discover and monitor all your NGINX instances with a single configuration file.

Each integration has its own [specific configuration YAML](#integration-config-links). Our NGINX integration default config file looks like this:

```yaml
discovery:
  command:
    # Use the following optional arguments:
    # --namespaces: Comma separated list of namespaces to discover pods on
    # --port: Port used to connect to the kubelet. Default is 10255
    # --tls: Use secure (TLS) connection
    # Custom Example:
    # exec: /var/db/newrelic-infra/nri-discovery-kubernetes --namespaces namespace1,namespace2 --port 10250 --tls
    # Default
    exec: /var/db/newrelic-infra/nri-discovery-kubernetes --tls --port 10250
    match:
      label.app: nginx
integrations:
  - name: nri-nginx
    env:
      STATUS_URL: http://${discovery.ip}/status
      STATUS_MODULE: discover
      METRICS: 1
```

The above config enables the following:

-   Runs `nri-discovery-kubernetes` to query the data for the node we are currently on.
-   Parses the data that comes back and looks for any Kubernetes pod that has a Kubernetes container with an `app=` label with value `nginx`.
-   For any matches, it attempts to run the NGINX integration. The status URL is built from:
    -   The pod's IP address
    -   The status page is pulled from the label on K8s pod called `status_url`

This automatic discovery works the same as the [container auto-discovery](https://docs.newrelic.com/docs/integrations/host-integrations/installation/container-auto-discovery) used by the infrastructure agent. For more advanced options, see [Container auto-discovery](https://docs.newrelic.com/docs/integrations/host-integrations/installation/container-auto-discovery).

### Add a service YAML to the Kubernetes integration config [#configmap]

The integration will recognize the above snippet, if you place it inside a designated `ConfigMap`. If you're using our [Kubernetes integration v3](https://docs.newrelic.com/docs/kubernetes-pixie/kubernetes-integration/advanced-configuration/k8s-version2/changes-since-v3), a `ConfigMap` with a name ending in `-integrations-cfg` should already exist.

If you're using Kubernetes integration version 2, see [Add a service YAML](https://docs.newrelic.com/docs/kubernetes-pixie/kubernetes-integration/advanced-configuration/k8s-version2/overview/#add-service-yaml).

Locate the config map and add the modified snippet to it, so it ends up looking like this:

```yaml
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: newrelic-infrastructure-integrations-cfg
  namespace: newrelic
data:
  nginx-config.yml: |
    discovery:
      command:
        # Use the following optional arguments:
        # --namespaces: Comma separated list of namespaces to discover pods on
        # --port: Port used to connect to the kubelet. Default is 10255
        # --tls: Use secure (TLS) connection
        # Custom Example:
        # exec: /var/db/newrelic-infra/nri-discovery-kubernetes --namespaces namespace1,namespace2 --port 10250 --tls
        # Default
        exec: /var/db/newrelic-infra/nri-discovery-kubernetes --tls --port 10250
        match:
          label.app: nginx
    integrations:
      - name: nri-nginx
        env:
          STATUS_URL: http://${discovery.ip}/status
          STATUS_MODULE: discover
          METRICS: 1
```

If you're using our [Kubernetes integration v3](https://docs.newrelic.com/docs/kubernetes-pixie/kubernetes-integration/advanced-configuration/k8s-version2/changes-since-v3), this `ConfigMap` will already be mounted in the required container.

Please note that the same `ConfigMap` can hold multiple config files, which is recommended to keep modifications to the manifests at a minimum.

```yaml
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: newrelic-infrastructure-integrations-cfg
  namespace: newrelic
data:
  nginx-config.yml: |
    discovery: # ...
    integrations:
      - name: nri-nginx
        # ...
  redis-config.yml: |
    discovery: # ...
    integrations:
      - name: nri-redis
        # ...
```

## How to use the reported data [#how-to-use-data]

You can learn more about how to find and use your Kubernetes data [here](https://docs.newrelic.com/docs/kubernetes-pixie/kubernetes-integration/understand-use-data/find-use-your-kubernetes-data/) and you can review our K8sServiceSample data schema [here](/attribute-dictionary/?event=K8sServiceSample).