Install Kubernetes events integration

The New Relic Kubernetes events integration watches for events happening in your Kubernetes clusters and sends those events to New Relic. To visualize your event data, use the Kubernetes cluster explorer in New Relic One platform, or use the infrastructure events UI.

Requirements

The Kubernetes events integration requires:

  • New Relic Kubernetes integration installed and activated.
  • Kubernetes pod with the label app.kubernetes.io/name=nri-kube-events and two containers, one used to capture events in the Kubernetes cluster and the other to forward the events to New Relic.

Install the Kubernetes events integration

To install the New Relic Kubernetes events integration in a Kubernetes environment:

  1. Download the integration manifest YAML file:

    curl -O https://download.newrelic.com/infrastructure_agent/integrations/kubernetes/nri-kube-events-latest.yaml
  2. Edit the nri-kube-events-latest.yaml manifest file, adding both a cluster name, to identify your Kubernetes cluster (required), and your New Relic license key (required).

    clusterName: "YOUR_CLUSTER_NAME"
    [...]
    	- name: "NRIA_LICENSE_KEY"
    	value: "YOUR_LICENSE_KEY" 
  3. Deploy the integration in your Kubernetes cluster:

    kubectl apply -f nri-kube-events-latest.yaml
  4. To confirm that the integration has been configured correctly, wait a few minutes, then go to one.newrelic.com > Query builder, and run the following NRQL query to see if data has been reported:

    FROM InfrastructureEvent SELECT count(*) WHERE clusterName = 'YOUR_CLUSTER_NAME' since 1 hour ago

View your Kubernetes events

Once you have successfully installed the Kubernetes event integration, you can view and query your events in New Relic.

To add events to your New Relic One dashboard:

  1. Add a chart to a new or existing dashboard or create a chart using the New Relic chart builder.
  2. When creating or updating your chart, select the chart type table and use the following event query, replacing YOUR_CLUSTER_NAME with the name of your cluster:

    FROM InfrastructureEvent SELECT
    event.involvedObject.kind,event.involvedObject.name,event.reason,event.message 
    WHERE clusterName = 'YOUR_CLUSTER_NAME' limit 100

Query events

Here are a few examples to query your data:

Get attributes names

To see all the attributes names, use the following query:

FROM InfrastructureEvent SELECT keyset() WHERE category = 'kubernetes'

See event details

To see details about the latest events in a cluster, use the following query, replacing YOUR_CLUSTER_NAME with the name of your cluster:

FROM InfrastructureEvent SELECT event.involvedObject.kind, 
event.involvedObject.name, event.type, event.message, event.reason WHERE 
category = 'kubernetes' AND clusterName='YOUR_CLUSTER_NAME'​

The events collected by New Relic will have the exact same attributes as given by Kubernetes. For a reference of these attributes, see the Kubernetes documentation's event v1 core reference.

Search events in New Relic

To search events in the New Relic Infrastructure UI:

  1. Navigate to the Infrastructure event page: Go to one.newrelic.com > Infrastructure > Events.
  2. In Category, select kubernetes.
  3. Use the search Search events field to look for specific events. To focus on a specific set of events, select or change the filter set.

View events in the Kubernetes cluster explorer

To view events in the Kubernetes cluster explorer, use either of these methods:

Then, to view the pod details:

  1. In the Kubernetes cluster explorer, select a pod.
  2. Select Show pod events.

Optional: Collect metrics of the event collector

To collect metric data for the event collector itself, install the New Relic Prometheus OpenMetrics integration. To get started:

  1. Install the New Relic Prometheus OpenMetrics integration.
  2. Label the deployment of your Kubernetes events integration with your configured scrape label (default is prometheus.io/scrape=true).

Optional: Define custom attributes

To add custom attributes to the events sent by the integration these need to be specified as environment variables following the nomenclature NRI_KUBE_EVENTS_attributeKey=attributeValue. These environment variables should be defined as part of the spec for the kube-events container.

For example, to add the attribute environment with value staging to all the events, you need to add the following to your manifest:

env: 
  - name: NRI_KUBE_EVENTS_environment 
    value: dev

Remember that the environment should be specified in the spec for the kube-events container, making your manifest look something like:

...
kind: Deployment
...
spec:
  ...
  template:
    ...
    spec:
      containers:
        - name: kube-events
          ...
          env:
            - name: NRI_KUBE_EVENTS_environment
              value: dev

Troubleshooting

Here are some troubleshooting tips when using the Kubernetes events integration.

Get logs on event collection

To check the logs of our event collector:

kubectl logs deploy/nr-kube-events kube-events

To add more details, enable verbose mode by adding verbose: "true" in the config section of you configuration file, either before or after the clusterName.

Get logs on sending events

To check the logs of the agent responsible for sending the events to New Relic:

kubectl logs deploy/nr-kube-events infra-agent

For more help

If you need more help, check out these support and learning resources: