Kubernetes logs

The Kubernetes integration does not produce debug level logs. If you are doing a more in-depth investigation on your own or with New Relic Support, you can enable verbose mode.

• To get verbose logging details using Helm and the nri-bundle chart, add the following configuration to your values.yaml file. You can also enable verbose logging for all subcharts:

  # To enable verboseLog for newrelic-infrastructure only
  # newrelic-infrastructure:
  #   verboseLog: true
  
  # To enable verboseLog for all sub-charts
  # global:
  #   verboseLog: true

  Copy

  Once you have added the desired configuration, upgrade the solution with the following command:

  bash

  Copy

  $
  helm upgrade <RELEASE_NAME> newrelic/nri-bundle \
  >
  --namespace <NEWRELIC_NAMESPACE> \
  >
  -f values-newrelic.yaml \
  >
  [--version fixed-chart-version]

  After collecting your verbose logs, revert the change in your values.yaml file, and run the upgrade command again.

• To get verbose logging details using a manifest file, you need to set two different environment variables:

  1. NRIA_VERBOSE="1" for all agent and forwarder containers; this enables agent verbose logs. You can set this below NRIA_LICENSE_KEY.

  2. set NRI_KUBERNETES_VERBOSE=true" in all components of the integration; this enables verbose logs for the integration.

     env:

     Copy
  • name: NRIA_LICENSE_KEY valueFrom: secretKeyRef: key: licenseKey name: newrelic-bundle-newrelic-infrastructure-license
  • name: NRIA_VERBOSE value: "1"

    Copy

  Once you have edited the manifest, upgrade the solution with the following command:

  bash

  Copy

  $
  kubectl apply -f your_newrelic_k8s.yaml -n <NEWRELIC_NAMESPACE>

  After collecting your verbose logs, revert the change in the manifest, and apply the manifest again.

There are three different components of the integration that are in charge of scraping ksm, controlplane, and kubelet. In each instance, there are two containers running, with one scraping the data and the other one forwarding it. The agent in the kubelet component is also in charge of scraping node data and running integrations. The diagram below shows the integration installed in a cluster with three nodes.

To fetch logs, get the name of the pods you want to retrieve logs from:

$
kubectl get pods -n <NEWRELIC_NAMESPACE>

Retrieve the logs:

$
kubectl logs <POD_NAME> --all-containers --prefix

To retrieve logs from the previous execution, add the --previous flag. To get logs from only one of the containers, remove the --all-containers --prefix flag and specify the container with the --container option.

When you use the Kubernetes integration, the infrastructure agent is distributed as a Docker image that contains both the infrastructure agent and the Kubernetes integration. The Docker image is tagged with a version, and the infrastructure agent also has its own version.

Once the agent is sending data to New Relic, you can retrieve the versions of the infrastructure agent for Kubernetes (the Docker image) you are running in your clusters by using the following NRQL query:

FROM K8sContainerSample SELECT uniqueCount(entityId) WHERE containerName LIKE 'agent' FACET clusterName, containerImage

If the agent isn't reporting any data:

Get the versions of the New Relic integration for Kubernetes that you are running in a cluster using kubectl:

$
kubectl get pods --all-namespaces -l app.kubernetes.io/name=newrelic-infrastructure -o jsonpath="{.items..spec..containers..image}"

To retrieve the version of kube-state-metrics running on your clusters, run the following NRQL query:

FROM K8sContainerSample SELECT uniqueCount(entityId) WHERE containerName LIKE '%kube-state-metrics%' facet clusterName, containerImage