Kubernetes integration release notesRSS

Notes

Follow standard procedures to install or update the New Relic integration for Kubernetes.

Changelog

• Added: Support for OpenShift 4.5 and Kubernetes versions 1.18.X
• Changed: Upgraded Docker base image newrelic/infrastructure-bundle to v1.4.2.
  For more information on the release please see the New Relic Infrastructure Bundle release notes.
• Changed: When querying the summary endpoint from Kubelet to get the Node or Pod network metrics, if the default network interface is not eth0 then summary endpoint for Kubelet doesn't return the metrics as we expect them. We rely on them being a direct member of the "network" object. See rxBytes, txBytes and rxErrors in the following example metrics:

"network": {
 "time": "2020-06-04T10:01:15Z",
 "name": "eth0",
 "rxBytes": 207909096,
 "rxErrors": 0,
 "txBytes": 8970981,
 "txErrors": 0,
 "interfaces": [
  {
   "name": "eth0",
   "rxBytes": 207909096,
   "rxErrors": 0,
   "txBytes": 8970981,
   "txErrors": 0
  },
  {
   "name": "ip6tnl0",
   "rxBytes": 0,
   "rxErrors": 0,
   "txBytes": 0,
   "txErrors": 0
  },
  {
   "name": "tunl0",
   "rxBytes": 0,
   "rxErrors": 0,
   "txBytes": 0,
   "txErrors": 0
  }
 ]
}

This scenario only happens when the default interface is eth0. Kubernetes
source code has it hardcoded that eth0 is the default. In the following
example you can see that we only have network metrics inside the interfaces
list, in this case there is no eth0 on the and the default interface is ens5:

"network": {
 "time": "2020-06-04T10:01:15Z",
 "name": "",
 "interfaces": [
  {
   "name": "ens5",
   "rxBytes": 207909096,
   "rxErrors": 42,
   "txBytes": 8970981,
   "txErrors": 24
  },
  {
   "name": "ip6tnl0",
   "rxBytes": 0,
   "rxErrors": 0,
   "txBytes": 0,
   "txErrors": 0
  },
  {
   "name": "tunl0",
   "rxBytes": 0,
   "rxErrors": 0,
   "txBytes": 0,
   "txErrors": 0
  }
 ]

In cases like this, the integration will look for the default interface
inside the interfaces list and use those values. The default interface name
is retrieved from the network route file (default /proc/net/route).

When running the unprivileged version of the integration we don't have access
to the route file, the integration won't be able to get the default interface
name and won't send network metrics for the unless there's a network
interface called eth0.

For Pods, this issue is mainly present when using hostNetwok since they
shared the same network interfaces with the Node.

Notes

Follow standard procedures to install or update the New Relic integration for Kubernetes.

Changelog

• Added: Support for OpenShift 4.4 and Kubernetes versions 1.17.X
• Changed: Upgraded Docker base image newrelic/infrastructure-bundle to v1.4.1.
  For more information on the release please see the New Relic Infrastructure Bundle release notes.
• Changed: When retrieving control plane metrics from the API server to generate the K8sApiServerSample, the integration now uses by default the secure port localhost:443 with the service account's bearer authentication. If the query on the secure port fails, it will fallback automatically to the non-secure one localhost:8080, this should preserve the same behavior as previous versions. If your cluster uses a different secure port to expose its metrics you can use the configuration option API_SERVER_ENDPOINT_URL to specify it.

Notes

Follow standard procedures to install or update the New Relic integration for Kubernetes.

Changelog

• Changed: Upgraded Docker base image newrelic/infrastructure-bundle to v1.4.0.
  For more information on the release please see the New Relic Infrastructure Bundle release notes.

Notes

Follow standard procedures to install the New Relic integration for Kubernetes events.

Changelog

• Updated the version of the k8s-events-forwarder sidecar container to 1.12.0 of the infrastructure agent. You can read the release notes for the infrastructure agent for more information.

Notes

Follow standard procedures to install or update the New Relic integration for Kubernetes.

Changelog

• Fixes a bug that was swapping the values for Node Allocatable Resources with the Node Capacity Resources in the K8sNodeSample.

Notes

Follow standard procedures to install or update the New Relic integration for Kubernetes.

Changelog

• Kubernetes 1.16 is now officially supported!
  • The minimum supported version of kube-state-metrics for this release is 1.9.5, according to the KSM compatibility matrix.
• Added container throttling metrics to the K8sContainerSample:
  • containerCpuCfsPeriodsDelta: Delta change of elapsed enforcement period intervals.
  • containerCpuCfsThrottledPeriodsDelta: Delta change of throttled period intervals.
  • containerCpuCfsThrottledSecondsDelta: Delta change of duration the container has been throttled.
  • containerCpuCfsPeriodsTotal: Number of elapsed enforcement period intervals.
  • containerCpuCfsThrottledPeriodsTotal: Number of throttled period intervals.
  • containerCpuCfsThrottledSecondsTotal: Total time duration the container has been throttled.
• Added container mmap byte usage metrics to the K8sContainerSample:
  • containerMemoryMappedFileBytes: Size of memory mapped files in bytes.

Notes

Follow standard procedures to install the New Relic integration for Kubernetes events.

Changelog

• Updated the version of the k8s-events-forwarder sidecar container from 1.11.24 to 1.11.45 of the infrastructure agent.

Notes

Follow standard procedures to install or update the New Relic integration for Kubernetes.

Changelog

• Changed: Upgraded the base image newrelic/infrastructure-bundle to v1.3.9. For more information on the release please see the New Relic Infrastructure Bundle release notes.

The Kubernetes integration v1.21.0 brings several changes related to node allocatable resources.

• Cluster explorer
• Cluster dashboard
• Kubernetes metrics

Changes to the cluster explorer

We added support for allocatable resources to the Kubernetes cluster explorer:

• The cluster explorer entity list now calculates CPU usage by summing the allocatableCpuCores from the K8sNodeSample instead of using processorCount from SystemSample. Similarly, memory usage is now calculated by summing allocatableMemoryBytes instead of memoryTotalBytes. Pods' columns now refer to running pods vs. allocatable pods.

  

• In addition to resource consumption, the node's tooltip in the cluster explorer now shows the amount of allocatable pods used by the node.

  

• The node card, which appears when clicking a node in cluster explorer, now shows resource consumption (CPU, memory, and storage) against the allocatable resources.

  • CPU usage aggregates cpuUsedCores, cpuRequestedCores, and cpuLimitCores from K8sContainerSample, and allocatableCpuCores from the K8sNodeSample.
  • Memory usage aggregates memoryWorkingSetBytes, memoryRequestedBytes, and memoryLimitBytes from K8sContainerSample, and allocatableMemoryBytes from K8sNodeSample.
  • Storage usage relies on fsUsedBytes and fsCapacityBytes fromK8sNodeSample.

• A new pods widget has been added to the node card that shows the amount of pods used against the total allocatable pods of the node.

  

If you run the unprivileged version of the Kubernetes integration, you can now see the node metrics. Previously, this was only possible with the privileged version.

Changes to the cluster dashboard

The cluster explorer dashboard, which appears when clicking Kubernetes dashboard, has been updated to account for resource consumption against allocatable resources.

• CPU widgets displays cpuUsedCores and allocatableCpuCores from K8sNodeSample. As for Memory usage, it now shows memoryWorkingSetBytes and allocatableMemoryBytes.
• Disk widgets use fsUsedBytes and fsCapacityBytes from K8sNodeSample.
• A new pods widget has been added that shows the amount of pods used against the total allocatable pods of the cluster.

Changes to Kubernetes metrics

• Node allocatable resources and capacity are retrieved from the API server and added to the K8sNodeSample as capacity<ResourceName> and allocatable<ResourceName>.
• The Kubernetes server version is now retrieved from the API server and cached with the APIServerCacheK8SVersionTTL config option. The version is added to K8sClusterSample as clusterK8sVersion, and to the inventory.
• Add support for static pods status for Kubernetes server versions 1.15 or newer.
• In older versions of Kubernetes, the kubeletes wouldn’t sync the status of the static pods with the API server and would always return a Pending status. This behavior was fixed in v1.15 of Kubernetes. Our integration now detects what version of Kubernetes the cluster is running and, if it’s 1.15 or newer, it adds the status of the static pods in K8sPodSample.

Other changes

• We have upgraded the newrelic/infrastructure-bundle base image to v1.3.8. For more information, see the New Relic Infrastructure Bundle release notes.