The Kubernetes integration v1.21.0 brings several changes related to node allocatable resources.
Changes to the cluster explorer
We added support for allocatable resources to the Kubernetes cluster explorer:
The cluster explorer entity list now calculates CPU usage by summing the
allocatableCpuCores
from theK8sNodeSample
instead of usingprocessorCount
fromSystemSample
. Similarly, memory usage is now calculated by summingallocatableMemoryBytes
instead ofmemoryTotalBytes
. Pods' columns now refer to running pods vs. allocatable pods.In addition to resource consumption, the node's tooltip in the cluster explorer now shows the amount of allocatable pods used by the node.
The node card, which appears when clicking a node in cluster explorer, now shows resource consumption (CPU, memory, and storage) against the allocatable resources.
- CPU usage aggregates
cpuUsedCores
,cpuRequestedCores
, andcpuLimitCores
fromK8sContainerSample
, andallocatableCpuCores
from theK8sNodeSample
. - Memory usage aggregates
memoryWorkingSetBytes
,memoryRequestedBytes
, andmemoryLimitBytes
fromK8sContainerSample
, andallocatableMemoryBytes
fromK8sNodeSample
. - Storage usage relies on
fsUsedBytes
andfsCapacityBytes
fromK8sNodeSample
.
- CPU usage aggregates
A new pods widget has been added to the node card that shows the amount of pods used against the total allocatable pods of the node.
If you run the unprivileged version of the Kubernetes integration, you can now see the node metrics. Previously, this was only possible with the privileged version.
Changes to the cluster dashboard
The cluster explorer dashboard, which appears when clicking Kubernetes dashboard, has been updated to account for resource consumption against allocatable resources.
- CPU widgets displays
cpuUsedCores
andallocatableCpuCores
fromK8sNodeSample
. As for Memory usage, it now showsmemoryWorkingSetBytes
andallocatableMemoryBytes
. - Disk widgets use
fsUsedBytes
andfsCapacityBytes
fromK8sNodeSample
. - A new pods widget has been added that shows the amount of pods used against the total allocatable pods of the cluster.
Changes to Kubernetes metrics
- Node allocatable resources and capacity are retrieved from the API server and added to the
K8sNodeSample
ascapacity<ResourceName>
andallocatable<ResourceName>
. - The Kubernetes server version is now retrieved from the API server and cached with the
APIServerCacheK8SVersionTT
L config option. The version is added toK8sClusterSample
asclusterK8sVersion
, and to the inventory. - Add support for static pods status for Kubernetes server versions 1.15 or newer.
- In older versions of Kubernetes, the kubeletes wouldn’t sync the status of the static pods with the API server and would always return a
Pending
status. This behavior was fixed in v1.15 of Kubernetes. Our integration now detects what version of Kubernetes the cluster is running and, if it’s 1.15 or newer, it adds the status of the static pods inK8sPodSample
.
Other changes
- We have upgraded the
newrelic/infrastructure-bundle
base image to v1.3.8. For more information, see the New Relic Infrastructure Bundle release notes.
Effective Wednesday, 12 August 2020, Kubernetes integration v1.7 or lower will be deprecated. To avoid losing data, upgrade to the latest version. For more information, read this note or contact your account team.
Background
In order to provide the best possible experience for monitoring Kubernetes, we are merging the Kubernetes Cluster Explorer into New Relic, where you’ll enjoy all the latest features.
What is happening?
We will support Kubernetes integrations v1.7 or lower until Wednesday, 12 August 2020.
Starting on July 1st the Kubernetes Cluster Explorer will only be available at one.newrelic.com > All capabilities > Kubernetes Cluster Explorer. The Kubernetes Cluster Explorer will only be compatible with Kubernetes integrations v.1.8 and newer from Wednesday, August 12th.
Therefore in order to enjoy the latest features and to be fully compatible with the New Relic Kubernetes Cluster Explorer, you must upgrade your Kubernetes integration to the latest version.
What do you need to do?
Follow the instructions here to upgrade to the latest version of the Kubernetes integration.
What happens if you don’t make any changes to your account?
The Kubernetes integration will still send data to the New Relic platform, though the Kubernetes Cluster Explorer will not work for Clusters running Kubernetes integration versions v1.7 or older after Wednesday, 12 August. In other words, the integration will still send data, however you will be unable to view that data in the Cluster Explorer.
Notes
Follow standard procedures to install the New Relic integration for Kubernetes events.
Changelog
- Updated base image for the nri-kube-events container to alpine
3.11
. - Updated the version of the k8s-events-forwarder sidecar container from
1.5.31
to1.11.24
of the Infrastructure agent. - Updated the Deployment
apiVersion
fromapps/v1beta2
toapps/v1
. - Updated the ClusterRoleBinding
apiVersion
fromrbac.authorization.k8s.io/v1beta1
torbac.authorization.k8s.io/v1
. - Updated the ClusterRole
apiVersion
fromrbac.authorization.k8s.io/v1beta1
torbac.authorization.k8s.io/v1
.
Notes
Follow standard procedures to install or update the New Relic integration for Kubernetes.
Changelog
- Changed: Upgraded the base image newrelic/infrastructure-bundle to v1.3.5. For more information on the release please see the New Relic Infrastructure Bundle release notes.
Notes
Follow standard procedures to install or update the New Relic integration for Kubernetes.
Changelog
- Added: new label combination to discover the Kubernetes controller manager:
app=controller-manager
controller-manager=true
- Changed: Upgraded the base image newrelic/infrastructure-bundle to v1.3.4. For more information on the release please see the New Relic Infrastructure Bundle release notes.
Notes
Follow standard procedures to install or update the New Relic integration for Kubernetes.
Changelog
- Changed: Upgraded the base image newrelic/infrastructure-bundle to v1.3.2. For more information on the release please see the New Relic Infrastructure Bundle release notes.
Notes
Follow standard procedures to install or update the New Relic integration for Kubernetes.
Changelog
- Changed: Upgraded the base image newrelic/infrastructure-bundle to v1.3.0. For more information on the release please see the New Relic Infrastructure Bundle release notes.
Notes
Follow standard procedures to install or update the New Relic integration for Kubernetes.
Changelog
- Added: Support for completely avoid querying Kube State Metrics. It's behind the
DISABLE_KUBE_STATE_METRICS
environment variable and its default value isfalse
. Note that disabling this will imply missing metrics that are collected from KSM and possibly missing features in the Kubernetes Cluster Explorer. Please, refer to our official documentation on this configuration option for more information.
Notes
Follow standard procedures to install or update the New Relic integration for Kubernetes.
Changelog
- Added: support for querying Kube State Metrics (KSM) instances behind kube-rbac-proxy. This feature only works when paired with the label-based KSM discovery (see release notes of version 1.11.0 for more information). This adds two new environment variables that allow deeper configuration of the feature:
KUBE_STATE_METRICS_SCHEME
: Whether the query should use thehttp
orhttps
protocols. Defaults tohttp
and doesn't introduce a breaking change in behavior.KUBE_STATE_METRICS_PORT
: The port in which thekube-rbac-proxy
instance is listening on. You can confirm this by describing your KSM pod. Defaults to8080
and doesn't introduce a breaking change in behavior.
- Added: Added 4 new environment variables to explicitly set the control plane component base URLs. These variables can be used for Kubernetes configurations, such as OpenShift, that are different than the defaults defined in the section Discovery of master nodes and control plane components. See the section OpenShift 4.x Configuration for more information.
SCHEDULER_ENDPOINT_URL
: The base URL of the scheduler pod metrics endpoint. The default is the empty string. In this case, the default configuration will be used.ETCD_ENDPOINT_URL
: The base URL of the ETCD pod metrics endpoint. The default is the empty string. In this case, the default configuration will be used.CONTROLLER_MANAGER_ENDPOINT_URL
: The base URL of the controller manager pod metrics endpoint. The default is the empty string. In this case, the default configuration will be used.API_SERVER_ENDPOINT_URL
: The base URL of the API server pod metrics endpoint. The default is the empty string. In this case, the default configuration will be used.
- Fixed: A bug preventing attributes named like
selector.<label_name>
from being added to some entries in theK8sServiceSample
. - Changed: The integration now uses newrelic/infrastructure-bundle as the base image. The tag used is for the version 1.2.0. For more information on this image please see the New Relic Infrastructure Bundle releases notes.