Kubernetes 1.26 What's new?

Kubernetes 1.26 is about to be released, and it comes packed with novelties! Where do we begin?

This release brings 37 enhancements, on par with the 40 in Kubernetes 1.25 and the 46 in Kubernetes 1.24. Of those 37 enhancements, 11 are graduating to Stable, 10 are existing features that keep improving, 16 are completely new, and one is a deprecated feature.

Watch out for all the deprecations and removals in this version!

Two new features stand out in this release that have the potential to change the way users interact with Kubernetes: Being able to provisioning volumes with snapshots from other namespaces.

There are also new features aimed at high performance workloads, like science researching or machine learning: Better what physical CPU cores your workloads run on.

Also, other features will make life easier for cluster administrators, like support for OpenAPIv3.

We are really hyped about this release!

There is plenty to talk about, so let's get started with what’s new in Kubernetes 1.25.

Kubernetes 1.26 – Editor’s pick:

These are the features that look most exciting to us in this release (ymmv):

#3294 Provision volumes from cross-namespace snapshots

The VolumeSnapshot feature allows Kubernetes users provision volumes from volume snapshots, providing great benefits for users and applications, like enabling database administrators to snapshot a database before any critical operation, or the ability to develop and implement backup solutions.

Starting in Kubernetes 1.26 as an Alpha feature, users will be able to create a PersistentVolumeClaim from a VolumeSnapshot across namespaces, breaking the initial limitation of having both objects in the same namespace.

This enhancement comes to eliminate the constraints that prevented users and applications from operating on fundamental tasks, like saving a database checkpoint when applications and services are in different namespaces.

Víctor Hernando - Sr. Technical Marketing Manager at Sysdig

#3488 CEL for admission control

Finally, a practical implementation of the validation expression language from Kubernetes 1.25!

By defining rules for the admission controller as Kubernetes objects, we can start forgetting about managing webhooks, simplifying the setup of our clusters. Not only that, but implementing Kubernetes security is a bit easier now.

We love to see these user-friendly improvements. They are the key to keep growing Kubernetes adoption.

Víctor Jiménez Cerrada - Content Engineering Manager at Sysdig

#3466 Kubernetes component health SLIs

Since Kubernetes 1.26, you can configure Service Level Indicator (SLI) metrics for the Kubernetes components binaries. Once you enable them, Kubernetes will expose the SLI metrics in the /metrics/slis endpoint - so you won't need a Prometheus exporter. This can take Kubernetes monitoring to another level making it easier to create health dashboards and configure PromQL alerts to assure your cluster's stability.

Jesús Ángel Samitier - Integrations Engineer at Sysdig

#2371 cAdvisor-less, CRI-full container and Pod stats

Currently, to gather metrics from containers, such as CPU or memory consumed, Kubernetes relies on cAdvisor. This feature presents an alternative, enriching the CRI API to provide all the metrics from the containers, allowing more flexibility and better accuracy. After all, it's the Container Runtime who best knows the behavior of the container.

This feature represents one more step on the roadmap to remove cAdvisor from Kubernetes code. However, during this transition, cAdvisor will be modified not to generate the metrics added to the CRI API, avoiding duplicated metrics with possible different and incoherent values.

David de Torres Huerta – Engineer Manager at Sysdig

#3063 Dynamic resource allocation

This new Kubernetes release introduces a new Alpha feature which will provide extended resource management for advanced hardware. As a cherry on top, it comes with a user-friendly API to describe resource requests. With the increasing demand to process different hardware components, like GPU or FPGA, and the need to set up initialization and cleanup, this new feature will speed up Kubernetes adoption in areas like scientific research or edge computing.

Javier Martínez - Devops Content Engineer at Sysdig

#3545 Improved multi-numa alignment in Topology Manager

This is yet another feature aimed at high performance workloads, like those involved in scientific computing. We are seeing the new CPU manager taking shape since Kubernetes 1.22 and 1.23, enabling developers to keep their workloads close to where their data is stored in memory, improving performance. Kubernetes 1.26 goes a step further, opening the door to further customizations for this feature. After all, not all workloads and CPU architectures are the same.

The future of HPC on Kubernetes is looking quite promising, indeed.

Vicente J. Jiménez Miras – Security Content Engineer at Sysdig

#3335 Allow StatefulSet to control start replica ordinal numbering

StatefulSets in Kubernetes often are critical backend services, like clustered databases or message queues.
This enhancement, seemingly a trivial numbering change, allows for greater flexibility and enables new techniques for rolling cross-namespace or even cross-cluster migrations of the replicas of the StatefulSet without any downtime. While the process might seem a bit clunky, involving careful definition of PodDisruptionBudgets and the moving of resources relative to the migrating replica, we can surely envision tools (or existing operators enhancements) that automate these operations for seamless migrations, in stark contrast with the cold-migration strategy (shutdown-backup-restore) that is currently possible.

Daniel Simionato - Security Content Engineer at Sysdig

#3325 Auth API to get self user attributes

This new feature coming to alpha will simplify cluster Administrator's work, especially when they are managing multiple clusters. It will also assist in complex authentication flows, as it lets users query their user information or permissions inside the cluster.

Also, this includes whether you are using a proxy (Kubernetes API server fills in the userInfo after all authentication mechanisms are applied) or impersonating (you receive the details and properties for the user that was impersonated), so you will have your user information in a very easy way.

Miguel Hernández - Security Content Engineer at Sysdig

#3352 Aggregated Discovery

This is a tiny change for the users, but one step further on cleaning the Kubernetes internals and improving its performance. Reducing the number of API calls by aggregating them (or at least on the discovery part) is a nice solution to a growing problem. Hopefully, this will provide a small break to cluster administrators.

Devid Dokash - Content Engineering Intern at Sysdig

Deprecations

A few beta APIs and features have been removed in Kubernetes 1.26, including:

Deprecated API versions that are no longer served, and you should use a newer one:

• CRI v1alpha2, use v1 (containerd version 1.5 and older are not supported).
• flowcontrol.apiserver.k8s.io/v1beta1, use v1beta2.
• autoscaling/v2beta2, use v2.

Deprecated. Implement an alternative before the next release goes out:

• In-tree GlusterFS driver.
• kubectl --prune-whitelist, use --prune-allowlist instead.
• kube-apiserver --master-service-namespace.
• Several unused options for kubectl run: --cascade, --filename, --force, --grace-period, --kustomize, --recursive, --timeout, --wait.
• CLI flag pod-eviction-timeout.
• The apiserver_request_slo_duration_seconds metric, use apiserver_request_sli_duration_seconds.

Removed. Implement an alternative before upgrading:

• Legacy authentication for Azure and Google Cloud is deprecated.
• The userspace proxy mode.
• Dynamic kubelet configuration.
• Several command line arguments related to logging.
• in-tree OpenStack (cinder volume type), use the CSI driver.

Other changes you should adapt your configs for:

• Pod Security admission: the pod-security warn level will now default to the enforce level.
• kubelet: The default cpuCFSQuotaPeriod value with the cpuCFSQuotaPeriod flag enabled is now 100µs instead of 100ms.
• kubelet: The --container-runtime-endpoint flag cannot be empty anymore.
• kube-apiserver: gzip compression switched from level 4 to level 1.
• Metrics: Changed preemption_victims from LinearBuckets to ExponentialBuckets.
• Metrics: etcd_db_total_size_in_bytes is renamed to apiserver_storage_db_total_size_in_bytes.
• Metrics: kubelet_kubelet_credential_provider_plugin_duration is renamed kubelet_credential_provider_plugin_duration.
• Metrics: kubelet_kubelet_credential_provider_plugin_errors is renamed kubelet_credential_provider_plugin_errors.
• Removed Windows Server, Version 20H2 flavors from various container images.
• The e2e.test binary no longer emits JSON structs to document progress.

You can check the full list of changes in the Kubernetes 1.26 release notes. Also, we recommend the Kubernetes Removals and Deprecations In 1.26 article, as well as keeping the deprecated API migration guide close for the future.

#281 Dynamic Kubelet configuration

Feature group: node

After being in beta since Kubernetes 1.11, the Kubernetes team has decided to deprecate DynamicKubeletConfig instead of continuing its development.

This feature was marked for deprecation in 1.21, then removed from the Kubelet in 1.24. Now in 1.26, it has been completely removed from Kubernetes.

Kubernetes 1.26 API

#3352 Aggregated discovery

Stage: Net new to Alpha
Feature group: api-machinery
Feature gate: AggregatedDiscoveryEndpoint Default value: false

Every Kubernetes client like kubectl needs to discover what APIs and versions of those APIs are available in the kubernetes-apiserver. For that, they need to make a request per each API and version, which causes a storm of requests.

This enhancement aims to reduce all those calls to just two.

Clients can include as=APIGroupDiscoveryList to the Accept field of their requests to the /api and /apis endpoints. Then, the server will return an aggregated document (APIGroupDiscoveryList) with all the available APIs and their versions.

#3488 CEL for admission control

Stage: Net new to Alpha
Feature group: api-machinery

Feature gate: ValidatingAdmissionPolicy Default value: false

Building on #2876 CRD validation expression language from Kubernetes 1.25, this enhancement provides a new admission controller type (ValidatingAdmissionPolicy) that allows implementing some validations without relying on webhooks.

These new policies can be defined like:

 apiVersion: admissionregistration.k8s.io/v1alpha1
 kind: ValidatingAdmissionPolicy
 metadata:
   name: "demo-policy.example.com"
 Spec:
   failurePolicy: Fail
   matchConstraints:
     resourceRules:
     - apiGroups:   ["apps"]
       apiVersions: ["v1"]
       operations:  ["CREATE", "UPDATE"]
       resources:   ["deployments"]
   validations:
     - expression: "object.spec.replicas <= 5"

This policy would deny requests for some deployments with 5 replicas or less.

Discover the full power of this feature in the docs.

#1965 kube-apiserver identity

Stage: Graduating to Beta
Feature group: api-machinery
Feature gate: APIServerIdentity Default value: true

In order to better control which kube-apiservers are alive in a high availability cluster, a new lease / heartbeat system has been implemented.

Read more in our "What's new in Kubernetes 1.20" article.

Apps in Kubernetes 1.26

#3017 PodHealthyPolicy for PodDisruptionBudget

Stage: Net new to Alpha
Feature group: apps
Feature gate: PDBUnhealthyPodEvictionPolicy Default value: false

A PodDisruptionBudget allows you to communicate some minimums to your cluster administrator to make maintenance tasks easier, like "Do not destroy more than one of these" or "Keep at least two of these alive".

However, this only takes into account if the pods are running, not if they are healthy. It may happen that your pods are Running but not Ready, and a PodDisruptionBudget may be preventing its eviction.

This enhancement expands these budget definitions with the status.currentHealthy, status.desiredHealthy, and spec.unhealthyPodEvictionPolicy extra fields to help you define how to manage unhealthy pods.

$ kubectl get poddisruptionbudgets example-pod
apiVersion: policy/v1
kind: PodDisruptionBudget
[...]
status:
  currentHealthy: 3
  desiredHealthy: 2
  disruptionsAllowed: 1
  expectedPods: 3
  observedGeneration: 1
  unhealthyPodEvictionPolicy: IfHealthyBudget

#3335 Allow StatefulSet to control start replica ordinal numbering

Stage: Net new to Alpha
Feature group: apps
Feature gate: StatefulSetStartOrdinal Default value: false

StatefulSets in Kubernetes currently number their pods using ordinal numbers, with the first replica being 0 and the last being spec.replicas.

This enhancement adds a new struct with a single field to the StatefulSet manifest spec, spec.ordinals.start, which allows to define the starting number for the replicas controlled by the StatefulSet.

This is useful, for example, in cross-namespace or cross-cluster migrations of StatefulSet, where a clever use of PodDistruptionBudgets (and multi-cluster services) can allow a controlled rolling migration of the replicas avoiding any downtime to the StatefulSet.

#3329 Retriable and non-retriable Pod failures for Jobs

Stage: Graduating to Beta
Feature group: apps
Feature gate: JobPodFailurePolicy Default value: true
Feature gate: PodDisruptionsCondition Default value: true

This enhancement allows us to configure a .spec.podFailurePolicy on the Jobs's spec that determines whether the Job should be retried or not in case of failure. This way, Kubernetes can terminate Jobs early, avoiding increasing the backoff time in case of infrastructure failures or application errors.

Read more in our "What's new in Kubernetes 1.25" article.

#2307 Job tracking without lingering Pods

Stage: Graduating to Stable
Feature group: apps
Feature gate: JobTrackingWithFinalizers Default value: true

With this enhancement, Jobs will be able to remove completed pods earlier, freeing resources in the cluster.

Read more in our "Kubernetes 1.22 - What's new?" article.

Kubernetes 1.26 Auth

#3325 Auth API to get self user attributes

Stage: Net new to Alpha
Feature group: auth
Feature gate: APISelfSubjectAttributesReview Default value: false

This new feature is extremely useful when a complicated authentication flow is used in a Kubernetes cluster, and you want to know all your userInfo, after all authentication mechanisms are applied.

Executing kubectl alpha auth whoami will produce the following output:

apiVersion: authentication.k8s.io/v1alpha1
kind: SelfSubjectReview
status:
  userInfo:
    username: jane.doe
    uid: b79dbf30-0c6a-11ed-861d-0242ac120002
    groups:
    - students
    - teachers
    - system:authenticated
    extra:
      skills:
      - reading
      - learning
      subjects:
      - math
      - sports

In summary, we are now allowed to do a typical /me to know our own permissions once we are authenticated in the cluster.

#2799 Reduction of secret-based service account tokens

Stage: Graduating to Beta
Feature group: auth
Feature gate: LegacyServiceAccountTokenNoAutoGeneration Default value: true

API credentials are now obtained through the TokenRequest API, are stable since Kubernetes 1.22, and are mounted into Pods using a projected volume. They will be automatically invalidated when their associated Pod is deleted.

Read more in our "Kubernetes 1.24 - What's new?" article.

Network in Kubernetes 1.26

#3453 Minimizing iptables-restore input size

Stage: Net new to Alpha
Feature group: network
Feature gate: MinimizeIPTablesRestore Default value: false

This enhancement aims to improve the performance of kube-proxy. It will do so by only sending the rules that have changed on the calls to iptables-restore, instead of the whole set of rules.

#1669 Proxy terminating endpoints

Stage: Graduating to Beta
Feature group: network
Feature gate: ProxyTerminatingEndpoints Default value: true

This enhancement prevents traffic drops during rolling updates by sending all external traffic to both ready and not ready terminating endpoints (preferring the ready ones).

Read more in our "Kubernetes 1.22 - What's new?" article.

#2595 Expanded DNS configuration

Stage: Graduating to Beta
Feature group: network
Feature gate: ExpandedDNSConfig Default value: true

With this enhancement, Kubernetes allows up to 32 DNS in the search path, and an increased number of characters for the search path (up to 2048), to keep up with recent DNS resolvers.

Read more in our "Kubernetes 1.22 - What's new?" article.

#1435 Support of mixed protocols in Services with type=LoadBalancer

Stage: Graduating to Stable
Feature group: network
Feature gate: MixedProtocolLBService Default value: true

This enhancement allows a LoadBalancer Service to serve different protocols under the same port (UDP, TCP). For example, serving both UDP and TCP requests for a DNS or SIP server on the same port.

Read more in our "Kubernetes 1.20 - What's new?" article.

#2086 Service internal traffic policy

Stage: Graduating to Stable
Feature group: network
Feature gate: ServiceInternalTrafficPolicy Default value: true

You can now set the spec.trafficPolicy field on Service objects to optimize your cluster traffic:

• With Cluster, the routing will behave as usual.
• When set to Topology, it will use the topology-aware routing.
• With PreferLocal, it will redirect traffic to services on the same node.
• With Local, it will only send traffic to services on the same node.

Read more in our "Kubernetes 1.21 - What's new?" article.

#3070 Reserve service IP ranges for dynamic and static IP allocation

Stage: Graduating to Stable
Feature group: network
Feature gate: ServiceIPStaticSubrange Default value: true

This update to the --service-cluster-ip-range flag will lower the risk of having IP conflicts between Services using static and dynamic IP allocation, and at the same time, keep the compatibility backwards.

Read more in our "What's new in Kubernetes 1.24" article.

Kubernetes 1.26 Nodes

#2371 cAdvisor-less, CRI-full container and Pod stats

Stage: Major change to Alpha
Feature group: node
Feature gate: PodAndContainerStatsFromCRI Default value: false

This enhancement summarizes the efforts to retrieve all the stats about running containers and pods from the Container Runtime Interface (CRI), removing the dependencies from cAdvisor.

Starting with 1.26, the metrics on /metrics/cadvisor are gathered by CRI instead of cAdvisor.

Read more in our "Kubernetes 1.23 - What's new?" article.

#3063 Dynamic resource allocation

Stage: Net new to Alpha
Feature group: node
Feature gate: DynamicResourceAllocation Default value: false

Traditionally, the Kubernetes scheduler could only take into account CPU and memory limits and requests. Later on, the scheduler was expanded to also take storage and other resources into account. However, this is limiting in many scenarios.

For example, what if the device needs initialization and cleanup, like an FPGA; or what if you want to limit the access to the resource, like a shared GPU?

This new API covers those scenarios of resource allocation and dynamic detection, using the new ResourceClaimTemplate and ResourceClass objects, and the new resourceClaims field inside Pods.

apiVersion: v1
 kind: Pod
[...]
 spec:
   resourceClaims:
   - name: resource0
     source:
       resourceClaimTemplateName: resource-claim-template
   - name: resource1
     source:
       resourceClaimTemplateName: resource-claim-template
[...]

The scheduler can keep track of these resource claims, and only schedule Pods in those nodes with enough resources available.

#3386 Kubelet evented PLEG for better performance

Stage: Net new to Alpha
Feature group: node
Feature gate: EventedPLEG Default value: false

The aim of this enhancement is to reduce the CPU usage of the kubelet when keeping track of all the pod states.

It will partially reduce the periodic polling that the kubelet performs, instead relying on notifications from the Container Runtime Interface (CRI) as much as possible.

If you are interested in the implementation details, you may want to take a look at the KEP.

#3545 Improved multi-NUMA alignment in topology manager

Stage: Net new to Alpha
Feature group: node
Feature gate: TopologyManagerPolicyOptions Default value: false
Feature gate: TopologyManagerPolicyBetaOptions Default value: false
Feature gate: TopologyManagerPolicyAlphaOptions Default value: false

This is an improvement for TopologyManager to better handle Non-Uniform Memory Access (NUMA) nodes. For some high-performance workloads, it is very important to control in which physical CPU cores they run. You can significantly improve performance if you avoid memory jumping between the caches of the same chip, or between sockets.

A new topology-manager-policy-options flag for kubelet will allow you to pass options and modify the behavior of a topology manager.

Currently, only one alpha option is available:

• When prefer-closest-numa-nodes=true is passed along, the Topology Manager will align the resources on either a single NUMA node or the minimum number of NUMA nodes possible.

As new options may be added in the future, several feature gates have been added so you can choose to focus only on the stable ones:

• TopologyManagerPolicyOptions: Will enable the topology-manager-policy-options flag and the stable options.
• TopologyManagerPolicyBetaOptions: Will also enable the beta options.
• TopologyManagerPolicyAlphaOptions: Will also enable the alpha options.

Related: #2902 CPUManager policy option to distribute CPUs across NUMA nodes in Kubernetes 1.23.
Related: #2625 New CPU Manager Policies in Kubernetes 1.22.

#2133 Kubelet credential provider

Stage: Graduating to Stable
Feature group: node
Feature gate: KubeletCredentialProviders Default value: true

This enhancement replaces in-tree container image registry credential providers with a new mechanism that is external and pluggable.

Read more in our "Kubernetes 1.20 - What's new?" article.

#3570 Graduate to CPUManager to GA

Stage: Graduating to Stable
Feature group: node
Feature gate: CPUManager Default value: true

The CPUManager is the Kubelet component responsible for assigning pod containers to sets of CPUs on the local node.

It was introduced in Kubernetes 1.8, and graduated to beta in release 1.10. For 1.26, the core CPUManager has been deemed stable, while experimentation continues with the additional work on its policies.

Related: #3545 Improved multi-numa alignment in Topology Manager in Kubernetes 1.26.
Related: #2625 New CPU Manager Policies in Kubernetes 1.22.

#3573 Graduate DeviceManager to GA

Stage: Graduating to Stable
Feature group: node
Feature gate: DevicePlugins Default value: true

The DeviceManager in the Kubelet is the component managing the interactions with the different Device Plugins.

Initially introduced in Kubernetes 1.8 and moved to beta stage in release 1.10, the Device Plugin framework saw widespread adoption and is finally moving to GA in 1.26.

This framework allows the use of external devices (e.g., NVIDIA GPUs, AMD GPUS, SR-IOV NICs) without modifying core Kubernetes components.

Scheduling in Kubernetes 1.26

#3521 Pod scheduling readiness

Stage: Net new to Alpha
Feature group: scheduling
Feature gate: PodSchedulingReadiness Default value: false

This enhancement aims to optimize scheduling by letting the Pods define when they are ready to be actually scheduled.

Not all pending Pods are ready to be scheduled. Some stay in a miss-essential-resources state for some time, which causes extra work in the scheduler.

The new .spec.schedulingGates of a Pod allows to identify when they are ready for scheduling:

apiVersion: v1
 kind: Pod
[...]
 spec:
   schedulingGates:
   - name: foo
   - name: bar
[...]

When any scheduling gate is present, the Pod won't be scheduled.

You can check the status with:

$ kubectl get pod test-pod
 NAME       READY   STATUS            RESTARTS   AGE
 test-pod   0/1     SchedulingGated   0          7s

#3094 Take taints/tolerations into consideration when calculating PodTopologySpread skew

Stage: Graduating to Beta
Feature group: scheduling
Feature gate: NodeInclusionPolicyInPodTopologySpread Default value: true

As we discussed in our "Kubernetes 1.16 - What's new?" article, the topologySpreadConstraints fields, along with maxSkew, allow you to spread your workloads across nodes. A new NodeInclusionPolicies field allows taking into account NodeAffinity and NodeTaint when calculating this pod topology spread skew.

Read more in our "What's new in Kubernetes 1.25" article.

Kubernetes 1.26 storage

#3294 Provision volumes from cross-namespace snapshots

Stage: Net new to Alpha
Feature group: storage
Feature gate: CrossNamespaceVolumeDataSource Default value: false

Prior to Kubernetes 1.26, users were able to provision volumes from snapshots thanks to the VolumeSnapshot feature. While this is a great and super useful feature. it had some limitations, like the inability to bind a PersistentVolumeClaim to VolumeSnapshots from other namespaces.

This enhancement breaks this limitation and allows Kubernetes users to provision volumes from snapshots across namespaces.

If you want to use the cross-namespace VolumeSnapshot feature, you’ll have to first create a ReferenceGrant object, and then a PersistentVolumeClaim binding to the VolumeSnapshot. Here, you’ll find a simple example of both objects for learning purposes.

---
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: ReferenceGrant
metadata:
  name: test
  namespace: default
spec:
  from:
  - group: ""
    kind: PersistentVolumeClaim
    namespace: nstest1
  to:
  - group: snapshot.storage.k8s.io
    kind: VolumeSnapshot
    name: testsnapshot
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: testvolumeclaim
  namespace: nstest1
spec:
  storageClassName: mystorageclass
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi
  dataSourceRef2:
    apiGroup: snapshot.storage.k8s.io
    kind: VolumeSnapshot
    name: testsnapshot
    namespace: default
  volumeMode: Filesystem

#2268 Non-graceful node shutdown

Stage: Graduating to Beta
Feature group: storage
Feature gate: NodeOutOfServiceVolumeDetach Default value: true

This enhancement addresses node shutdown cases that are not detected properly, where the pods that are part of a StatefulSet will be stuck in terminating status on the shutdown node and cannot be moved to a new running node.

The pods will be forcefully deleted in this case, trigger the deletion of the VolumeAttachments, and new pods will be created on a different running node so that application can continue to function.

Read more in our "Kubernetes 1.24 - What's new?" article.

#3333 Retroactive default StorageClass assignement

Stage: Graduating to Beta
Feature group: storage
Feature gate: RetroactiveDefaultStorageClass Default value: false

This enhancement helps manage the case when cluster administrators change the default storage class. All PVCs without StorageClass that were created while the change took place will retroactively be set to the new default StorageClass.

Read more in our "What's new in Kubernetes 1.25" article.

#1491 vSphere in-tree to CSI driver migration

Stage: Graduating to Stable
Feature group: storage
Feature gate: CSIMigrationvSphere Default value: false

As we covered in our "What's new in Kubernetes 1.19" article, the CSI driver for vSphere has been stable for some time. Now, all plugin operations for vspherevolume are now redirected to the out-of-tree 'csi.vsphere.vmware.com' driver.

This enhancement is part of the #625 In-tree storage plugin to CSI Driver Migration effort.

#1885 Azure file in-tree to CSI driver migration

Stage: Graduating to Stable
Feature group: storage
Feature gate: InTreePluginAzureDiskUnregister Default value: true

This enhancement summarizes the work to move Azure File code out of the main Kubernetes binaries (out-of-tree).

Read more in our "Kubernetes 1.21 - What's new?" article.

#2317 Allow Kubernetes to supply pod's fsgroup to CSI driver on mount

Stage: Graduating to Stable
Feature group: storage
Feature gate: DelegateFSGroupToCSIDriver Default value: false

This enhancement proposes providing the CSI driver with the fsgroup of the pods as an explicit field, so the CSI driver can be the one applying this natively on mount time.

Read more in our "Kubernetes 1.22 - What's new?" article.

Other enhancements in Kubernetes 1.26

#3466 Kubernetes component health SLIs

Stage: Net new to Alpha
Feature group: instrumentation
Feature gate: ComponentSLIs Default value: false

There isn't a standard format to query the health data of Kubernetes components.

Starting with Kubernetes 1.26, a new endpoint /metrics/slis will be available on each component exposing their Service Level Indicator (SLI) metrics in Prometheus format.

For each component, two metrics will be exposed:

• A gauge, representing the current state of the healthcheck.
• A counter, recording the cumulative counts observed for each healthcheck state.

With this information, you can check the overtime status for the Kubernetes internals, e.g.:

kubernetes_healthcheck{name="etcd",type="readyz"}

And create an alert for when something's wrong, e.g.:

kubernetes_healthchecks_total{name="etcd",status="error",type="readyz"} > 0

#3498 Extend metrics stability

Stage: Net new to Alpha
Feature group: instrumentation
Feature gate: N/A

Metrics in Kubernetes are classified as alpha or stable. The stable ones are guaranteed to be maintained, providing you with the information to prepare your dashboards so they don't break unexpectedly when you upgrade your cluster.

In Kubernetes 1.26, two new classes are added:

• beta: For metrics related to beta features. They may change or disappear, but they are in a more advanced development state than the alpha ones.
• internal: Metrics for internal usage that you shouldn't worry about, either because they don't provide useful information for cluster administrators, or because they may change without notice.

You can check a full list of available metrics in the documentation.

Related: #1209 Metrics stability enhancement in Kubernetes 1.21.

#3515 OpenAPI v3 for kubectl explain

Stage: Net new to Alpha
Feature group: cli
Environment variable: KUBECTL_EXPLAIN_OPENAPIV3 Default value: false

This enhancement allows kubectl explain to gather the data from OpenAPIv3 instead of v2.

In OpenAPIv3, some data can be represented in a better way, like CustomResourceDefinitions (CDRs).

Internal work is also being made to improve how kubectl explain prints the output.

Related: #2896 OpenAPI v3 in Kubernetes 1.24.

#1440 kubectl events

Stage: Graduating to Beta
Feature group: cli
Feature gate: N/A

A new kubectl events command is available that will enhance the current functionality of kubectl get events.

Read more in our "Kubernetes 1.23 - What's new?" article.

#3031 Signing release artifacts

Stage: Graduating to Beta
Feature group: release
Feature gate: N/A

This enhancement introduces a unified way to sign artifacts in order to help avoid supply chain attacks. It relies on the sigstore project tools, and more specifically cosign. Although it doesn’t add new functionality, it will surely help to keep our cluster more protected.

Read more in our "Kubernetes 1.24 - What's new?" article.

#3503 Host network support for Windows pods

Stage: Net new to Alpha
Feature group: windows
Feature gate: WindowsHostNetwork Default value: false

There is a weird situation in Windows pods where you can set hostNetwork=true for them, but it doesn't change anything. There isn't any platform impediment, the implementation was just missing.

Starting with Kubernetes 1.26, the kubelet can now request that Windows pods use the host's network namespace instead of creating a new pod network namespace.

This will come handy to avoid port exhaustion where there's large amounts of services.

#1981 Support for Windows privileged containers

Stage: Graduating to Stable
Feature group: windows
Feature gate: WindowsHostProcessContainers Default value: true

This enhancement brings the privileged containers feature available in Linux to Windows hosts.

Privileged containers have access to the host, as if they were running directly on it. Although they are not recommended for most of the workloads, they are quite useful for administration, security, and monitoring purposes.

Read more in our "Kubernetes 1.22 - What's new?" article.

---

That’s all for Kubernetes 1.26, folks! Exciting as always; get ready to upgrade your clusters if you are intending to use any of these features.

If you liked this, you might want to check out our previous ‘What’s new in Kubernetes’ editions:

• Kubernetes 1.26 - What's new?
• Kubernetes 1.25 - What's new?
• Kubernetes 1.24 - What's new?
• Kubernetes 1.23 - What's new?
• Kubernetes 1.22 - What's new?
• Kubernetes 1.21 - What's new?
• Kubernetes 1.20 - What's new?
• Kubernetes 1.19 - What's new?
• Kubernetes 1.18 - What's new?
• Kubernetes 1.17 - What's new?
• Kubernetes 1.16 - What's new?
• Kubernetes 1.15 - What's new?
• Kubernetes 1.14 - What's new?
• Kubernetes 1.13 - What's new?
• Kubernetes 1.12 - What's new?

Get involved in the Kubernetes community:

• Visit the project homepage.
• Check out the Kubernetes project on GitHub.
• Get involved with the Kubernetes community.
• Meet the maintainers on the Kubernetes Slack.
• Follow @KubernetesIO on Twitter.

And if you enjoy keeping up to date with the Kubernetes ecosystem, subscribe to our container newsletter, a monthly email with the coolest stuff happening in the cloud-native ecosystem.