Not everything here is new in 1.37.

Kubernetes v1.37 was released on 26 August 2026, and the coverage around it has a shape worth noticing. Several changes are attributed to this release that do not belong to it: the kubelet refusing to start on cgroup v1 (that default flipped in v1.35), static pods losing their Secret and ConfigMap references (on by default since v1.34), and kube-proxy's ipvs mode being removed (it has carried a deprecation notice since v1.35, and removal is targeted at v1.43). One change is misreported in the opposite direction: containerd 1.x still runs against a v1.37 kubelet, and the cliff is v1.38. Meanwhile the one change in this release that can genuinely leave pods stuck in ContainerCreating on upgrade day — SELinuxMount reaching GA and switching on by default — gets a paragraph, because it is invisible to anyone whose clusters do not run SELinux.

The distinction is not pedantry, because each column implies different work. If the cgroup change surprises you on 1.37, you were already two releases late and the fix is a reboot per node rather than a rollback. If you rebuild your container runtime in a panic because somebody told you containerd 1.x had stopped working, you will have spent a maintenance window on a deadline that is still one release away — real, but not today. And if the SELinux change surprises you on 1.37, you have pods that will not start, and the audit that would have found them is materially harder to run once you are past the upgrade. So this article does two things: it separates the ledger honestly, and then it goes deep on the parts that need work — the SELinux audit and opt-out in full, the real ipvs and containerd timelines, and a runbook that puts the irreversible steps in the right order.

The failure modes, and which release each one belongs to

Start with what you would actually see, because none of these failures announces its own cause. A pod sits in ContainerCreating forever on one node while an identical pod runs fine on another — that is an SELinux label conflict, and the deciding factor is which pod got to the volume first. A node comes back from an upgrade and its kubelet will not start at all — that is almost certainly cgroup v1, and it has been true since v1.35. A monitoring agent that has run as a static pod since 2021 fails on exactly one node, the one you just upgraded — that is the Secret reference it was never supposed to be able to use, and the default that took it away landed back in v1.34.^[sneak]

The pattern worth internalising is that a version upgrade surfaces every change since the release you were last confident about, not just the changes in the release you are moving to. Clusters do not upgrade one minor at a time in practice; they sit on 1.34 for a year and then move. Service ExternalIPs is the current example of this: it was deprecated in v1.36, where the API server began warning on every use, and people arriving at v1.37 from earlier are meeting that warning for the first time. Nothing about it is removed yet — kube-proxy support is expected to go off by default no earlier than v1.40, with removal no earlier than v1.43 — which is exactly how a change three releases out becomes an outage: the warning arrives in a release nobody read the notes for. That is why the second section of this article is a ledger rather than a release-note summary.^[extip]

The ledger: new in 1.37, versus already landed

Here is the honest accounting. The left column is what the release team's own sneak peek lists for v1.37; the right columns are where the change actually originates, or where it is still heading. Everything in the "earlier" rows is something you should already have dealt with, and if you have not, the upgrade window is when you will find out; everything in the "later" rows is a date to put in a calendar, not work for this week.^[sneak]

The single most useful sentence in this article: if your nodes do not run SELinux in enforcing mode, the largest section here does not apply to you at all. The kubelet skips the entire SELinux code path when SELinux is unavailable or disabled in the kernel. Check that first — it takes one command — because it determines whether this is a half-day of audit work or a ten-minute read.

Two of these deserve a note on how to verify them yourself rather than taking anyone's word for it. The feature gates reference publishes the per-release state of SELinuxMount, SELinuxChangePolicy and KubeProxyIPVS, which is the fastest way to check a claim about which release flipped which default. And the release-team blog for each version carries its own deprecation list; reading three of them takes twenty minutes and is a better use of an upgrade window than most of what goes into one.^[gates]

Where you are actually standing

Before any of this becomes actionable you need four numbers, and they are independent of one another: the kubelet version per node, whether SELinux is enforcing on that node, which cgroup version it runs, and which mode kube-proxy is in. In a fleet of any size the answers are not uniform — node pools roll forward on their own schedules, and the version skew policy explicitly permits a kubelet to lag the API server, so a mixed fleet is a supported configuration rather than a sign of neglect.^[skew]

# Four numbers decide how much of this article applies to you, and they are
# independent of each other. Ask for all four rather than assuming.

# 1. Control plane and kubelet versions. Node pools drift; this is normal.
kubectl get nodes -o custom-columns=\
'NODE:.metadata.name,KUBELET:.status.nodeInfo.kubeletVersion,'\
'RUNTIME:.status.nodeInfo.containerRuntimeVersion,'\
'KERNEL:.status.nodeInfo.kernelVersion,OS:.status.nodeInfo.osImage'
# NODE     KUBELET   RUNTIME              KERNEL          OS
# node-01  v1.36.4   containerd://2.3.2   6.8.0-51        Ubuntu 24.04.3 LTS
# node-02  v1.35.9   containerd://1.7.28  5.15.0-118      Ubuntu 22.04.5 LTS   <- two problems

# 2. Is SELinux actually in play? If the answer is "no" on every node, the
#    largest section of this article does not apply to you at all.
for n in $(kubectl get nodes -o name); do
  printf '%-22s ' "${n#node/}"
  kubectl debug "$n" -q -it --image=busybox --profile=general -- \
    chroot /host sh -c 'getenforce 2>/dev/null || echo "not installed"' 2>/dev/null
done
# node-01  Enforcing        <- section "SELinux" applies
# node-02  not installed    <- it does not

# 3. Which cgroup version. v2 shows cgroup2fs; v1 shows tmpfs.
kubectl debug node/node-01 -q -it --image=busybox --profile=general -- \
  chroot /host stat -fc %T /sys/fs/cgroup
# cgroup2fs

# 4. Which kube-proxy mode. This is the one people are most often wrong about,
#    because the answer usually predates everyone currently on the team.
kubectl -n kube-system get configmap kube-proxy \
  -o jsonpath='{.data.config\.conf}' | grep -E '^\s*mode:'
#   mode: "ipvs"

# Clean up the debug pods. `kubectl debug node/...` names them
# node-debugger-<node>-<suffix> and applies no label, so there is nothing to
# select on - match the name, or they accumulate silently.
kubectl get pods -o name | grep '^pod/node-debugger-' | xargs -r kubectl delete

The other number that matters is how much runway the version you are on actually has. Kubernetes supports the three most recent minor releases, roughly fourteen months each, with the last two months in maintenance mode where only critical security fixes land. That schedule is the reason upgrades are not optional, and it is worth having in front of you when someone proposes deferring this one.^[k8srel]

SELinuxMount goes GA: the change that stops pods

This is the section that justifies the article. SELinuxMount reaches GA in v1.37 and is enabled by default. The change is a performance win and the mechanism is elegant: instead of the container runtime walking a volume and relabelling every inode — which on a large or remote filesystem is genuinely slow — the kubelet mounts the volume with -o context=<label> and the kernel applies the label to every inode on that mount in constant time. The problem is a direct consequence of the mechanism. A mount can hold exactly one SELinux context. Under recursive relabelling, two pods with different labels could share a volume; under a context mount they cannot, and one of them will sit in ContainerCreating until the other is gone.^{[selblog][kep1710]}

The blast radius is narrower than that sounds, and it is worth working out precisely rather than assuming the worst. Five conditions all have to hold before a single volume changes behaviour, and the one people miss is the CSI driver: the kubelet only uses a context mount when the driver has declared it can take one, by setting seLinuxMount: true on its CSIDriver object. A driver that leaves the field unset — and unset is not true — keeps the old recursive behaviour and is completely unaffected by the flip. The in-tree volume types that support the mount option are fc, iscsi and rbd; everything else in tree relabels recursively regardless.^[csidriver]

# The blast radius of the SELinuxMount change is not "clusters with SELinux".
# It is the intersection of three things, and all three have to be true before
# a single pod is at risk.

# (a) SELinux enforcing on the node. Checked above. If not, stop here.

# (b) A CSI driver that has opted in. The kubelet only uses the mount option
#     when the driver declares it can take one. Drivers that do not set this
#     keep the old recursive relabel and are unaffected by the flip.
kubectl get csidrivers \
  -o custom-columns='DRIVER:.metadata.name,SELINUXMOUNT:.spec.seLinuxMount'
# DRIVER                    SELINUXMOUNT
# ebs.csi.aws.com           true      <- volumes on this driver change behaviour
# efs.csi.aws.com           false     <- unchanged
# csi.trident.netapp.io     <none>    <- unset is not true; unchanged

# The in-tree volume types that support the mount option are fc, iscsi and rbd.
# Everything else in tree relabels recursively regardless.

# (c) Two pods with different SELinux labels sharing one volume. This is the
#     part you cannot infer from a manifest, because the label is often
#     assigned by the runtime rather than written down. The cheapest proxy is
#     to find the volumes that more than one workload mounts at all:
kubectl get pods -A \
  -o jsonpath='{range .items[*]}{range .spec.volumes[?(@.persistentVolumeClaim)]}'\
'{.persistentVolumeClaim.claimName}{"\t"}{end}{.metadata.namespace}{"\n"}{end}' \
  | awk -F'\t' 'NF>1' | sort | uniq -c | sort -rn | awk '$1>1'
#   3 shared-media   default
#   2 build-cache    ci

# That list is a starting point, not an answer. The answer comes from the
# controller below, which knows the labels.

# One more thing worth knowing before you panic: a pod that mounts a volume
# through different subPaths used to be able to share it across labels too.
# That case also stops working, and it is rare enough that upstream says it has
# never been seen in practice.

Two sharing patterns break, and only one of them occurs in the wild. The first is two pods sharing a volume through different subPaths with different labels, which upstream describes as very niche and says it has never seen in practice. The second is a privileged pod and an unprivileged pod sharing a volume — still uncommon, but observed in real applications, and the one to go looking for. The KEP's own upgrade story is worth reading if you are the person who has to sign off on this, because it is the closest thing to an official runbook that exists.^[kepstory3]

The audit that has to happen before the upgrade

Kubernetes v1.36 shipped a controller for exactly this problem and almost nobody has switched it on, because it is opt-in and because the thing it warns about had not happened yet. selinux-warning-controller runs inside kube-controller-manager behind --controllers=*,selinux-warning-controller, watches every pod in the cluster, and reports each pair of pods that share a volume in a way SELinuxMount will not permit. It reports them even when the pods are on different nodes, on the correct reasoning that the scheduler may put them together tomorrow.^[k8s136]

# Kubernetes v1.36 shipped a controller whose entire job is to find these
# conflicts before the upgrade turns them into stuck pods. It is off by
# default and it is the single most useful thing in this article.
#
# It runs inside kube-controller-manager. `*` keeps every default controller
# and adds this one; listing it alone would disable all the others.

# kubeadm clusters: edit the static pod manifest on each control plane node.
sudo vi /etc/kubernetes/manifests/kube-controller-manager.yaml
#   spec:
#     containers:
#     - command:
#       - kube-controller-manager
#       - --controllers=*,bootstrapsigner,tokencleaner,selinux-warning-controller
#                                                      ^^^^^^^^^^^^^^^^^^^^^^^^^
# The kubelet restarts the pod when the file changes. Give it a minute.

# Note the second requirement, which is easy to miss: you must NOT have
# explicitly disabled the SELinuxChangePolicy feature gate. It is GA and on by
# default, so this only bites clusters carrying an old --feature-gates line.
kubectl -n kube-system get pod -l component=kube-controller-manager \
  -o jsonpath='{.items[*].spec.containers[*].command}' | tr ',' '\n' \
  | grep -E 'feature-gates|controllers='

# Confirm it is running before you trust its silence:
kubectl -n kube-system logs -l component=kube-controller-manager --tail=200 \
  | grep -i 'selinux'
# ... "Starting controller" controller="selinux-warning-controller"

# Enabling it has one privacy consequence worth stating: the metric it emits
# carries namespace names as labels, so it can leak namespace names to anyone
# who can read kube-controller-manager metrics. Upstream's assumption is that
# only cluster administrators can.

Then read both metrics, because they answer different questions and neither is sufficient alone. The controller's selinux_warning_controller_selinux_volume_conflict carries the conflicting pod names and namespaces as labels — that is your work list. The kubelet's volume_manager_selinux_volume_context_mismatch_warnings_total has no pod-name label at all, but it is the honest count of pods that would actually fail, emitted while SELinuxMount is still disabled. That last clause is the whole point of doing this on v1.36: the warnings counter is non-zero while everything still works. After the upgrade the same measurement appears as ..._errors_total, and by then the pods are stuck rather than at risk.^[selblog]

# There are two metrics and they answer two different questions. You need both:
# one tells you WHICH pods, the other tells you HOW MANY will actually fail.

# --- (1) From kube-controller-manager: which pods conflict, by name ---------
# Reported even when the pods are on different nodes, because the scheduler
# may put them together tomorrow.
kubectl get --raw /metrics 2>/dev/null | true   # via your metrics stack, or:
kubectl -n kube-system exec -it \
  "$(kubectl -n kube-system get pod -l component=kube-controller-manager \
     -o name | head -1)" -- \
  curl -sk https://127.0.0.1:10257/metrics \
  | grep '^selinux_warning_controller_selinux_volume_conflict'
# selinux_warning_controller_selinux_volume_conflict{
#   pod1_name="my-other-pod",pod1_namespace="default",
#   pod1_value="system_u:object_r:container_file_t:s0:c0,c1",
#   pod2_name="my-pod",pod2_namespace="default",
#   pod2_value="system_u:object_r:container_file_t:s0:c0,c2",
#   property="SELinuxLabel"} 1

# --- (2) From the kubelet: how many pods would actually fail ---------------
# Emitted while SELinuxMount is still DISABLED, which is exactly the window
# you are in before the upgrade. It has no pod-name label - that is what the
# controller above is for - but it is the honest count.
kubectl get --raw "/api/v1/nodes/node-01/proxy/metrics" \
  | grep '^volume_manager_selinux_volume_context_mismatch_warnings_total'
# volume_manager_selinux_volume_context_mismatch_warnings_total{...} 2

# After the upgrade the same measurement lives under a different name, and by
# then the pods are already stuck rather than merely at risk:
#   volume_manager_selinux_volume_context_mismatch_errors_total

# The whole point of doing this on v1.36 is that the warnings metric is
# non-zero while everything still works. Read it while it is still cheap.

For every workload the metrics name, you have two choices: fix the sharing, or opt that pod out. The opt-out is a Pod field, spec.securityContext.seLinuxChangePolicy, and it has been stable API since v1.36 — which means you can apply it now, on your current version, with no behavioural change, and it will do the right thing the moment the upgrade lands. That is the cheapest insurance in this entire release. Setting it to Recursive keeps the old behaviour for that pod and costs you the performance win, which for a workload that was sharing a volume across labels is a trade you were already making.^{[seccontext][mutadm]}

# For every workload the metrics named, you have two options: fix the sharing,
# or opt that pod out of the mount-option path. The opt-out is a Pod field and
# it is stable API as of Kubernetes v1.36.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: legacy-shared-cache
spec:
  template:
    spec:
      securityContext:
        # Three values:
        #   unset       - follow the cluster default. On v1.37 with SELinuxMount
        #                 enabled, that means mount options.
        #   MountOption - use mount options explicitly. Only valid while the
        #                 SELinuxMount feature gate is on.
        #   Recursive   - the pre-1.37 behaviour: the runtime relabels every
        #                 file. Slower on large volumes, and it is what lets two
        #                 differently labelled pods share one volume.
        seLinuxChangePolicy: Recursive
      containers:
        - name: app
          image: registry.internal.example.com/app:1.4.2
---
# Applying this to every affected workload by hand does not scale, and both
# the SELinux blog and the KEP say so. Prefer a policy. In-tree:

apiVersion: admissionregistration.k8s.io/v1beta1
kind: MutatingAdmissionPolicy
metadata:
  name: selinux-recursive-optout
spec:
  matchConstraints:
    resourceRules:
      - apiGroups:   [""]
        apiVersions: ["v1"]
        operations:  ["CREATE"]
        resources:   ["pods"]
  # Scope this to the namespaces the metrics actually named. A cluster-wide
  # opt-out works, but it also throws away the performance win for every
  # workload that was never at risk.
  matchConditions:
    - name: only-flagged-namespaces
      expression: "request.namespace in ['default', 'ci']"
  failurePolicy: Fail
  reinvocationPolicy: IfNeeded
  mutations:
    - patchType: ApplyConfiguration
      applyConfiguration:
        expression: >
          Object{ spec: Object.spec{
            securityContext: Object.spec.securityContext{
              seLinuxChangePolicy: "Recursive" } } }

Applying that field to every affected Deployment and StatefulSet by hand does not scale past a small cluster, and the SELinux blog says so directly, naming MutatingAdmissionPolicy, mutating webhooks, Kyverno and Gatekeeper as the ways to do it in bulk. One piece of advice on scope: resist the temptation to apply Recursive cluster-wide as a blanket precaution. It works, and it throws away the performance improvement for every workload that was never at risk — which, in most clusters, is nearly all of them. Scope it to the namespaces the metrics actually named.^[kyverno]

Static pods and their Secret references: this one happened in 1.34

This is the failure most likely to be blamed on 1.37 by somebody who upgraded from 1.33. Static pods are managed by the kubelet from a directory on disk rather than created through the API server, so they were never supposed to be able to read API objects at all; a defect let them reference Secrets and ConfigMaps through fields like configMapRef and secretRef. The restriction that closes it is the PreventStaticPodAPIReferences feature gate, and it has defaulted to on since v1.34 — so on any cluster that actually passed through 1.34, 1.35 and 1.36, this has already happened and the affected pods already broke. The v1.37 sneak peek announced that the gate itself was being removed, taking the opt-out with it; the feature-gates reference shipped with v1.37 still lists PreventStaticPodAPIReferences as a beta gate defaulting to true, so the escape hatch may technically still be there. Plan as if it is not. It was a defect rather than a feature, it is not coming back, and grepping the static pod directories costs less than discovering this per node.^{[staticpod][iss140226]}

# Static pods are managed by the kubelet from a directory on disk, not by the
# API server. They were never supposed to be able to read API objects; a bug
# let them, through envFrom.configMapRef, envFrom.secretRef, valueFrom and
# volume references. The gate that closes it - PreventStaticPodAPIReferences -
# has defaulted to true since v1.34, so this is only "new in 1.37" for a
# cluster that skipped releases. The v1.37 sneak peek says the gate was
# removed in this release; the shipped v1.37 feature-gates reference still
# lists it as Beta/true. Plan as if the opt-out is gone: it closed a defect,
# and it is not coming back. Find the references before the upgrade.

# Where the manifests live. Do not assume /etc/kubernetes/manifests: read it
# from the kubelet's own configuration.
sudo grep -E '^staticPodPath:' /var/lib/kubelet/config.yaml
# staticPodPath: /etc/kubernetes/manifests

# The audit, per node. Any hit is a pod that will fail to start on v1.37.
sudo grep -rnE 'configMapRef|secretRef|configMapKeyRef|secretKeyRef|(configMap|secret):' \
  /etc/kubernetes/manifests/
# /etc/kubernetes/manifests/node-exporter.yaml:24:            secretRef:
# /etc/kubernetes/manifests/node-exporter.yaml:25:              name: scrape-creds

# Fleet-wide, without logging into every box. Note that the control plane's
# own static pods (kube-apiserver, etcd, kube-scheduler,
# kube-controller-manager) are generated by kubeadm and do not use these
# references, so a clean result there is expected rather than reassuring.
for n in $(kubectl get nodes -o name); do
  printf '%-22s ' "${n#node/}"
  kubectl debug "$n" -q -it --image=busybox --profile=general -- \
    chroot /host sh -c \
    'grep -rlE "configMapRef|secretRef|configMapKeyRef|secretKeyRef" \
       /etc/kubernetes/manifests/ 2>/dev/null | tr "\n" " " || true' 2>/dev/null
  echo
done

# The fix is to stop being a static pod, or stop needing the reference:
#   * A DaemonSet is the right answer for almost everything that is a static
#     pod today for historical reasons. It can read Secrets normally.
#   * If it has to stay static, put the value in the manifest, or bind-mount a
#     file from the host and read it from there. Both are worse than a
#     DaemonSet, and both work.

The fix is almost always to stop being a static pod. A great many static pods exist for historical reasons — they were the way to run a node-level agent before DaemonSets were as capable as they are now — and a DaemonSet can read Secrets normally, gets rolling updates, and shows up in the places people look. Where something genuinely has to stay static, the options are to inline the value in the manifest or to bind-mount a file from the host and read it from there. Both are worse than a DaemonSet and both work. Note separately that kubectl run --filename/-f is deprecated in this release too, on the grounds that the pod it produced was always built purely from the command-line arguments — that one affects scripts, not clusters.^[iss138671]

kube-proxy ipvs: deprecated since 1.35, not removed

Now the change that is most over-reported. kube-proxy's ipvs mode has carried a deprecation notice since v1.35, not since v1.37. What v1.37 adds is the KubeProxyIPVS feature gate, itself marked deprecated, alongside the warning kube-proxy logs at startup. Nothing stops working, no feature gate flips, and no traffic is affected. The reasoning behind the deprecation is worth understanding because it explains why there was never a fix: the kernel's ipvs API cannot express everything a Kubernetes Service needs, so ipvs mode has always fallen back to iptables underneath for parts of the job. KEP-3866 puts it bluntly in a section heading — the ipvs mode of kube-proxy will not save us.^{[kep5495][kep3866]}

The destination is nftables mode, which has been GA since v1.33 and is the recommended mode for Linux nodes. Clusters do not migrate on their own; you have to set mode: "nftables" explicitly. The kernel requirement is real but not onerous — every kernel too old to support nftables mode leaves LTS by the end of 2026 — and the nftables bugfixes were backported to the 1.33 and 1.34 branches specifically so that ipvs users on older releases could migrate without upgrading Kubernetes first.^[nftblog]

# What actually happens on v1.37 if you run ipvs mode: kube-proxy logs the
# same deprecation warning it has logged since v1.35. That is all. Nothing
# stops working, no feature gate flips, and no traffic is affected. What v1.37
# adds is the KubeProxyIPVS gate, itself marked deprecated - the switch that
# will later be used to turn the mode off by default.
kubectl -n kube-system logs -l k8s-app=kube-proxy --tail=50 | grep -i deprecat
# W0826 ... "ipvs mode of kube-proxy is deprecated and will be removed in a
#            future release; see KEP-5495"

# The reason is worth knowing, because it explains why there is no fixing it:
# the kernel's ipvs API cannot express everything a Kubernetes Service needs,
# so ipvs mode has always fallen back to iptables underneath for parts of the
# job. It was never the clean escape from iptables it was sold as.

# The destination is nftables mode, GA since v1.33. Clusters never migrate on
# their own - you have to set it.

# --- migrating, on a kubeadm cluster ---------------------------------------
# 1. Check the kernel. nftables mode wants a reasonably modern kernel; every
#    kernel too old for it leaves LTS by the end of 2026.
kubectl get nodes -o jsonpath='{range .items[*]}{.status.nodeInfo.kernelVersion}{"\n"}{end}' \
  | sort -u

# 2. Change the mode in the ConfigMap.
kubectl -n kube-system get configmap kube-proxy -o yaml > /tmp/kube-proxy.bak.yaml
kubectl -n kube-system patch configmap kube-proxy --type merge -p \
  "$(printf '{"data":{"config.conf":%s}}' \
     "$(kubectl -n kube-system get cm kube-proxy -o jsonpath='{.data.config\.conf}' \
        | sed 's/^\(\s*mode:\).*/\1 "nftables"/' | jq -Rs .)")"

# 3. Roll the DaemonSet one node at a time and watch, rather than all at once.
kubectl -n kube-system rollout restart daemonset/kube-proxy
kubectl -n kube-system rollout status daemonset/kube-proxy --timeout=10m

# 4. Verify from the data plane, not the control plane. A Service that
#    resolves but does not connect is the failure mode here.
kubectl run nft-check --rm -it --restart=Never --image=busybox -- \
  sh -c 'wget -qO- --timeout=5 http://kubernetes.default.svc/healthz || echo FAILED'

# Rolling back is the same edit in reverse; keep /tmp/kube-proxy.bak.yaml.
# Do this as its own change, on its own day. Bundling a proxy-mode migration
# into a version upgrade means that when connectivity breaks you will not know
# which one did it.

One piece of sequencing advice, offered with feeling: do the proxy-mode migration in its own maintenance window, on its own day, not bundled into the version upgrade. Both changes touch the data plane, and when connectivity breaks in a window that contained both, you will spend the outage working out which one did it instead of fixing it. There is no time pressure here to justify combining them — the deprecation policy guarantees a beta-or-better feature a long runway, and the KEP's own graduation criteria put the feature gate defaulting to false at v1.40 and removal at v1.43.^{[kubeproxycfg][deprecpolicy]}

cgroup v1: this one happened in 1.35

The cgroup v1 story is the one most often misattributed, and getting it right changes what you do about it. The kubelet setting failCgroupV1 has defaulted to true since Kubernetes v1.35. A node still on cgroup v1 has therefore been refusing to start its kubelet since then, unless somebody added failCgroupV1: false — which a lot of people did, at speed, during the v1.35 upgrade, and then never revisited. So the useful question on the way to v1.37 is not "will this break" but "who is still carrying the override, and for how much longer".^{[kep5573][kubeletcfg]}

# This is the change most often misattributed to v1.37. The kubelet setting
# `failCgroupV1` has defaulted to TRUE since Kubernetes v1.35. A node still on
# cgroup v1 has therefore been refusing to start its kubelet since v1.35,
# unless somebody added the override - which many people did, in a hurry, and
# then forgot.

# So the useful question on the way to v1.37 is not "will this break" but
# "who is still carrying the override?"
for n in $(kubectl get nodes -o name); do
  printf '%-22s ' "${n#node/}"
  kubectl debug "$n" -q -it --image=busybox --profile=general -- \
    chroot /host sh -c \
    'printf "cgroup=%s override=%s\n" \
       "$(stat -fc %T /sys/fs/cgroup)" \
       "$(grep -c failCgroupV1 /var/lib/kubelet/config.yaml 2>/dev/null)"' 2>/dev/null
done
# node-01  cgroup=cgroup2fs override=0     <- fine
# node-02  cgroup=tmpfs     override=1     <- v1, running on borrowed time

# The override itself, for reference. It is a stopgap and upstream says so:
#   apiVersion: kubelet.config.k8s.io/v1beta1
#   kind: KubeletConfiguration
#   failCgroupV1: false

# v1.37 still honours it. KEP-5573 removes cgroup v1 support outright in a
# later release, and no date has been committed to, so the honest planning
# assumption is "the next one that suits SIG Node" rather than a fixed month.

# What you lose in the meantime is not theoretical. In-place pod resizing and
# tiered memory protection - both features people are actively asking for -
# depend on cgroup v2 and simply do not work on a v1 node.

# Switching the node is a kernel command line change and a reboot:
#   systemd.unified_cgroup_hierarchy=1
# ...and then the container runtime's cgroup driver has to agree with the
# kubelet's. That is a longer job than it looks, which is why it has its own
# article.

v1.37 still honours the override, and KEP-5573 removes cgroup v1 support outright in a later release with no date committed to — so the honest planning assumption is "whichever release suits SIG Node" rather than a month you can put in a plan. What you give up in the meantime is not theoretical: in-place pod resizing and tiered memory protection both depend on cgroup v2 and simply do not function on a v1 node, and those are features teams are actively asking for. The switch itself is a kernel command line change and a reboot, plus making the container runtime's cgroup driver agree with the kubelet's — which is more work than it sounds and has its own article.^[cgroups]

containerd 1.x: the cliff is 1.38, not now

This one is misreported in the other direction, and getting it wrong costs you a maintenance window you did not need to spend. Support for containerd 1.x was not removed in v1.36, and it is not removed in v1.37. A v1.37 kubelet still runs against it: with KubeletCgroupDriverFromCRI enabled the kubelet asks the runtime for its cgroup driver over the RuntimeConfig CRI RPC, containerd 1.y does not implement that RPC, and the kubelet quietly falls back to its own --cgroup-driver value while incrementing the cri_losing_support metric. That fallback was originally scheduled to disappear in v1.37 and was deferred by one release, explicitly to align with containerd v1.7's own support window. The Kubernetes runtime documentation now says it plainly: in v1.38 the fallback is dropped, and older containerd versions fail against newer kubelets.^{[k8s134][ctrrel]}

# The change most often stated backwards. containerd 1.x was NOT removed in
# v1.36, and it still runs against a v1.37 kubelet. What it runs on is a
# fallback: the kubelet asks the runtime for its cgroup driver over the
# RuntimeConfig CRI RPC, containerd 1.y does not implement that RPC, and the
# kubelet falls back to its own --cgroup-driver value. That fallback was
# scheduled to go in v1.37 and was deferred one release to align with
# containerd v1.7's support window, so it disappears in v1.38.

# The audit is a metric, not a spreadsheet. Every node relying on the fallback
# increments this, so scrape it rather than walking nodes by hand.
kubectl get --raw /metrics | grep '^cri_losing_support'
# cri_losing_support{version="1.38.0"} 1

# Then confirm which nodes, and with what:
kubectl get nodes -o custom-columns=\
'NODE:.metadata.name,KUBELET:.status.nodeInfo.kubeletVersion,'\
'RUNTIME:.status.nodeInfo.containerRuntimeVersion' | sort -k3
# NODE     KUBELET   RUNTIME
# node-02  v1.37.0   containerd://1.7.28   <- works today, fails on 1.38
# node-01  v1.37.0   containerd://2.3.2

crictl version
# RuntimeName:        containerd
# RuntimeVersion:     v2.3.2
# RuntimeApiVersion:  v1

# Two things make this less comfortable than "one release of runway" sounds:
# containerd 1.7's own extended support ends in September 2026, and
# containerd's published Kubernetes support matrix has no row for 1.37 yet -
# the last row is 1.36 (2.3.0+, 2.2.0+). Book the migration before 1.38, in
# its own window. Landing two runtime-level changes together means that when a
# node comes back wrong you will be bisecting instead of fixing.

So the deadline is real and it is exactly one release away — a better position than the panic implies, and a worse one than doing nothing suggests. Two things sharpen it. The runtime side of the clock runs out first: containerd 1.7 is the LTS branch this hinges on, and its extended support ends in September 2026, which is now. And containerd's own Kubernetes support matrix currently has no row for Kubernetes 1.37 at all — the last published row is 1.36, listing 2.3.0+ and 2.2.0+ — so anybody quoting you a containerd-blessed version for 1.37 is extrapolating rather than citing. Practically: scrape cri_losing_support, which is already being emitted by every node that needs the fallback and is a better audit than walking nodes by hand, then confirm with containerRuntimeVersion. Do the runtime migration before 1.38 and in its own window. Landing two runtime-level changes together means that when a node comes back wrong you will be bisecting rather than fixing.^[runtimes]

metrics.k8s.io finally reaches v1

The good news in this release, and it is genuinely good. metrics.k8s.io graduates to v1 after nearly nine years in beta. This is the API behind kubectl top and behind the HorizontalPodAutoscaler's CPU and memory metrics, which makes it one of the most widely used interfaces in Kubernetes and an odd thing to have left in beta for that long. The graduation recognises stability rather than introducing change: no functional differences are expected, and both v1 and v1beta1 remain served during the transition.^{[kep5207][metricspipe]}

# The good news in this release. metrics.k8s.io graduates to v1 after nearly
# nine years in beta. Both versions stay served during the transition, so
# there is nothing to do on upgrade day - this is a thing you can adopt on
# your own schedule rather than a thing that happens to you.

kubectl get --raw /apis/metrics.k8s.io | jq -r '.versions[].groupVersion'
# metrics.k8s.io/v1
# metrics.k8s.io/v1beta1

# What consumes it: `kubectl top`, and the HorizontalPodAutoscaler's cpu and
# memory metrics. Both keep working without changes.
kubectl top nodes
kubectl top pods -A --sort-by=memory | head

# Where it matters is code you own. Anything that talks to the API directly -
# a custom autoscaler, a capacity report, a dashboard backend - should move
# off v1beta1 while both are available rather than after one is not.
kubectl get --raw /apis/metrics.k8s.io/v1/nodes | jq '.items[0]'

# Find the clients still on the beta path, from the API server's own counters:
kubectl get --raw /metrics \
  | grep 'apiserver_requested_deprecated_apis\|metrics.k8s.io.*v1beta1' | head

# There is no removal date for v1beta1 yet. Kubernetes' deprecation policy
# guarantees a beta API at least nine months or three releases after
# deprecation, so this is a housekeeping item, not a deadline.

So there is nothing to do on upgrade day, which is the point — this is a thing you adopt on your own schedule rather than a thing that happens to you. Where it does matter is code you own: a custom autoscaler, a capacity report, a dashboard backend, anything that talks to the API directly should move off the beta path while both are available rather than after one is not. There is no removal date for v1beta1 yet, and the deprecation policy guarantees a beta API at least nine months or three releases, so treat it as housekeeping rather than a deadline.^[hpa]

The rest of the release, briefly

Three more things in this release are worth knowing about even though none of them will affect an upgrade. All three are graduations rather than removals, and the first is the one to watch.^[kep4960]

• Kubelet in a user namespace — rootless mode — reaches beta. Node components have traditionally run as root on the host. This lets them run as an unprivileged user on the host while still appearing as root inside a Linux user namespace, which limits the blast radius of a vulnerability in a node component. Beta here means the gate is on by default, but enabling it does not by itself run the kubelet in a user namespace — there is host configuration behind it — and the change did not make the release announcement. Treat it as something to try on a test node rather than something that has happened to you. It is also not the same feature as user namespaces for pods, which went beta in v1.35 and GA in v1.36.
• SELinux volume relabelling for ReadWriteOncePod volumes was already GA in v1.36. That is SELinuxMountReadWriteOncePod, a narrower gate than the SELinuxMount this article is about, and it is why some clusters have been mounting with a context option for a while without anything breaking — RWOP volumes cannot be shared by definition, so the conflict this article describes cannot arise there.
• Volume health monitoring restarts at alpha. An initial implementation landed in v1.21 and never graduated; KEP-1432 resets it behind the CSIVolumeHealth gate and introduces four CSI RPCs — ControllerListVolumeHealth, ControllerGetVolumeHealth, NodeGetVolumeHealth and NodeGetStorageHealth — reporting into PersistentVolumeClaim.status.healthStatus, Pod.status.volumeHealth and CSINode.status.storageHealth. The vocabulary is deliberately small and machine-readable: Inaccessible, DataLoss and Degraded, with reason and message on the condition for driver-specific detail.

Alpha means off by default and not for production, but the health monitoring work is worth tracking if you have ever had to cross-reference a hung mount against a storage vendor's dashboard to work out what was wrong. It is the first machine-readable answer Kubernetes has offered to that question.^{[kep1432][userns]}

The upgrade, in order

The order matters more than the commands. Everything that is genuinely hard about this upgrade is hard before it, and the one step that becomes materially more expensive afterwards is the SELinux audit — so it goes first, by weeks rather than by minutes.^[kubeadmup]

# The order matters more than the commands, and the SELinux audit has to come
# first because it is the only step that is materially harder after the
# upgrade than before it.

# --- WEEKS BEFORE, on v1.36 ------------------------------------------------
# 1. Turn on selinux-warning-controller, read both metrics, apply the opt-outs.
# 2. Audit static pods for Secret and ConfigMap references. Move them to
#    DaemonSets where you can.
# 3. Get every node onto containerd 2.x and cgroup v2, if any are not.
#    Both of these are already overdue rather than upcoming.
# 4. Decide about ipvs - and then do it in a DIFFERENT window.

# --- THE DAY ---------------------------------------------------------------
# Control plane first, one node at a time. Nothing here is 1.37-specific;
# it is the standard kubeadm sequence and it is standard because it works.
sudo apt-mark unhold kubeadm && sudo apt-get update \
  && sudo apt-get install -y kubeadm='1.37.0-*' && sudo apt-mark hold kubeadm

sudo kubeadm upgrade plan
sudo kubeadm upgrade apply v1.37.0        # first control plane node
# sudo kubeadm upgrade node               # every other control plane node

# Then the kubelet and kubectl on that same node:
sudo apt-mark unhold kubelet kubectl && sudo apt-get update \
  && sudo apt-get install -y kubelet='1.37.0-*' kubectl='1.37.0-*' \
  && sudo apt-mark hold kubelet kubectl
sudo systemctl daemon-reload && sudo systemctl restart kubelet

# --- WORKER NODES, ONE AT A TIME -------------------------------------------
NODE=node-02
kubectl drain "$NODE" --ignore-daemonsets --delete-emptydir-data --timeout=15m
# ... upgrade kubeadm, run `kubeadm upgrade node`, upgrade kubelet, restart ...
kubectl uncordon "$NODE"

# Then STOP and look, before the next node. The failure this release can
# produce is a pod stuck in ContainerCreating, and it is per-node:
kubectl get pods -A --field-selector spec.nodeName="$NODE" \
  -o wide | grep -v Running | grep -v Completed

# The version skew policy is what makes the staged rollout legal: a kubelet
# may be up to three minor versions behind the API server, so a fleet halfway
# through this is a supported configuration rather than a risk in itself.

One reassurance about doing this gradually: the version skew policy permits a kubelet to run up to three minor versions behind the API server, so a fleet halfway through this rollout is a supported configuration rather than a risk in its own right. Take the time. Upgrade one node, look at it properly, and only then continue — because the failure this release can produce is per-node and shows up as a pod that never starts rather than as an error anyone pages you about.^[skew]

Rolling back, and what does not roll back

Rollback deserves a straight answer rather than a reassuring one, and on this upgrade the answer has three parts. The reversible changes are genuinely reversible: the kube-proxy mode is a ConfigMap and a DaemonSet restart, and seLinuxChangePolicy is a Pod field that behaves identically on v1.36 and v1.37 — which is precisely why applying it before the upgrade costs nothing and buys you the whole rollback story.^[kubeadmup]

# Rollback deserves a straight answer. Most of this release rolls back; one
# part of it does not roll back in the way people assume.

# --- What rolls back cleanly ----------------------------------------------
# The kube-proxy mode change: it is a ConfigMap and a DaemonSet restart.
kubectl -n kube-system apply -f /tmp/kube-proxy.bak.yaml
kubectl -n kube-system rollout restart daemonset/kube-proxy

# The seLinuxChangePolicy opt-out: it is a Pod field. Setting it to Recursive
# is safe on v1.36 and v1.37 alike, which is why applying it BEFORE the
# upgrade costs nothing and buys the whole rollback story.

# --- What does not ---------------------------------------------------------
# The control plane. `kubeadm upgrade` has no downgrade path: going back means
# restoring the etcd snapshot you took before you started, which means losing
# everything written to the cluster since. If you did not take one, you do not
# have a rollback - you have a forward fix.
sudo ETCDCTL_API=3 etcdctl \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key \
  snapshot save "/var/backups/etcd-pre-1.37-$(date +%F).db"

# The static pod references. If a static pod was relying on a secretRef, the
# reference is gone on 1.37 and comes back on a downgrade - but the downgrade
# is the control plane operation above, so in practice the fix is forward:
# move it to a DaemonSet.

# --- The one that surprises people ----------------------------------------
# Downgrading the kubelet does NOT un-stick a pod that failed to mount because
# of an SELinux label conflict, because the pod that WON the volume is still
# holding it with its own context. Terminate one of the two, or set
# seLinuxChangePolicy: Recursive on both and let them share again. Version
# numbers are not the lever here; the Pod field is.

The control plane is the part that does not roll back. kubeadm upgrade has no downgrade path; going backwards means restoring the etcd snapshot you took before you started, and losing everything written to the cluster since. If you did not take one, you do not have a rollback — you have a forward fix, which is a different conversation to be having at two in the morning. And there is one failure that surprises people: downgrading the kubelet does not un-stick a pod that failed to mount because of an SELinux label conflict, because the pod that won the volume is still holding it with its own context. The lever there is the Pod field, not the version number.^[selblog]

Verifying, rather than hoping

Verification on this upgrade has a specific shape, because the characteristic failure leaves everything green. The node is Ready. The kubelet is healthy. The control plane is fine. There is simply a pod that never finishes creating, on one node, belonging to one team. So checking that the cluster is up proves nothing at all — you have to check the things that can be silently wrong.^[chlog]

#!/usr/bin/env bash
# Post-upgrade verification. Exits non-zero when something is wrong, so it can
# run between nodes in a pipeline rather than being read by a person at 3am.
set -uo pipefail
rc=0
fail() { printf '  FAIL  %s\n' "$*"; rc=1; }
pass() { printf '  ok    %s\n' "$*"; }

echo '== versions =='
kubectl version -o json | jq -r '.serverVersion.gitVersion'

echo '== every node Ready and on the expected version =='
bad=$(kubectl get nodes --no-headers | awk '$2!="Ready"{print $1}')
[ -z "$bad" ] && pass 'all nodes Ready' || fail "not Ready: $bad"

echo '== no pod stuck creating (the SELinux failure mode) =='
stuck=$(kubectl get pods -A --no-headers \
  | awk '$4=="ContainerCreating"{print $1"/"$2}')
[ -z "$stuck" ] && pass 'nothing in ContainerCreating' || fail "stuck: $stuck"

echo '== SELinux context mismatches that became real failures =='
for n in $(kubectl get nodes -o name); do
  v=$(kubectl get --raw "/api/v1/nodes/${n#node/}/proxy/metrics" 2>/dev/null \
      | awk '/^volume_manager_selinux_volume_context_mismatch_errors_total/{s+=$2} END{print s+0}')
  [ "$v" = "0" ] && pass "${n#node/}: 0 mismatch errors" \
                 || fail "${n#node/}: $v SELinux mismatch errors"
done

echo '== static pods all running =='
sp=$(kubectl get pods -A -o json \
  | jq -r '.items[] | select(.metadata.annotations["kubernetes.io/config.source"]=="file")
           | select(.status.phase!="Running") | .metadata.namespace+"/"+.metadata.name')
[ -z "$sp" ] && pass 'static pods Running' || fail "static pods not Running: $sp"

echo '== the resource metrics API answers on both versions =='
kubectl get --raw /apis/metrics.k8s.io | jq -e \
  '[.versions[].groupVersion] | index("metrics.k8s.io/v1")' >/dev/null \
  && pass 'metrics.k8s.io/v1 served' || fail 'metrics.k8s.io/v1 missing'
kubectl top nodes >/dev/null 2>&1 && pass 'kubectl top works' || fail 'kubectl top broken'

echo '== service traffic actually flows =='
kubectl run verify-net --rm -i --restart=Never --image=busybox --timeout=60s -- \
  sh -c 'wget -qO- --timeout=5 http://kubernetes.default.svc/healthz' >/dev/null 2>&1 \
  && pass 'in-cluster Service reachable' || fail 'in-cluster Service unreachable'

exit $rc

Run it between nodes rather than at the end. The script exits non-zero, which means it can sit in a pipeline step instead of being read by a person at three in the morning, and the two checks worth keeping even after the upgrade is behind you are the ContainerCreating sweep and the SELinux mismatch counter. Both are cheap, and both catch a class of problem that otherwise waits until someone notices their workload never came back.^[k8spatch]

The order to do this in

Compressed, this is a smaller upgrade than the article implies — provided you did the work that belonged to the two releases before it. The decision table below is really a question about which of those you skipped.^[sneak]

1. Answer the SELinux question first, in one command. If no node runs SELinux in enforcing mode, skip the largest section here entirely and treat 1.37 as a routine upgrade. If any node does, everything below applies.
2. Turn on selinux-warning-controller on v1.36 and read both metrics. The controller names the conflicting pods; the kubelet counts the ones that will actually fail. Do this weeks ahead, because it is materially harder after the upgrade and the pods are stuck by then rather than merely at risk.
3. Apply seLinuxChangePolicy: Recursive to the workloads the metrics named, by policy rather than by hand. It is stable API on v1.36, changes nothing today, and does the right thing the moment the upgrade lands. Scope it to the namespaces involved rather than the whole cluster.
4. Grep every node's static pod directory for secretRef and configMapRef. This has been the default since v1.34, so on a cluster that stepped through the releases it has already happened; on one that jumped, it is waiting. Move what you find to a DaemonSet, which is almost always what it should have been.
5. Clear the cgroup v1 debt from 1.35, and put the containerd migration in the calendar before 1.38. A node on cgroup v1 has been carrying an override for two releases. A node on containerd 1.x still works today and stops working one release from now. Both get their own window — and leave the ipvs to nftables migration for a different day entirely.

Two of the debts above have their own runbooks, because neither is a five-minute job: migrating containerd 1.7 to 2.x, which is the containerd 1.x removal from the runtime side, and migrating from cgroup v1 to cgroup v2, which is the audit and conversion behind the failCgroupV1 override. If the same maintenance window is also rebuilding your ingress, moving from ingress-nginx to the Gateway API covers that migration. And if this is the point at which someone in the room asks whether the whole thing is worth it, when not to use Kubernetes is the other side of that argument, made honestly.

Frequently asked questions

When is Kubernetes 1.37 released, and what actually breaks?

v1.37 was released on 26 August 2026. Only one change in it can stop a workload: SELinuxMount reaches GA and is enabled by default, which can leave pods in ContainerCreating where two differently labelled pods share a volume on an SELinux-enforcing node with a CSI driver that opted in. The rest of what is genuinely new is quiet — the KubeProxyIPVS feature gate is added and immediately marked deprecated, metrics.k8s.io graduates to v1 with both versions still served, and kubectl run --filename/-f is deprecated, which affects scripts rather than clusters. Everything else being attributed to this release landed earlier or has not landed yet: the cgroup v1 kubelet failure is v1.35, the static pod Secret restriction is v1.34, ipvs mode has been deprecated since v1.35 and is removed in v1.43, and containerd 1.x still works — its CRI fallback is dropped in v1.38.

Does Kubernetes 1.37 remove kube-proxy's ipvs mode?

No, and it did not deprecate it either — that happened in v1.35, and kube-proxy has been logging a warning at startup ever since. What v1.37 adds is the KubeProxyIPVS feature gate, itself marked deprecated, which is the switch that will later turn the mode off. Nothing stops working and no traffic is affected. KEP-5495 sets out the timeline: the gate is expected to default to false by v1.40, with support removed entirely by v1.43. The reason for the deprecation is that the kernel's ipvs API cannot express everything a Kubernetes Service needs, so ipvs mode always fell back to iptables underneath for parts of the job. The migration target is nftables mode, GA since v1.33 — and clusters never switch on their own, so you have to set mode: "nftables" yourself. Do it in a separate maintenance window from the version upgrade.

Why are my pods stuck in ContainerCreating after upgrading to 1.37?

If the node runs SELinux in enforcing mode, the likely cause is a volume-sharing conflict introduced by SELinuxMount going GA. Volumes are now mounted with -o context=<label> rather than being relabelled recursively, and a mount can hold exactly one SELinux context — so two pods with different labels sharing one volume on one node can no longer coexist. One sits in ContainerCreating until the other terminates. The classic case is a privileged pod and an unprivileged pod sharing a volume. Confirm it by reading the kubelet metric volume_manager_selinux_volume_context_mismatch_errors_total on that node. The fix is to set spec.securityContext.seLinuxChangePolicy: Recursive on the affected pods — note that downgrading the kubelet will not release the volume, because the pod that won it is still holding the context.

How do I find SELinux volume conflicts before I upgrade?

Enable the selinux-warning-controller that shipped in v1.36 by passing --controllers=*,selinux-warning-controller to kube-controller-manager, and make sure you have not explicitly disabled the SELinuxChangePolicy feature gate. Then read two metrics. selinux_warning_controller_selinux_volume_conflict carries the conflicting pod names and namespaces as labels — that is your work list, and it reports conflicts even when the pods are on different nodes, because the scheduler may co-locate them later. volume_manager_selinux_volume_context_mismatch_warnings_total, emitted by the kubelet while SELinuxMount is still disabled, has no pod-name label but gives the honest count of pods that would actually fail. You need both. Do this on v1.36, while the warnings counter is non-zero and everything still works.

What does seLinuxChangePolicy: Recursive actually do, and is it safe to set now?

It opts that pod out of the mount-option path and keeps the pre-1.37 behaviour: the container runtime walks the volume and relabels every file. The cost is the performance win — on a large or remote volume, recursive relabelling is genuinely slow — and the benefit is that two differently labelled pods can share the volume again. It is stable Pod API as of v1.36, so setting it today changes nothing about how your cluster behaves right now and does the right thing the moment the upgrade lands. That makes it the cheapest insurance in this release. Apply it through a MutatingAdmissionPolicy or a policy engine rather than by editing every Deployment, and scope it to the namespaces the conflict metrics actually named rather than the whole cluster — a blanket Recursive throws away the improvement for every workload that was never at risk.

My cluster does not use SELinux. Does any of this apply?

Almost none of it. When SELinux is unavailable or disabled in the kernel, the kubelet skips the entire SELinux code path, so the SELinuxMount change is a no-op for you and the largest section of this article can be ignored. What still applies is the static pod restriction — grep every node's static pod directory for secretRef and configMapRef, which has been the default since v1.34 and therefore only bites clusters that skipped releases — plus the cgroup v2 debt from v1.35 if you have not cleared it, and the containerd 2.x migration, which is not urgent on 1.37 but has to be done before 1.38. Check getenforce on every node rather than assuming, though: a mixed fleet where one node pool ships an SELinux-enabled image is more common than people expect.

Does 1.37 stop the kubelet from starting on cgroup v1?

That change is not from 1.37. The kubelet's failCgroupV1 setting has defaulted to true since v1.35, so a node on cgroup v1 has been failing to start its kubelet since then unless somebody added failCgroupV1: false. v1.37 still honours that override. KEP-5573 removes cgroup v1 support outright in a later release, but no date has been committed to, so the planning assumption should be "whichever release suits SIG Node" rather than a month. The reason not to sit on the override is that in-place pod resizing and tiered memory protection both require cgroup v2 and simply do not work without it. Switching a node means a kernel command line change (systemd.unified_cgroup_hierarchy=1), a reboot, and making the runtime's cgroup driver agree with the kubelet's.

Do I need containerd 2.0 for Kubernetes 1.37?

No — and this is the claim most often stated backwards. containerd 1.x still runs against a v1.37 kubelet. It does so through a fallback: the kubelet asks the runtime for its cgroup driver over the RuntimeConfig CRI RPC, containerd 1.y does not implement that RPC, and the kubelet falls back to its own --cgroup-driver value while incrementing the cri_losing_support metric. That fallback was scheduled for removal in v1.37 and deferred one release to align with containerd v1.7's support window, so it disappears in v1.38 — at which point older containerd versions do fail against newer kubelets. Two caveats that make this less comfortable than it sounds: containerd 1.7's own extended support ends in September 2026, and containerd's published Kubernetes support matrix has no row for 1.37 yet, so nobody can currently cite an officially blessed pairing. Audit with cri_losing_support and containerRuntimeVersion, and book the migration before you take 1.38 — in its own window, not bundled with a version upgrade.

What replaces a static pod that reads a Secret?

A DaemonSet, in almost every case. Static pods are managed by the kubelet from a directory on disk rather than created through the API server, so reading API objects was never supposed to work — a defect allowed it through fields like configMapRef and secretRef. The PreventStaticPodAPIReferences gate that closes the defect has defaulted to on since v1.34, so this is not new in 1.37 — it only feels new to clusters that jumped several releases. The v1.37 sneak peek said the gate was being removed outright; the feature-gates reference shipped with v1.37 still lists it as a beta gate defaulting to true. Either way, do not plan around the opt-out. Most static pods exist for historical reasons, from before DaemonSets were as capable as they are now; a DaemonSet reads Secrets normally, gets rolling updates, and appears where people look for it. If something genuinely has to remain static, inline the value in the manifest or bind-mount a file from the host and read it from there. Both are worse than a DaemonSet and both work. Find them before the upgrade rather than one node at a time.

Should I upgrade straight from 1.34 or 1.35 to 1.37?

You can — the version skew policy permits a kubelet up to three minor versions behind the API server, and kubeadm handles the control plane a version at a time — but the risk is not the mechanics, it is the reading. Skipping releases means every deprecation from the versions you skipped arrives at once, and they are documented per release rather than cumulatively. Coming from 1.35 you also inherit the v1.36 changes: the Service ExternalIPs deprecation warning, SELinuxMountReadWriteOncePod and SELinuxChangePolicy reaching GA, and pod user namespaces going stable. Coming from 1.34 you additionally inherit the failCgroupV1 default flip from v1.35 — which will stop the kubelet outright on a cgroup v1 node. Read the release-team blog post for every version you are stepping over. Three of them take about twenty minutes and are a better use of an upgrade window than most of what goes into one.

Can I roll back a 1.37 upgrade?

Partly, and the parts matter. The kube-proxy mode change rolls back cleanly — it is a ConfigMap and a DaemonSet restart. seLinuxChangePolicy is a Pod field that behaves the same on v1.36 and v1.37, which is why applying it beforehand costs nothing. The control plane does not roll back: kubeadm upgrade has no downgrade path, so going backwards means restoring the etcd snapshot you took before starting and losing everything written since. Take that snapshot. And one failure surprises people: downgrading the kubelet does not un-stick a pod that failed to mount because of an SELinux conflict, because the pod that won the volume still holds the mount's context. The lever is seLinuxChangePolicy on both pods, not the version number.

Is metrics.k8s.io v1beta1 going away?

Not yet, and not without notice. metrics.k8s.io graduates to v1 in v1.37 after nearly nine years in beta, and both v1 and v1beta1 remain served during the transition specifically so that adoption can happen at your own pace. No functional changes are expected — the graduation recognises stability rather than introducing it. kubectl top and the HorizontalPodAutoscaler carry on working without any action. What is worth doing is moving code you own — a custom autoscaler, a capacity report, a dashboard backend — off the beta path while both are available. Kubernetes' deprecation policy guarantees a beta API at least nine months or three releases after deprecation, and v1beta1 has not been given a removal date at all, so this is housekeeping rather than a deadline.

Sources

Primary sources first. The release team's sneak peek and the v1.37 changelog are the only authoritative statements about what is in this release; the KEPs are the only place the multi-release timelines are written down, which is what makes them the antidote to a summary that says "removed in 1.37" about something that is removed in 1.43. Where this article corrects something — that the cgroup failure is a v1.35 change, that the static pod restriction is a v1.34 change, that ipvs has been deprecated since v1.35 rather than removed in v1.37, and that the containerd cliff is ahead of you in v1.38 rather than behind you in v1.36 — the disagreement is with the secondary coverage, not with the project. One disagreement is internal to the project and worth naming: the sneak peek says the PreventStaticPodAPIReferences gate was removed in this release, and the feature-gates reference published with v1.37 still lists it. The reference page is the one to trust for what shipped.

1. Kubernetes v1.37 Sneak Peek - the release team's own list of what is deprecated, removed and breaking in this release, published 31 July 2026. This is the document that separates "new in 1.37" from "still in progress", and it carries its own caveat that the information reflects the state of the release before the release date
2. Kubernetes CHANGELOG-1.37.md - the authoritative record once the release is cut on 26 August 2026. Where this article and the changelog disagree after that date, the changelog is right and this page is a snapshot of the plan
3. SELinux Volume Label Changes goes GA (and likely implications in v1.37) - the pre-announcement by the feature's own authors. It contains the five conditions for a mount-option relabel, the two conflict scenarios, the seLinuxChangePolicy opt-out, the selinux-warning-controller and the recommended upgrade path. This is the single most important source for this article
4. KEP-1710: Speed up SELinux volume relabeling using mounts. The enhancement proposal behind SELinuxMount, including why a mount can hold only one context and therefore why volume sharing across differently labelled pods stops working
5. KEP-1710, "Story 3: cluster upgrade" - the upgrade scenario written by the authors, which is the closest thing to an official runbook for this change
6. Kubernetes - Configure a Security Context for a Pod or Container: seLinuxOptions, the seLinuxChangePolicy field, efficient SELinux volume relabeling and the selinux-warning-controller
7. Kubernetes API reference - CSIDriver: the seLinuxMount field a driver has to set to true before the kubelet will mount its volumes with a context option. Drivers that do not set it keep the old recursive behaviour, which is why the blast radius of this change is driver-specific
8. Kubernetes - Feature Gates: the per-release state of SELinuxMount, SELinuxChangePolicy, SELinuxMountReadWriteOncePod and KubeProxyIPVS. The table is the fastest way to check a claim about which release flipped which default
9. KEP-5495: Deprecate ipvs mode in kube-proxy. The deprecation timeline this article quotes - warning now, feature gate defaulting to false by v1.40, removal by v1.43 - comes from its graduation criteria and nowhere else
10. KEP-5495 README on GitHub: the same document at its source, including the graduation criteria table with the release numbers
11. KEP-3866: nftables kube-proxy backend, including the section titled "The ipvs mode of kube-proxy will not save us" - the technical argument that ipvs mode never stopped depending on iptables underneath, which is the reason for the deprecation
12. NFTables mode for kube-proxy - the introduction to the mode that replaces both iptables and ipvs, and the migration notes for moving to it
13. Kubernetes - kube-proxy configuration (v1alpha1) reference: the `mode` field this article tells you to read and change, and the rest of the KubeProxyConfiguration surface
14. KEP-5573: Remove CGroup v1 support. The staged removal plan behind the kubelet's failCgroupV1 setting, and the statement that the override is temporary
15. Kubernetes - About cgroup v2: how to check which version a node is on, the requirements for cgroup v2, and the features that depend on it
16. Kubernetes - kubelet configuration (v1beta1) reference: failCgroupV1 and the rest of the KubeletConfiguration fields this article edits
17. Kubernetes - Static Pods: what they are, why the kubelet manages them directly rather than through the API server, and therefore why referencing API objects from one was never supposed to work
18. kubernetes/kubernetes issue 140226 - the discussion behind prohibiting Secret and ConfigMap references from static pods, and the fate of the PreventStaticPodAPIReferences feature gate that allowed an opt-out. The gate has defaulted to on since v1.34; the v1.37 sneak peek says it was removed in this release while the shipped v1.37 feature-gates reference still lists it, so the reference page is the one to trust
19. kubernetes/kubernetes issue 138671 - the deprecation of `kubectl run --filename/-f`, on the grounds that the pod it produces is always built purely from the command-line arguments
20. KEP-5207: metrics.k8s.io API definition. The enhancement that graduates the resource metrics API to stable after nearly nine years in beta, and the statement that v1 and v1beta1 both remain available during the transition
21. Kubernetes - Resource metrics pipeline: what metrics.k8s.io actually serves, who serves it, and how `kubectl top` and the HorizontalPodAutoscaler consume it
22. Kubernetes - Horizontal Pod Autoscaling: the largest consumer of the resource metrics API, and the reason its graduation matters beyond `kubectl top`
23. KEP-2033 / KEP-4960: Kubelet in UserNS, also known as rootless mode. Graduating to beta in v1.37, which lets node components run as an unprivileged user on the host while still appearing as root inside the namespace
24. Kubernetes - User namespaces for pods: the workload-level feature that reached GA in v1.36, distinct from the rootless kubelet but built on the same kernel mechanism
25. KEP-1432: Volume health monitoring. Reset to alpha in v1.37 with four new CSI RPCs and three new status fields, after an initial implementation in v1.21 that never graduated
26. Kubernetes v1.34: Of Wind & Will - the release that opened the containerd 1.x end-of-support discussion and the CRI cgroup-driver work behind it. Read alongside the container runtimes page, which records where that timeline actually ended up: the fallback is dropped in v1.38, not v1.36
27. Kubernetes v1.36 release announcement - the release that made user namespaces GA, graduated SELinuxMountReadWriteOncePod, and shipped the selinux-warning-controller that this article tells you to switch on
28. Kubernetes v1.36: Deprecation and removal of Service ExternalIPs - note that despite the title, v1.36 only deprecates the field and starts emitting warnings; kube-proxy support goes off by default no earlier than v1.40 and removal is no earlier than v1.43. A good illustration both of why reading one release's notes is not enough, and of how a headline turns into a rumour
29. Kubernetes - Releases: the supported branches and their end-of-life dates. The source for the support window this article uses to argue about how much runway a cluster actually has
30. Kubernetes - Patch releases: the cadence, the support period and the maintenance-mode window at the end of each minor release's life
31. Kubernetes - Version skew policy: how far the kubelet may lag the API server, which is what makes a staged node upgrade legal in the first place
32. Kubernetes - Upgrading kubeadm clusters: the control plane first, then one node at a time, with drain and uncordon around each. The sequence the runbook in this article slots into
33. Kubernetes - Deprecation policy: the rules that govern how long a deprecated feature must survive before removal, which is why an ipvs deprecation in v1.37 cannot become a removal before v1.43
34. Kubernetes - Container runtimes: installing and configuring containerd or CRI-O, including the cgroup driver requirement that ties this article to the cgroup v2 migration. It is also the page that settles the containerd question, stating that older containerd versions still work today through the kubelet's cgroup-driver fallback and that in Kubernetes 1.38 that fallback is dropped and they will fail with newer kubelets
35. containerd - Versioning and release: the release-status table and the Kubernetes/containerd support matrix. As of September 2026 the matrix stops at Kubernetes 1.36 (2.3.0+, 2.2.0+) and has no row for 1.37, and containerd 1.7's extended LTS support ends in September 2026
36. Kubernetes - Releases: the release and support dates quoted in this article, including v1.35 on 17 December 2025, v1.36 on 22 April 2026 and v1.37 on 26 August 2026
37. Kubernetes - MutatingAdmissionPolicy: the in-tree way to apply the seLinuxChangePolicy opt-out across a namespace or a cluster without editing every workload by hand
38. Kyverno: a policy engine the SELinux blog names explicitly as a way to apply the opt-out fleet-wide. Listed because the alternative - patching every Deployment and StatefulSet individually - does not scale past a small cluster
39. Gateway API v1.6: TCPRoute and UDPRoute graduate to Standard. Out of scope for this article but on the same upgrade window for most clusters, and the reason the ingress migration keeps appearing in the same maintenance plan