Kubernetes DaemonSet OnDelete Update Management

Kubernetes DaemonSet OnDelete Update Management is the practice of operating DaemonSets configured with spec.updateStrategy.type: OnDelete, a strategy that hands complete manual control over template propagation to the operator by disabling all automatic Pod replacement. Under OnDelete, updating a DaemonSet's Pod template changes what future Pods will look like, but has zero immediate effect on any Pod currently running — nothing happens on any node until an operator explicitly deletes that node's existing Pod, at which point its replacement is created using the newer template.

This inversion of control — from the controller driving the rollout automatically to the operator driving it Pod by Pod — makes OnDelete a deliberate choice for daemons where the risk of an unattended, automatic, cluster-wide update is considered too high relative to the operational overhead of manual control.

Why Choose OnDelete Over RollingUpdate

Extremely Sensitive Infrastructure

Daemons whose failure has an outsized blast radius — a CNI plugin governing Pod networking, a CSI node plugin governing volume mounts — are the most common candidates for OnDelete, since a bad RollingUpdate rollout could silently propagate a broken version across a large fraction of the cluster before anyone notices, whereas OnDelete forces a human decision point at every single node.

Coordinated Multi-Component Rollouts

Some daemon upgrades need to be coordinated with a corresponding control-plane component change (a CNI version bump paired with a compatible controller version), where updating a node's daemon Pod ahead of the paired component being ready would cause a functional mismatch. OnDelete allows an operator to sequence the two changes precisely rather than relying on independent, automatic timers.

Environments Without Automated Rollout Tooling Yet

Teams still building out their CI/CD maturity for cluster-level infrastructure changes sometimes default new, unproven DaemonSets to OnDelete until they have enough confidence (and tooling) to safely automate rollouts via RollingUpdate.

Operating an OnDelete DaemonSet

Applying the Template Change

kubectl set image daemonset/codartium-cni-agent cni-agent=codartium/cni-agent:2.1.0

This updates the DaemonSet's stored template immediately, visible via kubectl get daemonset -o yaml, but produces no Pod changes on any node until deletion is triggered.

Manually Propagating Node by Node

kubectl delete pod -l app=codartium-cni-agent --field-selector spec.nodeName=node-worker-01
# verify health on node-worker-01 before continuing
kubectl delete pod -l app=codartium-cni-agent --field-selector spec.nodeName=node-worker-02

A deliberate, one-node-at-a-time deletion sequence — verifying each replacement Pod's health before moving to the next node — is the standard operating pattern, effectively reimplementing a manual rolling update with explicit human checkpoints between steps.

Scripting a Paced Rollout

for node in $(kubectl get nodes -o jsonpath='{.items[*].metadata.name}'); do
  pod=$(kubectl get pods -l app=codartium-cni-agent --field-selector spec.nodeName=$node -o jsonpath='{.items[0].metadata.name}')
  kubectl delete pod "$pod"
  kubectl wait --for=condition=ready pod -l app=codartium-cni-agent --field-selector spec.nodeName=$node --timeout=60s
done

Scripting the manual process while still gating each step on the previous node's readiness preserves the safety benefit of OnDelete while reducing the tedium of doing it by hand across a large node population.

Tracking Rollout Progress Under OnDelete

No Native Progress Signal

Because OnDelete does not drive the rollout itself, .status.updatedNumberScheduled still reports how many nodes have the current template, but this number only advances as fast as an operator (or their script) deletes Pods — there is no automatic timeline the way there is under RollingUpdate.

kubectl get daemonset codartium-cni-agent -o jsonpath='{.status.updatedNumberScheduled}/{.status.desiredNumberScheduled}'

Identifying Which Nodes Still Need Updating

kubectl get pods -l app=codartium-cni-agent -o json | \
  jq -r '.items[] | select(.spec.containers[0].image != "codartium/cni-agent:2.1.0") | .spec.nodeName'

Explicitly comparing each Pod's running image against the target version is the most reliable way to track partial rollout progress under OnDelete, since the DaemonSet's own status fields summarize counts but do not enumerate exactly which nodes remain outdated.

Switching Between Strategies

Changing updateStrategy.type between OnDelete and RollingUpdate on an existing DaemonSet takes effect for future updates only — it does not retroactively reconcile any drift that accumulated while the previous strategy was in effect, so switching from OnDelete to RollingUpdate after a partial manual rollout will cause the controller to immediately finish propagating the already-in-progress template change to any remaining outdated nodes.

Example

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: codartium-cni-agent
spec:
  updateStrategy:
    type: OnDelete
  selector:
    matchLabels:
      app: codartium-cni-agent
  template:
    metadata:
      labels:
        app: codartium-cni-agent
    spec:
      hostNetwork: true
      containers:
        - name: cni-agent
          image: codartium/cni-agent:2.0.0