Kubernetes Daemon Workloads

Kubernetes Daemon Workloads are workloads that must run exactly once per node, or once per each node in a selected subset, rather than being scaled independently of the cluster's size. This pattern is implemented through the DaemonSet controller, which ties the number of running Pods directly to the number of eligible nodes, automatically adding a Pod when a new node joins and removing it when a node leaves, rather than being driven by a fixed replica count.

The DaemonSet Controller

Node-Bound Scaling

A DaemonSet does not specify a replicas field, because its scale is not an independent decision, it is determined entirely by the number of nodes matching its scheduling constraints. As the cluster grows or shrinks, the DaemonSet controller reconciles the set of running Pods to match the current set of eligible nodes, without any user action required.

Scheduling Behavior

Historically, DaemonSet Pods were placed directly by the DaemonSet controller itself, bypassing the default scheduler. In current Kubernetes versions, DaemonSet Pods are scheduled by the standard kube-scheduler like any other Pod, but with a required node affinity automatically added that pins each Pod to its target node, preserving the one-Pod-per-node guarantee while benefiting from standard scheduling features such as taints and tolerations.

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: codartium-node-agent
spec:
  selector:
    matchLabels:
      app: codartium-node-agent
  template:
    metadata:
      labels:
        app: codartium-node-agent
    spec:
      tolerations:
        - key: node-role.kubernetes.io/control-plane
          effect: NoSchedule
      containers:
        - name: agent
          image: codartium/node-agent:1.1.0
          resources:
            requests:
              cpu: "50m"
              memory: "64Mi"

Restricting to a Subset of Nodes

A nodeSelector or node affinity rule on the Pod template limits a DaemonSet to a specific subset of nodes, such as those with a particular hardware feature, rather than every node in the cluster.

spec:
  template:
    spec:
      nodeSelector:
        hardware.codartium.io/gpu: "true"

Common Use Cases

Log and Metrics Collection

A node-level log or metrics collection agent must observe every container running on its node, which requires a single collector process per node rather than a pool of independently scheduled collectors that might cluster unevenly across the cluster's machines.

Network and Storage Plugins

Container Network Interface (CNI) and Container Storage Interface (CSI) plugins frequently run as DaemonSets, since networking and storage integration must be established on every node before other workloads on that node can function correctly.

Node-Level Security and Compliance Agents

Intrusion detection agents, node hardening tools, and compliance scanners commonly run as DaemonSets, since their purpose is to observe or enforce policy on the node itself rather than to serve application traffic.

Update Strategies

RollingUpdate

By default, updating a DaemonSet's Pod template triggers a rolling update, replacing Pods on each node one at a time (or in small batches, controlled by maxUnavailable), avoiding simultaneous disruption of the daemon across the entire cluster.

spec:
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1

OnDelete

The OnDelete strategy leaves existing Pods untouched when the template changes; new Pods only pick up the updated template once their old Pod is manually deleted, giving an operator explicit control over the timing of each node's update, useful when node-level agents require careful, staged rollout.

Interaction with Node Taints

Because certain nodes, such as control-plane nodes, are commonly tainted to repel ordinary workloads, DaemonSets that must run everywhere, including on those nodes, explicitly configure matching tolerations, ensuring infrastructure-critical daemons such as network plugins are present even on nodes otherwise reserved for control plane components.

kubectl get daemonsets --all-namespaces
kubectl rollout status daemonset/codartium-node-agent
kubectl describe daemonset codartium-node-agent

Relationship to Other Controllers

DaemonSet occupies a distinct niche among workload controllers: where Deployment and StatefulSet scale according to an explicit desired count, and Job and CronJob scale according to a unit of work, DaemonSet scales according to cluster topology itself, making it the natural choice whenever a workload's correct cardinality is defined by "once per node" rather than any independently chosen number.