✦ For everyone, free.

Practical knowledge for real and everyday life

Home

Kubernetes DaemonSet Placement Management

Kubernetes DaemonSet ensures consistent node placement across clusters through policies that manage pod distribution and node affinity rules.

Kubernetes DaemonSet Placement Management is the set of scheduling mechanisms used to control precisely which nodes a DaemonSet's Pods land on, spanning nodeSelector, node affinity and anti-affinity, and taints and tolerations. While the default, unconstrained behavior of a DaemonSet is to place one Pod on every node in the cluster, real clusters are rarely uniform — they mix control-plane and worker nodes, different hardware profiles, different availability zones, and nodes reserved for specific purposes — and placement management is what lets a DaemonSet's coverage be scoped precisely to the subset of nodes where its workload actually belongs.

Placement decisions for DaemonSets are, in effect, answering the question "which nodes should this daemon consider its territory," and getting that scoping right is what prevents both under-coverage (a daemon missing from nodes it should run on) and over-coverage (a daemon wastefully or even harmfully running where it should not).


nodeSelector: Simple Label-Based Targeting

Basic Usage

template.spec.nodeSelector restricts a DaemonSet's Pods to only nodes carrying all of the specified labels, evaluated as an implicit AND across every key-value pair listed.

spec:
  template:
    spec:
      nodeSelector:
        node-role: gpu-worker

This is the simplest and most common placement mechanism for daemons that only make sense on a specific class of node — a GPU monitoring agent scoped to nodes labeled node-role: gpu-worker, for instance, rather than attempting (and failing) to run on every node in the cluster.


Node Affinity: Expressive Targeting Rules

requiredDuringSchedulingIgnoredDuringExecution

For placement logic beyond simple exact-match labels, nodeAffinity supports expressions using operators like In, NotIn, Exists, and Gt/Lt, evaluated against node labels at scheduling time.

spec:
  template:
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: node.kubernetes.io/instance-type
                    operator: In
                    values: ["c5.large", "c5.xlarge"]

This is used when placement logic needs to express something more nuanced than a single label match — for example, targeting several acceptable instance types, or excluding a specific subset of nodes by label rather than only including a specific subset.

The "IgnoredDuringExecution" Distinction

The suffix IgnoredDuringExecution reflects that node affinity is evaluated only at scheduling time; if a node's labels change after a Pod has already been placed there, the existing Pod is not automatically evicted purely because it would no longer satisfy the (now-changed) affinity rule. Reconciling that drift requires the DaemonSet controller's own logic (which does actively remove Pods from nodes that no longer match current scheduling constraints, distinct from raw node affinity evaluation) — this is precisely what surfaces as numberMisscheduled in DaemonSet status.


Taints and Tolerations: Exclusion by Default, Opt-In by Toleration

The Default-Exclusion Model

Taints work opposite to nodeSelector and affinity: rather than opting a Pod into a subset of nodes, a taint on a node opts every Pod out by default, unless that Pod carries a matching toleration. This inversion is why DaemonSets intended for full cluster coverage — including control-plane and any specially tainted nodes — must explicitly declare tolerations for every taint they need to override.

tolerations:
  - key: "node-role.kubernetes.io/control-plane"
    operator: "Exists"
    effect: "NoSchedule"
  - operator: "Exists"
    effect: "NoExecute"

A Blanket Toleration for True Full Coverage

Some infrastructure daemons (log collectors, certain CNI components) use a toleration with no key specified and operator: Exists, which matches every taint regardless of key or value — an intentional, broad opt-in appropriate specifically for daemons whose function genuinely needs to be present on absolutely every node, taints notwithstanding.


Combining Mechanisms

Real DaemonSet placement configurations typically combine several of these mechanisms together: a nodeSelector or nodeAffinity rule narrowing to the intended class of nodes, plus tolerations specifically for any taints present on that same class of nodes (a GPU node pool that is both labeled gpu: "true" and tainted to repel ordinary workloads, requiring both a matching nodeSelector and a matching toleration for the daemon to be scheduled there at all).

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: codartium-gpu-agent
spec:
  selector:
    matchLabels:
      app: codartium-gpu-agent
  template:
    metadata:
      labels:
        app: codartium-gpu-agent
    spec:
      nodeSelector:
        gpu: "true"
      tolerations:
        - key: "nvidia.com/gpu"
          operator: "Exists"
          effect: "NoSchedule"
      containers:
        - name: gpu-agent
          image: codartium/gpu-agent:latest

Verifying Placement Correctness

kubectl get pods -l app=codartium-gpu-agent -o wide
kubectl get nodes -l gpu=true

Comparing the set of nodes labeled for a given class against the set of nodes actually running the corresponding daemon's Pods confirms placement rules are producing the intended coverage before relying on the daemon in production.