✦ For everyone, free.

Practical knowledge for real and everyday life

Home

Kubernetes Pod Eviction Resource Management

Kubernetes Pod Eviction Resource Management ensures pods are removed when resource limits are exceeded, maintaining cluster stability and efficient resource allocation.

Kubernetes Pod Eviction Resource Management is the broader practice of distinguishing and managing the two fundamentally different eviction pathways in Kubernetes — node-pressure eviction, triggered involuntarily by the kubelet when a node runs critically low on resources, and API-initiated eviction, triggered voluntarily through the Eviction API by operators, node drains, or cluster autoscalers, and governed by PodDisruptionBudget objects. While both pathways ultimately terminate a running Pod, they differ in cause, in whether they respect disruption budgets, and in what remediation or prevention strategy actually applies to each.

Conflating these two eviction types is a common source of confusion during incident investigation — a Pod's disappearance attributed to "eviction" requires determining which pathway was actually responsible before any meaningful corrective action can be taken.


Node-Pressure Eviction (Involuntary)

Triggered by the Kubelet, Not the API Server

Node-pressure eviction is initiated directly by the kubelet in response to MemoryPressure, DiskPressure, or PIDPressure conditions on that specific node — it does not go through the Eviction API subresource and is not subject to PodDisruptionBudget protection at all, since it represents an emergency resource-reclamation action the node itself must take regardless of any disruption tolerance a workload has declared.

kubectl get pod codartium-app -o jsonpath='{.status.reason}'

A Pod terminated this way shows Reason: Evicted with a message describing which resource threshold triggered it, distinct from OOMKilled (a per-container limit violation, not a node-wide condition).

PDB Does Not Apply

Because node-pressure eviction addresses an urgent, node-local emergency, PodDisruptionBudget constraints are bypassed entirely — a Pod belonging to a workload with minAvailable: 100% configured can still be evicted under genuine memory pressure, since the alternative (letting the node fail entirely) would be worse for every workload on that node, not just the one PDB-protected Pod.


API-Initiated Eviction (Voluntary)

The Eviction Subresource

Voluntary eviction goes through the Pod's eviction subresource, most commonly invoked by kubectl drain during planned node maintenance, or by a cluster autoscaler consolidating workloads before removing underutilized nodes.

kubectl drain node-worker-05 --ignore-daemonsets --delete-emptydir-data

PodDisruptionBudget Governs This Pathway

Unlike node-pressure eviction, API-initiated eviction requests are checked against any applicable PodDisruptionBudget — a request that would violate the budget's minAvailable or maxUnavailable constraint is rejected by the API server, causing the initiating operation (like kubectl drain) to retry later rather than proceeding immediately.

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: codartium-api-pdb
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: codartium-api

Distinguishing Which Pathway Caused an Eviction

Reading the Termination Reason

kubectl get events --field-selector reason=Evicted --sort-by=.lastTimestamp
kubectl get events --field-selector reason=EvictionByNodeDrain 2>/dev/null

Node-pressure evictions carry a message referencing the specific resource condition (memory, disk, pid) that triggered them; API-initiated evictions are typically associated with a node drain operation or autoscaler activity visible in cluster-level events, giving distinguishable audit trails for each pathway.

Checking for Correlated Node Conditions

A cluster of evictions correlated with a MemoryPressure or DiskPressure condition appearing on the affected node around the same time strongly indicates the involuntary pathway; evictions with no corresponding node condition, occurring alongside a node cordon or drain event, point to the voluntary pathway instead.


Remediation Differs by Pathway

For Node-Pressure Eviction

Remediation focuses on the resource sizing and node-level capacity issues discussed in resource pressure management — adjusting requests/limits, investigating unexpected consumption growth, or provisioning additional node capacity.

For API-Initiated Eviction

Remediation focuses on PodDisruptionBudget configuration — ensuring budgets are neither so strict they block legitimate, necessary maintenance indefinitely, nor so loose they fail to protect genuine availability requirements during voluntary disruptions.

kubectl get pdb -n codartium-team
kubectl describe pdb codartium-api-pdb -n codartium-team

Example

apiVersion: v1
kind: Pod
metadata:
  name: codartium-eviction-example
  labels:
    app: codartium-api
spec:
  containers:
    - name: app
      image: codartium/app:latest
      resources:
        requests:
          cpu: "500m"
          memory: "512Mi"
        limits:
          cpu: "500m"
          memory: "512Mi"