✦ For everyone, free.

Practical knowledge for real and everyday life

Home

Kubernetes Resource Pressure Management

Kubernetes Resource Pressure Management ensures optimal cluster performance by monitoring and balancing resource usage across nodes and workloads.

Kubernetes Resource Pressure Management is the kubelet's node-local mechanism for detecting when a node is running critically low on memory, disk, or process IDs, and responding by reclaiming resources — first through garbage collection, and if that proves insufficient, through active Pod eviction — before the node's own stability is threatened. Resource pressure is distinct from ordinary scheduling-time resource accounting: it is a runtime, node-local condition that can arise even on a node whose Pods all passed their original scheduling fit checks cleanly, typically because actual usage has grown beyond what was accounted for at placement time, or because system-level consumption unrelated to any specific Pod has grown unexpectedly.

Because resource pressure responses directly terminate running Pods, understanding how pressure is detected, and how eviction targets are chosen once it is, is essential to correctly interpreting Pod terminations that were not caused by any application-level crash or explicit deletion.


Node Conditions Signaling Pressure

MemoryPressure, DiskPressure, PIDPressure

The kubelet continuously monitors node-level resource availability and sets corresponding conditions — MemoryPressure, DiskPressure, PIDPressure — to True once availability drops below configured thresholds, providing the first observable signal that eviction may be imminent.

kubectl describe node node-worker-05 | grep -A 3 "Conditions"

Soft and Hard Eviction Thresholds

Pressure detection distinguishes soft thresholds (triggering eviction only after a configurable grace period, giving transient dips a chance to resolve on their own) from hard thresholds (triggering eviction immediately, with no grace period, reserved for genuinely critical resource exhaustion).

--eviction-soft=memory.available<500Mi
--eviction-soft-grace-period=memory.available=1m30s
--eviction-hard=memory.available<100Mi

The Reclamation Sequence

Garbage Collection Before Eviction

Before evicting any running Pod, the kubelet first attempts less disruptive reclamation — removing unused container images and dead container filesystems for disk pressure, for instance — since this can often relieve pressure without affecting any running workload at all.

Eviction as the Last Resort

Only if garbage collection and other non-disruptive reclamation fail to bring the node back under its threshold does the kubelet proceed to actively select and terminate Pods, prioritizing the least disruptive choices available given the node's current Pod population.


Eviction Ordering

QoS Class as the Primary Ranking Factor

BestEffort Pods are evicted first, followed by Burstable Pods (ranked by how far their actual usage exceeds their request, proportionally), with Guaranteed Pods evicted only as an absolute last resort — this ordering directly reflects the resource commitment strength each QoS class represents.

priorityeviction usageactual requestdeclared

Among Pods of equal QoS class, this usage-to-request ratio serves as the tiebreaker, with Pods exceeding their request most significantly evicted first.

Priority as a Secondary Factor

PriorityClass also factors into eviction ordering as an additional signal beyond QoS, generally causing lower-priority Pods within the same QoS tier to be preferred as eviction targets over higher-priority ones, giving operators an additional lever to protect specific critical workloads beyond QoS class alone.


Graceful vs. Immediate Termination

Grace Periods Under Soft Thresholds

Evictions triggered by soft thresholds respect the Pod's normal termination grace period, allowing the application an opportunity to shut down cleanly, consistent with how any ordinary Pod deletion behaves.

Immediate Termination Under Hard Thresholds

Evictions triggered by hard thresholds may bypass the normal grace period entirely, terminating Pods immediately given the more severe, time-critical nature of the underlying resource exhaustion the hard threshold represents.


Diagnosing Pressure-Driven Evictions

kubectl get pod codartium-app -o jsonpath='{.status.reason}'
kubectl describe pod codartium-app | grep -A 3 "Status:"

A Pod showing Status: Failed with Reason: Evicted (as opposed to OOMKilled, which is a per-container cgroup limit violation) indicates the Pod was terminated as part of node-level pressure relief, not because it exceeded its own declared limit — this distinction matters for correctly diagnosing whether the fix is adjusting that specific Pod's resource declarations or addressing broader node-level capacity issues.

kubectl get events --field-selector reason=Evicted --sort-by=.lastTimestamp

Example

apiVersion: v1
kind: Pod
metadata:
  name: codartium-pressure-example
spec:
  containers:
    - name: app
      image: codartium/app:latest
      resources:
        requests:
          cpu: "500m"
          memory: "512Mi"
        limits:
          cpu: "500m"
          memory: "512Mi"