✦ For everyone, free.

Practical knowledge for real and everyday life

Home

Kubernetes Container Restart Behavior

Kubernetes manages container restarts automatically based on lifecycle policies and failure conditions, ensuring application reliability and system stability.

Kubernetes Container Restart Behavior is the precise set of rules the kubelet applies when deciding whether, when, and how quickly to restart a container that has stopped running. This behavior is governed by the Pod-level restartPolicy, an internal exponential backoff algorithm, and the exit status the container reported, and it operates entirely at the kubelet's discretion without any involvement from the scheduler or higher-level controllers.


The restartPolicy Field

The Three Values

restartPolicy is set once per Pod and applies to every container within it (subject to the restartable init container exception). It accepts exactly three values:

  • Always: restart the container regardless of exit code, including on successful exit. This is the default and the appropriate choice for long-running services.
  • OnFailure: restart only if the container exits with a non-zero code. Appropriate for batch work that should retry on error but stop on success.
  • Never: do not restart under any circumstance, regardless of exit code.
apiVersion: v1
kind: Pod
metadata:
  name: restart-behavior-example
spec:
  restartPolicy: OnFailure
  containers:
    - name: worker
      image: registry.example.com/worker:1.0.0

Exponential Backoff

Why Backoff Exists

Immediately restarting a container that has just crashed risks a tight crash loop that consumes node CPU and I/O without any chance of success, especially if the underlying cause (a missing dependency, a bad configuration) has not changed. The kubelet instead delays each successive restart attempt by a growing interval.

Backoff Growth and Reset

The delay starts small, typically around ten seconds, and doubles with each consecutive failure up to a capped maximum, commonly five minutes. A container that stays Running successfully for a sustained period resets this backoff counter, so a single transient failure does not permanently slow future restarts.

kubectl describe pod restart-behavior-example
Waiting
  Reason: CrashLoopBackOff
  Message: back-off 40s restarting failed container=worker

CrashLoopBackOff as an Observed State

Not a Failure Mode Itself

CrashLoopBackOff is not an error in the traditional sense; it is the kubelet correctly applying backoff to a container that keeps failing. The actual root cause is always in the container's own exit behavior, application error, missing configuration, failed dependency, and must be diagnosed by inspecting logs and the lastState.terminated block rather than treating the backoff message itself as the problem.

lastState:
  terminated:
    exitCode: 1
    reason: Error
    message: "connection refused: database unreachable"

Restart Scope

Container-Level, Not Pod-Level

Restart behavior operates on individual containers, not the Pod as a whole. In a multi-container Pod, one container restarting repeatedly does not force its siblings to restart, and the Pod object itself is never replaced by a restart; its name, UID, and IP address remain constant throughout.

In-Place, No New Scheduling

A restart reuses the existing Pod sandbox and node assignment entirely. It does not go through the scheduler again, does not re-pull volumes from scratch (though it may re-pull the image depending on imagePullPolicy), and does not generate a new Pod object.


Interaction With Probes

Liveness Probe Failures Trigger the Same Path

A liveness probe failure results in the kubelet killing the container and routing it through the identical restart-and-backoff mechanism as a natural process exit, meaning repeated liveness failures produce the same CrashLoopBackOff symptom as an application that crashes on its own.

containers:
  - name: app
    livenessProbe:
      httpGet:
        path: /healthz
        port: 8080
      failureThreshold: 3
      periodSeconds: 10

Restart Behavior Diagram

time 10s 20s 40s 80s capped max

Each successive failed attempt widens the gap shown above, which is why a container stuck in CrashLoopBackOff for several minutes may show only a handful of restart attempts rather than dozens, since the backoff interval quickly dominates the observation window.