Kubernetes Reliability Model

Kubernetes Reliability Model is the underlying conceptual framework that unifies every specific reliability mechanism into a single coherent design philosophy: failure is treated as a normal, expected, continuous condition rather than an exception, and reliability emerges from the same declarative reconciliation loop that drives every other aspect of the system, applied specifically to detecting and correcting departures from a desired healthy state.

Failure as the Normal Case, Not the Exception

Designing for Continuous, Expected Failure

The Kubernetes reliability model does not attempt to prevent failure; it assumes failure, of individual containers, of nodes, of entire availability zones, will happen continuously and routinely, and designs every mechanism around detecting and correcting it quickly rather than around achieving an unattainable failure-free state.

The Same Reconciliation Loop, Applied to Health

Just as a custom controller reconciles a custom resource's desired state against observed state, Kubernetes's built-in workload controllers (the ReplicaSet controller in particular) continuously reconcile "how many healthy replicas should exist" against "how many healthy replicas currently exist," creating or replacing pods as needed, applying the same watch-driven, level-triggered reconciliation pattern used throughout the extension model directly to the problem of maintaining availability.

if observedHealthyReplicas < desiredReplicas {
    createReplacementPod()
}

Mean Time Between Failures and Mean Time to Recovery

The Two Levers Available

Overall availability is a function of how frequently failures occur (mean time between failures, MTBF) and how quickly the system recovers from each one (mean time to recovery, MTTR); Kubernetes's reliability model focuses almost entirely on minimizing MTTR through fast automated detection and correction, since MTBF is largely determined by underlying infrastructure quality outside Kubernetes's direct control.

Probes as the Detection Half of the Loop

Liveness, readiness, and startup probes exist specifically to shrink the detection portion of MTTR, the time between a component actually failing and the system becoming aware of it, since a failure that goes undetected for minutes provides no opportunity for automated correction during that entire window.

Redundancy as Probabilistic Risk Reduction

The Mathematics of Independent Failure

If a single replica has some independent probability of failure over a given window, running multiple replicas reduces the probability that all replicas fail simultaneously, provided those replicas' failures are genuinely independent, which is precisely what failure-domain spread (topology spread constraints, anti-affinity) is designed to ensure by preventing replicas from sharing a single point of failure.

Why Correlated Failure Undermines Redundancy

Three replicas on the same physical node provide no real redundancy against a node failure, since their failure probabilities are not independent but perfectly correlated; the reliability model's emphasis on spread and anti-affinity exists precisely to convert nominal redundancy (multiple replicas) into actual redundancy (multiple replicas whose failure probabilities are genuinely uncorrelated).

Fault Domain Hierarchy

Nested Layers of Independent Failure

The reliability model recognizes a hierarchy of fault domains, container, pod, node, rack, availability zone, region, each layer representing a boundary within which failures are likely correlated and across which they are more likely independent; different reliability mechanisms target different layers of this hierarchy, restart policies at the container layer, rescheduling at the node layer, topology spread at the zone layer.

Container -> Pod -> Node -> Zone -> Region

Graceful Degradation Over Binary Availability

Partial Capacity Loss, Not Total Outage

The reliability model favors mechanisms that degrade capacity gracefully under partial failure, losing one replica out of five reduces capacity by twenty percent rather than causing a full outage, over architectures where any single failure causes complete unavailability; pod disruption budgets formalize this preference by defining exactly how much capacity loss is tolerable during planned disruption.

Relationship to Reliability and Availability Scope and Areas

The reliability model is the conceptual foundation beneath every specific mechanism enumerated under reliability and availability areas: probes, restart policies, disruption budgets, topology spread, and node-level eviction are all concrete implementations of the same underlying philosophy, treat failure as continuous and expected, minimize detection and recovery time, convert nominal redundancy into genuinely independent redundancy across the fault domain hierarchy, and favor graceful degradation over all-or-nothing availability.