Kubernetes Stateful Workloads

Kubernetes Stateful Workloads are applications run on Kubernetes that depend on persistent identity, ordered lifecycle operations, or durable storage tied to a specific replica, in contrast to the interchangeable, ephemeral Pods typical of stateless services. Because Kubernetes was originally designed around disposable, identical replicas, running stateful applications, databases, message queues, distributed file systems, requires a distinct set of primitives that provide the stability these applications assume.

What Makes a Workload Stateful

Identity Sensitivity

A stateful workload cares about which specific instance it is: a database replica may be configured as a primary or a specific follower, and swapping its identity with another replica's identity would break replication topology. Stateless workloads, by contrast, treat every replica as fully interchangeable.

Storage Sensitivity

A stateful workload typically requires that its data survive the replacement of the Pod that produced it. Losing the underlying storage when a Pod is rescheduled would mean losing the application's actual state, not just a disposable runtime process.

Ordering Sensitivity

Some stateful systems require operations, especially startup and shutdown, to happen in a defined order across replicas, for example, initializing a primary node before followers attempt to join a cluster.

StatefulSet as the Primary Primitive

Stable Network Identity

Each Pod managed by a StatefulSet is assigned a predictable name composed of the StatefulSet name and an ordinal index, and a corresponding stable DNS entry is published through a headless Service, allowing other components to address a specific replica by a name that survives rescheduling.

apiVersion: v1
kind: Service
metadata:
  name: codartium-db
spec:
  clusterIP: None
  selector:
    app: codartium-db
  ports:
    - port: 5432

Per-Replica Persistent Storage

Through volumeClaimTemplates, each replica in a StatefulSet is bound to its own PersistentVolumeClaim, created once and reused across rescheduling events, so that a replaced Pod reattaches to the same underlying storage rather than starting with an empty volume.

Ordered, Graceful Deployment and Scaling

Pods are created and made ready one at a time, in ascending ordinal order, by default, and terminated in descending order during scale-down, giving applications with join or leave sequencing requirements the guarantees they need without requiring custom orchestration logic outside the cluster.

kubectl scale statefulset codartium-db --replicas=5
kubectl get pods -l app=codartium-db -o wide

Operators for Complex Stateful Systems

Beyond Generic Primitives

Many stateful systems have operational requirements too specific for a generic controller to express, coordinated failover, backup scheduling, schema migrations, or cluster topology changes that must be sequenced carefully. Operators extend Kubernetes with Custom Resource Definitions and dedicated controllers that encode this operational knowledge directly, exposing a higher-level API that abstracts the underlying StatefulSets, Services, and storage objects.

Operator Pattern in Practice

An operator watches a custom resource representing the desired state of an application, such as a PostgresCluster, and translates it into the lower-level Kubernetes objects needed to realize it, while also handling ongoing operational tasks such as automated failover or rolling version upgrades in a way specific to that application's requirements.

apiVersion: databases.codartium.io/v1
kind: PostgresCluster
metadata:
  name: codartium-primary
spec:
  version: "15"
  instances: 3
  storage:
    size: 50Gi

Storage Considerations for Stateful Workloads

Access Modes

PersistentVolumes declare access modes describing how many nodes may mount them simultaneously: ReadWriteOnce, mountable by a single node; ReadOnlyMany, mountable read-only by many nodes; and ReadWriteMany, mountable read-write by many nodes, a capability not supported by all storage backends.

StorageClasses and Dynamic Provisioning

A StorageClass defines a class of storage, backed by a specific provisioner and set of parameters, allowing PersistentVolumeClaims to trigger on-demand creation of appropriately sized and configured volumes rather than requiring volumes to be provisioned manually in advance.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast-ssd
provisioner: codartium.io/csi-driver
parameters:
  type: ssd
reclaimPolicy: Retain

Reclaim Policy

The reclaimPolicy of a PersistentVolume determines what happens to the underlying storage once its claim is released: Delete removes the underlying storage resource, while Retain preserves it for manual recovery, a choice with direct consequences for data durability guarantees on stateful workloads.

Trade-offs of Running State on Kubernetes

Running stateful workloads on Kubernetes provides the same declarative management, scheduling flexibility, and automated recovery available to stateless workloads, but requires more careful attention to storage backend characteristics, backup strategy, and, for complex systems, the adoption of an operator that encodes the correct operational behavior for that specific technology rather than relying on generic Pod replacement semantics alone.