Kubernetes etcd Architecture

Kubernetes etcd Architecture is the internal structure of etcd as a distributed, consistent key-value store: how its data is organized, how its members reach agreement on the cluster's canonical state through the Raft consensus protocol, and how the Kubernetes API server maps its object model onto etcd's underlying key space. Because etcd is the single source of truth for every object in a cluster, its own architecture directly determines the durability and consistency guarantees the rest of Kubernetes can rely on.

Raft Consensus

Roles Among Members

An etcd cluster's members each hold one of three roles at any given time: a single leader, responsible for accepting and replicating writes; followers, replicating the leader's log and responding to reads; and, transiently during an election, candidates, competing to become the new leader after the previous one becomes unreachable.

Log Replication and Commitment

A write is formally committed only once the leader has replicated it to a majority of members and received their acknowledgment; only after commitment is the write applied to the key-value store and visible to clients, guaranteeing that a committed write survives the failure of any minority of members.

Leader Election

If a leader fails to send heartbeats within a defined timeout, followers transition to candidates and initiate an election, requesting votes from other members; a candidate receiving a majority of votes becomes the new leader, a process designed to complete quickly to minimize the write-unavailable window.

Data Model

Multi-Version Key-Value Store

etcd formally stores data as a multi-version, ordered key-value map: every write creates a new revision rather than overwriting history in place, allowing etcd to serve consistent point-in-time reads and to support the watch mechanism by replaying the sequence of revisions since a given point.

etcdctl get /registry/pods/codartium-team/codartium-api-abc123
etcdctl get /registry --prefix --keys-only | head

Kubernetes' Key Layout

The API server formally maps each object to a key under a fixed prefix structure, /registry/<resource-type>/<namespace>/<name> for namespaced resources, with the object's serialized representation, typically protobuf, stored as the corresponding value.

Watch Mechanism

Streaming Change Notifications

etcd formally exposes a watch API that streams key changes to subscribed clients as they are committed, the mechanism the API server itself relies on internally to populate its own watch cache and, in turn, serve watch requests from Kubernetes clients without each one opening a separate connection to etcd directly.

Compaction

Because every write creates a new revision rather than discarding history, etcd's storage would grow unboundedly without periodic compaction, a maintenance operation that formally removes revisions older than a configured retention point, bounding storage growth at the cost of no longer being able to read arbitrarily old historical states.

etcdctl compact <revision>
etcdctl defrag

Cluster Membership and Quorum

Static and Dynamic Membership

An etcd cluster's membership can be configured statically at bootstrap or adjusted dynamically at runtime through explicit member-add and member-remove operations, with every membership change itself subject to the same Raft consensus protocol governing ordinary data writes.

Quorum Loss

If a majority of members become permanently unavailable, the remaining minority formally cannot elect a leader or accept writes, even though it may still serve stale reads in some configurations; recovering from quorum loss requires either restoring failed members or performing a formal disaster-recovery procedure from a prior snapshot.

Snapshots and Recovery

Point-in-Time Snapshots

etcd formally supports taking a consistent snapshot of its entire key space at a given revision, the basis for both backup procedures and for bootstrapping new members without replaying the cluster's complete write history from the beginning.

etcdctl snapshot save /backup/etcd-snapshot.db
etcdctl snapshot status /backup/etcd-snapshot.db

Restoring from Snapshot

Recovery from catastrophic member loss formally involves restoring a snapshot into a new data directory and reinitializing the cluster's membership around it, a distinct, deliberate operational procedure rather than an automatic recovery path the cluster performs on its own.

Why etcd Is Architected This Way

Building the cluster's single source of truth on Raft consensus, rather than a simpler but weaker replication scheme, is what formally guarantees that every etcd member, and therefore every downstream API server replica reading from it, observes the same linearizable sequence of writes, the specific consistency property the rest of Kubernetes' reconciliation model depends on to behave correctly under concurrent, distributed access.