Kubernetes Control Plane Leader Election

Kubernetes Control Plane Leader Election is the specific algorithm and timing parameters by which multiple replicas of a control plane component agree on exactly one active leader at a time, using nothing beyond an ordinary API object, a Lease, and the same optimistic-concurrency write guarantees available to any other client, rather than a dedicated consensus protocol of its own.

The Lease Object as the Lock

A Lease Is Just Another API Object

Leader election formally relies on a Lease object, an ordinary, namespaced API resource with no special server-side logic beyond what any other object receives; the "lock" it represents exists purely by convention, in how client code interprets and updates its fields, not through any dedicated locking primitive built into the API server.

apiVersion: coordination.k8s.io/v1
kind: Lease
metadata:
  name: kube-scheduler
  namespace: kube-system
spec:
  holderIdentity: cp-2_a1b2c3d4
  leaseDurationSeconds: 15
  renewTime: "2024-05-12T09:14:03.120000Z"
  leaseTransitions: 3

The Acquisition Algorithm

Attempting to Become Leader

Each candidate replica formally attempts to create the Lease object if it does not exist, or, if it exists but its renewTime is older than its leaseDurationSeconds, attempts to update it, setting holderIdentity to its own identity; this update is submitted with the Lease's current resourceVersion, so the API server's ordinary optimistic-concurrency check ensures only one candidate's concurrent attempt can succeed.

kubectl -n kube-system get lease kube-scheduler -o jsonpath='{.spec.holderIdentity}'

Why No Two Replicas Can Simultaneously Win

Because the API server formally rejects any write whose submitted resourceVersion no longer matches the object's current value, two replicas racing to acquire an expired lease will have exactly one succeed and one fail, with the losing replica simply observing the winner's holderIdentity in its own next read and remaining a standby.

Renewal by the Current Leader

Continuous Heartbeat Writes

The current leader formally renews its lease well before it would expire, repeatedly updating renewTime (and its own resourceVersion) at an interval shorter than leaseDurationSeconds, functioning as a heartbeat that other replicas can observe to confirm the current leader remains active.

Stepping Down on Renewal Failure

If the current leader fails to renew successfully, most commonly because it has lost connectivity to kube-apiserver, it formally treats this as a signal to stop acting as leader itself, ceasing its own reconciliation work even before another replica has necessarily taken over, avoiding a scenario where a partitioned former leader continues acting despite having lost its lock.

Configurable Timing Parameters

leaseDuration, renewDeadline, retryPeriod

Three formally distinct timing parameters govern the algorithm's behavior: leaseDuration, how long a lease remains valid without renewal; renewDeadline, the maximum time the current leader will attempt renewal before giving up and stepping down; and retryPeriod, how frequently standby replicas check whether the lease has become available.

leaderElection:
  leaseDuration: 15s
  renewDeadline: 10s
  retryPeriod: 2s

The Tradeoff These Parameters Encode

Shorter durations formally produce faster failover after a genuine leader failure but increase the risk of unnecessary leadership churn under transient network jitter; longer durations reduce churn risk but extend the window during which no leader is actively reconciling after a real failure, a tradeoff each component's configuration must balance deliberately.

Observability of the Election State

Tracking Leadership Transitions

The Lease's leaseTransitions field formally increments each time holderIdentity changes, providing a simple counter that reveals how often leadership has actually changed hands, useful for distinguishing a stable, rarely failing-over control plane component from one experiencing frequent, potentially concerning leadership churn.

kubectl -n kube-system get lease kube-controller-manager -o jsonpath='{.spec.leaseTransitions}'
kubectl -n kube-system describe lease kube-scheduler

Why Leader Election Is Built on Ordinary API Primitives

Implementing leader election entirely through Lease objects and the API server's existing optimistic-concurrency guarantees, rather than through a separate, dedicated distributed-locking service, is what allows any custom controller to adopt the exact same reliable coordination mechanism used by the built-in scheduler and controller manager, using nothing beyond the client libraries already required to interact with the Kubernetes API in the first place.