Kubernetes Controller Architecture

Kubernetes Controller Architecture is the internal design pattern shared by every reconciliation loop in Kubernetes, whether a built-in loop running inside kube-controller-manager or a custom controller managing a Custom Resource Definition: an informer-backed cache of watched objects, a work queue of items needing reconciliation, and a worker loop that repeatedly pulls items from that queue and drives them toward their desired state. This shared internal shape is what allows built-in and third-party controllers to be reasoned about, and built, using the exact same architecture.

The Informer Pattern

Local Cache Backed by a Watch

A controller does not issue a fresh API read for every reconciliation; instead, it constructs an informer, a component that establishes a watch against the API server for a given resource type, maintains a local, continuously updated in-memory cache of that resource's objects, and exposes lightweight, indexed lookups against that cache to the controller's own logic.

List-Then-Watch and Periodic Resync

An informer formally initializes its cache with a full list of the target resource, then transitions to consuming the watch stream's incremental events; independently, it performs a periodic full resync, re-enqueuing every cached object for reconciliation regardless of whether it changed, guarding against silently missed watch events.

# Illustrative informer configuration (conceptual, not a real manifest)
informer:
  resource: pods
  resync_period: 30s

The Work Queue

Decoupling Observation from Processing

Rather than reconciling an object the instant a watch event arrives, an informer's event handlers formally enqueue the object's key into a work queue; a separate pool of worker goroutines pulls from this queue, decoupling the rate at which changes are observed from the rate at which they are processed.

Deduplication

The work queue formally deduplicates keys already pending processing: if the same object changes multiple times before a worker gets to it, only one reconciliation is triggered, using the latest cached state at the time processing occurs, rather than one reconciliation per individual change event.

Rate Limiting and Retry

Upon a reconciliation failure, a controller formally re-adds the failed item to the queue with an exponentially increasing backoff delay, preventing a persistently failing reconciliation from consuming worker capacity in a tight retry loop while still eventually retrying as the underlying condition may resolve.

The Reconciliation Function

Read Current State, Compute Desired Action

A controller's core reconciliation function formally follows the same shape regardless of what it manages: read the target object's current spec and the current state of whatever it controls, compute the difference, and issue whatever create, update, or delete calls are needed to close that difference, then return.

# Illustrative reconciliation shape (conceptual)
function reconcile(key):
  obj = cache.get(key)
  actual = observe_related_state(obj)
  desired = obj.spec
  if actual != desired:
    apply_corrective_actions(actual, desired)

Idempotency as a Formal Requirement

Because a reconciliation function may be invoked repeatedly for the same object, due to resyncs, retries, or overlapping events, it is formally required to be idempotent: invoking it multiple times in immediate succession against unchanged state must produce no additional side effects beyond the first invocation.

Shared Informers Across Multiple Controllers

Avoiding Redundant Watches

Where a single process, such as kube-controller-manager, runs many controllers that each need to observe the same resource type, a shared informer factory formally provides one underlying informer and cache per resource type, with multiple controllers registering their own event handlers against it, avoiding redundant watch connections and duplicate caches for the same data.

kubectl -n kube-system logs deployment/kube-controller-manager | grep -i "starting"

Why Controllers Are Architected This Way

Building every controller on the same informer-cache-queue-reconcile pattern is what formally makes controller behavior predictable and composable: any controller built this way tolerates missed events (via resync), tolerates transient failures (via requeue with backoff), and tolerates concurrent modification (via idempotent reconciliation), properties that follow directly from the architecture rather than needing to be independently reimplemented by each controller author.