Kubernetes Nodes and Runtime

Kubernetes Nodes and Runtime refers to the worker-side machinery of a cluster: the nodes that provide compute capacity, and the software stack running on each node that actually executes containers on behalf of the control plane. While the control plane decides what should run and where, it is the node and its runtime components that carry those decisions out, translating a PodSpec into running processes isolated by namespaces, cgroups, and container images.

Nodes

Node Registration and Identity

A node, whether a physical machine or a virtual machine, becomes part of a cluster by running the kubelet and registering with the control plane. Upon registration, the node reports its capacity, in terms of CPU, memory, ephemeral storage, and the maximum number of Pods it can host, along with metadata such as its operating system, architecture, and kubelet version. The control plane tracks each node's state through a Node object, which includes conditions such as Ready, MemoryPressure, DiskPressure, and PIDPressure.

Node Status and Heartbeats

Each node periodically reports a heartbeat to the API server, either as a full status update or a lightweight lease object, allowing the Node Controller to detect when a node has become unreachable. If heartbeats stop arriving within a configured threshold, the node is marked NotReady, and after a further grace period, Pods scheduled on it are evicted and rescheduled elsewhere.

Node Taints and Conditions

Nodes can be tainted to repel Pods that do not explicitly tolerate the taint, a mechanism commonly used to reserve dedicated nodes for specific workloads, such as those requiring GPUs, or to automatically mark nodes as unschedulable when they enter an unhealthy condition.

kubelet

The kubelet is the primary agent running on every node and is the component most directly responsible for the node's contribution to reconciliation.

PodSpec Reconciliation

The kubelet watches the API server for Pods assigned to its node and continuously compares the desired container state described in each PodSpec against the actual state of containers on the node, starting, stopping, or restarting containers as needed to converge the two.

Health Probing

The kubelet executes three categories of health probes against containers:

• Liveness probes, which determine whether a container should be restarted.
• Readiness probes, which determine whether a container is ready to receive traffic and should be included in Service endpoints.
• Startup probes, which delay the other two probes until an application has finished its initial startup sequence.

Resource Enforcement

The kubelet enforces the resource requests and limits declared in a Pod's specification, working with the underlying operating system's cgroup mechanism to allocate CPU shares and memory ceilings, and to evict Pods under node resource pressure according to a defined eviction policy.

Container Runtime and the CRI

The Container Runtime Interface

Kubernetes does not implement container execution itself; instead, the kubelet communicates with a container runtime through a standardized gRPC interface, the Container Runtime Interface (CRI). This decoupling allows any CRI-compliant runtime to be used interchangeably without modifying Kubernetes itself.

Common Runtimes

• containerd: A lightweight, industry-standard runtime originally extracted from Docker, widely used as the default runtime in managed Kubernetes offerings.
• CRI-O: A runtime built specifically to satisfy the CRI, designed to be minimal and tightly scoped to Kubernetes' requirements.

Runtime Responsibilities

The container runtime is responsible for pulling and unpacking container images from a registry, creating and managing the low-level namespaces and control groups that isolate each container, and interfacing with a lower-level runtime specification implementation, such as runc, to actually launch container processes.

# Example of runtime-related inspection commands
crictl images
crictl ps
crictl inspect <container-id>

Networking and Storage at the Node Level

CNI Plugins

Each node runs a Container Network Interface (CNI) plugin responsible for allocating a Pod IP address, configuring the Pod's network namespace, and connecting it to the cluster's overall networking fabric so that it can reach and be reached by other Pods.

CSI Plugins

Where Pods require persistent storage, node-level Container Storage Interface (CSI) plugins are responsible for attaching, mounting, and formatting volumes so that they appear inside the Pod's filesystem as specified in its volume mounts.

Node Lifecycle Summary

lifecycle:
  - join: kubelet starts, registers Node object with API server
  - ready: node reports capacity, becomes schedulable
  - scheduled: scheduler binds pods; kubelet starts containers via CRI
  - monitored: kubelet reports status; heartbeats maintain Ready condition
  - degraded: resource pressure triggers eviction of lower-priority pods
  - unreachable: missed heartbeats mark node NotReady; pods rescheduled elsewhere
  - drained: node cordoned and drained for maintenance or removal

This lifecycle illustrates that a node is not a static resource but an actively monitored participant in the cluster's reconciliation process, continuously reporting its state and executing the portion of desired state assigned to it by the control plane.