Kubernetes Cluster Communication Architecture

Kubernetes Cluster Communication Architecture is the set of formally distinct communication paths connecting nodes to the control plane and the control plane back to nodes, each secured and routed differently, tracing why cluster communication is not a single symmetric channel but two directionally distinct patterns: node-to-control-plane traffic, which is comparatively simple and always initiated outward from the node, and control-plane-to-node traffic, which is architecturally harder to secure and has evolved through several distinct solutions.

Node-to-Control-Plane Communication

The Primary, Simple Direction

The overwhelming majority of cluster communication formally flows from nodes toward the control plane: the kubelet on every node initiates outbound connections to the API server to report status and watch for assigned Pods, and kube-proxy similarly initiates outbound watches against Services and EndpointSlices.

Authentication of Outbound Connections

Because this direction is always node-initiated, it is formally secured using standard client-server TLS: the node presents client certificates (or another supported credential) to authenticate itself to the API server, and the API server's own serving certificate, validated against a known certificate authority, authenticates the API server to the node.

cat /var/lib/kubelet/pki/kubelet-client-current.pem
kubectl get csr

API Server Serving Endpoint

Nodes formally reach the API server through a single, well-known endpoint, typically fronted by a load balancer in highly available deployments, meaning individual nodes need no awareness of how many API server replicas exist behind that endpoint.

Control-Plane-to-Node Communication

Why This Direction Is Architecturally Harder

Traffic originating from the control plane toward a node, most notably the API server calling a kubelet directly for exec, logs, or port-forward operations, is formally harder to secure than the reverse direction because nodes commonly sit behind firewalls, NAT, or private networks not directly reachable from the control plane's own network position.

Direct Connection (Unsecured Network Assumption)

In its simplest form, the API server formally connects directly to a kubelet's HTTPS endpoint on a known port; this approach assumes the network path between control plane and node is itself trusted or otherwise secured, an assumption that does not hold in many production topologies.

curl -k https://<node-ip>:10250/pods

SSH Tunneling (Legacy Approach)

An older approach formally routes API-server-to-node traffic through an SSH tunnel established from the control plane to each node, providing an encrypted, authenticated path across untrusted network segments without requiring nodes to be directly reachable on arbitrary ports from the control plane.

Konnectivity Service

The current recommended approach formally introduces a Konnectivity service: an agent runs on each node (or as a proxy layer) and initiates an outbound connection to a Konnectivity server co-located with the control plane; the API server then routes its outbound-to-node traffic through this already-established, node-initiated tunnel, converting the architecturally difficult control-plane-to-node direction into a reuse of the simpler node-to-control-plane connection pattern.

apiVersion: apiserver.k8s.io/v1beta1
kind: EgressSelectorConfiguration
egressSelections:
  - name: cluster
    connection:
      proxyProtocol: GRPC
      transport:
        uds:
          udsName: /etc/kubernetes/konnectivity-server/konnectivity-server.socket

Communication Within the Control Plane Itself

API-Mediated Coordination

As established elsewhere, control plane components formally never communicate with each other directly; scheduler, controller manager, and cloud controller manager all reach the API server the same way any node would, an outbound, client-authenticated connection, meaning the node-to-control-plane pattern described above applies internally to the control plane's own components as well.

kubectl -n kube-system logs deployment/kube-scheduler | grep -i "connected"

Practical Consequences for Cluster Design

Firewall and Network Policy Implications

Because control-plane-to-node traffic is architecturally the harder direction to secure, cluster network design formally needs to account for it explicitly, either by ensuring direct reachability on the kubelet's serving port, tunneling via SSH, or deploying Konnectivity, rather than assuming the same simple outbound-only rule set that suffices for node-to-control-plane traffic is sufficient in both directions.

kubectl get pods -n kube-system -l k8s-app=konnectivity-agent

Why Communication Is Architected Asymmetrically

Treating node-to-control-plane and control-plane-to-node as formally distinct communication patterns, rather than a single bidirectional channel, reflects the genuine asymmetry of typical cluster network topologies: nodes can almost always reach a well-known control plane endpoint, but the control plane cannot always assume the reverse is true, and the evolution from direct connection to SSH tunneling to Konnectivity reflects successive attempts to close that gap without requiring every deployment to expose nodes directly to the control plane's network.