25 Kubernetes Autoscaling

Kubernetes Autoscaling is the set of mechanisms Kubernetes provides for automatically adjusting the amount of compute capacity allocated to workloads and to the cluster itself, in response to observed demand, reducing the need for manual capacity planning while balancing performance against resource cost.

Layers of Autoscaling

Three Independent Dimensions

Kubernetes autoscaling operates across three largely independent dimensions: the number of Pod replicas for a given workload, the resource requests and limits assigned to individual Pods, and the number of nodes available in the cluster as a whole.

Why Layering Matters

These layers interact: scaling the number of Pod replicas increases aggregate resource demand, which may in turn require the cluster to scale out its number of nodes, meaning effective autoscaling strategies typically combine multiple layers rather than relying on just one.

Horizontal Pod Autoscaler

Scaling Replica Count

The Horizontal Pod Autoscaler (HPA) automatically adjusts the number of replicas for a Deployment, ReplicaSet, or StatefulSet based on observed metrics, most commonly average CPU utilization relative to the Pod's resource requests.

Custom and External Metrics

Beyond basic CPU and memory metrics, the HPA can scale based on custom application-level metrics, such as request queue length, or external metrics sourced from outside the cluster, allowing scaling decisions to reflect the actual bottleneck of a given workload rather than only generic resource usage.

Vertical Pod Autoscaler

Scaling Resource Requests

The Vertical Pod Autoscaler (VPA) adjusts the resource requests and limits of individual containers based on their observed historical usage, aiming to right-size Pods that were initially given requests that are too high or too low for their actual workload.

Interaction with Horizontal Scaling

Because changing a Pod's resource requests typically requires recreating the Pod, the Vertical Pod Autoscaler is generally not combined with the Horizontal Pod Autoscaler on the same metric simultaneously, since the two could otherwise interact in conflicting ways.

Cluster Autoscaler

Scaling Node Count

The Cluster Autoscaler adjusts the number of nodes in the cluster by adding nodes when Pods cannot be scheduled due to insufficient capacity, and removing nodes when they are underutilized and their workloads can be safely consolidated elsewhere.

Cloud Provider Integration

The Cluster Autoscaler typically integrates with the underlying cloud provider's infrastructure APIs to provision or deprovision actual virtual machines, meaning its behavior and configuration options vary somewhat depending on the environment in which the cluster runs.

Scaling Policies and Stability

Avoiding Thrashing

Autoscalers apply stabilization windows and cooldown periods to avoid rapidly scaling up and down in response to short-lived spikes or noise in observed metrics, which would otherwise cause instability and unnecessary churn in running Pods.

Scaling Bounds

Both the Horizontal Pod Autoscaler and Cluster Autoscaler are typically configured with minimum and maximum bounds, ensuring that scaling decisions remain within an operator-defined safe range rather than growing or shrinking without limit.

Autoscaling Layers Diagram