Kubernetes Autoscaling

Kubernetes Autoscaling is the set of controllers and mechanisms that automatically adjust the amount of compute capacity allocated to workloads and the cluster itself, in response to observed demand, without requiring an operator to manually resize deployments or provision additional nodes. Autoscaling operates at three largely independent levels: the number of Pod replicas serving a workload, the resource requests and limits assigned to individual containers, and the number of nodes available in the cluster, each addressed by a distinct autoscaler.

Horizontal Pod Autoscaler

Purpose and Mechanism

The Horizontal Pod Autoscaler (HPA) adjusts the replica count of a Deployment, ReplicaSet, or StatefulSet based on observed metrics, most commonly average CPU or memory utilization relative to each Pod's resource requests, scaling out under increased load and back in as demand subsides.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: codartium-api-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: codartium-api
  minReplicas: 3
  maxReplicas: 25
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

Desired Replica Calculation

The HPA controller periodically recalculates the desired replica count from the ratio of current metric value to target metric value, applied against the current replica count, rounding up and respecting the configured minimum and maximum bounds.

Custom and External Metrics

Beyond built-in resource metrics, the HPA can scale based on custom metrics exposed by workloads themselves, such as queue depth or requests per second, or external metrics sourced from systems outside the cluster, obtained through the metrics APIs registered with the cluster's metrics aggregation layer.

metrics:
  - type: Pods
    pods:
      metric:
        name: requests_per_second
      target:
        type: AverageValue
        averageValue: "50"

Scaling Behavior and Stabilization

The behavior field allows fine control over how aggressively the HPA scales up or down, including stabilization windows that prevent rapid oscillation (flapping) by requiring a metric to remain elevated or depressed for a period before acting on it.

spec:
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
    scaleUp:
      stabilizationWindowSeconds: 0

Vertical Pod Autoscaler

Purpose

The Vertical Pod Autoscaler (VPA) addresses a different dimension of scaling: rather than changing the number of replicas, it adjusts the CPU and memory requests and limits assigned to a workload's containers, based on observed historical usage, correcting for requests that were set too low, causing throttling or eviction, or too high, wasting cluster capacity.

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: codartium-worker-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: codartium-worker
  updatePolicy:
    updateMode: "Auto"

Update Modes

VPA supports several modes: Off, which only produces recommendations without applying them; Initial, which applies recommendations only at Pod creation; and Auto, which evicts and recreates Pods to apply updated resource values as usage patterns change over time.

Interaction with HPA

Using CPU- or memory-based HPA scaling and VPA on the same workload's same resource dimension simultaneously is discouraged, since the two controllers would compete: VPA changing per-Pod resource requests would shift the utilization ratio the HPA is measuring against, producing unstable, conflicting scaling behavior.

Cluster Autoscaler

Purpose

The Cluster Autoscaler operates at the infrastructure level, adding nodes to the cluster when Pods cannot be scheduled due to insufficient capacity, and removing nodes that are significantly underutilized and whose Pods could be safely rescheduled elsewhere, keeping the supply of node capacity aligned with aggregate demand.

Scale-Up Trigger

A scale-up is triggered when one or more Pods remain unschedulable due to insufficient resources, at which point the Cluster Autoscaler determines which node group could accommodate the pending Pods and requests additional nodes from the underlying infrastructure provider.

Scale-Down and Safety Constraints

Scale-down considers node utilization over a sustained period and respects safety constraints, such as Pod Disruption Budgets, local storage, and Pods that cannot be evicted, ensuring nodes are only removed when their workloads can be safely relocated without violating availability guarantees configured elsewhere in the cluster.

kubectl get hpa
kubectl describe hpa codartium-api-hpa
kubectl get vpa
kubectl -n kube-system logs deployment/cluster-autoscaler

Combining the Three Layers

A cluster commonly runs all three autoscalers together in a complementary configuration: the Cluster Autoscaler ensures sufficient node capacity exists, the HPA adjusts replica counts of individual workloads within that capacity in response to load, and the VPA, typically limited to workloads not using HPA on the same metric, ensures each replica is sized appropriately for its actual resource consumption.