Kubernetes Autoscaling Guidelines

Kubernetes Autoscaling Guidelines describe the practices for automatically adjusting the number of Pod replicas, the resources allocated to individual Pods, and the number of nodes in a cluster, in response to actual demand, so that a workload has enough capacity to handle load without a human manually resizing it, while avoiding the cost and instability of persistently over-provisioned capacity.

The Three Autoscaling Dimensions

Horizontal Pod Autoscaler

The Horizontal Pod Autoscaler (HPA) adjusts the number of replicas of a Deployment or StatefulSet based on observed metrics — commonly CPU or memory utilization relative to the Pod's resource request, but also custom or external metrics via the metrics adapter API. Horizontal scaling is the preferred first lever for stateless workloads because it adds redundancy along with capacity, unlike vertical scaling which concentrates more load onto the same number of instances.

Vertical Pod Autoscaler

The Vertical Pod Autoscaler (VPA) adjusts a container's resource requests and limits over time based on observed usage, in one of several modes: Off (recommendation only), Initial (sets requests at Pod creation only), or Auto (actively updates running Pods, which currently requires evicting and recreating the Pod to apply new values). VPA is most valuable for workloads that can't scale horizontally easily — a single-replica batch job, or a StatefulSet with a fixed, small replica count — where getting the per-instance sizing right matters more than adding instances.

Cluster Autoscaler

The Cluster Autoscaler adds nodes when pending Pods can't be scheduled due to insufficient capacity, and removes nodes when they're significantly underutilized and their Pods could be rescheduled elsewhere. It reacts to the consequences of HPA and VPA decisions rather than to application metrics directly — if HPA scales out and there's no room, Cluster Autoscaler is what actually creates the capacity to satisfy that scale-out.

Combining HPA and VPA Safely

The Conflict Between Horizontal and Vertical Scaling on the Same Metric

Running HPA and VPA against the same metric (both reacting to CPU utilization, for instance) on the same workload creates a feedback conflict — VPA resizes the Pod's request, which changes the utilization percentage HPA is computing against, causing both controllers to react to each other's changes rather than to genuine demand. The common safe pairing is VPA managing memory (which HPA rarely targets well due to memory's non-compressible nature) while HPA manages replica count based on CPU or request-rate metrics.

Multi-Metric HPA

An HPA can scale against multiple metrics simultaneously, using the largest resulting replica recommendation across all of them, which is useful when a workload's bottleneck resource varies by traffic pattern (CPU-bound at times, but queue-depth-bound during backlogs) and a single metric wouldn't capture the whole picture.

Metric Selection

Utilization-Based vs. Custom Metrics

CPU/memory utilization is the simplest metric to configure but is often a lagging or indirect proxy for the thing that actually matters — request latency, queue depth, requests-per-second per Pod. Custom metrics (via Prometheus Adapter or similar) let HPA scale against the metric that actually reflects user-facing load, at the cost of additional operational complexity in maintaining that metrics pipeline.

External Metrics for Queue-Driven Workloads

For workloads consuming from an external queue (not itself a Kubernetes object), external metrics let HPA scale replica count based on queue depth or message backlog reported by that external system, which is essential for consumer workloads where Pod-level CPU utilization says almost nothing about whether more consumers are actually needed.

Scaling Behavior Tuning

Stabilization Windows

behavior.scaleDown.stabilizationWindowSeconds (and its scale-up counterpart) prevents the HPA from reacting to brief metric spikes or dips by requiring a sustained trend over the configured window before acting, avoiding a "flapping" pattern where replica count oscillates rapidly in response to noisy, short-lived load fluctuations.

Scale-Down Caution

Scale-down should generally be more conservative than scale-up — arriving slightly early to a load spike is far cheaper than losing capacity during a load spike that hasn't fully subsided yet. policies under behavior.scaleDown can cap the rate and magnitude of replica reduction per stabilization window to enforce this asymmetry explicitly.

Minimum and Maximum Replica Bounds

minReplicas should reflect a workload's genuine floor for redundancy (never 1, for anything expected to tolerate a single Pod failure without a user-visible gap), and maxReplicas should reflect a deliberate ceiling informed by downstream dependency capacity — scaling a frontend workload without bound can simply shift the bottleneck (and the outage) to a downstream database or third-party API that can't absorb the resulting load.

Cluster Autoscaler Interaction

Provisioning Lead Time

New node provisioning takes real time — cloud instance boot, kubelet registration, image pulls — during which pending Pods wait unscheduled. Workloads with steep, sudden traffic ramps should account for this lag, either through over-provisioned baseline capacity, pre-warmed node pools, or a scale-up stabilization window tuned to trigger earlier rather than reactively at the moment of overload.

Pod Disruption Budgets Constrain Scale-Down

Cluster Autoscaler respects PodDisruptionBudgets when deciding whether a node can be drained for removal, meaning an overly strict PDB can prevent legitimate node consolidation and leave the cluster running more (and more expensive) nodes than actually necessary.

Example Configuration

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: codartium-api-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: codartium-api
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 65
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Pods
          value: 2
          periodSeconds: 60

Practical Consequences

Well-tuned autoscaling produces a workload that absorbs traffic growth automatically, maintains redundancy at all times, and keeps infrastructure cost proportional to actual demand rather than peak-provisioned-forever capacity. Poorly tuned autoscaling commonly manifests as replica counts flapping under noisy metrics, scale-out arriving too late to prevent a user-visible latency spike because node provisioning lead time was never accounted for, or a scaled-out frontend simply moving an outage downstream to a database that was never sized to handle the traffic the frontend was now capable of generating.