Kubernetes Resource Metrics Usage

Kubernetes Resource Metrics Usage is the practice of collecting, querying, and acting on actual CPU and memory consumption data — as distinct from the declared requests and limits Pods specify — using the Metrics API served by the Metrics Server, the cluster-standard source of lightweight, real-time resource usage data consumed by kubectl top, the Horizontal Pod Autoscaler, and the Vertical Pod Autoscaler. Where requests and limits describe intended resource behavior, metrics usage describes what actually happened, and the gap between the two is precisely the information needed to validate whether resource declarations are well-sized, and to drive autoscaling decisions that respond to real, current load.

The Metrics API is intentionally minimal — providing current CPU and memory usage per Pod and per node without historical retention — leaving longer-term trend analysis and alerting to a full monitoring stack layered on top of it.

The Metrics Server

Lightweight, Point-in-Time Collection

Metrics Server periodically scrapes resource usage data from every kubelet's /stats/summary endpoint across the cluster, aggregates it, and exposes it through the Kubernetes API server's Metrics API — it holds no historical data, only the most recently collected snapshot, keeping its footprint deliberately small for a component many autoscaling decisions depend on.

kubectl top nodes
kubectl top pods --all-namespaces

Not a Full Monitoring Solution

Metrics Server is explicitly not intended to replace a full observability stack (Prometheus and its ecosystem, or a commercial monitoring platform) — it answers "what is current usage right now" for autoscaling and quick operational checks, not "how has usage trended over the past month" for capacity planning or historical analysis.

kubectl top

Node-Level Usage

kubectl top nodes

NAME              CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
node-worker-01    1850m        46%    12Gi            75%
node-worker-02    900m         22%    6Gi              37%

Comparing actual usage against a node's Allocatable capacity gives a real-time view of headroom remaining, complementing the request-based accounting the scheduler uses for placement decisions.

Pod-Level and Container-Level Usage

kubectl top pods -n codartium-team
kubectl top pods -n codartium-team --containers

The --containers flag breaks usage down per container within multi-container Pods, useful for identifying which specific container (a sidecar versus the main application) is responsible for a Pod's overall resource footprint.

Comparing Usage Against Requests and Limits

Validating Request Sizing

kubectl top pod codartium-app --containers
kubectl get pod codartium-app -o jsonpath='{.spec.containers[0].resources.requests}'

Comparing these two outputs side by side is the most direct way to validate whether a container's declared request reflects its actual steady-state consumption, and is the standard first step in any resource right-sizing exercise.

Detecting Approaching Limit Violations

Regularly checking usage against declared limits (rather than waiting for an OOMKilled event to occur) allows proactive limit adjustment before a container is actually terminated for exceeding it — usage trending consistently close to a memory limit is a leading indicator worth acting on before it becomes an incident.

Metrics as Autoscaler Input

Horizontal Pod Autoscaler

The HPA queries the Metrics API (or a custom/external metrics adapter for non-CPU/memory signals) to compare current average utilization across a workload's Pods against a target, scaling replica count up or down accordingly — resource metrics usage is the foundational data source this entire autoscaling mechanism depends on.

kubectl get hpa codartium-api -o jsonpath='{.status.currentCPUUtilizationPercentage}'

Vertical Pod Autoscaler

The VPA similarly consumes historical usage data (through its own metrics history component, going beyond what the base Metrics Server retains) to recommend or automatically apply request/limit adjustments, directly automating the right-sizing exercise that manual kubectl top comparison would otherwise require doing by hand.

Limitations to Keep in Mind

No Built-In Historical Retention

Because Metrics Server retains no history, any trend analysis, alerting on sustained usage patterns, or capacity forecasting requires a separate metrics pipeline (Prometheus scraping cAdvisor or kubelet metrics endpoints directly) rather than relying on the Metrics API alone.

Sampling Interval Smoothing

Metrics Server's periodic scrape interval means very short, transient spikes in usage may not be fully captured in any single query — workloads with bursty, sub-minute usage patterns benefit from a dedicated monitoring pipeline with finer-grained sampling for accurate characterization.

Example

kubectl top pod codartium-app --containers
kubectl get hpa codartium-api
kubectl get pod codartium-app -o jsonpath='{.spec.containers[0].resources}'