Kubernetes External Metrics Usage

Kubernetes External Metrics Usage covers the External Metrics API, external.metrics.k8s.io, the aggregated API extension point that exposes metrics describing state entirely outside the Kubernetes cluster — a cloud queue's message count, a managed database's connection count, a third-party service's usage figures — identified purely by metric name and label selector, with no requirement that the metric relate to any Kubernetes object at all.

How the External Metrics API Differs From Custom Metrics

No Object Association Required

Where the Custom Metrics API always associates a metric with a specific Kubernetes object (typically a pod), the External Metrics API carries no such requirement; a metric is identified solely by its name and an optional label selector, since the data it represents may have no natural mapping to anything running inside the cluster.

kubectl get --raw "/apis/external.metrics.k8s.io/v1beta1/namespaces/payments/queue_messages_visible?labelSelector=queue-name%3Dorders"

A Simpler, Flatter Query Shape

Because there is no object to scope the query against, External Metrics API requests are simpler in structure than Custom Metrics API requests, taking a namespace (for RBAC scoping purposes, even though the metric itself may not be namespace-specific in the underlying system) plus the metric name and selector.

Implementing an External Metrics Adapter

Bridging an External System Into Kubernetes

An external metrics adapter authenticates to the relevant external system, queries or subscribes to the specific metric of interest, and exposes it through the External Metrics API in the format the API expects, effectively translating an external system's native monitoring interface into Kubernetes-native metric data.

apiVersion: v1beta1
kind: ExternalMetricValueList
items:
- metricName: queue_messages_visible
  metricLabels:
    queue-name: orders
  timestamp: "2024-01-15T10:30:00Z"
  value: "142"

Credential Scoping for the External System

Because the adapter must authenticate against the external system to retrieve data, scoping its credentials to read-only access limited to the specific metrics it needs — often via workload identity federation rather than static long-lived credentials — follows the same least-privilege principles applied to any other cluster identity with external system access.

Consuming External Metrics

Direct Queries for Verification

Querying the External Metrics API directly, independent of any HPA relying on the same data, isolates whether a scaling problem originates in the adapter and its connection to the external system versus in the HPA's own configuration or calculation.

kubectl get --raw "/apis/external.metrics.k8s.io/v1beta1/namespaces/payments/queue_messages_visible"

Consumption Through HPA

The primary consumer of external metrics is the HorizontalPodAutoscaler's External metric type, which retrieves the named metric's value and uses it (as an absolute Value or an AverageValue divided across replicas) in its scaling calculation, alongside any other configured metrics.

metrics:
- type: External
  external:
    metric:
      name: queue_messages_visible
      selector:
        matchLabels:
          queue-name: orders
    target:
      type: AverageValue
      averageValue: "30"

Access Control Considerations

RBAC Over External Metric Access

Reading external metrics requires RBAC permission against the external.metrics.k8s.io API group, and because these metrics can reveal information about external system state (queue backlogs indicating processing delays, for instance), scoping access to only the identities that genuinely need it — the HPA controller and specific operational tooling — is appropriate.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: external-metrics-reader
rules:
- apiGroups: ["external.metrics.k8s.io"]
  resources: ["*"]
  verbs: ["get", "list"]

Operational Considerations

Dependency on External System Availability

Because the External Metrics API's data ultimately comes from a system outside the cluster's own control, its availability and latency characteristics are inherited directly — an external system outage or degraded performance directly translates into missing or stale metric data, which any consumer (particularly an HPA relying on it for scaling decisions) must be designed to tolerate gracefully.

Distinguishing External Metrics Problems From Cluster Problems

When an HPA using External metrics fails to scale as expected, checking the adapter's own logs and the external system's direct API or dashboard first isolates whether the issue lies outside the cluster entirely, avoiding time spent investigating Kubernetes-side configuration for a problem that originates in the external dependency.

Choosing External Versus Object Metrics Appropriately

A signal that happens to be exposed as a Kubernetes-adjacent resource (a ConfigMap populated by an exporter, for instance) might be more naturally modeled as an Object metric rather than External, reserving External specifically for data with no reasonable Kubernetes object representation at all.