Kubernetes Availability Observation Basics

Kubernetes Availability Observation Basics is the specific set of metrics and signals used to verify that the reliability mechanisms covered throughout this knowledge area, probes, disruption budgets, topology spread, replica counts, are actually functioning as intended, distinct from the general-purpose observability infrastructure covered as its own knowledge area, focusing narrowly on the signals that answer "is my reliability configuration actually working."

Replica and Readiness Metrics

Observing the Gap Between Desired and Ready

kube_deployment_status_replicas_available{deployment="web"}
/ kube_deployment_spec_replicas{deployment="web"}

A sustained ratio below 1.0 indicates the workload is not achieving its declared replica reliability target, the first and most direct signal that something covered under replica reliability, insufficient cluster capacity, a failing readiness probe, an unschedulable pod template, is not functioning as configured.

kube_pod_container_status_restarts_total

A rising restart count, tracked per pod over time, is the direct observational counterpart to the pod recovery mechanisms covered elsewhere, and a sudden spike specifically correlates with the container-level recovery layer being exercised more heavily than expected.

Disruption Budget Health

Confirming a Budget Has Actual Headroom

kube_poddisruptionbudget_status_disruptions_allowed{poddisruptionbudget="web-pdb"}
kube_poddisruptionbudget_status_current_healthy{poddisruptionbudget="web-pdb"}

disruptions_allowed reaching zero indicates a PodDisruptionBudget has no remaining headroom, precisely the condition that causes a node drain to stall, discussed under disruption budget basics; alerting on this value reaching zero for a sustained period surfaces the problem before an unrelated maintenance operation is attempted and unexpectedly blocked.

Node Condition Tracking

Watching for Node-Level Failure Signals

kube_node_status_condition{condition="Ready", status="true"}
kube_node_spec_taint{key="node.kubernetes.io/unreachable"}

Tracking node readiness conditions and the presence of automatically applied taints directly observes the node failure tolerance mechanisms in action, letting an operator distinguish "a single node is currently unreachable and being handled automatically" from "multiple nodes across a zone are unreachable simultaneously," a materially different and more urgent situation.

Topology Spread Verification

Confirming Actual Distribution, Not Just Configuration

count(kube_pod_info{pod=~"web-.*"}) by (node)

Configuring a topology spread constraint does not itself guarantee it remains satisfied indefinitely, since replacement pods created after node failures could theoretically concentrate if capacity elsewhere becomes constrained; periodically querying actual pod distribution across nodes or zones directly verifies the spread constraint's intended effect is holding in practice, not merely declared in configuration.

Rollout Progress Observation

Detecting a Stuck Rollout Programmatically

kube_deployment_status_condition{condition="Progressing", status="false"}

Alerting directly on the Progressing condition transitioning to False provides an automated equivalent of manually running kubectl rollout status, surfacing a stuck rollout, the specific condition covered under rollout availability basics, without requiring an operator to check it manually after every deployment.

Error Budget Burn Rate

Connecting Reliability Mechanisms to the SLO Framework

(1 - (sum(rate(http_requests_total{status!~"5.."}[1h])) / sum(rate(http_requests_total[1h])))) 
/ (1 - 0.999)

A burn rate metric, computing how quickly an SLO's error budget is being consumed relative to its target window, ties the low-level mechanism signals above (restarts, disruption headroom, rollout progress) to the higher-level availability model's SLO framework, letting an operator distinguish "a minor, budget-compatible blip" from "a burn rate that will exhaust the entire error budget well before the window ends" using the same underlying reliability-mechanism telemetry.

Synthetic Probing as an External Verification Layer

Confirming Availability From Outside the Cluster

apiVersion: v1
kind: ConfigMap
metadata:
  name: blackbox-targets
data:
  targets.yaml: |
    - targets: ["https://myapp.example.com/healthz"]

A synthetic probe run from outside the cluster, distinct from the in-cluster readiness probes covered under readiness availability, verifies the entire path, ingress, load balancer, Service, pod, actually delivers a successful response from a client's true vantage point, catching failures in components between the client and the pod that in-cluster readiness signals alone cannot observe.

Relationship to the Reliability Model, Availability Model, and Broader Observability

Availability observation basics is the narrow, mechanism-specific instrumentation layer that verifies the reliability model's failure-detection-and-correction loop and the availability model's SLI/SLO framework are functioning as designed, distinct from and complementary to the general-purpose metrics, logging, and tracing infrastructure covered under the separate observability knowledge area: where general observability answers broad questions about system behavior, availability observation basics answers the narrower, reliability-specific question of whether every mechanism covered throughout this knowledge area is actually delivering the guarantee it was configured to provide.