15.2.2.2 Stats Container Metrics

Stats container metrics, accessed through Docker's underlying statistics API rather than the docker stats command itself, expose the raw, unprocessed cgroups counters that the CLI tool formats into percentages and human-readable units, and working with this raw API directly is necessary for building any custom monitoring tool, exporter, or automated check that needs more control than the CLI's fixed display provides.

The underlying API endpoint

docker stats is itself a client built on top of a daemon API endpoint that returns raw statistics as JSON, and that same endpoint can be queried directly, either through the Docker CLI's lower-level access or by calling the daemon's API directly:

docker run -d --name my-api my-api:1.4.0
curl --unix-socket /var/run/docker.sock http://localhost/containers/my-api/stats?stream=false

The stream=false parameter requests a single snapshot rather than the continuously streaming response the endpoint returns by default, which is the equivalent of the CLI's --no-stream flag at the API level.

Structure of the raw response

The raw JSON response contains nested objects for CPU, memory, network, and block I/O statistics, each with considerably more detail than the CLI's summarized display shows:

{
  "cpu_stats": {
    "cpu_usage": { "total_usage": 1234567890 },
    "system_cpu_usage": 9876543210,
    "online_cpus": 4
  },
  "precpu_stats": {
    "cpu_usage": { "total_usage": 1230000000 },
    "system_cpu_usage": 9870000000
  },
  "memory_stats": {
    "usage": 220200960,
    "limit": 536870912,
    "stats": { "cache": 10485760 }
  }
}

Notably, the response includes both cpu_stats (the current reading) and precpu_stats (the previous reading), which exist specifically because calculating a CPU percentage requires comparing two points in time, a detail the CLI handles internally but that any custom tooling built directly on this API needs to replicate itself.

Calculating CPU percentage from raw counters

The CPU percentage shown by docker stats is derived through a specific formula combining the current and previous CPU usage figures along with the number of available CPUs, and replicating this calculation correctly is one of the more error-prone parts of building custom tooling against the raw API:

function calculateCpuPercent(stats) {
  const cpuDelta = stats.cpu_stats.cpu_usage.total_usage - stats.precpu_stats.cpu_usage.total_usage;
  const systemDelta = stats.cpu_stats.system_cpu_usage - stats.precpu_stats.system_cpu_usage;
  const onlineCpus = stats.cpu_stats.online_cpus;
  return (cpuDelta / systemDelta) * onlineCpus * 100;
}

A common mistake when implementing this calculation independently is using cpu_stats alone without subtracting the precpu_stats baseline, which produces a nonsensical, dramatically inflated percentage rather than the actual rate of CPU consumption between the two sampled points.

Memory statistics and the cache distinction

The raw memory statistics object includes a detailed breakdown beyond the single combined figure the CLI displays, including the cache component that is worth subtracting when a more accurate view of genuine application memory pressure is needed:

function calculateMemoryUsage(stats) {
  const rawUsage = stats.memory_stats.usage;
  const cache = stats.memory_stats.stats.cache || 0;
  return rawUsage - cache;
}

Building a custom exporter that reports this cache-excluded figure, rather than the raw combined usage, can produce a metric that more accurately reflects actual application memory pressure than what docker stats shows by default, which is one of the practical reasons to work with the raw API rather than relying solely on the CLI's summarized output.

Streaming statistics for continuous collection

For a custom collector or exporter that needs continuous data rather than periodic polling, the same endpoint supports a streaming mode that delivers a new JSON object as each update becomes available, without needing to repeatedly open new connections:

curl --unix-socket /var/run/docker.sock http://localhost/containers/my-api/stats

import docker
client = docker.from_env()
container = client.containers.get('my-api')
for stat in container.stats(stream=True, decode=True):
    print(stat['memory_stats']['usage'])

Using an official Docker SDK, such as the Python or Go client libraries, rather than constructing raw HTTP requests by hand, generally simplifies working with this streaming interface considerably, since the SDK handles connection management and JSON parsing automatically.

Building a custom exporter

A common reason to work with this raw API directly is building a custom Prometheus exporter or similar integration that needs to expose container metrics in a specific format not provided by an existing tool like cAdvisor, or that needs to combine container metrics with additional, application-specific context not available from the Docker API alone:

from prometheus_client import Gauge, start_http_server
import docker

cpu_gauge = Gauge('container_cpu_percent', 'CPU percent', ['container'])
client = docker.from_env()

def collect():
    for container in client.containers.list():
        stats = container.stats(stream=False)
        cpu_gauge.labels(container=container.name).set(calculate_cpu_percent(stats))

This level of customization is rarely necessary for typical deployments, where cAdvisor and a standard Prometheus or similar pipeline already cover the common case well, but it becomes valuable when integrating container resource data with a proprietary or unusual monitoring system that has no existing Docker integration.

Common mistakes

• Calculating CPU percentage using only the current reading without the previous baseline, producing an incorrect, dramatically inflated figure.
• Treating the raw memory usage field as equivalent to genuine application memory pressure without accounting for the included cache component.
• Building custom tooling against the raw streaming API when an existing, well-tested tool like cAdvisor already covers the requirement adequately.
• Polling the snapshot endpoint at an unnecessarily high frequency for a custom collector, adding load to the Docker daemon without a corresponding benefit in metric freshness.
• Not using an official SDK for the target language, leading to more fragile, manually constructed HTTP and JSON handling than necessary.

Working with stats container metrics through the raw API is the right approach specifically when building genuinely custom monitoring tooling that an existing solution does not already cover, and doing so correctly requires understanding the two-point CPU calculation, the cache-inclusive memory figure, and the streaming versus snapshot modes the underlying endpoint provides, all of which docker stats and tools like cAdvisor already handle correctly on a typical deployment's behalf.