15.1.1.4 Container Log Rotation

Container log rotation is the mechanism that caps how much disk space a container's locally stored log output is allowed to consume, by limiting individual log file size and the number of historical files retained, preventing what would otherwise be unbounded growth for any long-running container under Docker's default logging configuration.

Why rotation is necessary by default

Docker's default json-file logging driver does not rotate or limit log file size unless explicitly configured. A long-running container with even a moderate volume of log output will, left unconfigured, accumulate a single, ever-growing log file for as long as the container exists, which for a production service running for months can eventually consume a significant and unpredictable amount of host disk space:

docker run -d my-api
docker inspect --format='{{.LogPath}}' my-api

ls -lh "$(docker inspect --format='{{.LogPath}}' my-api)"

Checking the actual size of a long-running container's log file directly is a useful way to confirm whether rotation has been configured at all, since an unexpectedly large file is a clear sign that it has not.

Configuring rotation per container

Rotation for the json-file and local drivers is controlled through two options: the maximum size of an individual log file before rotation occurs, and the maximum number of rotated files to retain:

docker run -d --log-opt max-size=10m --log-opt max-file=5 my-api

services:
  api:
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "5"

With these settings, the driver rotates to a new file once the current one reaches 10 megabytes, and retains at most 5 such files, deleting the oldest once a new rotation would exceed that count; the total maximum disk usage for this container's logs is therefore bounded at roughly 50 megabytes.

Setting a sensible daemon-wide default

Configuring rotation at the Docker daemon level applies it as the default for every container that does not explicitly override it, which is considerably more reliable than depending on every individual container definition to set rotation limits consistently:

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}

systemctl restart docker

A daemon-wide default is one of the more impactful, low-effort production hardening steps available, since it protects against unbounded log growth even for containers started by someone who forgot or did not know to set rotation explicitly.

Sizing rotation limits appropriately

The right size and file count depend on the container's actual log volume and how much recent history needs to remain available through docker logs before it is rotated away. A high-throughput service producing verbose output needs more generous limits than a quiet, low-traffic one to avoid losing recent, potentially relevant log history too quickly:

docker logs --since 1h my-api | wc -l

Measuring a container's actual recent log volume is a reasonable way to calibrate rotation settings: if an hour of normal operation already produces close to the configured max-size, the rotated history retained by max-file may cover a much shorter window of real time than the configuration might suggest at a glance.

What rotation does and does not protect against

Rotation bounds the steady-state disk usage of a single container's local log storage, but it does not protect against a sudden, sharp spike in log volume exhausting disk space before rotation has a chance to act, particularly if the rotation check happens at write time but the spike itself is large enough to fill remaining disk in a very short window:

df -h /var/lib/docker

Monitoring overall host disk usage independently of per-container log rotation settings remains necessary, since rotation limits the eventual steady-state footprint but does not eliminate the need for genuine disk capacity monitoring and alerting.

Rotation when using a remote-native logging driver

Drivers that send log output directly to a remote destination, rather than storing it locally, are not subject to the same local-disk rotation concern, since no local log file accumulates in the first place:

docker run -d --log-driver=awslogs --log-opt awslogs-group=my-app-logs my-api

In this case, retention and storage limits are managed entirely by the remote destination's own retention policy rather than by Docker's local rotation options, which is one of the practical trade-offs in choosing a remote-native driver over a locally stored one.

Rotation and centralized log collection working together

In a setup using a node-level collector that reads json-file logs from disk and forwards them onward, rotation settings need to be generous enough that the collector has a reasonable chance to read a log file's contents before it is rotated away and deleted, particularly if the collector experiences any delay or backlog:

[INPUT]
    Name tail
    Path /var/lib/docker/containers/*/*.log
    Read_from_Head true

A max-file setting that is too aggressive relative to the collector's typical read interval risks losing log content that was rotated out of local storage before the collector had a chance to forward it to its centralized destination.

Common mistakes

• Leaving log rotation entirely unconfigured, both per-container and at the daemon level, allowing unbounded local log growth on any long-running container.
• Setting rotation limits without measuring actual log volume, resulting in a retained history window that covers far less real time than expected.
• Assuming rotation alone is sufficient disk protection, without separately monitoring overall host disk usage for sudden spikes.
• Configuring rotation limits too aggressively relative to a log collector's read interval, causing the collector to occasionally miss content that was rotated away before it could be read.
• Forgetting that remote-native logging drivers are not subject to local rotation settings at all, since they do not retain a local log file in the first place.

Container log rotation is a small, easily overlooked configuration detail with outsized consequences if neglected; setting sensible, daemon-wide default limits, sized to actual log volume and coordinated with any log collection pipeline reading from local storage, prevents what is otherwise one of the most common and entirely avoidable causes of unexpected host disk exhaustion.