15.1.2.5 Fluentd Log Driver

The fluentd logging driver sends a container's stdout and stderr output directly to a Fluentd or Fluent Bit instance using the Fluentd forward protocol, removing the need for a separate node-level collector to tail and parse local log files, since the container's own log stream is delivered straight to the collection layer as it is produced.

Basic configuration

The driver requires the address of a reachable Fluentd or Fluent Bit instance accepting the forward protocol, typically running as another container on the same Docker network or as a service elsewhere on the network:

docker run -d --log-driver=fluentd --log-opt fluentd-address=fluentd-host:24224 my-api

services:
  api:
    logging:
      driver: fluentd
      options:
        fluentd-address: "fluentd:24224"

  fluentd:
    image: fluent/fluentd
    ports:
      - "24224:24224"

Because the driver pushes log data directly to Fluentd rather than relying on Fluentd to tail local files, this pattern works the same way whether Fluentd is running as a sidecar, a node-level collector, or a remote aggregation endpoint reachable over the network.

Tagging for routing within Fluentd

Fluentd routes incoming log data based on a tag, and the driver supports templating the tag from container metadata, which is what allows a single Fluentd instance receiving logs from many different containers to route, filter, or transform each container's logs differently based on which container produced them:

docker run -d --log-driver=fluentd \
  --log-opt fluentd-address=fluentd-host:24224 \
  --log-opt tag="docker.{{.Name}}" \
  my-api

<match docker.my-api>
  @type elasticsearch
  index_name my-api-logs
</match>

<match docker.worker>
  @type elasticsearch
  index_name worker-logs
</match>

Without a meaningful tag template, every container's logs arrive at Fluentd with the same generic tag, making it difficult to apply different processing rules to different services even though they are all flowing through the same collector instance.

Buffering and retry behavior

The fluentd driver buffers log messages locally and retries delivery if the configured Fluentd endpoint is temporarily unreachable, which protects against losing log data during a brief network interruption or a Fluentd restart, up to the configured buffer limits:

docker run -d --log-driver=fluentd \
  --log-opt fluentd-address=fluentd-host:24224 \
  --log-opt fluentd-buffer-limit=5242880 \
  --log-opt fluentd-retry-wait=1s \
  --log-opt fluentd-max-retries=30 \
  my-api

By default, the driver buffers in memory, which means the buffer itself does not survive a container restart; for workloads where this matters, the application or driver configuration needs to account for the possibility of losing buffered-but-undelivered log data if the container itself is restarted while the Fluentd endpoint is unreachable.

Blocking versus non-blocking delivery

A particularly important behavior to understand is what happens when the buffer fills because Fluentd remains unreachable for an extended period: by default, the driver blocks the container's own log writes once the buffer is full, which can in turn block the application itself if it is not designed to handle a slow or blocked write to its own stdout:

docker run -d --log-driver=fluentd \
  --log-opt fluentd-address=fluentd-host:24224 \
  --log-opt fluentd-async=true \
  my-api

Enabling asynchronous mode avoids this blocking behavior, dropping log messages rather than blocking the application when the buffer is full, which is generally the safer choice for production application containers, since losing some log messages during an extended outage of the logging backend is almost always preferable to the application itself becoming unresponsive because its log writes are blocked.

No local read-back support

Like several other remote-forwarding drivers, fluentd does not support docker logs, since the driver's entire purpose is delivering output directly to Fluentd rather than retaining a separately queryable local copy:

docker logs my-api

Error response from daemon: configured logging driver does not support reading

Any quick, local log inspection during debugging needs to go through whatever interface Fluentd or its downstream destination provides, or through Docker's dual logging feature if local read access is still needed alongside the Fluentd forwarding.

Fluent Bit as a lighter-weight alternative endpoint

Fluent Bit, a lighter-weight reimplementation compatible with the Fluentd forward protocol, is commonly used as the direct receiving endpoint for the fluentd driver specifically because of its lower resource footprint, while still forwarding onward to a heavier Fluentd instance or directly to a final aggregation backend:

services:
  fluent-bit:
    image: fluent/fluent-bit
    ports:
      - "24224:24224"

This two-tier pattern, lightweight Fluent Bit receiving directly from containers via the driver, then forwarding to a more capable but heavier downstream system, is a common way to balance per-host resource usage against the processing capability needed for routing, filtering, and final delivery.

Resilience considerations for the receiving endpoint

Because containers using this driver depend on their configured Fluentd endpoint being reachable, the endpoint itself needs to be treated as critical infrastructure with its own restart policy and monitoring, since its unavailability directly affects every container configured to forward logs to it:

services:
  fluentd:
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "fluentd", "--dry-run"]

Common mistakes

• Leaving the driver in its default blocking mode for an application container, risking the application itself becoming unresponsive if the Fluentd endpoint is unreachable for an extended period.
• Using a generic, non-templated tag, making it difficult to route or filter logs differently once many containers are all forwarding through the same Fluentd instance.
• Forgetting that the fluentd driver disables docker logs, and not enabling dual logging when local read access is still needed.
• Treating the receiving Fluentd or Fluent Bit endpoint as a low-priority component, when in practice its availability directly affects every container configured to depend on it.
• Not tuning buffer size and retry settings to match the actual tolerance for log loss versus blocking during an extended endpoint outage.

The fluentd driver is a strong choice for delivering container logs directly to a Fluentd-compatible collection layer without a separate file-tailing step, provided asynchronous mode is enabled to avoid blocking application writes during an outage, tags are templated meaningfully for downstream routing, and the receiving endpoint itself is treated with the same operational care as any other critical infrastructure component.