16.3.2.5 Build Cache Bloat

Build cache bloat is the accumulation of BuildKit's own cached build layers and mount caches, stored separately from images and containers, and because this cache exists specifically to make repeated builds faster, it grows in direct proportion to how actively a project is being built and iterated on, frequently becoming the single largest category of reclaimable space on a host used for regular development or CI build activity.

Why build cache grows faster than it might seem like it should

Every distinct combination of instruction and input that BuildKit has ever processed can potentially be retained as a separate cache entry, which means a project with frequently changing dependencies, multiple branches being built in parallel, or many slightly different build configurations can accumulate a surprisingly large number of cache entries relatively quickly:

docker builder du

ID                                              RECLAIMABLE   SIZE
a1b2c3d4e5f6                                    true          1.2GB
f6e5d4c3b2a1                                    true          890MB
...

Unlike image layers, which are deduplicated and shared across images with identical content, build cache entries can multiply more readily, since even small variations between builds, a slightly different dependency version, a different build argument, can each produce their own distinct, separately retained cache entries.

Checking overall build cache size

The aggregate figure in docker system df is a useful starting point for understanding how much of overall disk usage is attributable to build cache specifically, separate from images and containers:

docker system df

TYPE            TOTAL   ACTIVE   SIZE      RECLAIMABLE
Build Cache     342     0        24.6GB    24.6GB (100%)

A build cache reporting 100% reclaimable, as shown here, is expected when no build is currently in progress, since cache entries are only considered "active" while actually being used by a running build; this figure does not mean the cache is useless, only that none of it is needed at this exact moment, which is different from saying none of it would be useful for a future build.

Pruning with age-based filtering

Removing all build cache indiscriminately discards entries that might still produce a useful cache hit on the very next build, which defeats much of the purpose of having a persistent cache in the first place; filtering by age balances reclaiming space against retaining cache likely to still be useful soon:

docker builder prune --filter "until=72h"

docker builder prune -af

The age-filtered version removes only cache entries unused for the specified duration, which is generally the more appropriate default for routine maintenance, reserving the unfiltered, aggressive prune for situations where maximum space reclamation is needed immediately, accepting the cost of losing cache hits on upcoming builds that would otherwise have benefited from the now-removed entries.

Cache mounts as a separate, additional accumulation source

BuildKit's cache mount feature, used to persist package manager download caches across builds, accumulates its own separate storage outside of the standard layer cache, and this category needs to be checked and managed somewhat independently, since standard build cache pruning commands do not necessarily address cache mount storage the same way:

RUN --mount=type=cache,target=/root/.npm npm install

docker builder prune --filter type=exec.cachemount

Cache mounts can accumulate substantially for a project with frequently changing dependencies, since the underlying package manager cache itself grows as more distinct package versions pass through it over the project's history, even though the cache mount mechanism's purpose is specifically to make this accumulation worthwhile by speeding up future installs.

Multiple builder instances multiplying cache storage

docker buildx supports creating multiple, independent builder instances, each maintaining its own separate cache, which means a host or CI environment using several builder instances for different purposes can have build cache effectively multiplied across each one, rather than sharing a single, unified cache pool:

docker buildx ls

NAME       DRIVER     STATUS
default    docker     running
multiarch  docker-container   running

docker buildx du --builder multiarch

Checking and pruning cache for every active builder instance individually, rather than assuming a single prune command addresses all of them, is necessary on hosts that have accumulated more than one builder instance over time, particularly relevant for setups that created a dedicated multi-architecture builder instance separate from the default one.

CI runner persistent cache growing unbounded

CI environments that explicitly persist build cache across separate job runs, to avoid starting from an empty cache on every single CI build, need their own deliberate cache size management, since a persistent CI cache volume or directory can otherwise grow indefinitely across the full history of every build the CI system has ever run:

build:
  cache:
    paths:
      - .buildx-cache

docker buildx build --cache-to=type=local,dest=.buildx-cache,mode=max .

A CI-specific cache size limit, or a scheduled, separate cleanup job specifically targeting the CI system's own persisted cache storage location, addresses this independently of whatever cleanup happens on individual developer machines, since the two accumulate cache through entirely separate mechanisms and lifecycles.

Multi-platform builds multiplying cache size

Building for multiple target platforms simultaneously, through docker buildx build --platform, generally produces and caches separate build artifacts and intermediate layers for each target architecture, which means multi-platform build cache can be noticeably larger than the equivalent single-platform build's cache, simply because there is genuinely more, architecture-specific content being cached:

docker buildx build --platform=linux/amd64,linux/arm64 -t my-api .
docker builder du

This is expected and not itself a sign of misconfiguration, but it is worth factoring into capacity planning for any host or CI runner regularly performing multi-platform builds, since the storage requirement is genuinely higher than single-platform builds would produce.

Common mistakes

• Assuming build cache reported as 100% reclaimable means it is entirely unnecessary, rather than recognizing this reflects only that no build is currently using it at this exact moment.
• Pruning build cache aggressively and indiscriminately as routine maintenance, discarding entries that would have produced a useful cache hit on the very next build.
• Overlooking cache mounts as a separate accumulation source not necessarily addressed by standard build cache pruning commands.
• Not checking and pruning cache across every active buildx builder instance individually, missing accumulation in a secondary or multi-architecture builder instance.
• Not accounting for multi-platform builds genuinely requiring more total cache storage than single-platform builds, mistaking the larger size for a problem rather than an expected consequence.

Build cache bloat is a natural, expected consequence of having a cache that actively makes builds faster, and managing it well means using age-based filtering rather than indiscriminate full pruning for routine cleanup, checking cache mounts and every active builder instance separately, and planning CI-specific persistent cache storage with its own deliberate size limits rather than letting it accumulate across the system's entire build history unchecked.