16.1.3.1 Stale Build Layer

A stale build layer, in the sense of an already-built and published image rather than the build process's own cache decisions, is a layer whose content no longer reflects the current, intended state of whatever it was supposed to capture, most commonly a base image that has not been re-pulled since it was last updated upstream, producing a deployed image that is silently running on outdated, sometimes vulnerable, underlying software despite every build appearing to succeed normally.

The base image staleness problem

A FROM instruction referencing a mutable tag, such as node:20 rather than a specific digest, resolves to whatever that tag currently points to locally, which may not be the latest version actually published under that tag if the local Docker installation has not pulled an updated copy recently:

FROM node:20

docker build -t my-api .

Without an explicit pull, this build can silently reuse a node:20 image cached locally from weeks or months earlier, even though the upstream node:20 tag has since been updated with security patches, missing those updates entirely while the build itself reports no error or warning of any kind.

Forcing a fresh base image pull

The --pull flag forces Docker to check for and retrieve a newer version of the base image before proceeding with the build, rather than relying on whatever copy happens to already exist locally:

docker build --pull -t my-api .

Including this flag as a standard part of any build pipeline, rather than only the first time an image is built, ensures base image updates are picked up consistently rather than depending on whichever machine happens to be running the build and what it happens to have cached locally already.

Pinning by digest to make staleness an explicit, deliberate choice

Referencing a base image by its specific content digest, rather than a mutable tag, removes the ambiguity entirely: a digest always refers to the exact same, immutable content, which means staleness becomes a deliberate decision (continuing to use an old digest) rather than something that happens silently due to local caching behavior:

FROM node:20@sha256:3f29a8c1d8e2b4f6a9c7d5e8b1f3a6c9d2e5f8b1a4c7d0e3f6a9c2d5e8b1f4a7

docker pull node:20
docker inspect node:20 --format '{{index .RepoDigests 0}}'

Updating to a newer base image then becomes an explicit, version-controlled change to the digest reference itself, fully visible in code review and the commit history, rather than something that happens invisibly whenever a build machine's local cache happens to be refreshed.

CI environments and persistent build caches

CI runners that reuse a persistent Docker layer cache across many separate build jobs can compound this problem: a base image pulled once and cached at the start of a CI environment's lifetime can remain in use across many subsequent builds, all silently running against the same potentially outdated base image, until the CI cache itself is cleared or expires:

build:
  script:
    - docker build --pull -t my-api .

Explicitly including --pull in the CI pipeline's own build step, rather than relying on whatever the runner's persistent cache happens to already contain, ensures each CI build at least checks for a base image update, regardless of how long the runner's own cache has been accumulating without a full reset.

Auditing currently deployed images for base image staleness

For images already built and deployed, checking how old the actual base image layers are, relative to what is currently available upstream, surfaces staleness that would otherwise go unnoticed until a security scan or an unrelated incident draws attention to it:

docker image inspect my-api:1.4.0 --format '{{.Created}}'

docker scout cves my-api:1.4.0

Image scanning tools that check for known vulnerabilities in the specific package versions present within an image's layers are a more direct and reliable way to surface base image staleness with real security consequences, compared to only checking a build timestamp, which does not by itself indicate whether anything security-relevant has actually changed upstream since that image was built.

Rebuilding to refresh a stale layer

Once staleness is identified, the fix is rebuilding with an updated base image reference, whether that means bumping a pinned digest to a newer one or simply re-pulling a mutable tag, and then redeploying the resulting, refreshed image:

docker pull node:20
docker build --pull --no-cache -t my-api .

Using --no-cache alongside --pull for this specific rebuild ensures that not only the base image itself but every subsequent layer is genuinely re-executed against the fresh base, rather than potentially reusing other cached layers that might mask whether the update actually took full effect throughout the build.

Scheduling regular base image refresh as routine maintenance

Rather than only addressing base image staleness reactively, after a vulnerability scan flags it, scheduling a regular, routine rebuild and redeploy cycle specifically to pick up base image updates keeps this from becoming a recurring, individually investigated incident:

0 3 * * 1 docker build --pull --no-cache -t my-api . && docker push registry.example.com/my-api:latest

A weekly or similarly scheduled rebuild, even without any application code change, ensures base image security patches are incorporated on a predictable cadence rather than only whenever someone happens to notice or a scan happens to flag an issue.

Common mistakes

• Referencing base images by mutable tag without ever forcing a fresh pull, silently building against whatever happens to be cached locally indefinitely.
• Not including --pull in CI pipeline build steps, allowing a runner's persistent cache to serve an increasingly outdated base image across many builds over time.
• Treating digest pinning as a one-time setup rather than something that needs periodic, deliberate updates to actually benefit from upstream security patches.
• Relying only on a build's own success as evidence that everything in the resulting image is current, rather than using an image scanning tool to directly check for known vulnerabilities in the actual installed package versions.
• Addressing base image staleness only reactively after a scan or incident, rather than scheduling routine, regular rebuilds specifically to pick up upstream updates.

A stale build layer in the context of base images is a silent, easily overlooked risk precisely because nothing in a normal build process surfaces it as an error, and the combination of explicit --pull usage, deliberate digest pinning with scheduled updates, and periodic vulnerability scanning is what actually closes this gap rather than leaving base image freshness to chance.