✦ For everyone, free.

Practical knowledge for real and everyday life

Home

16.2.1.3 Runtime Executable Missing

A focused guide to Runtime Executable Missing, connecting core concepts with practical Docker and container operations.

Runtime executable missing covers a specific, often confusing variant of a missing-command error: the binary itself is genuinely present in the image at the correct path with the correct permissions, yet the container still fails to start it, reporting "no such file or directory," because the binary depends on a dynamically linked library that is not actually present, a problem most commonly encountered when mixing binaries built against glibc with minimal Alpine-based images that use musl instead.

The confusing error message

When a dynamically linked executable cannot find a required shared library, the error Docker reports is misleadingly identical in wording to a genuinely missing executable, even though the binary file itself is right there in the filesystem:

docker run --rm my-api:1.4.0 ls -la /app/server
-rwxr-xr-x 1 root root 8421376 Jun 1 12:00 /app/server
docker run --rm my-api:1.4.0 /app/server
exec /app/server: no such file or directory

The binary is confirmed present and executable, yet attempting to run it produces the same error message normally associated with a genuinely missing file, which is specifically because the kernel's exec mechanism reports this identical error when a binary's required dynamic linker or a required shared library cannot be found, not just when the binary file itself is absent.

glibc versus musl as the most common cause

Alpine Linux, the most widely used minimal base image, uses musl as its C standard library implementation, while most other distributions, and most prebuilt binaries distributed for general Linux use, are built against glibc; a binary compiled against glibc generally cannot run correctly on a musl-based system, since the two are not binary-compatible despite both implementing a similar C standard library interface:

FROM alpine:3
COPY --from=some-glibc-built-binary /app/server /app/server
exec /app/server: no such file or directory

This is one of the most frequent causes of this specific error pattern, and it commonly arises when a multi-stage build's earlier stage uses a glibc-based image (such as a standard node or golang image) but the final stage switches to Alpine for a smaller final image size, without accounting for the binary compatibility gap between the two C library implementations.

Diagnosing with ldd

The ldd utility, where available, lists a binary's dynamic library dependencies and reports whether each one can actually be resolved, which directly confirms whether a missing shared library, rather than a missing binary, is the actual cause:

docker run --rm --entrypoint sh some-glibc-image -c "ldd /app/server"
linux-vdso.so.1 (0x00007ffd...)
libc.so.6 => not found

A library reported as not found is the direct, conclusive confirmation of this specific problem; running this check against a temporary container based on a glibc-compatible image (rather than the actual Alpine-based final image, where ldd may behave differently or the binary may not run at all to even attempt the check) clarifies exactly which dependency is missing.

Resolving the glibc compatibility gap

Several approaches address this once diagnosed: switching the final stage to a glibc-based minimal image instead of Alpine, installing a glibc compatibility layer on top of Alpine, or, where the build process allows it, statically linking the binary so it has no dynamic library dependencies at all:

FROM debian:bookworm-slim
COPY --from=build /app/server /app/server
FROM alpine:3
RUN apk add --no-cache gcompat
go build -ldflags="-linkmode external -extldflags -static" -o server .

Static linking, where the language and build tooling support it well, is often the cleanest solution, since it produces a binary with no external shared library dependencies at all, making it portable across glibc and musl-based images interchangeably without any compatibility shim needed.

Scratch and distroless images amplify this risk

Building the final stage from scratch (an entirely empty base image) or a distroless image (minimal, with no shell or package manager) removes even more of the supporting filesystem a dynamically linked binary might assume is present, which means any dynamic dependency not explicitly copied into the final image will fail in exactly this way:

FROM scratch
COPY --from=build /app/server /server
ENTRYPOINT ["/server"]
exec /server: no such file or directory

For these minimal image types, statically linked binaries are generally the only reliable option, since there is no package manager available within the final image to install a missing shared library after the fact, the way apk add gcompat can address the issue on a still-functional Alpine base.

Verifying the fix

After applying a fix, whether switching base images, adding a compatibility layer, or statically linking, directly confirming the binary actually runs in the target image is the most reliable verification, rather than assuming the fix worked based on the build succeeding alone:

docker build -t my-api .
docker run --rm my-api /app/server --version

A successful invocation that produces expected output confirms the dynamic linking issue has actually been resolved, rather than merely assuming success because the build itself completed without error, since the build process for copying a binary between stages does not itself validate that the binary will actually execute correctly in its new, final environment.

Common mistakes

  • Treating a "no such file or directory" error as confirmation the binary itself is missing, without first verifying the file's actual presence and considering a dynamic library dependency issue instead.
  • Copying a binary built in a glibc-based build stage directly into an Alpine-based (musl) final stage without accounting for the binary compatibility gap between the two C library implementations.
  • Assuming scratch or distroless final images will work with any binary copied into them, without confirming the binary is statically linked or that every dynamic dependency has been explicitly included.
  • Not using ldd (against a compatible image) to directly confirm which specific shared library is actually missing before attempting a fix.
  • Assuming a successful build guarantees a runnable binary in the final image, without explicitly testing execution after the build completes.

Runtime executable missing errors that turn out not to be genuinely missing executables almost always trace back to a dynamic linking incompatibility, most commonly glibc versus musl across a multi-stage build's different base images, and resolving them reliably means either matching C library implementations consistently across stages, adding an explicit compatibility layer, or removing the dependency on dynamic linking altogether through static compilation.