19.2.5.5 Exec Live Debugging

Live debugging with docker exec means inspecting and diagnosing a container while it is actively running its production or development workload, without restarting it or altering its behavior. The exec mechanism injects a new process into the live container, giving direct access to the same filesystem, network, environment, and process tree that the application itself operates in. This makes it possible to investigate issues in their exact runtime context.

Opening a Debug Shell Without Disturbing the Application

docker exec -it my-app bash

Or with sh for minimal images:

docker exec -it my-app sh

The application's primary process keeps running. The debug shell runs alongside it as a separate process in the same container namespace.

Inspecting the Running Process

docker exec my-app ps aux

PID   USER     COMMAND
1     app      /app/server --port 8080
72    app      [worker-1]
73    app      [worker-2]

To see detailed information about the main process:

docker exec my-app cat /proc/1/cmdline | tr '\0' ' '
docker exec my-app cat /proc/1/status

Examining Memory Usage

docker exec my-app cat /proc/1/status | grep VmRSS

This shows the resident set size (actual RAM consumed) of the application's PID 1. Alternatively, docker stats on the host shows per-container memory usage.

For a running shell inside the container:

docker exec my-app free -h
docker exec my-app cat /proc/meminfo | grep -E "MemFree|Cached|MemAvailable"

Diagnosing Network Issues

All network diagnostics run inside the container's network namespace, so they reflect the container's view — not the host's:

# Check if the application is listening on the expected port
docker exec my-app ss -tlnp

# Test connectivity to a dependent service
docker exec my-app curl -sv http://database:5432

# Check DNS resolution
docker exec my-app nslookup database
docker exec my-app cat /etc/resolv.conf
docker exec my-app cat /etc/hosts

# Trace a network path
docker exec my-app traceroute database

Examining Environment Variables at Runtime

docker exec my-app env | sort
docker exec my-app printenv DATABASE_URL

This shows the exact environment the application receives, including variables injected via -e, --env-file, or Docker secrets.

Checking Configuration Files

docker exec my-app cat /app/config.yaml
docker exec my-app cat /etc/nginx/nginx.conf

If a container misbehaves after deployment with a new configuration, comparing the running container's config with the expected config identifies discrepancies.

Inspecting Open Files and Sockets

docker exec my-app lsof -p 1

This lists all open files, sockets, and pipes held by PID 1. It is particularly useful for diagnosing file descriptor leaks, unexpected open connections, or missing file references.

docker exec my-app lsof | grep "ESTABLISHED"

Shows all established network connections the application currently holds.

Log File Inspection

For applications that write logs to files rather than stdout:

docker exec my-app tail -f /app/logs/app.log
docker exec my-app grep -i "error" /app/logs/app.log | tail -20
docker exec my-app grep "exception" /app/logs/error.log | wc -l

CPU Profiling and Strace

For deep diagnostics, install tools temporarily inside the container:

# Alpine
docker exec -u root my-app apk add --no-cache strace

# Debian/Ubuntu
docker exec -u root my-app apt-get install -y strace

Then trace system calls of the application process:

docker exec -u root my-app strace -p 1 -e trace=network

Note: strace on PID 1 inside the container may require the container to have been started with --cap-add SYS_PTRACE.

Sending a Test Request from Inside the Container

docker exec my-app curl -s http://localhost:8080/api/status
docker exec my-app wget -qO- http://localhost:8080/health

This tests the application from inside its own network namespace, ruling out external networking or firewall issues.

Checking the Application's Open Connections to Dependencies

docker exec my-app ss -tnp | grep ESTABLISHED

State       Recv-Q  Send-Q  Local Address:Port  Peer Address:Port
ESTAB       0       0       172.17.0.2:54321    172.17.0.3:5432    users:("("server",pid=1,fd=7))

This shows the application has an active TCP connection to 172.17.0.3:5432 (the database container).

Writing and Running a Quick Test Script

docker exec my-app sh -c "cat > /tmp/test.sh << 'EOF'
#!/bin/sh
echo 'Testing database connection...'
pg_isready -h database -U app
echo 'Testing cache connection...'
redis-cli -h cache PING
EOF
chmod +x /tmp/test.sh && /tmp/test.sh"

Debugging Without a Shell (Minimal Images)

Distroless or scratch-based images contain no shell. Debugging options are limited:

• Add a debug or nonroot-debug variant of the image that includes a shell for development use.
• Use docker cp to copy diagnostic scripts into the container.
• Check logs via docker logs instead.
• Use nsenter on the host to enter the container's namespaces directly.

From the host (requires the container's PID):

CONTAINER_PID=$(docker inspect --format '{{.State.Pid}}' my-app)
sudo nsenter -t $CONTAINER_PID -n -- ss -tlnp

This enters only the network namespace of the container and runs ss without needing any tooling inside the container itself.