Kubernetes Container Resource Management

Kubernetes Container Resource Management is the container-level granularity at which Kubernetes actually declares and enforces resource requests and limits — the fact that resources is specified per-container rather than per-Pod, and the consequences that follow from aggregating multiple independently-configured containers into a single scheduled and enforced unit. While requests and limits are the mechanisms, and QoS and eviction are the outcomes, container-level resource management is specifically about how those mechanisms behave when a Pod contains more than one container — main containers, sidecars, and init containers — each with its own resource profile that must be reasoned about both individually and in aggregate.

Because the Pod, not the container, is the unit the scheduler places and the kubelet manages holistically, container-level resource declarations only become meaningful once combined correctly across every container sharing that Pod.

Aggregating Multiple Main Containers

Summing for Scheduling

For ordinary (non-init) containers running concurrently within a Pod, the scheduler's fit calculation uses the sum of every container's individual request — a Pod with an application container requesting 500m CPU and a logging sidecar requesting 50m CPU has an effective 550m CPU request for scheduling purposes, and the node must have that much allocatable capacity available.

containers:
  - name: app
    resources:
      requests:
        cpu: "500m"
        memory: "512Mi"
  - name: sidecar
    resources:
      requests:
        cpu: "50m"
        memory: "64Mi"

Independent Enforcement per Container

At runtime, each container's limit is enforced independently through its own cgroup — one container hitting its memory limit and being OOM-killed does not, by itself, affect a sibling container's own resource allocation or cause it to be terminated, though the Pod as a whole is affected since a container restart (subject to restartPolicy) changes the Pod's overall running state.

Init Containers and Resource Calculation

Sequential Execution, Non-Additive Resources

Init containers run sequentially, one at a time, before any main container starts — because they never run concurrently with each other or with the main containers, their resource requests are not simply summed the way main container requests are.

The Effective Calculation

The Pod's effective resource request is the maximum of: the largest single init container's request, and the sum of all main container requests — reflecting that at any given moment, only one init container (at most) is running, while all main containers run together.

A Pod with one large init container requesting 2 CPU (for a heavy data-loading step) and main containers together requesting only 500m CPU has an effective request of 2 CPU, since the scheduler must guarantee enough capacity for the largest single moment of the Pod's resource needs, including its init phase.

Sidecar Containers and Native Sidecar Support

Ordinary Sidecars Count as Main Containers

A sidecar implemented as a regular container (not using native sidecar support) is summed into the Pod's aggregate request exactly like any other main container, running for the Pod's entire lifetime alongside the primary application container.

Native Sidecars (Restartable Init Containers)

Kubernetes' native sidecar mechanism — an init container marked with restartPolicy: Always — starts before the main containers but continues running alongside them for the Pod's lifetime, meaning its resource request is included in the "sum of concurrently running containers" side of the calculation above, alongside the main containers, rather than only counting toward the "largest single init container" side.

initContainers:
  - name: log-shipper
    restartPolicy: Always
    resources:
      requests:
        cpu: "50m"
        memory: "64Mi"

Per-Container Limits and Isolation

Why Per-Container Limits Matter for Multi-Tenant Containers Within a Pod

Setting an explicit limit on every container within a multi-container Pod — rather than relying on a single generously-sized main container limit to implicitly cover everything — ensures a misbehaving sidecar (a memory leak in a logging agent, for instance) cannot silently consume resources that were intended for the primary application container, since each container's cgroup enforcement is independent.

Consistent Resource Profiles Across Companion Containers

Sidecar containers reused across many different Pod templates (a standard logging or metrics sidecar deployed alongside many different applications) benefit from a consistent, centrally-maintained resource profile, since inconsistent sizing across different teams' adoption of the same sidecar image complicates capacity planning at the cluster level.

Example

apiVersion: v1
kind: Pod
metadata:
  name: codartium-container-resource-example
spec:
  initContainers:
    - name: data-loader
      image: codartium/data-loader:latest
      resources:
        requests:
          cpu: "1"
          memory: "1Gi"
  containers:
    - name: app
      image: codartium/app:latest
      resources:
        requests:
          cpu: "500m"
          memory: "512Mi"
        limits:
          cpu: "1"
          memory: "1Gi"
    - name: log-sidecar
      image: codartium/log-shipper:latest
      resources:
        requests:
          cpu: "50m"
          memory: "64Mi"
        limits:
          cpu: "100m"
          memory: "128Mi"