The Runtime Stack, and What Isolation Is Worth

Typing docker run invokes a stack of separate programs, each with a narrow job. Knowing the split matters because production systems increasingly use only part of it.

The client parses your command and calls an API. It does no container work at all.

The daemon implements that API: image building, network setup, volume management, and orchestration of the layers below.

A container supervisor, typically containerd, manages image pull and unpack, storage, and the lifecycle of running containers. It is what most orchestrators talk to directly, skipping the daemon entirely.

A low-level runtime, typically runc, does the actual work from the first lesson: create namespaces, configure cgroups, pivot the root, drop capabilities, exec. It runs once per container start and then exits.

That last point surprises people. runc is not a long-running process babysitting your container; it sets everything up, executes your program, and leaves. The container is your process, exactly as the first lesson said.

So the stack is a chain of increasingly specific tools, and the boundaries between them are standardised.

Those boundaries are specifications rather than implementation details, which is why the ecosystem has competing tools that interoperate.

The Open Container Initiative was established in June 2015 under the Linux Foundation by Docker and others, expressly to create open industry standards for container formats and runtimes. It maintains three specifications: runtime-spec, image-spec and distribution-spec, covering how a container is run, how an image is laid out, and how images are distributed.

runc was donated by Docker as the reference implementation of the runtime specification, and it became the foundation for containerd, Podman and others.

The practical effect is substitutability at every layer. An image built by one tool runs under another. A registry serving the distribution spec serves any client. A different runtime can be dropped in beneath the same supervisor, which is exactly how the sandboxed runtimes later in this lesson are deployed.

That standardisation is why "Docker" is now the wrong word for most of what people mean. Docker built the ecosystem and then donated its core, and much production infrastructure runs OCI images under containerd and runc without Docker involved at all.

The traditional architecture has a security property that is easy to miss and hard to overstate.

The daemon runs as root and exposes a socket. Anyone able to talk to that socket can ask it to start a container, and a container can be started with the host's root filesystem mounted and privileges retained.

So access to the daemon socket is equivalent to root on the host. Adding a user to the group that can reach it is granting them root, without that being obvious from the command that does it.

The same applies to mounting the socket into a container to let it manage other containers, a common pattern in build agents. That container is now able to take over the host.

Two responses exist.

Rootless mode runs the whole stack as an unprivileged user, using the user namespace so that root inside the container maps to your normal user outside. A compromise is then confined to your privileges rather than the host's.

Daemonless tools such as Podman remove the long-running privileged process entirely, running containers as child processes of the invoking user.

Both are meaningful hardening, and both are still constrained by the property the next step covers.

Five mechanisms, and they are genuinely layered defence: each covers something the others do not.

Capabilities split root's authority into distinct privileges, so a container can be allowed to bind a low port without being allowed to load kernel modules. Runtimes drop most by default.

seccomp filters system calls, refusing ones a normal application never needs. Default profiles block a substantial fraction of the syscall surface, and that is the most effective single control, because it directly shrinks what an attacker can reach.

Linux Security Modules apply mandatory access control policies that constrain what a process may touch regardless of file permissions.

The bottom box is the qualification on all of it. Every one of these is enforced by the same kernel the container is trying to escape. A kernel vulnerability reachable through an allowed syscall defeats the entire stack at once, because all five are features of the thing that has been compromised.

That is the structural difference from a hypervisor, and no amount of hardening within the model changes it.

Several common options disable the isolation entirely, and they appear in real deployments because they make a problem go away.

--privileged grants all capabilities, disables seccomp and the security module, and gives access to host devices. A privileged container is a root process on the host with a different filesystem view. It is not a weakened boundary; it is no boundary.

Mounting the daemon socket is equivalent, as the previous step covered.

--pid=host or --net=host removes that namespace, so the container sees host processes or the host's real network stack including its loopback interface.

Mounting / or /proc from the host gives filesystem access to everything.

--cap-add=SYS_ADMIN is close to privileged on its own, since that capability covers a very large set of operations including mounting.

These exist for legitimate reasons: a monitoring agent may genuinely need host visibility. The problem is that they are also the first search result when something does not work, and they get applied to make an error disappear without the trade being understood.

The review question is simple and worth asking every time: does this container still have a boundary, and if not, is that deliberate?

Escaping a container requires a kernel vulnerability. Getting code into one requires only that someone add a dependency, and in practice that is the more common path.

The surface is wide. A base image contains an operating system's worth of packages, most unused and all potentially vulnerable. Application dependencies pull transitive dependencies nobody reviewed. A tag can be repointed at different content, as the images lesson established. And a compromised registry account replaces an image everyone pulls.

The controls are unglamorous and effective.

Scan images for known vulnerabilities in packages and dependencies, in the build pipeline rather than after deployment.

Minimise the base. Distroless or scratch images have almost no packages, so almost nothing to be vulnerable, and no shell for an attacker to use.

Pin by digest rather than tag, so what you tested is what runs.

Sign images and verify signatures at deployment, which is what makes a compromised registry insufficient on its own.

Generate a bill of materials so that when a vulnerability is announced you can answer whether you are affected in minutes rather than days.

The last one is the highest-leverage. Most incident cost is not remediation; it is discovering which of your images contain the affected package.

For workloads where a kernel escape is unacceptable, the answer is to stop sharing a kernel, and several designs do this while keeping the container interface.

Sandboxed runtimes intercept the container's system calls in userspace and service most of them without reaching the host kernel, dramatically shrinking the exposed surface. The cost is a compatibility and performance penalty on syscall-heavy work.

Lightweight virtual machines run each container in a stripped-down VM with a minimal device model, restoring the hypervisor boundary while keeping start-up times close to a container's. This is the common answer for multi-tenant platforms running untrusted code.

Both plug in below the supervisor, which is exactly what the OCI runtime specification made possible: they implement the same interface as runc, so nothing above them changes.

The decision rule is about who wrote the code.

Running your own workloads, containers with sensible hardening are appropriate: everything in the container is code you already trust, and the isolation is defence in depth. Running untrusted or multi-tenant code, a shared kernel is the wrong boundary, and the extra cost of a VM boundary is the price of that being someone else's code.

Stating it that way avoids both errors: treating containers as a security boundary they are not, and dismissing them as insecure when the threat model does not require more.

Four lessons, and the arc runs from a claim to its qualification.

A container is not a thing; it is a set of kernel features applied to a process, all of which predate containers. What made the technology matter was packaging: an image format that is immutable, content-addressed and shareable, which is the same construction as Git's object store and yields the same rules about deletion and identity.

The operational consequences follow from disposability: data, configuration and logs must live outside something designed to be thrown away.

And the security answer is a genuine both-and rather than a hedge. Containers provide real isolation, enforced by a shared kernel with an enormous interface. That is defence in depth for code you trust and an insufficient boundary for code you do not, and knowing which situation you are in is the whole of the decision.