Docker Fundamentals¶

What a container actually is¶

A container is not a lightweight virtual machine — there's no hypervisor, no guest kernel. A container is a set of regular Linux processes, running on the same kernel as the host, that the kernel has been told to isolate using three separate mechanisms:

• Namespaces — give the process its own isolated view of a global resource. A PID namespace makes the container's first process see itself as PID 1, unaware that on the host it's actually PID 48213. A network namespace gives the container its own network interfaces, routing table, and port space — a process inside can bind to port 80 without conflicting with something already on port 80 on the host, because it's binding to a different port 80 that only this namespace sees. There's also a mount namespace (its own view of the filesystem), a UTS namespace (its own hostname), and others.
• Control groups (cgroups) — limit and account for how much of a resource a process can use: CPU shares, memory ceiling, block I/O bandwidth. This is what makes docker run --memory=512m actually mean something — the kernel will OOM-kill the container's process if it tries to exceed 512MB, the same way it would kill any process that exceeds a cgroup memory limit.
• A union filesystem (overlayfs, in modern Docker) — layers multiple read-only filesystem layers underneath one writable layer, giving the container what looks like a normal single filesystem while actually being assembled from several stacked, reusable pieces.

This is the whole reason containers start in milliseconds while VMs take seconds to minutes: there's no kernel to boot, no hardware to emulate — just a process that the kernel is told to isolate before it starts running. It's also why a container can never run a different kernel than its host (a Linux container needs a Linux host kernel) — the isolation is namespace/cgroup-based, not a full virtualized machine.

Images vs. containers¶

An image is a read-only template: a stack of filesystem layers plus metadata (the default command to run, exposed ports, environment variables). A container is a running instance of an image — the image's layers, mounted read-only, with one additional writable layer on top for anything the running process changes.

Every layer above L1 only stores the diff from the layer below it — if a Dockerfile's RUN apt-get install python3 only adds files, that layer is small regardless of how large the full resulting filesystem looks. This layering is also what lets 10 different containers, all built from the same base image, share layers L1–L3 on disk instead of each having their own copy — only the writable layer and anything unique to each container's specific build steps takes additional space.

Deleting a container's writable layer on docker rm throws away everything the process wrote at runtime — this is why a database running in a container with no volume attached loses all its data the moment the container is removed. Persisting data across container lifecycles needs an explicit volume, which lives outside the container's layer stack entirely.

Why layer caching matters, concretely¶

Docker builds an image by executing a Dockerfile's instructions top to bottom, and caches the result of each instruction — if a later build has the exact same instruction, with the exact same preceding layers, it reuses the cached layer instead of re-running it. The cache is invalidated at the first instruction that changes, and everything after it must rebuild too, even if those specific instructions didn't change.

Concretely, given:

FROM python:3.12-slim
COPY . /app
RUN pip install -r /app/requirements.txt
CMD ["python", "/app/main.py"]

Every single source-code change (editing main.py) invalidates the COPY . /app layer — and every layer after it, including the pip install, because Docker has no way to know the requirements didn't change without re-running the diff check, and the cache invalidation rule is "if this instruction's layer changed, everything downstream reruns." On a project with many dependencies, this means every code edit triggers a full dependency reinstall, often taking far longer than the actual code change did.

Reordering fixes this directly:

FROM python:3.12-slim
COPY requirements.txt /app/requirements.txt
RUN pip install -r /app/requirements.txt
COPY . /app
CMD ["python", "/app/main.py"]

Now requirements.txt is copied and installed before the rest of the source code. Editing main.py only invalidates the final COPY . /app layer — the pip install layer, unchanged, is reused straight from cache. This single reordering is the highest-leverage Docker performance habit: put what changes least at the top of the Dockerfile, what changes most at the bottom.

A minimal end-to-end example¶

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["python", "main.py"]

docker build -t my-app:1.0 .
docker run --name my-app-container -p 8000:8000 my-app:1.0

docker build reads the Dockerfile in the current directory (. is the build context — everything under it is sent to the Docker daemon and available to COPY), executes each instruction as a new layer, and tags the result my-app:1.0. docker run creates a container from that image, with -p 8000:8000 mapping the host's port 8000 to the container's port 8000 (host:container) — without this flag, the container's network namespace is completely isolated and nothing outside it could reach port 8000 at all.

Common pitfall¶

Treating the build context carelessly — running docker build . from a directory that contains a .git folder, node_modules, or large log files — sends all of that to the Docker daemon on every build, slowing builds down and bloating layers that accidentally COPY more than intended. A .dockerignore file (same syntax as .gitignore) excludes these before the context is even sent — see Dockerfile Best Practices for the full checklist, including this one.