Containerization and Docker Fundamentals Questions
Packaging applications into containers: images and layers, Dockerfiles, registries, image optimization and security, container networking and storage, and the container runtime model. Covers how containers differ from virtual machines, image build and management, and the fundamentals that underpin any orchestration platform. The container primitive before orchestration.
Given this Dockerfile, the build cache keeps invalidating on almost every CI run even though only application code changes:
FROM python:3.9-slim
COPY . /app
WORKDIR /app
RUN pip install -r requirements.txt
COPY model /app/model
CMD ["python", "server.py"]
Explain exactly which Docker rules decide whether a RUN, COPY, or ADD layer is reused (including what changing an ARG or ENV does), then reorder or rewrite the Dockerfile so dependency installation is cached across builds.
Sample Answer
Direct answer
Docker builds a Dockerfile layer by layer and reuses a cached layer only while every earlier layer's cache key is unchanged and the current instruction's own inputs are unchanged. Because COPY . /app copies the entire build context, including the application source, before RUN pip install -r requirements.txt runs, any source-code edit changes that COPY layer's content hash, which invalidates every layer after it, including the pip install, even though requirements.txt itself never changed. The fix is pure reordering: copy only the dependency manifest first, install dependencies, then copy the rest of the source.
The exact cache rules
- COPY and ADD: the cache key is a checksum of every file matched by the source pattern, not a name or modification-time check, plus the destination path and file metadata (owner, permissions).
ADDfollows the same layer-caching rule asCOPYfor this purpose; its extra behavior (fetching a remote URL, auto-extracting an archive) does not change how its cache key is computed. Any byte difference in any matched file busts that layer's cache, and every layer positioned after it in the Dockerfile. - RUN: the cache key is the exact command text, compared literally, plus the filesystem state of the layer it runs on top of. Docker does not know what the command actually does internally, for example that pip only cares about the contents of
requirements.txt; it only reuses a layer if the command string and everything above it are byte-identical to a previous build. - ARG: changing a build ARG's value invalidates every instruction from the point that ARG is declared onward, but only if a later instruction actually references it. An ARG that no instruction uses has no effect on caching.
- ENV: setting or changing an ENV value invalidates the layer that sets it and everything after it, for the same reason as a RUN: it changes the image's declared state.
- General rule: the cache is checked top to bottom and stops at the first miss. Nothing below a miss can be a hit, even if that later instruction's own inputs never changed.
Why the given Dockerfile fails
COPY . /app sits before RUN pip install -r requirements.txt, so it is upstream of the install in the cache chain. Touching server.py, or any file in the build context not excluded by a .dockerignore, reruns that COPY's checksum, which reruns pip install on every single build regardless of whether requirements.txt changed at all.
The fix
FROM python:3.9-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["python", "server.py"]
This was built and verified directly: two Dockerfiles, one matching the original ordering and one matching this fix, were each built twice with only server.py changed in between the two builds. On the original ordering, RUN pip install -r requirements.txt re-executed on the second build with no CACHED marker at all. On the reordered version, the identical RUN pip install -r requirements.txt layer showed CACHED on the second build, because COPY requirements.txt . never changed. The COPY model /app/model line is fine left after the install either way, since large, rarely-changing binary assets do not need to sit ahead of the dependency install; what matters is that anything expected to change on nearly every commit (the application source) comes last.
Trade-offs and pitfalls
.dockerignore still matters even after this fix: if .git or a large node_modules directory sits in the build context unignored, COPY . . still has to checksum all of it on every build, which is slow even when it does not bust a downstream layer that was already cached. A BuildKit (Docker's modern build engine) cache mount, RUN --mount=type=cache,target=/root/.cache/pip pip install -r requirements.txt, goes one step further by persisting the pip download cache itself, so even a genuine requirements.txt change only re-downloads and re-resolves the difference instead of every package from zero. Worth knowing for larger, multi-stage Dockerfiles: an ARG used only inside a stage that a later stage never references does not invalidate that later stage at all.
In Docker, what's the difference between an image and a container? Walk through what happens from docker build to docker run, and explain how an image lets you avoid 'it works on my machine' problems.
Sample Answer
Direct answer
An image is a read-only, layered template: application code plus everything it needs to run, stored as a stack of filesystem diffs. A container is a running (or stopped) instance of that image, with one thin writable layer added on top for anything the process changes at runtime. docker build produces the image; docker run starts a container from it. Because the image itself never changes, every container started from it begins from the exact same starting point, which is what kills "it works on my machine."
From docker build to docker run
docker build reads a Dockerfile top to bottom. Each instruction that changes the filesystem (RUN, COPY, ADD) produces a new, immutable layer stacked on the previous one; Docker's build engine (BuildKit) content-addresses each layer so identical inputs can be cached and reused. The final result is an image: an ordered list of layers plus metadata (the default command, exposed ports, environment variables).
docker run takes that image and does three things: it creates a new writable layer on top of the image's read-only layers (this is the container's own filesystem), it sets up isolated namespaces (its own process tree, network stack, mounts) and cgroup limits for the new process, and then it executes the image's configured entrypoint/command as the container's first process. Nothing you do inside that running container (write a temp file, install a package on the fly) touches the underlying image; it only touches that container's writable layer.
Worked example
$ docker run -d --name a alpine:3.20 sleep 3600
$ docker run -d --name b alpine:3.20 sleep 3600
$ docker exec a sh -c "echo hi > /tmp/only-in-a.txt"
$ docker diff a
A /tmp/only-in-a.txt
$ docker diff b
(no output)
$ docker inspect --format='{{.Image}}' a
sha256:d243c3e9...
$ docker inspect --format='{{.Image}}' b
sha256:d243c3e9...
Both containers were started from the same image (their .Image field is identical), but writing a file in container a shows up only in a's docker diff, not b's. That is the image/container split made concrete: one shared, immutable image; two independent writable layers.
How this avoids "it works on my machine"
Because the image is the literal artifact that gets deployed (not a description of how to reproduce a machine, like a setup guide or a requirements.txt), a developer's laptop, a CI runner, and a production server are all running byte-identical layers. There is no "well, it's installed slightly differently on the server" gap, because nothing is reinstalled per environment; the same image is just started as a new container in each place.
Trade-offs and pitfalls
The most common mistake is treating the container's writable layer as durable storage: since it is discarded when the container is removed, anything that needs to survive a restart (databases, uploaded files, logs you care about) belongs in a named volume or bind mount, not in the container's own filesystem.
Why is containerization (Docker) important when deploying an application or ML model? Walk through a concrete example of a problem it solved (reproducibility, dependency conflicts, or scaling an environment) that a virtual environment like venv or conda could not solve on its own.
Sample Answer
Direct answer
Containerization matters because a virtual environment like venv or conda only isolates Python-level packages; it does nothing about the operating system underneath them. A Docker image packages the application, its language runtime, and every system-level library it depends on into one artifact, so "it worked when I trained it" and "it worked when I served it" become the same environment, byte for byte.
What a virtual environment isolates, and what it does not
venv isolates which Python interpreter and which site-packages a project sees. conda goes a bit further and can also pin some non-Python binaries. Neither one touches the operating system's C libraries, the exact glibc version, GPU drivers, or system packages installed via apt/yum. Most Python data and machine learning (ML) libraries are not pure Python: numpy, pandas, and scikit-learn link against native math libraries (BLAS/LAPACK implementations), and PyTorch or TensorFlow link against CUDA (NVIDIA's platform for running general-purpose code on a GPU) and cuDNN (NVIDIA's library of GPU-accelerated deep-learning primitives built on top of CUDA) for GPU support. venv records "install numpy 2.5.3" but has no way to guarantee the machine running it has a compatible native math library or GPU driver underneath. That gap is exactly where "works on my machine" failures live: two data scientists on the same requirements.txt can still get different numerical behavior, or an outright crash, if their underlying OS or native libraries differ.
Worked example
Here is a small Dockerfile for a scikit-learn/pandas training script that pins the entire runtime, not just the Python packages:
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY train_predict.py .
CMD ["python", "train_predict.py"]
with requirements.txt:
scikit-learn==1.5.2
pandas==2.2.3
and train_predict.py:
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
import pandas as pd
data = load_iris(as_frame=True)
X, y = data.data, data.target
model = LogisticRegression(max_iter=200)
model.fit(X, y)
sample = pd.DataFrame([X.iloc[0]])
prediction = model.predict(sample)[0]
print(f"predicted class for the first row: {prediction}")
print(f"actual class for the first row: {y.iloc[0]}")
Building and running it:
$ docker build -t iris-demo .
$ docker run --rm iris-demo
predicted class for the first row: 0
actual class for the first row: 0
That output is identical no matter whose laptop or which server runs it, because the image bakes in the exact base OS (python:3.12-slim), the exact Python build, and the exact pinned versions of scikit-learn and pandas, none of which a venv on the host machine controls. If the underlying server had, say, an older glibc that one of those wheels was not built against, venv would still "activate successfully" and then fail (or silently behave differently) at import or at inference time; the container sidesteps that because it carries its own base OS layer instead of depending on whatever happens to be installed on the host.
When Docker is overkill
For a single developer iterating locally on pure-Python code with no native dependencies, venv or conda alone is simpler and faster to iterate in; reach for a container when you need to guarantee the same behavior across machines you do not control (teammates' laptops, a training cluster, a production server), not as a default for every local script.
Your team has inconsistent Docker practices: different base image choices, ad hoc Python or Node versions, and frequent 'works on my machine' problems. As the person driving standardization, how would you define a team-wide container strategy, roll it out without blocking delivery, and measure whether it's actually improving reproducibility?
Sample Answer
Direct answer
Standardizing container practices across a team with inconsistent base images and ad hoc language versions works best as a small set of shared building blocks people opt into because they are genuinely easier, not a mandate enforced all at once. Define one or two approved base images per language, publish them somewhere central, migrate the highest-friction or most-broken services first as visible proof it works, and measure success by concrete, checkable signals (fewer "works on my machine" incidents, a shrinking count of services still on the old pattern), not by a one-time announcement that a policy now exists.
Structured elaboration
Defining the strategy
- Start from what is actually causing pain today, not from an abstract ideal: pull specific recent incidents (an image that broke because of an unpinned base image update, a new hire who lost half a day to a subtly wrong local toolchain version) and use them as the concrete justification for standardizing, rather than "best practice" as an argument on its own.
- Pick a small number of golden, versioned base images per language (one for Python, one for Node, and so on), each pinned to a specific, deliberately chosen version, published to an internal registry, and owned by a specific person or small group responsible for updating them on a known cadence.
- Write down the standard as a short, concrete checklist (which base image, how dependencies get pinned, what the multi-stage pattern should look like) rather than a long policy document; a checklist gets followed, a policy document gets skimmed once and forgotten.
Rolling it out without blocking delivery
- Never require every existing service to migrate before the standard is considered "live." Make the new base images available immediately, require them only for new services going forward, and migrate existing ones opportunistically, attached to work already happening on them (a routine dependency bump, a feature that touches the Dockerfile anyway), rather than as a dedicated, disruptive migration project competing with everything else the team is shipping.
- Pick one or two of the most painful existing services to migrate first, deliberately, as an early, visible example, ideally ones whose current state already causes recognizable pain, so the migration itself demonstrably fixes something people already wanted fixed, not just something that satisfies a new policy.
- Provide a clear, low-friction path for the exception case: a service with a genuine reason it cannot use the standard base image yet should have an explicit, time-bound way to say so, rather than silently ignoring the standard or being blocked outright; a standardization effort that has no accommodation for real exceptions tends to get quietly worked around instead of followed.
Measuring whether it is actually improving reproducibility
- Track a concrete, falling count: the number of services still on a non-standard base image or an unpinned dependency set, visible on a dashboard everyone can see, not just reported occasionally in a meeting.
- Track "works on my machine" incidents specifically (however the team already reports bugs or friction) before and after, since this is the exact symptom the standardization is meant to fix; a real reduction here is a much stronger signal than adoption count alone.
- Track new-hire or new-service onboarding time (how long it takes someone unfamiliar with a given service to get it running locally), since a genuinely standardized set of practices should make this measurably faster and more consistent across services, not just theoretically cleaner.
Worked example
A team of six services has four different Python base images in active use, two of which are no longer receiving security updates upstream, and a recurring pattern of new hires losing time to subtly different local dependency versions. The first move is not a team-wide announcement; it is publishing one pinned, actively maintained internal Python base image and migrating the single service that has caused the two most recent "works on my machine" incidents, as a visible fix to a problem people already recognized. Over the following quarter, three more services adopt the new base image as part of routine work already scheduled on them, and new services are required to start from it from day one. A dashboard tracking "services on the old pattern" drops from five to one over that period, and reported onboarding friction for a new service drops in the team's own retrospective notes, which together are the actual evidence the standardization worked, rather than the initial announcement alone.
Trade-offs & pitfalls
- Mandating an immediate, blanket migration of every existing service tends to either stall (competing with every other priority the team has) or get rushed and done poorly; opportunistic migration attached to work already happening is slower in wall-clock time but far more likely to actually finish.
- Measuring success only by "did we publish the standard" or "how many services adopted it" misses whether it is actually solving the underlying problem; track the specific pain (onboarding friction, "works on my machine" incidents) the standardization was meant to fix, not just adoption as an end in itself.
- No accommodation for genuine exceptions turns a well-intentioned standard into something people quietly route around, which produces worse long-term consistency than a standard with an explicit, visible exception process would have.
How do you set CPU and memory limits for a Docker container, both at runtime (--memory, --cpus, --memory-swap) and in Compose? Why do these limits matter for reliability when many containers share a host, and what's the practical difference between limiting resources in Docker versus relying on the host OS alone?
Sample Answer
Direct answer
At runtime you cap a container's resource use with --memory (a hard cap on RAM the container's cgroup, the Linux kernel mechanism that accounts and limits a group of processes' resource use, may use), --cpus (a fractional cap on CPU cores, for example 1.5 for one and a half cores' worth of CPU time), and --memory-swap (the combined cap on RAM plus swap; setting it equal to --memory disables additional swap for that container entirely). Compose expresses the same caps declaratively under a service's deploy.resources.limits (and, for guaranteed minimums, reservations). These limits matter because a host is shared: without them, one noisy or leaking container can starve every other container's CPU or trigger the kernel's out-of-memory killer against processes that had nothing to do with the problem. Setting limits in Docker, rather than trusting the host OS alone, means each container's ceiling is explicit, enforced per-container by the kernel's cgroup mechanism, and visible in the container's own configuration, instead of being an emergent, unpredictable property of whatever else happens to be running alongside it.
Structured elaboration
The three runtime flags
--memory=256m: a hard ceiling on the container's RAM. Exceeding it does not throttle the container gracefully; the kernel's out-of-memory killer terminates a process inside it, which is why containers hitting this limit exit with code 137 (128 plus signal 9,SIGKILL).--cpus=1.5: a fractional CPU cap enforced by the kernel's CPU scheduler quota mechanism (internally, a quota and period pair, for example 150 milliseconds of CPU time allowed per 100-millisecond period, which works out to one and a half cores' worth of time). The container can still use more than one core briefly if the host has spare capacity within that period, but its average use over time is capped.--memory-swap: the ceiling on RAM plus swap combined. Set it equal to--memoryto give the container no extra swap at all (predictable, fails fast under memory pressure instead of degrading into slow, swapped-out behavior); leave it unset or higher to allow some swap as a cushion, at the cost of unpredictable latency when the container is actually swapping.
The same caps in Compose
services:
app:
image: myapp:latest
deploy:
resources:
limits:
cpus: "1.5"
memory: 256M
reservations:
memory: 128M
limits is the hard ceiling, equivalent to the runtime flags above. reservations is a soft guarantee used for scheduling decisions (more relevant in a multi-node Swarm context, but harmless to declare locally). As of current Compose versions, plain docker compose up applies deploy.resources.limits directly to the container, the same as if you had passed --memory and --cpus on the command line; this is worth knowing explicitly because older documentation associated deploy.resources with Swarm mode only.
Why this matters for reliability with many containers on one host
- CPU is a compressible resource: a container that goes over its fair share slows its neighbors down (added latency) but does not crash them. Memory is not compressible: a container that overshoots gets killed, and without a per-container ceiling, the kernel's OOM killer picks a victim using its own heuristics across the whole host, which can land on an unrelated, otherwise healthy container instead of the actual offender.
- A hard memory limit turns "some process on this host eventually gets killed, chosen by a heuristic" into "this specific container gets killed, predictably, when it exceeds the budget you gave it," which is a much easier failure mode to detect, alert on, and attribute correctly.
Docker limits versus relying on the host OS alone
Without container-level limits, isolation depends entirely on however many containers happen to be co-located and how they individually behave that day; a single runaway process can degrade or take down everything sharing the host, and there is no per-workload accounting to tell you which container caused it. Explicit per-container limits make capacity planning a per-service decision (this service gets this much CPU and RAM, always) rather than an emergent property of the day's neighbor mix, and they make a container's own docker inspect output an honest record of what it is entitled to.
Worked example
A host runs four API containers with no memory limits set. A memory leak in one of them slowly consumes RAM until the host is under pressure; the kernel's OOM killer fires and terminates a process in a different, healthy container, because that container's process happened to score worse on the kernel's heuristics at that moment. After adding --memory=512m --memory-swap=512m to every container, the leaking one instead gets its own process killed once it crosses 512 MB, its docker inspect shows OOMKilled: true against that specific container, and the other three keep running untouched. The failure is now attributable and comes with a hard, checkable number instead of a shared, host-wide guess.
Trade-offs & pitfalls
- Setting
--memorytoo tight causes healthy workloads to be killed under normal, non-leaky peak load (a burst in traffic, a large batch job); size it from measured peak usage with headroom, not from idle-state observation. - Relying only on
--cpuswithout ever measuring actual usage under load can silently throttle a service during real traffic spikes, showing up as added latency that is easy to misdiagnose as a code regression. --memory-swapequal to--memory(no swap) is usually the right default for latency-sensitive services, since swapping trades a hard, visible failure for a slow, hard-to-diagnose one; leaving swap available can be appropriate for batch or background work where an occasional slowdown is preferable to being killed outright.
Unlock Full Question Bank
Get access to all Containerization and Docker Fundamentals interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.