Containerization and Docker Fundamentals Questions

Direct answer

For a service running in a container, pip with a lockfile-generating tool (Poetry, or in 2026 increasingly uv) is the better default: it produces small, fast-to-build images and plays well with Docker's layer caching. Reach for conda only when you specifically need conda-forge's precompiled, ABI-matched native stack (BLAS/CUDA combinations that don't ship as pip wheels) and are willing to pay for a larger image and a slower dependency solve. Whatever you pick, compiled dependencies (CUDA wheels, custom C/C++ or Rust extensions) get built once in a dedicated build stage and shipped as an artifact, never compiled fresh inside the runtime image or at container start.

Structured elaboration

Terms first: a wheel is Python's prebuilt binary package format (.whl); manylinux is the tagging convention that marks a wheel as portable across Linux distributions with a compatible glibc, which is what lets pip install torch fetch a working binary instead of compiling from source; the ABI (application binary interface) is the low-level contract, function layouts, calling conventions, that a compiled extension and the interpreter or CUDA runtime it links against must agree on (mismatch it and you get an import-time crash, not a helpful error message); CUDA is NVIDIA's GPU programming toolkit and runtime, and BLAS is a standard, low-level library interface for linear-algebra operations like matrix multiplication; a lockfile pins every resolved dependency, direct and transitive, to an exact version and ideally a hash, so the same input always resolves to the same set of packages.

By 2026 a fourth option is common enough to name even though the question didn't ask for it: uv, a Rust-based, pip-compatible installer and resolver that produces a single cross-platform lockfile (uv.lock) and resolves noticeably faster than Poetry on large dependency graphs. It doesn't change the underlying trade-off above (it's still "pip-family," so the conda-vs-pip split still applies), it's simply a faster, more current implementation of the pip and Poetry side of that split, and it's a reasonable default for new projects.

Handling compiled dependencies (CUDA wheels, custom native extensions)

• Match the wheel's CUDA build to your runtime, not the other way around. A wheel like torch==2.x+cu124 is compiled against CUDA 12.4's ABI; installing it on an image whose CUDA driver or toolkit is older will fail at import or, worse, silently fall back to running on CPU. Pin the CUDA version in your base image tag (for example nvidia/cuda:12.4.1-runtime-ubuntu22.04) and the wheel's CUDA suffix together, and treat that pair as one unit to upgrade, not two independent version bumps.
• For a custom native extension (a C, C++, or Rust module wrapped for Python), use a multi-stage Dockerfile: a builder stage on the CUDA devel image (which has the compiler toolchain and headers) builds the extension into a wheel; the runtime stage, based on the much smaller CUDA runtime image (no compiler, no headers), only ever installs that already-built wheel. This keeps compilers and headers out of the production image, shrinking it and reducing its attack surface, and it turns the compiled artifact into something you can cache and reuse across builds instead of recompiling every time.

flowchart LR
    A[Builder stage: CUDA devel image plus compilers] --> B[Compile native extension into a wheel]
    B --> C[Runtime stage: CUDA runtime image, no compilers]
    D[Locked requirements file: poetry.lock or uv.lock] --> C
    C --> E[Slim production image]

Keeping resolution reproducible and upgrades safe in CI

• Never let a CI build re-resolve dependencies from loose version ranges. Resolve once (locally, or in a dedicated update job), commit the lockfile, and have every ordinary CI run simply install from that lockfile with hash verification. That turns "does this build reproducibly" into a solved problem: the lockfile is the single source of truth for what "this build" means.
• Pin the base image by digest, not just a mutable tag, so a tag repoint upstream can't silently change your build.
• Separate the update path from the build path: a scheduled job (Renovate, Dependabot, or a manual poetry update or uv lock --upgrade) opens a pull request that bumps the lockfile and nothing else, that pull request runs the full test matrix, including any CUDA-dependent tests on a GPU runner, and a human or a required check approves the merge. Ordinary feature-branch CI never touches dependency versions at all.

Worked example

A team ships a FastAPI inference service with a custom CUDA kernel wrapped via pybind11. The Dockerfile has two stages: FROM nvidia/cuda:12.4.1-devel-ubuntu22.04 AS builder installs the CUDA toolkit headers and a compiler, then runs pip wheel . -w /wheels to produce custom_op-1.0.0-cp311-cp311-linux_x86_64.whl. The second stage, FROM nvidia/cuda:12.4.1-runtime-ubuntu22.04, copies only /wheels/*.whl and the project's uv.lock, then runs uv pip install --no-cache /wheels/*.whl -r requirements.lock.txt. The devel image's compiler toolchain and headers never make it into the shipped image; the runtime stage only carries the CUDA runtime libraries the compiled kernel links against at inference time, plus the pure-Python dependencies pinned in the lockfile. A nightly job runs uv lock --upgrade on a separate branch, and only that branch's CI is allowed to change what the lockfile pins.

Trade-offs & pitfalls

• Mixing conda install and pip install in the same environment is a common wrong turn: conda's solver doesn't know about packages pip installed afterward, so a later conda update can silently break binary linkage that pip had put in place (a NumPy build expecting conda's BLAS, next to a pip-installed package expecting a different one).
• A requirements.txt with unpinned or loosely pinned transitive dependencies isn't reproducible even if the top-level packages are pinned; a transitive C-extension dependency can float to a new version with a different ABI and only fail on the CUDA-enabled CI runner, not a developer's laptop.
• Recompiling a native extension inside the final production build step, instead of shipping a pre-built wheel from a builder stage, means what CI tested and what got deployed are two different compiled artifacts even if the source was identical, because compiler flags, library versions, or the timing of an upstream release can differ between the two builds.
• Both conda-lock and hash-pinned pip files solve reproducibility, but neither protects against a conda channel or PyPI index quietly yanking a version; vendoring or mirroring critical wheels internally is the fix once that has caused an outage.

Direct answer

An efficient multi-stage Node.js Dockerfile separates a build stage (install all dependencies with npm ci, build static assets) from a minimal runtime stage (install only production dependencies, copy in the built output), ordered so dependency installation is cached across builds when only source changes. Below is a Dockerfile that was built, run, and specifically re-built after a source-only change to confirm the caching claim, not just written to look right.

The Dockerfile

# Stage 1: install deps and build static assets
FROM node:20-slim AS build
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
RUN npm run build

# Stage 2: minimal runtime image
FROM node:20-slim AS runtime
RUN groupadd --system app && useradd --system --gid app app
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --omit=dev
COPY --from=build /app/dist ./dist
COPY server.js ./
RUN chown -R app:app /app
USER app
EXPOSE 3000
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
    CMD node -e "require('http').get('http://127.0.0.1:3000/', r => process.exit(r.statusCode === 200 ? 0 : 1)).on('error', () => process.exit(1))"
CMD ["node", "server.js"]

npm ci (not npm install) is used in both stages because it installs exactly what package-lock.json specifies and fails if the lockfile and package.json disagree, which is what makes the install reproducible across machines and CI runs. Both stages copy package.json/package-lock.json and run their own npm ci before copying the rest of the source, for the same cache-ordering reason as any other language: the dependency list changes far less often than the source. The runtime stage also creates a dedicated app system user and group and switches to it with USER app before the process starts, so the running container is non-root, and it declares a HEALTHCHECK that opens a real connection to the service rather than only checking the process is alive.

Verified build, cache behavior, and run

$ docker build -t node-app .
$ docker run -d --name napp -p 13000:3000 node-app
$ docker exec napp whoami   # confirms the container is running as the non-root app user, not root
app
$ curl http://127.0.0.1:13000/
<h1>built asset</h1>
$ docker images node-app --format '{{.Size}}'
310MB
$ docker inspect --format='{{.State.Health.Status}}' napp
healthy

Then, changing only server.js (not package.json or package-lock.json) and rebuilding:

=> CACHED [build 4/6] RUN npm ci
=> CACHED [runtime 5/8] RUN npm ci --omit=dev
=> [runtime 7/8] COPY server.js ./

Both npm ci steps, in both stages, stayed CACHED; only the layer copying the changed server.js file re-ran. That confirms the ordering requirement is actually met, not just plausible-looking in the Dockerfile.

Edge cases and trade-offs

Running npm ci twice, once per stage, means the runtime stage re-downloads production dependencies rather than reusing the build stage's node_modules directly; this is deliberate (it guarantees the runtime stage never accidentally ships a dev-only dependency that leaked into the build stage's tree) but it does mean two separate installs to keep in sync, and both must read the same lockfile or they can drift. If npm run build needs devDependencies to run (bundlers, TypeScript) but the build fails silently because a dependency was pruned too early, that is a sign the --omit=dev install happened in the wrong stage; it must stay in the runtime stage only. Finally, a package-lock.json that is missing or out of sync with package.json makes npm ci fail outright (by design), which is the correct, loud failure mode compared to npm install's tendency to silently resolve a different dependency tree than what was tested.

Direct answer

An out-of-memory kill during native compilation is almost always the build's own parallelism outrunning the runner's available memory, not a fundamentally broken build; the cheapest fix is usually capping how many compiler processes run at once, and the more structural fixes (bigger runners, split stages, prebuilt artifacts) trade cost, build time, or supply-chain trust for reliability.

Why this happens, and the mitigation options

• Reduce build parallelism first. Tools like make -jN or node-gyp commonly default to spawning one job per available CPU (nproc). Each job compiling a large C or C++ source file can use several hundred megabytes of resident memory on its own; on a runner with many virtual CPUs but limited memory (a common CI shape, since CPU and memory are not always scaled together), that default parallelism can request far more memory than the container's cgroup (Linux control group, the kernel mechanism enforcing the memory limit) allows. Explicitly capping parallelism (make -j2, MAKEFLAGS=-j2, or a build-tool-specific concurrency flag) is free, requires no infrastructure change, and directly targets the actual mechanism causing the kill.
• Increase runner memory or add swap. This works, but it treats the symptom: it costs more (bigger runners are billed higher, or compete for a scarcer pool of self-hosted capacity) and swap in particular trades an out-of-memory kill for severe slowdown once the compiler starts paging, since compilation is not swap-friendly (it repeatedly touches large working sets). It also tends to mask the problem until the dependency tree grows again and the runner hits the new, larger ceiling.
• Split the build into more stages. Structuring the Dockerfile so that memory-heavy compilation steps run in their own stage, sequentially rather than overlapping with other build work, reduces peak memory at any one instant, at the cost of a more complex Dockerfile and a build that requires understanding which specific step is actually memory-heavy rather than just recompiling everything the same way.
• Pull a prebuilt artifact instead of compiling. Using a distribution's prebuilt package, a language ecosystem's precompiled wheel (Python's manylinux wheels, for example) or a vendored prebuilt native addon (Node's prebuildify/N-API pattern) avoids the compilation step, and therefore the memory spike, entirely. This is usually the fastest and most memory-reliable option, but it shifts trust to a third party's binary and requires the prebuilt artifact to exactly match your target's C library and CPU architecture (glibc and musl are two different, binary-incompatible implementations of the C standard library that compiled programs link against; glibc-based distributions and musl-based ones like Alpine are not interchangeable here, and an architecture mismatch surfaces as a different failure, an illegal-instruction or missing-symbol crash at runtime instead of a build-time OOM).

Worked example

Consider an 8-virtual-CPU CI runner with a 4 GiB memory limit, building a native C extension whose compiler process (cc1 or equivalent) uses roughly 400 MiB of resident memory per translation unit while compiling. A build tool that defaults to -j$(nproc) will attempt to launch up to 8 concurrent compiler processes: 8 x 400 MiB = 3,200 MiB, about 3.13 GiB (3,200 / 1,024), just in compiler processes, before accounting for the shell, the build orchestrator itself, and any already-linked objects held in memory. That leaves under 1 GiB of headroom on a 4 GiB limit for everything else running in the container, and a slightly larger source file or one additional concurrent job pushes the cgroup over its limit. The kernel's out-of-memory (OOM) killer then terminates the offending process, which surfaces in CI logs as the build step exiting with code 137 (128 + 9, meaning the process was killed by SIGKILL). Capping parallelism to -j2 bounds the same math to roughly 800 MiB for compiler processes, comfortably inside the 4 GiB limit, without touching the source code or the runner's provisioned memory at all.

Trade-offs and pitfalls

Capping parallelism trades build duration for reliability: fewer concurrent compiler jobs means the compilation step takes longer in wall-clock terms (the exact amount depends on the host and is not something to promise a number for), but it costs nothing and directly targets the mechanism causing the failure, which is why it is usually the first thing to try, not the last. Throwing a bigger runner at the problem is the opposite: it is fast to apply but recurring cost that grows again as soon as dependencies do, and it teaches the team nothing about why the build is memory-hungry in the first place. Prebuilt artifacts are the fastest and most memory-reliable long-term fix but introduce a real pitfall if adopted carelessly: pulling a wheel or addon built against the wrong C library or CPU architecture converts a visible, fail-fast build-time OOM into a much harder-to-diagnose runtime crash on a machine that looks like it should work.

Direct answer

An image registry stores, versions, and serves container images the way a package repository serves libraries: docker push uploads layers and a manifest to it, docker pull (or a deployment step) downloads them. Docker Hub is the default public registry; AWS ECR (Elastic Container Registry), Azure Container Registry (ACR), and Google Artifact Registry are the major cloud-managed alternatives. An organization runs a private registry, rather than relying only on a public one, mainly for access control, reliability guarantees it does not get from a shared public service, and to keep proprietary images out of a public namespace entirely.

Naming note: Google Container Registry is retired

The question's third option, GCR (Google Container Registry, gcr.io), is retired for new use: Google shut down write access on March 18, 2025, so docker push to a Container Registry-hosted repository no longer works. Read access was not cut off on a fixed date the way write access was; Google migrated existing gcr.io image references to be served through Artifact Registry under that same hostname, so images already published there keep pulling successfully rather than disappearing. Google's current offering for new work is Artifact Registry, which also supports legacy gcr.io-style hostnames for images that were already there. A candidate who has used the older name should recognize it, but new work should target Artifact Registry directly, and the operational risk to plan for is push/CI credentials configured against Container Registry breaking, not already-published images becoming unreachable.

Why run a private registry

A public registry like Docker Hub is a fine default for open-source or non-sensitive images, but production teams usually want: access control (who can push a new version of a production image, and who can pull it, tied into the same identity system as everything else); rate-limit and reliability guarantees a shared public service does not promise for free-tier pulls at production scale; and simply not publishing proprietary application code as an image anyone on the internet can pull. Cloud-managed private registries (ECR, ACR, Artifact Registry) add tight integration with each cloud's identity and access management (IAM) system, so a compute service can be granted pull access without embedding long-lived credentials.

Tagging conventions

Two conventions dominate, and most mature pipelines use both for different purposes: semantic version tags (v1.4.2) are meant for humans and for deliberate promotion between environments (staging tests v1.4.2, then production is told to run v1.4.2), while commit-SHA tags (myapp:a1b2c3d) are generated automatically on every build and give perfect traceability back to the exact source commit, which a semantic version alone does not (two builds of "the same" v1.4.2 from a hotfixed branch would otherwise be indistinguishable). Many pipelines push both tags to the same underlying image so a human-readable release name and an exact build provenance trail both exist.

Authentication and access control in production

Pushing and pulling from a private registry requires authenticating first (docker login, or a cloud CLI's registry-login helper that exchanges cloud credentials for a short-lived registry token). In production, the strong pattern is to avoid long-lived, shared credentials entirely: a compute service (a CI runner, a Kubernetes node, a serverless container platform) is granted an IAM role or workload identity scoped to pull (and, for CI, push) specific repositories, and the registry issues short-lived tokens against that identity rather than a static username and password baked into a config file.

Direct answer

CMD sets the default command a container runs, and it can be fully overridden by whatever is passed after the image name on docker run. ENTRYPOINT sets the fixed executable that always runs; anything from CMD, or arguments passed to docker run, is appended to it as arguments instead of replacing it. Shell form (CMD myapp --flag) runs the command through /bin/sh -c; exec form (CMD ["myapp", "--flag"]) runs the binary directly, without a shell in between, which matters specifically because of how Linux delivers signals like SIGTERM.

When to prefer each

Prefer ENTRYPOINT when the image should behave like a fixed, single-purpose binary and you want docker run arguments to feel like command-line flags to that binary; the classic example is a CLI tool image like docker run mytool --version, where ENTRYPOINT ["mytool"] plus no CMD means any arguments the user supplies are passed straight through to mytool. Prefer plain CMD when the image is meant to run one sensible default command that a user might reasonably want to replace entirely, such as a base development image where CMD ["python", "app.py"] is the default but someone might run docker run myimage bash to get a shell instead; with only CMD set, that fully replaces the default command as intended.

Shell form, exec form, and why exec form matters for signals

docker stop sends SIGTERM to the container's PID 1 process, waits a grace period (10 seconds by default), and then sends SIGKILL if the process has not exited. In shell form, the actual application is often started as a child of /bin/sh -c "..."; whether the shell then forwards SIGTERM to that child, or the shell instead sits as PID 1 and the application never sees the signal at all, is implementation detail you should not rely on, and it varies by exactly what the command string does (a single simple final command is sometimes optimized away by the shell into a direct exec, but anything with multiple statements, pipes, or && may leave the shell resident as PID 1). Exec form removes that ambiguity entirely by making your application the literal PID 1 process, guaranteed, with no shell in between.

That said, there is a second, separate wrinkle worth knowing: being PID 1 inside a container's process namespace changes how the Linux kernel applies default signal dispositions. A process that has not explicitly installed a SIGTERM handler normally terminates on SIGTERM by default, but the kernel suppresses that default action specifically for PID 1, so an application with no explicit signal handler can ignore SIGTERM even in exec form. This was verified directly: a plain sleep 100 running as exec-form PID 1 with no init process ignored SIGTERM completely and rode out the full grace period before being SIGKILLed (docker stop -t 3 took the full 3.2 seconds), while the same binary run under docker run --init (where a tiny init process, not the application, is PID 1, and sleep runs as a normal child at PID 2) received SIGTERM's default action immediately and exited in about 0.1 seconds.

What this means in practice

Exec form is necessary but not sufficient for graceful shutdown: it guarantees your application (not a shell) receives the signal Docker sends, but if your application does not install its own SIGTERM handler, running it as PID 1 without an init process can still mean it never actually reacts to the signal and just waits out the timeout before being force-killed. The two-part fix is exec form plus either an explicit signal handler in the application, or docker run --init (or an init process like tini baked into the image) so the application is not PID 1 at all and gets ordinary signal-handling semantics.