I85PREP
⌘ K
DCA study materials
DCA study resources

Docker Certified Associate: 1.6 GB images and three nodes that distrust the registry

DCA · Put your knowledge to work in context.

All case studies

1.6 GB images and three nodes that distrust the registry

CI builds a Python API from a repo whose Dockerfile does COPY . /app and then RUN pip install -r requirements.txt. There is no .dockerignore, so .git and 900 MB of test fixtures enter the build context. Images are 1.6 GB, every commit invalidates the dependency layer, and builds take about nine minutes. Images are pushed to registry.internal:5000, served with a self-signed CA. A five-node swarm runs docker service create --with-registry-auth --image registry.internal:5000/api:1.4 api. Three nodes fail with x509: certificate signed by unknown authority; two nodes pull fine and were hand-built by an engineer who has since left.

1. Rewrite the build so the image is small and dependencies stay cached across commits. Explain the mechanism behind the cache behaviour.

Read the worked answer

Three independent defects

  1. COPY . /app precedes pip install, so any file change (a README, a test fixture) invalidates the layer that installs dependencies.
  2. No .dockerignore, so the whole context - .git, 900 MB of fixtures - is uploaded to the daemon before any RUN executes. That upload is most of the nine minutes.
  3. Single stage, so compilers, headers and pip caches ship in the runtime image.

Dockerfile

# syntax=docker/dockerfile:1
FROM python:3.12-slim AS builder
WORKDIR /app
COPY requirements.txt .
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install --prefix=/install -r requirements.txt

FROM python:3.12-slim
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1
WORKDIR /app
COPY --from=builder /install /usr/local
COPY app/ ./app/
USER 10001
CMD ["python", "-m", "app"]

.dockerignore

.git
**/__pycache__
tests/fixtures
*.md
.env

Mechanism

  • A layer is cached on the hash of its instruction and the content of its inputs. COPY requirements.txt changes only when requirements change, so pip install is reused for every code commit; COPY app/ invalidates only the cheap final layer.
  • Multi-stage keeps build tooling out of the shipped layer; pip --prefix copies installed packages only.
  • Expect roughly 1.6 GB → a few hundred MB, and nine minutes → seconds on a cache hit. Verify with docker image ls and docker history --no-trunc.

Don't "fix" caching by deleting files in a later layer (&& rm -rf) - the bytes remain in the earlier layer.

2. Explain why only three nodes reject the registry certificate, and give a remediation that survives future node additions. What does `--with-registry-auth` actually do here?

Read the worked answer

Who is failing. x509: certificate signed by unknown authority is emitted by the daemon on the node performing the pull, not by the swarm manager. Each dockerd validates the registry chain against its own trust store plus any files under /etc/docker/certs.d/<host>:<port>/. There is no cluster-wide trust propagation - the two working nodes simply have ca.crt in place from the earlier engineer.

What --with-registry-auth does. It forwards the credential from your local ~/.docker/config.json to the nodes so they can authenticate the pull. It does nothing for TLS trust. That is the trap: the flag looks like it should fix this and cannot.

Remediation that survives new nodes

  1. Best: remove the need for per-node configuration. Put the registry behind a reverse proxy (nginx/Traefik) with a publicly trusted certificate from ACME/Let's Encrypt and use registry.example.com:443. Zero node bootstrap work, works for any future node or CI runner.
  2. If the private CA stays, distribute it by machine, not by hand:
install -D -m 0644 ca.crt /etc/docker/certs.d/registry.internal:5000/ca.crt
systemctl restart docker     # live-restore: true keeps containers running

with "live-restore": true in /etc/docker/daemon.json, driven from cloud-init / Ansible / the node image, and a join-time smoke test (docker pull registry.internal:5000/api:<digest>) so a node that cannot pull never joins the swarm.

Avoid --insecure-registry in production: it disables verification for that host cluster-wide and hides the next misconfiguration rather than fixing this one.

3. How do you guarantee that what runs in the cluster is exactly the artifact CI produced, and how do you prove it after the fact?

Read the worked answer

Tags are mutable; digests are not. api:1.4 can be re-pushed by anyone with write access; api@sha256:… cannot be changed.

  1. Capture the digest at push time (write it to the build record):
docker build -t registry.internal:5000/api:1.4 .
docker push registry.internal:5000/api:1.4
docker image inspect --format '{{index .RepoDigests 0}}' registry.internal:5000/api:1.4
  1. Deploy by digest, not tag:
docker service create --with-registry-auth \
  --image registry.internal:5000/api@sha256:<digest> api

--with-registry-auth is required on every service create/update and every stack deploy that pulls from a private registry.

  1. Prove what is running:
docker service inspect --format '{{.Spec.TaskTemplate.ContainerSpec.Image}}' api
docker service ps api --no-trunc
docker image inspect <image-id> --format '{{.RepoDigests}}'

If the spec shows a tag rather than a digest, someone bypassed the pipeline - that is a policy finding, not a mystery.

  1. Close the other doors: registry tag-immutability so 1.4 cannot be overwritten; DOCKER_CONTENT_TRUST=1 (Notary) so only signed tags are pulled on nodes; and a scan gate (trivy image / docker scout cves) in CI before the push, so a vulnerable digest never becomes deployable.

Judgement call: digest pinning is what makes rollback deterministic - docker service rollback returns to a known digest rather than to whatever 1.3 points at today.