Skip to content

All lessons

Area 07 // DeploymentLesson 109 / 14015 min read

Docker

Docker packages your app together with its exact Python version, libraries and files into an image, a portable bundle that runs the same way as a container on your laptop, in CI and on any server, which removes "works on my machine" problems from deployment.

Think of it like this

Analogy

Shipping containers changed world trade because everything goes in the same standard box: a crane in Shanghai and a lorry in Rotterdam don't care whether it holds bananas or laptops. Docker does that for software. Your app, its Python version and every library go into one standard box (an image), and any machine with Docker can run it, without anyone reinstalling the right versions by hand.

The problem it solves

Your RAG app works on your laptop with Python 3.12 and exact library versions. The server has Python 3.10, a teammate has a different NumPy, and CI has neither. Installing things by hand on each machine is slow and error-prone, and "works on my machine" bugs appear at the worst moment: during a deploy. You need one artifact that carries its whole environment and runs the same everywhere (environments).

How it works, step by step

  1. Write a Dockerfile: a recipe of instructions (FROM, COPY, RUN, ENV, CMD).
  2. Start from a base image: python:3.12-slim gives a small Linux with Python preinstalled.
  3. Install dependencies before copying code: each instruction creates a cached layer; if requirements.txt hasn't changed, the slow pip install layer is reused and only the code layer rebuilds.
  4. Copy only what's needed: a .dockerignore keeps out .venv, tests, .env and caches.
  5. Harden it: run as a non-root user, add a HEALTHCHECK, set PYTHONUNBUFFERED=1 so logs appear immediately, never put secrets in the image (secrets).
  6. Build: docker build -t rag-app:v1 . produces an image.
  7. Run: docker run -p 8000:8000 -e LLM_PROVIDER=mock rag-app:v1 starts a container; config and secrets come from environment variables at run time.
  8. Tag and push to a container registry (GitHub Container Registry, Docker Hub, a cloud registry) with the git SHA as the tag, so deploys reference an exact version (CI/CD).
  9. Deploy the image: managed platforms (serverless and managed hosting), Docker Compose (Compose) or Kubernetes (Kubernetes) all run the same image.

See it in code

Install
Docker Desktop (Windows, macOS) or Docker Engine (Linux)
Needs
Docker installed. The file is examples/rag-app/Dockerfile in the course repository.

The capstone app's Dockerfile, then the real build log from the course's GitHub Actions run.

Dockerfile
FROM python:3.12-slim
# No .pyc files, logs straight to stdout.
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 PIP_NO_CACHE_DIR=1
WORKDIR /app
 
# Dependencies first: this layer is cached until requirements.txt changes.
COPY requirements.txt .
RUN pip install -r requirements.txt
 
# Then the code and data (they change often, so they come last).
COPY rag_app/ rag_app/
COPY data/ data/
 
# Don't run as root.
RUN useradd --create-home appuser
USER appuser
 
ARG APP_VERSION=dev
ENV APP_VERSION=${APP_VERSION} LLM_PROVIDER=mock
EXPOSE 8000
HEALTHCHECK --interval=10s --timeout=3s --start-period=10s --retries=3 \
  CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')" || exit 1
CMD ["uvicorn", "rag_app.app:app", "--host", "0.0.0.0", "--port", "8000"]
Output
$ docker build --build-arg APP_VERSION=31fab5b -t ghcr.io/<owner>/all-about-ai/rag-app:31fab5b .
#1 [internal] load build definition from Dockerfile     1.05kB
#3 [internal] load metadata for docker.io/library/python:3.12-slim    DONE 0.6s
#4 [internal] load .dockerignore                        123B
#5 [internal] load build context                        10.83kB
#6 [1/7] FROM docker.io/library/python:3.12-slim@sha256:5e2bdb1b...
...  (steps 2/7 to 7/7: WORKDIR, COPY requirements, pip install, COPY code, COPY data, useradd)
step finished in 30s
 
# a second build of the same files a few seconds later (by Docker Compose):
#12 [7/7] RUN useradd --create-home appuser
#12 CACHED
#13 naming to docker.io/library/rag-app-app:latest done

The build context was only 10.83 kB because .dockerignore kept out tests, evals and caches. The second build reused every layer from cache. The image was then pushed to GitHub Container Registry tagged with the commit SHA (31fab5b); the registry path is shortened here.

A real example

Example

One image from laptop to staging

In the course repository, the same image is used three times in one pipeline run: built once in CI, started with Docker Compose for a smoke test that called /ready and /ask, then pulled by the staging deploy job and started behind a health check (ship the RAG app). Nothing is reinstalled along the way, so if it passed the smoke test, the exact same bytes run in staging. When the first run failed on a Python import, the fix was in the code; the Dockerfile never needed to change.

The easy way

Start here

Copy the Dockerfile above, adjust the COPY lines and CMD for your app, add a .dockerignore, and run docker build then docker run -p 8000:8000. Open http://localhost:8000/docs to check it works.

When you grow: the production setup

Production images use pinned base image digests, multi-stage builds (build tools in one stage, a minimal runtime image in the next), dependency lock files with hashes, vulnerability scanning in CI, signed images and provenance (SBOMs), small images for faster pulls, read-only file systems, resource limits set by the orchestrator, and a registry with retention rules.

Trade-offs

What you gain

  • Same environment everywhere: laptop, CI, servers.
  • Fast, repeatable deploys and rollbacks by image tag.
  • Isolation between apps on one machine.

What it costs you

  • Another tool to learn; images can get large (ML libraries, model weights).
  • GPU containers need extra drivers and runtime setup.
  • Images need updating to pick up security fixes in the base layers.

When not to use it

Common mistakes

  • Copying the whole folder (including .env and .venv) into the image.
  • Putting COPY . . before pip install, so every code change reinstalls everything.
  • Running as root.
  • Baking API keys or model weights into the image.
  • Deploying :latest instead of a SHA tag.

Interview questions

Try answering out loud first, then open the model answer.

What is Docker and why use it for AI apps?

Model answer

Docker packages an application with its runtime and dependencies into an image that runs identically as a container anywhere Docker runs. For AI apps it pins Python and library versions (which matter a lot for ML), makes CI and production match development, enables fast deploys and rollbacks by image tag, and is the unit that managed platforms and Kubernetes run.

How do you write an efficient, secure Dockerfile for a Python service?

Model answer

Use a slim, pinned base image; copy the dependency file and install before copying code to maximise layer caching; use .dockerignore; consider multi-stage builds; run as a non-root user; add a health check; set unbuffered logging; pass config and secrets at run time via environment variables or secret mounts; scan the image; and tag with the commit SHA.

How should model weights be handled with containers?

Model answer

Usually not baked into the application image: store them in object storage or a model registry and download or mount them at start-up (with caching on the node), or use a separate model-serving image built for that purpose. This keeps app images small, avoids rebuilding for model changes and lets you version models independently.

Quiz

Pick an answer to see whether it’s right and why.

  1. 01What's the difference between an image and a container?
  2. 02Why copy requirements.txt and install dependencies before copying the code?
  3. 03Where should an API key used by the container come from?
  4. 04Why tag images with the git commit SHA?
  5. 05What does a HEALTHCHECK in the Dockerfile do?

0 of 5 answered

Key terms

Docker
A tool that packages an app with everything it needs into an image and runs it as a container, the same way on any machine.
Container
A lightweight, isolated package of an app and everything it needs to run, sharing the host's kernel. Docker is the most common tool. Also used as a sandbox.
Container image
A read-only package of an app plus its runtime, libraries and files, built from a Dockerfile and run as containers.
Dockerfile
A text file of instructions (FROM, COPY, RUN, CMD...) that Docker follows to build an image. Each instruction makes a cached layer.
Container registry
A server that stores and serves container images by name and tag, such as GitHub Container Registry, Docker Hub or a cloud registry.

Before and after

Read first

Read next

Progress is saved in this browser only.