Area 07 of 08

Deployment

Getting AI apps from your laptop to users and keeping them healthy: Git, Docker, CI/CD with eval gates, Kubernetes, GPUs, safe releases and rollbacks.

0 / 18 done

Printable cheat sheet for this area

107
Git and GitHub for deployment
Git records every change to your code as a commit, branches let you work on changes safely, and GitHub hosts the repository so teammates can review pull requests and automation can test and deploy each change, which makes git history the backbone of every deployment.
108
What deploying means, and environment configs
Deploying means taking a specific version of your app from your laptop to servers where real users can reach it, usually through separate environments (development, staging and production) that run the same code with different configuration supplied from outside the code.
CoversWhat deploying means (dev → staging → prod) · Environment Configs
109
Docker
Docker packages your app together with its exact Python version, libraries and files into an image, a portable bundle that runs the same way as a container on your laptop, in CI and on any server, which removes "works on my machine" problems from deployment.
110
Docker Compose
Docker Compose describes all the containers an app needs (the API, a database, a cache, a local model server) in one compose.yaml file, so a single command starts the whole stack with the right ports, settings, volumes and health checks, on a laptop, in CI or on a small server.
111
CI/CD pipelines
A CI/CD pipeline is an automated sequence of steps that runs on every change, testing it (continuous integration) and then building and releasing it (continuous delivery or deployment), so that only changes which pass every check ever reach users.
112
GitHub Actions
GitHub Actions runs automated workflows (tests, eval gates, image builds, deployments) on GitHub's machines whenever something happens in your repository, such as a push or pull request, using YAML files in .github/workflows that describe jobs made of steps.
113
Eval gates in CI
An eval gate is a pipeline step that runs your evaluation set against the new version and blocks the merge or release if quality metrics fall below fixed thresholds or drop compared with the current main branch, so prompt, model and retrieval changes can't silently make answers worse.
114
Infrastructure as code
Infrastructure as code means describing servers, databases, networks, buckets and permissions in version-controlled files that a tool such as OpenTofu or Terraform turns into real cloud resources, so environments are reproducible, reviewed like code and changed by plan and apply instead of by clicking in a console.
115
Serverless functions and the easy hosting route
Serverless functions run your code only when a request arrives and bill per use, while managed platforms (container hosts, Hugging Face Spaces and similar) run your app from a repo or image with little setup, which makes them the fastest way to ship most AI apps that call model APIs.
CoversServerless Functions · Deploying on Hugging Face Spaces / a managed platform
116
Ollama for local models
Ollama downloads open models and runs them on your own laptop or server with one command, exposing a simple local HTTP API, which is ideal for offline development, privacy-sensitive experiments and cheap testing, as long as you evaluate whether a small local model is actually good enough for the task.
117
GPU instances
A GPU instance is a rented cloud machine with one or more GPUs, used to run or fine-tune open models yourself; choosing one means checking that the model fits in GPU memory, estimating throughput and cost per token, and comparing that with simply calling a model API.
118
Kubernetes basics
Kubernetes runs containers across a cluster of machines and keeps them in the state you describe, the right number of healthy copies, reachable through a stable address, updated gradually and rolled back when a new version can't get ready, which makes it a common platform for serving AI apps and models at scale.
119
Autoscaling and load balancing
Autoscaling adds or removes copies of your service as demand changes, and a load balancer spreads incoming requests across those copies, so an AI app stays fast at peak without paying for idle capacity, as long as you scale on the right signal and balance by real load.
CoversAutoscaling · Load Balancing
120
Canary, blue-green and feature flags
Release strategies limit the damage of a bad release, a canary sends a small share of users to the new version first, blue-green keeps two full environments so you can switch traffic back instantly, and feature flags turn features on or off for chosen users without deploying.
CoversCanary Releases · Blue-green deployments · Feature Flags
121
Prompt versioning, model versioning and registries
Prompts, model choices and fine-tuned weights change an AI app's behaviour as much as code does, so each needs an immutable version, a record of how it was evaluated, and a registry with labels like "production" that you can move forward to release or back to roll back.
CoversPrompt Versioning · Model Versioning · Model registries and MLOps (MLflow, Hugging Face Hub)
122
Health checks and rollbacks
Health checks are small endpoints that tell the platform whether an instance is alive (liveness) and ready for traffic (readiness), and rollbacks return production to the last good version quickly when a release fails those checks or its metrics get worse.
CoversHealth Checks · Rollbacks
123
Alerts and dashboards
Dashboards show how your AI app is doing at a glance (traffic, errors, latency, cost and quality), and alerts notify a person only when something needs action, ideally based on service-level objectives and how fast the error budget is being used up.
124
Capstone: ship the RAG app
The deployment capstone takes the course's RAG app from a folder of Python to a shipped service, a Docker image, a GitHub Actions pipeline with tests and an eval gate, a staging deploy that waits for a health check, and an automatic rollback that is tested on every run, all with real runs from the course repository.