Skip to content

All lessons

Area 00 // FoundationsLesson 11 / 14013 min read

Where models come from, and what an AI engineer does

A few labs (OpenAI, Anthropic, Google, Meta, Mistral and others) train the big models and offer them either through paid APIs or as open weights you can download from hubs like Hugging Face, and an AI engineer's day is choosing between them and building reliable, safe, affordable products on top.

Think of it like this

Analogy

Think of car makers and taxi companies. A few huge factories (Toyota, Ford…) design and build engines; that's the AI labs training foundation models. Some factories only rent you rides (closed models through an API); others also sell you the car so you can drive and modify it yourself (open weights).

An AI engineer is like someone running a delivery business: you don't build engines. You choose which vehicles to use, plan routes, train drivers, track costs and make sure parcels arrive safely.

The problem it solves

There are hundreds of models with confusing names, new ones every month, and wildly different prices. Beginners either pick whatever is famous or get stuck comparing endlessly. Meanwhile it's unclear what the job "AI engineer" even involves, if not training models.

You need a simple map: who makes models, how you get access to them, how to choose, and what you'll actually do every day.

How it works: where models come from

  1. Labs pretrain foundation models on enormous datasets with thousands of GPUs, then post-train them to follow instructions (lesson 19).
  2. They release them in one of two ways:
    • Closed (API-only): you send requests, they run the model. As of November 2026: OpenAI (GPT-6 family), Anthropic (Claude Opus, Sonnet, Haiku, Fable), Google (Gemini 4).
    • Open weights: you download the model and run it anywhere. As of November 2026: Meta (Llama 4, and newer open weights under the Muse name), Mistral (Mistral Large 4, Mistral Small), Alibaba (Qwen), Google (Gemma), DeepSeek. Licences differ: some are Apache 2.0, some have usage restrictions.
  3. Hugging Face is the hub for open models: model files, model cards (what it's for, how it was trained, limits), datasets, leaderboards and hosted demos ("Spaces").
  4. Cloud platforms resell access: AWS Bedrock, Google Vertex AI, Microsoft Azure / Foundry host many models behind one bill, useful for companies already on that cloud.
  5. Local runners (Ollama, LM Studio, llama.cpp) and servers (vLLM) let you run open models on a laptop or your own GPUs.
Access routeExamplesGood for
Provider APIOpenAI, Anthropic, Google AI StudioBest quality, fastest start
Cloud platformBedrock, Vertex AI, Azure / FoundryEnterprise billing, data-residency, one contract
Open weights, self-hostedLlama, Mistral, Qwen via vLLM or OllamaPrivacy, control, fixed cost at high volume

How it works: an AI engineer's day

An AI engineer builds products on top of models. A typical week cycles through:

  1. Framing: is this a job for an LLM, classic ML or plain code? (lesson 1)
  2. Prompting and context: writing prompts, deciding what information the model sees (RAG, Area 2).
  3. Building: APIs, tool calling and agents (Areas 3 and 8).
  4. Evaluating: test sets and metrics so changes are measured, not guessed (Area 4).
  5. Scaling and cost: caching, routing, batching, serving (Area 5).
  6. Securing: prompt injection defences, permissions, privacy (Area 6).
  7. Shipping and monitoring: CI/CD, tracing, dashboards, rollbacks (Area 7).
The AI engineering loop. Most of the job is the loop, not the first version.

See it in code

Needs
Nothing: plain Python 3. Prices are as of November 2026 (from each provider's pricing page); check before relying on them.

Choosing a model is partly arithmetic. This compares what one job (10,000 support replies a day, about 1,500 tokens in and 300 out each) would cost per month on different models.

model_costs.py
# (provider, model ID, $ per 1M input tokens, $ per 1M output tokens), as of Nov 2026
MODELS = [
    ("OpenAI",    "gpt-6.1-sol",       2.00, 10.00),
    ("OpenAI",    "gpt-6-luna",        0.10,  0.50),
    ("Anthropic", "claude-sonnet-5-5", 2.00, 10.00),
    ("Anthropic", "claude-haiku-5-5",  0.10,  0.50),
    ("Google",    "gemini-4-hydrogen", 1.50,  7.50),
]
 
requests_per_day = 10_000
tokens_in, tokens_out = 1_500, 300        # per request
 
for provider, model, price_in, price_out in MODELS:
    per_request = tokens_in / 1e6 * price_in + tokens_out / 1e6 * price_out
    per_month = per_request * requests_per_day * 30
    print(f"{provider:9} {model:18} ${per_month:>8,.0f}/month")
Output
OpenAI    gpt-6.1-sol        $   1,800/month
OpenAI    gpt-6-luna         $      90/month
Anthropic claude-sonnet-5-5  $   1,800/month
Anthropic claude-haiku-5-5   $      90/month
Google    gemini-4-hydrogen  $   1,350/month

A 20× price difference between tiers. Whether the cheap model is good enough is exactly what evals answer (Area 4); that's the core of choosing models.

On the real platform

What this looks like in Hugging Face · See the real screen in the official docs

A model page on an open model hub. Simplified view of a model page on a hub such as Hugging Face.
Simplified view of a model page on a hub such as Hugging Face.All About AI (simplified mock-up, not a real screenshot). Made for this course.
  1. 1Organisation / model name: check who published it.
  2. 2Tags: task, size, licence and file format (prefer safetensors).
  3. 3The model card: intended use, training data, evals and limitations.
  4. 4The weight files: their size tells you how much memory you need.
  5. 5'Use this model' shows code for transformers, vLLM, Ollama and more.

Every open model on Hugging Face has a page with the same parts. Read them in this order before you use a model:

  1. Model card: what it's for, how it was trained, known limits and biases.
  2. Licence (top of the page): can you use it commercially? Some need you to accept terms first.
  3. Files and versions: the weights. Prefer .safetensors files over .bin/pickle files (why).
  4. Downloads and likes: a rough popularity signal, not a quality guarantee.
  5. "Use this model" button: ready-made code for Transformers, vLLM, Ollama and others.

A real example

Example

Choosing models for a legal-documents startup

A startup builds a tool that answers questions about contracts. The team:

  • Prototypes in a week with a top closed model through its API, to learn what "good" looks like.
  • Builds 200 test questions with lawyers and runs three models on them. A mid-priced model scores within 2% of the top one at a third of the cost.
  • Gets a bank as a customer that forbids sending documents to outside APIs, so for that customer they deploy an open-weights model on the bank's own GPUs, checked against the same 200 questions.

No one at the startup trained a model. All the value came from choosing, evaluating, retrieving the right clauses and shipping safely.

The easy way

Start here

Start with one good closed model through its API (fast, high quality) and one small local model through Ollama (free, private). Build your first features with the API model; use the local one to experiment without spending money. Switch or mix later, guided by evals.

When you grow: the production setup

Mature teams put an LLM gateway in front of all providers (lesson 82), route easy requests to cheap models and hard ones to strong models, keep a fallback provider for outages, version every prompt and model choice, and re-run evals whenever a provider releases a new model.

Trade-offs

What you gain

  • Many strong options at very different price points, competing on price and quality.
  • Open weights give privacy, control and no per-token bill.
  • APIs give the best quality with zero infrastructure.

What it costs you

  • The landscape changes every few months; knowledge and prices go stale fast.
  • Closed models can change behaviour or be retired with notice you don't control.
  • Self-hosting open models needs GPUs and operations skills.

When not to use it

  • Don't commit to one provider's special features everywhere in your code if you might need to switch; keep provider-specific code behind your own interface.

Common mistakes

  • Picking a model because it tops a public leaderboard rather than testing on your own tasks.
  • Assuming "open weights" means "open source" or "free for any use". Read the licence.
  • Calling the most expensive model for everything when a smaller one would pass your evals.
  • Hard-coding one provider throughout the codebase.
  • Believing an AI engineer must be a researcher. The job is engineering: reliability, measurement, cost and safety.

Interview questions

Try answering out loud first, then open the model answer.

How would you choose a model for a new feature?

Model answer

Start from requirements: quality bar, latency, cost per request at expected volume, context length, data privacy and residency, and licensing. Shortlist a few models across tiers (one strong closed model, one cheaper one, one open-weights option if privacy matters). Build a small eval set from real tasks, run all candidates, and compare quality, p95 latency and cost. Choose the cheapest that meets the bar, keep a fallback, and re-evaluate when new models ship.

Closed API models vs open-weights models: what are the trade-offs?

Model answer

Closed models usually lead on quality and need no infrastructure, but you pay per token, send data to a third party, depend on their availability and roadmap, and can't modify weights beyond offered fine-tuning. Open weights give control, privacy, customisation and fixed costs at scale, but you operate GPUs and serving, handle updates and security yourself, and the best open models may trail the frontier on some tasks.

What does an AI engineer do that a backend engineer doesn't?

Model answer

They handle non-deterministic components: designing prompts and context, retrieval pipelines, tool use and agents; building evaluation datasets and metrics because unit tests can't capture output quality; managing token cost and latency; defending against model-specific threats like prompt injection; and monitoring quality drift in production. Underneath, many skills are the same as backend work: APIs, data, reliability and deployment.

Quiz

Pick an answer to see whether it’s right and why.

  1. 01What does "open weights" mean?
  2. 02Which of these is typically API-only (closed)?
  3. 03What is Hugging Face mainly used for?
  4. 04A hospital wants AI on patient notes but data may not leave its network. What's a good fit?
  5. 05Which task is NOT a typical part of an AI engineer's day?
  6. 06What's the best way to choose between two models for your product?

0 of 6 answered

Key terms

Foundation model
A large model pretrained on broad data that can be adapted to many tasks, such as GPT, Claude, Gemini or Llama.
Open weights
A model whose trained parameters can be downloaded and run on your own hardware. The licence may still limit use.
Closed model
A model you can only use through its maker's API; the weights are not released.
Hugging Face
A platform hosting open models, datasets and demo apps (Spaces), plus popular libraries like Transformers.
AI engineer
An engineer who builds products on top of existing AI models, handling prompting, retrieval, tools, evaluation, cost, security and deployment.
Model provider
A company that offers access to models, through its own API (OpenAI, Anthropic, Google) or by hosting others' models (AWS Bedrock, Vertex AI, Azure).

Before and after

Read first

Read next

Progress is saved in this browser only.