Area 00 // FoundationsLesson 02 / 14014 min read
How a computer learns from examples
A computer learns by looking at many examples where the answer is known, adjusting itself until its guesses match those answers, and then using what it learned to answer new questions.
Think of it like this
Analogy
Imagine a new cook learning to season soup. Each day they make soup, the head chef tastes it and says "too salty" or "not enough". The cook adjusts a little each time. After a few weeks they don't need the chef any more: they can season a soup they've never made before.
The problem it solves
You want an app that guesses a flat's rent from its size. You could write rent = size × 20, but where does 20 come from? It's different in every city, it changes every year, and size isn't the only thing that matters.
Instead of guessing the rule, you can collect real rents and let the computer find the rule. If the market changes, you collect new examples and it finds the new rule. You never rewrite the code.
How it works: learning from examples
- Collect examples where the answer is known. Past flats with their real rent.
- Start with a guess. The model starts with a rule that's probably wrong, like "rent = size × 1".
- Measure how wrong it is. Compare its guesses to the real rents. The total wrongness is called the loss (more in lesson 4).
- Adjust a little to be less wrong. Nudge the number (1 → 1.5 → 2…) in the direction that lowers the loss.
- Repeat thousands of times until the guesses are close.
- Keep the result. The final numbers are the trained model.
How it works: data, features and labels
Every example has two parts:
- Features are what you know: size, number of bedrooms, distance to a station. For an email, the words in it. For a photo, its pixels.
- The label is the answer you want predicted: the rent, "spam" or "not spam", "cat" or "dog".
| Size (m²) | Bedrooms | Near station? | Rent (label) |
|---|---|---|---|
| 40 | 1 | yes | 1,050 |
| 65 | 2 | no | 1,300 |
| 90 | 3 | yes | 2,150 |
Each row is one example; together they are the training data. When examples have labels, it's called supervised learning. When they don't (say, grouping customers by similar habits), it's unsupervised learning. LLMs are trained in a third way, self-supervised: the label is simply the next word of real text, so no human has to label anything (see pretraining).
How it works: training vs inference
| Training | Inference | |
|---|---|---|
| What happens | Model learns from examples | Model answers a new input |
| How often | Once, or every few weeks/months | Every request, millions of times a day |
| Cost | Huge for big models (weeks on thousands of GPUs) | Small per request |
| Who does it for LLMs | OpenAI, Anthropic, Google, Meta… | You, every time you call the API |
As an AI engineer you'll spend almost all your time on inference: sending inputs to a trained model and building around its answers. You'll sometimes do small extra training (fine-tuning), but rarely train from scratch.
See it in code
- Needs
- Nothing: plain Python 3.
A complete (tiny) training and inference run. The model has one number, price_per_m2. Training nudges it until the guesses match real rents; inference uses it on a flat it has never seen.
# Training data: (size in m², real monthly rent). Features = size, label = rent.
examples = [(40, 1050), (55, 1400), (65, 1650), (90, 2300), (120, 3050)]
price_per_m2 = 1.0 # the model's only number, starting as a bad guess
learning_rate = 0.00001 # how big each nudge is
# ---- Training: repeat "guess, measure error, adjust" many times ----
for step in range(2000):
for size, real_rent in examples:
guess = size * price_per_m2
error = guess - real_rent # positive = guessed too high
price_per_m2 -= learning_rate * error * size # nudge to shrink the error
if step in (0, 10, 1999):
print(f"after round {step:>4}: price per m² = {price_per_m2:.2f}")
# ---- Inference: use the trained model on a new flat ----
new_flat = 75
print(f"predicted rent for {new_flat} m²: {new_flat * price_per_m2:.0f}")after round 0: price per m² = 7.88
after round 10: price per m² = 24.84
after round 1999: price per m² = 25.48
predicted rent for 75 m²: 1911The number moved from a silly 1.0 to about 25.48 per m², the best single price that fits the examples. That's all "learning" means here: finding numbers that make the guesses match the data. Real models have millions or billions of these numbers instead of one.
A real example
Example
How a photo app learns to find your dog
Your phone's gallery can find "dog" photos without you tagging any. Here's how that was built:
- Data: the company collected millions of photos that people had labelled ("dog", "beach", "car").
- Features: the pixels of each photo. Labels: the tags.
- Training: on big GPU clusters for days, a neural network adjusted itself until it tagged the training photos correctly.
- Inference: the trained model ships inside the app. When you take a new photo, it runs on your phone in milliseconds and adds the tag "dog".
Notice the split: training happened once, far away, at great cost. Inference happens on every photo, quickly and cheaply.
The easy way
Start here
Don't try to remember every term at once. Hold on to one sentence: "training finds the numbers; inference uses them." Run the snippet above and change new_flat, then change one rent in examples and see the learned price move.
When you grow: the production setup
In production you'll meet this split everywhere: a training pipeline that runs occasionally and stores a versioned model, and an inference service that loads that model and answers requests, monitored for quality as the world drifts away from the training data.
Trade-offs
What you gain
- No one has to write the rules; the model finds them in the data.
- Retraining on fresh examples adapts the model to change without new code.
- Once trained, inference is fast and cheap per request.
What it costs you
- You need many good, correctly labelled examples, which can be slow and expensive to collect.
- The model copies the patterns in the data, including mistakes and biases.
- It does poorly on situations that weren't in the examples.
When not to use it
- You can't get representative examples of the real situations the model will face.
- A clear formula already exists (the rent is fixed by law, VAT is 20%).
Common mistakes
- Mixing up features and labels: the label is only the thing you want predicted.
- Thinking a model keeps learning from every request by default. Most deployed models are frozen; they only change when someone retrains them.
- Collecting examples that don't match real use (only daytime photos, only one city's rents).
- Assuming more data always helps. Wrong or messy labels can make a model worse.
Interview questions
Try answering out loud first, then open the model answer.
Explain training vs inference, and why the difference matters for an AI engineer.
Model answer
Training is when a model's parameters are adjusted using labelled (or self-labelled) data to minimise a loss; it's compute-heavy and done rarely. Inference is running the frozen model on new inputs to get predictions; it happens on every request. For an AI engineer, almost all cost, latency and scaling work is about inference: choosing the model, batching, caching, streaming. Training only comes up for fine-tuning.
What are features and labels? Give an example.
Model answer
Features are the input variables the model uses; the label is the target it predicts. For fraud detection, features might be amount, merchant category, country and time since last purchase; the label is fraud / not fraud. For an LLM, the input text is the feature and the next token is the label.
What's the difference between supervised, unsupervised and self-supervised learning?
Model answer
Supervised learning uses examples with human-provided labels (spam / not spam). Unsupervised learning has no labels and finds structure, like clusters of similar customers. Self-supervised learning creates labels from the data itself, for example hiding the next word of a sentence and predicting it; that's how LLMs are pretrained on internet-scale text without anyone labelling it.
Quiz
Pick an answer to see whether it’s right and why.
0 of 6 answered
Key terms
- Training data
- The examples a model learns from. For supervised learning, each example has features and a label.
- Feature
- One piece of input information the model uses to make a prediction, such as a flat's size or the words in an email.
- Label
- The correct answer attached to a training example, the thing the model should learn to predict (a price, "spam", "cat").
- Training
- The phase where a model's internal numbers are adjusted, using examples, until its outputs match the right answers as closely as possible.
- Inference
- Using a trained model to produce an output for a new input. Every call to an LLM API is inference.
- Supervised learning
- Learning from examples that come with the right answers (labels). Unsupervised learning has no labels; self-supervised learning makes labels from the data itself.
- Model
- The thing you keep after training. A function, stored as numbers, that turns an input (an email, a prompt) into an output (a label, a reply).
Before and after
Progress is saved in this browser only.