Area 01 // LLMsLesson 21 / 14013 min read

Few-shot prompting and chain of thought

Few-shot prompting means showing the model a few worked examples of the task so it copies the pattern, and chain of thought means asking it to reason step by step before answering, which helps on multi-step problems.

On this page

Think of it like this
The problem it solves
How it works: few-shot prompting
How it works: chain of thought
See it in code
A real example
The easy way
Trade-offs
Common mistakes
Interview questions
Quiz
Key terms
Before and after

Think of it like this

Analogy

Few-shot is like showing a new waiter three filled-in order slips before their first shift: "this is how we write a table number, this is how we write 'no onions'". They copy the pattern instantly, far better than from a page of rules.

Chain of thought is the maths teacher saying "show your working". Writing the steps down doesn't just let the teacher check them; it helps the student get the right answer.

The problem it solves

Some tasks are easy to describe but the details are fiddly: exactly which label for borderline tickets, exactly what format, which tone. Instructions alone ("zero-shot") leave too much room for interpretation.

Other tasks need several steps: a word problem, checking a policy against a situation, planning. If the model blurts out an answer immediately, it often skips a step and gets it wrong.

How it works: few-shot prompting

Start zero-shot: clear instructions, no examples. If the results are good, stop.
Add 2–8 worked examples (few-shot prompting): each is an input and the exact output you want, either as fake user/assistant turns or as a labelled block in the prompt.
Make examples representative: cover each label or case, include a tricky borderline one, and vary length and wording.
Keep the format identical across examples and the real request; the model copies the format exactly.
Watch for over-copying: if outputs start echoing example content, vary the examples or reduce them.
For large example sets, retrieve the few most similar examples for each request ("dynamic few-shot").

How it works: chain of thought

Ask for reasoning before the answer (chain of thought): "Think step by step inside thinking tags, then write the final answer as ANSWER: …".
The model writes intermediate steps. Each step is now part of the context, so later steps can build on earlier ones.
Your code extracts only the final, clearly marked answer. Show the reasoning to users only if it helps them.
With reasoning models, thinking happens internally by design (next lesson). Don't spell out every step; give the goal, constraints and success criteria, and control depth with the provider's effort setting.
Measure. CoT adds tokens and latency. Keep it where your evals show accuracy improves.

Few-shot shows the pattern; chain of thought gives the model room to work.

See it in code

Needs: Nothing: plain Python 3.

Building a few-shot conversation, and parsing a chain-of-thought reply safely.

few_shot_cot.py

import re
 
# Few-shot: show the model worked examples of exactly the format you want.
EXAMPLES = [
    ("The app crashes when I upload a photo", "bug"),
    ("I was charged twice this month", "billing"),
    ("How do I change my email address?", "account"),
]
 
def few_shot_messages(ticket: str) -> list[dict]:
    msgs = [{"role": "system", "content": "Classify the ticket as bug, billing or account. Reply with one word."}]
    for text, label in EXAMPLES:                 # each example = a fake user turn + ideal answer
        msgs.append({"role": "user", "content": text})
        msgs.append({"role": "assistant", "content": label})
    msgs.append({"role": "user", "content": ticket})
    return msgs
 
print(len(few_shot_messages("My invoice shows the wrong VAT")), "messages, last:",
      few_shot_messages("My invoice shows the wrong VAT")[-1]["content"])
 
# Chain of thought: ask for reasoning first, then a clearly marked final answer,
# and have your code read only the final answer.
COT_INSTRUCTION = "Think step by step inside <thinking> tags, then give the result as 'ANSWER: <number>'."
model_reply = """<thinking>3 boxes of 12 is 36 pens. 36 minus 5 given away is 31.</thinking>
ANSWER: 31"""                                    # what a model might return
 
match = re.search(r"ANSWER:\s*(-?\d+)", model_reply)
print("final answer parsed by code:", int(match.group(1)) if match else None)

Output

8 messages, last: My invoice shows the wrong VAT
final answer parsed by code: 31

The few-shot list is 1 system message + 3 example pairs + the real ticket = 8 messages; pass it as messages= to any chat API (for Anthropic, move the system message to system=).

A real example

Example

Tagging insurance claims with examples instead of rules

An insurer wanted claims tagged as "water damage", "theft", "accident" or "other". The first prompt listed definitions, and accuracy on 300 labelled claims was 81%. Borderline cases (a burst pipe that flooded a car) went wrong.

Adding six examples, including two borderline ones with the agreed label, raised accuracy to 92%. Asking the model to think step by step added two more points but doubled the cost, so they used chain of thought only for claims over £10,000, where mistakes were expensive.

The easy way

Start here

Start zero-shot. If the format or labels wobble, add three good examples. If multi-step answers are wrong, add "think step by step, then give ANSWER: …" or switch to a reasoning model. Measure each change on the same test set.

When you grow: the production setup

At scale, teams keep a library of labelled examples, retrieve the most similar ones per request, refresh them from reviewed production data, and decide per route whether reasoning is worth its tokens, often using a cheap model without reasoning for easy cases and a reasoning model for hard ones (routing).

Trade-offs

What you gain

Few-shot teaches format and subtle judgement quickly, without training.
Chain of thought noticeably improves multi-step reasoning on many tasks.
Both are just prompt changes: easy to try, measure and roll back.

What it costs you

Examples and reasoning add tokens to every request.
Models can over-copy examples, including their mistakes and biases.
Visible reasoning can be wrong or misleading even when the final answer is right (and vice versa).

When not to use it

Simple tasks where zero-shot already scores well: examples just add cost.
Latency-critical paths (autocomplete, voice) where extra reasoning tokens are too slow.

Common mistakes

Using examples that all look the same or all share one label.
Changing the format between examples and the real request.
Parsing the whole chain-of-thought reply instead of a marked final answer.
Forcing detailed step-by-step scripts on reasoning models that already plan well.
Treating the model's written reasoning as a faithful explanation of how it decided.

Interview questions

Try answering out loud first, then open the model answer.

When would you use few-shot prompting instead of fine-tuning?

Model answer

When you need to steer format or judgement quickly, have only a handful of examples, need to iterate fast, or use a closed model. Few-shot is instant and reversible. Fine-tuning becomes worth it when you have many examples, the prompt is getting long and expensive at high volume, or few-shot accuracy plateaus below the bar.

Why does chain-of-thought prompting work, and what are its costs?

Model answer

Generating intermediate steps gives the model more computation per problem and conditions later tokens on earlier partial results, which reduces errors on multi-step tasks. Costs: more output tokens and latency, possible leakage of reasoning to users, and reasoning text that isn't guaranteed to reflect the true basis of the answer. Reasoning models internalise this with trained thinking, controlled by effort settings.

How do you pick few-shot examples?

Model answer

Choose examples that cover every class or case, include hard and borderline ones with the agreed answer, vary surface features, match the exact output format, and avoid ones that overlap with your test set. For large pools, retrieve the most similar examples per request. Validate the chosen set against an eval set and check for label or position bias.

Quiz

Pick an answer to see whether it’s right and why.

0 of 6 answered

Key terms

Few-shot prompting: Including a few worked examples (input and ideal output) in the prompt so the model copies the pattern.
Zero-shot prompting: Asking a model to do a task with instructions only, without any worked examples.
Chain of thought: Having a model write out intermediate reasoning steps before its final answer, which helps on multi-step problems.
Reasoning model: An LLM trained to think internally before answering, which improves maths, code and multi-step tasks at the cost of more tokens and time.

Before and after

Read first

020 Prompt engineering basics and system prompts