Area 01 // LLMsLesson 22 / 14014 min read
Reasoning models
Reasoning models are LLMs trained to think through a problem internally (planning, trying approaches, checking their work) before answering, which makes them much better at maths, code and multi-step tasks at the cost of more tokens and more time.
Think of it like this
Analogy
Ask two people "what's 17 × 24?". The first blurts out "about 400". The second says "give me a moment", works it out on scrap paper, checks it, and says "408".
A reasoning model is the second person. It uses a hidden scratchpad before answering. You wait a bit longer and pay for the scrap paper, but on hard questions it's right far more often.
The problem it solves
Standard chat models answer in one go, token by token. On tasks that need several dependent steps (a tricky bug, a proof, a plan with constraints, a tax calculation), one early slip ruins the answer. Chain-of-thought prompting helped, but it relied on clever prompts and the model wasn't trained to reason well.
Labs then trained models specifically to think before answering, rewarding them when the final answer was verifiably correct. These reasoning models changed what's possible for coding agents and complex analysis, but they also changed cost and latency maths.
How it works, step by step
- Training: start from a strong chat model, then use reinforcement learning on problems with checkable answers (maths with known results, code with unit tests). Long, careful reasoning that leads to correct answers gets rewarded (post-training).
- At inference, the model first writes reasoning tokens: exploring approaches, doing calculations, catching its own mistakes, then writes the final answer.
- The raw reasoning is usually hidden. Providers return the answer, sometimes with a summary of the thinking, and report reasoning token counts in usage.
- You pay for reasoning as output tokens. A 200-token answer might come with 5,000 reasoning tokens.
- You control depth with an effort setting. As of November 2026: OpenAI's Responses API takes
reasoning={"effort": "low" | "medium" | "high" ...}; Anthropic's Claude models think adaptively and takeoutput_config={"effort": "low" | ... | "max"}(on Claude Opus 5.5 thinking can't be turned off and the default effort is medium). Check each model's docs. - Prompt with goals, not scripts. Describe the outcome, constraints, available information and how success is judged. Let the model plan.
- Pick per task. Use reasoning (or higher effort) where your evals show it pays off; use fast non-reasoning settings elsewhere.
| Standard response | Reasoning response | |
|---|---|---|
| Latency | Seconds | Seconds to minutes on hard tasks |
| Cost | Visible output only | Visible output + reasoning tokens |
| Strong at | Chat, extraction, summaries, simple Q&A | Maths, code, planning, agents, analysis |
See it in code
- Needs
- Nothing for the cost calculator (plain Python 3). The API calls below need OPENAI_API_KEY or ANTHROPIC_API_KEY.
Reasoning tokens are where the money goes. Same question, three effort levels (token counts are typical shapes, not measurements; your evals will show the real numbers).
# Reasoning tokens are billed like output tokens, even when you never see them.
PRICE_IN, PRICE_OUT = 2.00, 10.00 # $ per 1M tokens (example mid-tier model, Nov 2026)
def cost(input_tokens, visible_output, reasoning_tokens):
out = visible_output + reasoning_tokens
return input_tokens / 1e6 * PRICE_IN + out / 1e6 * PRICE_OUT
# Same question, three effort levels (token counts are typical shapes, not measurements).
runs = {"low effort": 300, "medium effort": 2_500, "high effort": 12_000}
for name, reasoning in runs.items():
c = cost(input_tokens=1_000, visible_output=200, reasoning_tokens=reasoning)
print(f"{name:13}: {reasoning:>6,} reasoning tokens -> ${c:.4f} per request,"
f" ${c * 100_000:,.0f} per 100k requests")low effort : 300 reasoning tokens -> $0.0070 per request, $700 per 100k requests
medium effort: 2,500 reasoning tokens -> $0.0290 per request, $2,900 per 100k requests
high effort : 12,000 reasoning tokens -> $0.1240 per request, $12,400 per 100k requestsSetting effort in real calls (model IDs as of November 2026):
from openai import OpenAI
import anthropic
question = "A train leaves at 09:40 and arrives at 13:05. How long is the journey?"
r = OpenAI().responses.create(model="gpt-6.1-sol", input=question,
reasoning={"effort": "low"}) # low | medium | high
print(r.output_text, r.usage.output_tokens_details.reasoning_tokens)
m = anthropic.Anthropic().messages.create(
model="claude-opus-5-5", max_tokens=4000,
output_config={"effort": "low"}, # low ... max
messages=[{"role": "user", "content": question}])
print(next(b.text for b in m.content if b.type == "text"))A real example
Example
A coding assistant that routes by difficulty
A company's internal coding assistant handled everything with a reasoning model at high effort. Developers loved the quality but complained it took 40 seconds to answer "what does this function return?". The bill was also high.
They added a simple router: questions about a single small file go to the model at low effort (2–4 seconds), while "fix this failing test" or "refactor this module" go to high effort. Quality on hard tasks stayed the same in evals, median latency fell to 5 seconds, and costs dropped by about 60%.
The easy way
Start here
Start with your provider's default effort. If hard tasks fail in your evals, raise effort; if simple tasks are slow or expensive, lower it. Write prompts as clear goals with success criteria.
When you grow: the production setup
Production systems choose model and effort per route, set per-request token budgets, stream progress to users during long thinking, log reasoning token counts for cost dashboards, and re-tune effort when providers release new models, because the same effort label can behave differently across model versions.
Trade-offs
What you gain
- Much higher accuracy on maths, code, planning and agentic tasks.
- Less prompt engineering needed for multi-step problems.
- Effort settings give a single dial between speed and depth.
What it costs you
- More tokens and latency, sometimes far more.
- Hidden reasoning makes debugging harder (only summaries or counts are visible).
- Overthinks simple tasks, wasting money.
When not to use it
- High-volume simple tasks (classification, extraction, FAQ answers): use fast, cheap settings or smaller models.
- Strict real-time paths (voice, autocomplete) where seconds of thinking are unacceptable.
Common mistakes
- Budgeting by visible output and forgetting reasoning tokens.
- Using maximum effort everywhere "to be safe".
- Writing rigid step-by-step scripts that fight the model's own planning.
- Setting
max_tokenstoo low so thinking uses the budget and the answer gets cut off. - Assuming one effort setting transfers unchanged to a new model version.
Interview questions
Try answering out loud first, then open the model answer.
What are reasoning models and when would you use one?
Model answer
They're LLMs trained, typically with reinforcement learning on verifiable tasks, to produce extended internal reasoning before answering. They excel at multi-step problems: maths, coding, planning, complex analysis and agentic tool use. I'd use them where evals show accuracy gains justify extra cost and latency, and avoid them for simple, high-volume or latency-critical tasks, often routing between reasoning and non-reasoning paths.
How do reasoning models affect cost and latency planning?
Model answer
Reasoning tokens are billed as output and can dwarf the visible answer, and time to the final answer grows with thinking. Plan with measured reasoning token distributions per task, set max tokens with headroom for thinking, choose effort per route, stream progress for long tasks, and track cost per successful task rather than per request.
How does prompting change for reasoning models?
Model answer
Give them the objective, context, constraints and definition of done, and let them plan; avoid prescribing every step or adding "think step by step", which is redundant. Ask for the final output format explicitly. Provide examples sparingly. Adjust effort rather than piling on instructions, and validate with evals because older prompts tuned for chat models can underperform.
Quiz
Pick an answer to see whether it’s right and why.
0 of 6 answered
Key terms
- Reasoning model
- An LLM trained to think internally before answering, which improves maths, code and multi-step tasks at the cost of more tokens and time.
- Reasoning tokens
- Tokens a reasoning model generates while thinking before its answer. Usually hidden, but billed as output tokens.
- Effort (reasoning effort)
- A request setting that controls how much a model thinks before answering, trading cost and latency for depth.
- Chain of thought
- Having a model write out intermediate reasoning steps before its final answer, which helps on multi-step problems.
- Token
- A small chunk of text (a word, part of a word or a symbol) that a language model reads and writes one at a time. Usage and prices are counted in tokens.
Before and after
Progress is saved in this browser only.