Area 01 // LLMsLesson 23 / 14014 min read

Structured outputs

Structured outputs make a model reply in an exact machine-readable shape (usually JSON matching a schema you define), so your code can use the answer directly instead of trying to parse free text.

On this page

Think of it like this
The problem it solves
How it works, step by step
See it in code
A real example
The easy way
Trade-offs
Common mistakes
Interview questions
Quiz
Key terms
Before and after

Think of it like this

Analogy

Asking a model for free text is like asking a colleague to "send me the invoice details somehow": you get an email, a photo, a voice note. Structured outputs hand them a form with labelled boxes ("Vendor: ___", "Total: ___", "Currency: [GBP / EUR / USD]") and they're only allowed to fill in the boxes.

The form guarantees you get every box in the right format. It doesn't guarantee they copied the numbers correctly, so you still check the important ones.

The problem it solves

Most AI features feed code, not people: extract fields from an email into a database, classify a ticket and route it, return a list of tasks to show in a UI. If the model replies "Sure! Here's the info you asked for: …" or forgets a field or writes "about £130", your parser breaks, often in production at 2 a.m.

Asking for "JSON only" in the prompt mostly works, until it doesn't. You need a guarantee.

How it works, step by step

Define the shape with a JSON Schema, most easily as a Pydantic model in Python: field names, types, allowed values (Literal["GBP", "EUR", "USD"]), ranges, optional fields.
Pass it to the API. As of November 2026: OpenAI client.responses.parse(model=..., input=..., text_format=Invoice); Anthropic client.messages.parse(model=..., max_tokens=..., messages=[...], output_format=Invoice). Raw-schema variants exist too.
The provider uses constrained decoding: while generating, any token that would break the schema is blocked, so the output always parses and always has the required fields with the right types.
The SDK parses and validates it into a typed object: response.output_parsed (OpenAI) or response.parsed_output (Anthropic).
You still do output validation on values: is the total plausible, does the date make sense, does the ID exist? The schema guarantees shape, not truth.
Design schemas for honesty: allow null or "unknown" when information may be missing; otherwise the model is forced to invent something.

Shape is enforced by the API; correctness is checked by your code.

See it in code

Install: pip install pydantic openai anthropic
Needs: Python 3.10+. The validation part runs offline; the API calls need OPENAI_API_KEY or ANTHROPIC_API_KEY.

First, what validation catches (offline), then the same model used with both APIs.

invoice_schema.py

from typing import Literal
from pydantic import BaseModel, Field, ValidationError
 
# The shape you want back. The same class is what you pass to the API (see below).
class Invoice(BaseModel):
    vendor: str
    total: float = Field(ge=0)                       # must be zero or more
    currency: Literal["GBP", "EUR", "USD"]           # only these three allowed
    paid: bool
 
# Pretend these came back from a model.
good = '{"vendor": "Acme Ltd", "total": 129.5, "currency": "GBP", "paid": false}'
bad = '{"vendor": "Acme Ltd", "total": "about 130", "currency": "pounds"}'
 
invoice = Invoice.model_validate_json(good)          # parse + validate in one step
print("parsed:", invoice)
print("typed :", type(invoice.total).__name__, invoice.total * 2)
 
try:
    Invoice.model_validate_json(bad)
except ValidationError as e:
    print("rejected with", e.error_count(), "errors:")
    for err in e.errors():
        print(" -", err["loc"][0], "->", err["msg"])

Output

parsed: vendor='Acme Ltd' total=129.5 currency='GBP' paid=False
typed : float 259.0
rejected with 3 errors:
 - total -> Input should be a valid number, unable to parse string as a number
 - currency -> Input should be 'GBP', 'EUR' or 'USD'
 - paid -> Field required

With structured outputs turned on, the API never returns something like bad, because the decoding is constrained to the schema:

invoice_api.py

from openai import OpenAI
import anthropic
from invoice_schema import Invoice      # the class above
 
email = "Hi, invoice from Acme Ltd: £129.50, not yet paid."
 
r = OpenAI().responses.parse(model="gpt-6.1-sol", input=f"Extract the invoice:\n{email}",
                             text_format=Invoice)
print(r.output_parsed)                   # an Invoice object
 
m = anthropic.Anthropic().messages.parse(
    model="claude-sonnet-5-5", max_tokens=1024, output_format=Invoice,
    messages=[{"role": "user", "content": f"Extract the invoice:\n{email}"}])
print(m.parsed_output)                   # an Invoice object

A real example

Example

Turning 10,000 supplier emails a day into database rows

An accounts team receives invoices as emails and PDFs in every format. They extract vendor, invoice number, total, currency, due date and VAT with a structured-output call per document, using a Pydantic schema where due_date and vat may be null.

Before structured outputs, about 2% of replies failed to parse and silently dropped invoices. After, parsing failures went to zero. Their validation layer then checks business rules (total = net + VAT, invoice number not already paid) and sends anything that fails to a human queue. Shape is guaranteed by the API; truth is checked by code and people.

The easy way

Start here

Write a Pydantic model for the answer you want, pass it as text_format= (OpenAI) or output_format= (Anthropic), and use the returned object. Allow None for anything that might be missing.

When you grow: the production setup

Production pipelines version schemas alongside prompts, keep them small and flat (nesting and huge enums slow generation and confuse models), validate business rules after parsing, route failures to retries or humans, log validation error rates as a quality metric, and combine structured outputs with tool calling when the model must take actions rather than return data.

Trade-offs

What you gain

Output always parses and matches the schema: no fragile regex or retry loops for format.
Typed objects plug straight into code, databases and UIs.
Enums and constraints remove messy near-duplicate values.

What it costs you

Shape is guaranteed, correctness isn't: wrong values still need checks.
Very large or deeply nested schemas can slow responses or reduce quality.
Feature support and schema limits differ between providers and models.

When not to use it

The output is for people to read (an email draft, an explanation): free text is fine.
The task is open-ended exploration where forcing a shape loses useful nuance.

Common mistakes

Making every field required, which pushes the model to invent missing data.
Trusting values just because the JSON is valid.
Asking for "JSON only" in the prompt instead of using the structured-output feature.
Giant schemas with dozens of nested optional objects for one call; split into steps.
Changing the schema without updating evals and downstream code.

Interview questions

Try answering out loud first, then open the model answer.

How do you reliably get JSON out of an LLM?

Model answer

Use the provider's structured-output feature with a JSON Schema (via Pydantic), which applies constrained decoding so outputs always conform. Design the schema with enums, nullable fields for missing information and clear field descriptions. Parse into typed objects, then validate business rules and route failures to retry or human review. Prompt-only JSON or regex extraction is a fallback, not a strategy.

What is constrained decoding?

Model answer

At each generation step, the decoder masks out tokens that would make the partial output invalid under a grammar (e.g. a JSON Schema converted to a state machine), so only valid continuations can be sampled. The final output is guaranteed to parse and conform. It controls form, not semantic correctness.

Structured outputs vs tool calling: when do you use which?

Model answer

Structured outputs when you want the model's answer as data in a fixed shape (extraction, classification, a plan). Tool calling when the model should decide whether and which action to take, with arguments, and then continue after seeing the result. Tool arguments can also be schema-enforced (strict mode), so the two often appear together.

Quiz

Pick an answer to see whether it’s right and why.

0 of 6 answered

Key terms

Structured outputs: An API feature that forces a model's reply to match a JSON Schema you provide, so code can parse it reliably.
JSON Schema: A standard way to describe the shape of JSON data (fields, types, allowed values), used for structured outputs and tool arguments.
Pydantic: A Python library for defining data shapes as classes and validating data against them. Widely used for LLM outputs and APIs.
Constrained decoding: Restricting which tokens a model may generate at each step so the output always fits a grammar or schema.
Output validation: Checking a model's output (types, values, business rules, safety) before using it or showing it to users.

Before and after

Read first

020 Prompt engineering basics and system prompts