Area 03 // AI AgentsLesson 48 / 14015 min read

Tool calling

Tool calling is how agents act, and doing it well means designing clear tools, running several calls from one turn and returning all their results together, sending errors back to the model instead of crashing, and checking every argument and permission in your own code.

On this page

Think of it like this
The problem it solves
How it works, step by step
See it in code
A real example
The easy way
Trade-offs
Common mistakes
Interview questions
Quiz
Key terms
Before and after

Think of it like this

Analogy

A new assistant at a law firm can use the firm's systems: the case database, the calendar, the email system. A good office manager gives them a short, clear list of what each system is for, lets them make several requests at once, tells them plainly when something fails ("case number not found") so they can try again, and the systems themselves check permissions, whatever the assistant says.

That's good tool calling.

The problem it solves

You met the basics of tool calling in lesson 24. In agents, tools are used constantly, in long loops, often several at once. Small design mistakes compound: vague descriptions make the model pick the wrong tool, crashes on errors end the whole task, results split across messages confuse the model, huge results blow the context budget, and tools that trust the model's arguments become security holes.

How it works, step by step

Design tools around tasks, not your database. get_order_status(order_id) and issue_refund(order_id, amount), not run_sql(query). Few, distinct, well-named tools.
Write the description for a new colleague: what it does, when to use it, when not to, what the arguments mean, example values. This is the model's only guide.
Use strict schemas (strict: true, additionalProperties: false, required fields, enums) so arguments always parse (structured outputs).
Send tools with each request. The model replies with text, or with one or more tool calls (stop reason tool_use for Anthropic, function_call items for OpenAI).
Handle parallel tool calls: one turn may request several tools. Run independent ones concurrently, then return all results in one message, each tied to its call ID.
Validate and authorise inside the tool: types and ranges, and whether this user may do this action on this object. Never trust identity or permissions claimed in arguments.
Return errors as results, marked as errors with an actionable message ("unknown city 'Atlantis'; known cities: London, Madrid"). The model can then retry or explain.
Keep results small and structured: the fields the model needs, as JSON.
Loop until the model stops calling tools (agent loop).

One model turn, two tool calls, one combined reply.

See it in code

Needs: Nothing for this offline version (plain Python 3). Shapes follow the Anthropic Messages API; the full loop with a real model is in the agent capstone.

Running all tool calls from one turn, catching errors, and building the single reply message.

run_tool_calls.py

import json
 
def get_weather(city: str) -> dict:
    data = {"London": {"temp_c": 11, "rain": True}, "Madrid": {"temp_c": 19, "rain": False}}
    if city not in data:
        raise ValueError(f"unknown city {city!r}")
    return data[city]
 
TOOLS = {"get_weather": get_weather}
 
def run_tool_calls(tool_uses: list[dict]) -> list[dict]:
    """Run every tool call from ONE model turn and return ALL results in one user message."""
    results = []
    for call in tool_uses:
        try:
            output = TOOLS[call["name"]](**call["input"])
            results.append({"type": "tool_result", "tool_use_id": call["id"], "content": json.dumps(output)})
        except Exception as e:                       # errors go back to the model, not up the stack
            results.append({"type": "tool_result", "tool_use_id": call["id"],
                            "content": f"Error: {e}", "is_error": True})
    return results
 
# One model turn asked for two tools at once (parallel tool use), Anthropic Messages API shape.
model_turn = [
    {"type": "tool_use", "id": "toolu_1", "name": "get_weather", "input": {"city": "London"}},
    {"type": "tool_use", "id": "toolu_2", "name": "get_weather", "input": {"city": "Atlantis"}},
]
for r in run_tool_calls(model_turn):
    print(r)

Output

{'type': 'tool_result', 'tool_use_id': 'toolu_1', 'content': '{"temp_c": 11, "rain": true}'}
{'type': 'tool_result', 'tool_use_id': 'toolu_2', 'content': "Error: unknown city 'Atlantis'", 'is_error': True}

With the real API, the list above becomes messages.append({"role": "user", "content": results}) after appending the model's own turn (response.content) as an assistant message. Tool definitions are passed as tools=[{"name": ..., "description": ..., "input_schema": {...}, "strict": True}].

A real example

Example

A banking assistant's tool redesign

A bank's first assistant had one tool: query_database(sql). It was powerful and terrifying: the model sometimes wrote slow queries, once queried another customer's data during testing, and security refused to approve it.

The redesign had five narrow tools (get_balance, list_transactions(from, to), get_card_status, freeze_card, search_help), each checking the authenticated customer ID from the session, never from arguments. freeze_card required confirmation in the UI. Tool-selection accuracy in evals went up (clear names help), and security approved it.

The easy way

Start here

Write each tool as a normal Python function with a clear docstring and type hints, validate inside it, wrap the call in try/except that returns an error result, and return results for every call of a turn together. SDK helpers (Anthropic's tool runner, the OpenAI Agents SDK's @function_tool) build schemas from functions for you.

When you grow: the production setup

Production tool layers add per-tool timeouts and retries, idempotency keys for actions, rate limits, allow-lists, audit logs, human approval for risky actions, least-privilege credentials per tool, and evals that measure tool-call accuracy. Shared tools are often exposed through MCP so several apps reuse them.

Trade-offs

What you gain

Lets agents fetch real data and take real actions.
Parallel calls cut latency for independent lookups.
Error results let agents recover instead of failing.

What it costs you

Every call is another round-trip and more tokens.
More tools = more chances to pick the wrong one.
Action tools create real security risk.

When not to use it

Don't expose a tool for something your code should just do every time (call it yourself in a workflow).
Don't create general-purpose tools (raw SQL, shell, HTTP to anywhere) without sandboxing and strict controls.

Common mistakes

Vague or overlapping tool descriptions.
Returning tool results in separate messages, or dropping failed ones.
Letting exceptions crash the agent loop.
Trusting user IDs or permissions passed in arguments.
Dumping huge raw results into the context.

Interview questions

Try answering out loud first, then open the model answer.

How do you design tools for an LLM agent?

Model answer

Model tools on user-level tasks with narrow scopes and distinct names; write descriptions that say what, when and when not to use them; use strict JSON Schemas with enums and formats; enforce authorization and validation inside the tool using the real session identity; make actions idempotent; return concise structured results and actionable errors; and evaluate selection and argument accuracy on realistic tasks.

How do you handle parallel tool calls and tool errors?

Model answer

Execute all tool calls from a single model turn (concurrently when independent and safe), then return every result in one message, each linked to its call ID. Failures become error results with clear messages rather than exceptions, so the model can retry with corrected arguments, try another tool, or explain the problem. Timeouts and retries with backoff happen inside the tool layer.

Why not give the agent a raw SQL or shell tool?

Model answer

It maximises the blast radius: the model can be manipulated by prompt injection or simply make mistakes, issuing destructive, expensive or data-leaking commands, and fine-grained permission checks become nearly impossible. Narrow tools with built-in authorization, validation and limits are safer and also easier for models to use correctly. If general tools are necessary, sandbox them with read-only credentials, allow-lists and human approval.

Quiz

Pick an answer to see whether it’s right and why.

0 of 6 answered

Key terms

Tool calling (function calling): A model replying with a request to run one of your functions, with arguments; your app runs it and sends the result back.
Tool schema: The definition of a tool given to a model, with a name, a description and a JSON Schema for its arguments.
Tool result: The output of a tool your app ran, sent back to the model linked to the call's ID so it can continue.
Parallel tool calls: Several tool calls requested in one model turn. Run them (concurrently if safe) and return all results together.
Strict mode (tools): A setting that forces a model's tool arguments to match the JSON Schema exactly (types, required fields, no extras).

Before and after

Read first