Area 01 // LLMsLesson 24 / 14014 min read

Function and tool calling basics

Tool calling lets a model ask your code to run a function (look up an order, search a database, send an email) by replying with the function's name and arguments; your app runs it and sends the result back so the model can finish its answer.

On this page

Think of it like this
The problem it solves
How it works, step by step
See it in code
A real example
The easy way
Trade-offs
Common mistakes
Interview questions
Quiz
Key terms
Before and after

Think of it like this

Analogy

A hotel concierge can't personally check the restaurant's bookings system, but they have a phone and a list of numbers: "Restaurant: book a table (give date, time, people)". When a guest asks for dinner, the concierge decides which number to call and what to say, the restaurant does the actual booking, and the concierge tells the guest the result.

The model is the concierge. Your tools are the phone numbers with instructions. Your code is the restaurant actually doing the work.

The problem it solves

A model on its own only knows its training data and can only produce text. It can't check today's order status, read your database, look up the weather or create a support ticket. Prompting it to "pretend" leads to made-up answers.

Tool calling connects the model to real systems in a controlled way: the model decides what should happen, your code decides whether and how it happens.

How it works, step by step

Describe each tool with a tool schema: a name (get_order_status), a description of when to use it, and a JSON Schema for its arguments.
Send the tools with the request. The model sees the user's message and the list of tools.
The model decides. It either answers normally or replies with a tool call: the tool name, arguments as JSON, and a call ID. (It can request several calls at once.)
Your app runs the function. Parse the arguments, validate them, check the user is allowed, then call your real code.
Send the tool result back, linked by the call ID (OpenAI: function_call_output with call_id; Anthropic: a tool_result block with tool_use_id).
The model continues: it writes the final answer using the result, or calls another tool.
Repeat in a loop and you have the core of an agent.

The model asks; your code acts; the model answers.

See it in code

Needs: Nothing for this offline version (plain Python 3). The full loop with a real API is in the Tool Calling lesson of Area 3.

The whole round-trip with the model's reply simulated, so you can see every message shape. The shapes follow the OpenAI Responses API; Anthropic's are equivalent (tool_use and tool_result blocks).

tool_roundtrip.py

import json
 
# 1) Describe the tool: a name, what it does, and a JSON schema for its arguments.
TOOLS = [{
    "type": "function",
    "name": "get_order_status",
    "description": "Look up the delivery status of an order by its ID.",
    "parameters": {
        "type": "object",
        "properties": {"order_id": {"type": "string", "description": "e.g. A1234"}},
        "required": ["order_id"],
        "additionalProperties": False,
    },
}]
 
# 2) Your real code that the tool name maps to.
ORDERS = {"A1234": "out for delivery", "B5678": "delivered"}
def get_order_status(order_id: str) -> str:
    return ORDERS.get(order_id, "order not found")
 
# 3) What a model sends back when it decides to use the tool (shape from the OpenAI Responses API).
model_output = {"type": "function_call", "name": "get_order_status",
                "call_id": "call_1", "arguments": '{"order_id": "A1234"}'}
 
# 4) Your app runs the function (the model never runs code itself)...
args = json.loads(model_output["arguments"])
result = {"get_order_status": get_order_status}[model_output["name"]](**args)
 
# 5) ...and sends the result back so the model can write the final answer.
tool_result = {"type": "function_call_output", "call_id": model_output["call_id"], "output": result}
print("model asked for:", model_output["name"], args)
print("we send back   :", tool_result)

Output

model asked for: get_order_status {'order_id': 'A1234'}
we send back   : {'type': 'function_call_output', 'call_id': 'call_1', 'output': 'out for delivery'}

With the real API you'd pass tools=TOOLS to client.responses.create(...), loop over response.output for items whose type == "function_call", and send tool_result back in the next request's input.

A real example

Example

A travel chatbot that stopped inventing flight times

A travel site's chatbot answered "when does flight BA117 leave?" from its training data and was often wrong. The team added two tools: get_flight_status(flight_number, date) and search_flights(from, to, date), backed by their live flight API.

Now the model calls the tool, gets the real departure time and answers from it. Wrong-time complaints disappeared. They also learned to validate arguments: the model sometimes passed dates like "tomorrow", so the tool schema now requires YYYY-MM-DD and the app returns a clear error result when the format is wrong, which the model then fixes on its own.

The easy way

Start here

Start with one read-only tool that looks something up (orders, weather, a FAQ search). Give it a clear name and description, validate arguments, and return short, plain results. Add write actions (refunds, emails) only later, with human approval.

When you grow: the production setup

Production tool use adds: strict schemas, per-user authorization checks inside every tool, timeouts and retries, idempotency for actions (idempotency keys), audit logs of every call, least-privilege credentials, and evals for tool-call accuracy. Standard protocols like MCP let you share tools across apps.

Trade-offs

What you gain

Gives models live, accurate data and the ability to take real actions.
Keeps execution in your code, where you control permissions and validation.
Reduces hallucination for facts that live in your systems.

What it costs you

Every call adds a round-trip: more latency and tokens.
The model can pick the wrong tool or wrong arguments.
Tools that take actions create real security and safety risk.

When not to use it

The information is static and small: put it in the prompt instead.
The workflow is fixed (always do A then B): call the functions in code yourself instead of letting the model choose.

Common mistakes

Vague tool names and descriptions, so the model can't tell when to use them.
Trusting model-provided arguments without validation and permission checks.
Returning huge raw payloads (whole database rows, HTML pages) instead of short, relevant results.
Crashing on tool errors instead of returning an error result the model can act on.
Giving a tool more power than the feature needs (a "run any SQL" tool).

Interview questions

Try answering out loud first, then open the model answer.

How does function calling work end to end?

Model answer

The app sends the conversation plus tool definitions (name, description, JSON Schema). The model returns either text or one or more tool-call requests with arguments and IDs. The app validates and authorises each call, executes it, and returns results tagged with the matching IDs. The model then produces a final answer or further calls. Running this in a loop until the model stops calling tools is the basic agent pattern.

What are the security risks of tool calling and how do you mitigate them?

Model answer

The model can be manipulated (prompt injection) into calling tools with harmful arguments or leaking data. Mitigate with least-privilege tools, server-side authorization tied to the real user (never the model's claims), strict argument validation, allow-lists, rate limits, sandboxing for code execution, human approval for irreversible actions, and audit logging.

How do you design a good tool for an LLM?

Model answer

One clear purpose per tool, a descriptive name, a description saying when and when not to use it, a minimal strict argument schema with enums and formats, concise structured results, informative error messages the model can act on, idempotent behaviour for actions, and permissions scoped to the calling user. Evaluate tool selection and argument accuracy on realistic cases.

Quiz

Pick an answer to see whether it’s right and why.

0 of 6 answered

Key terms

Tool calling (function calling): A model replying with a request to run one of your functions, with arguments; your app runs it and sends the result back.
Tool schema: The definition of a tool given to a model, with a name, a description and a JSON Schema for its arguments.
Tool result: The output of a tool your app ran, sent back to the model linked to the call's ID so it can continue.
JSON Schema: A standard way to describe the shape of JSON data (fields, types, allowed values), used for structured outputs and tool arguments.
Agent: A system where a model decides, in a loop, which tools to call and what to do next until a task is done.

Before and after

Read first

023 Structured outputs