Area 01 // LLMsLesson 20 / 14015 min read

Prompt engineering basics and system prompts

Prompt engineering is writing instructions a model can't misunderstand, with a clear role, task, context, constraints and output format, and the system prompt is the standing set of instructions that frames every conversation.

On this page

Think of it like this
The problem it solves
How it works: prompt engineering basics
How it works: system prompts
See it in code
A real example
The easy way
Trade-offs
Common mistakes
Interview questions
Quiz
Key terms
Before and after

Think of it like this

Analogy

Imagine briefing a brilliant temp worker on their first day. They're fast and knowledgeable, but they know nothing about your company and will do exactly what you say. A good brief covers: who they are working for, what the job is, the materials (clearly labelled), the rules, what the finished work should look like, and what to do if they're stuck.

That brief is a prompt. The part you pin on the wall for their whole shift is the system prompt.

The problem it solves

Models are very capable but literal and context-free. Vague prompts get vague, inconsistent answers: too long one day, wrong format the next, confidently made-up facts when unsure. Instructions mixed into pasted documents get followed by accident. Prompts copied around a codebase drift apart.

Prompt engineering is the discipline of writing instructions that produce reliable behaviour, and managing them like code.

How it works: prompt engineering basics

Set the context. Who is the model working for, who is the reader, what's the situation? ("You help customers of Acme Bank. Readers are non-technical.")
State the task plainly, as a verb: classify, summarise, extract, draft, answer.
Provide the inputs, clearly separated with delimiters such as XML tags (<document>…</document>) or triple quotes, so data is never confused with instructions.
Give constraints: length, tone, language, what not to do, what sources to use.
Specify the output format exactly (a JSON shape, three bullets, a single word). For machine-readable output, use structured outputs.
Say what to do when unsure: "If the answer isn't in the documents, say you don't know."
Show examples when the format or judgement is subtle (few-shot).
Iterate with evidence: try it on many real inputs, not two favourites, and keep what improves your evals.
Template it. A prompt template keeps the fixed wording in one place and fills in slots in code. Store templates in version control (prompt versioning).

How it works: system prompts

The system prompt is a separate message (role system, or the system / instructions parameter) that frames the whole conversation: identity, rules, tone, tools, format.
Your app writes it; users don't see or change it. User messages carry the specific request.
Models give system instructions higher priority than user messages, which helps keep behaviour on track, though a determined user can still try to override them (prompt injection).
Keep it stable and put it first. Providers cache identical prompt beginnings, so a fixed system prompt is cheaper and faster on repeat requests (prompt caching).
Keep secrets out. Assume a system prompt can leak; never put API keys or private data in it.

What the model actually receives on every turn: system prompt first, then the conversation.

See it in code

Needs: Nothing: plain Python 3. Sending the messages to an API is the same as in lesson 10.

A system prompt plus a user-message template, assembled by code. Note the delimiters around the user's question and the trimming of stray whitespace.

prompt_template.py

from string import Template
 
# The system prompt: who the model is, rules, and output format. Written once.
SYSTEM = """You are the support assistant for Acme Bank.
- Answer only questions about Acme Bank accounts and cards.
- If you are not sure, say so and suggest contacting support.
- Reply in at most 3 short sentences."""
 
# A user-message template: fixed wording with slots filled per request.
USER_TEMPLATE = Template("""Customer tier: $tier
<question>
$question
</question>""")
 
def build_messages(question: str, tier: str) -> list[dict]:
    return [
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": USER_TEMPLATE.substitute(tier=tier, question=question.strip())},
    ]
 
msgs = build_messages("  How do I freeze my card?  ", tier="premium")
for m in msgs:
    print(f"--- {m['role']} ---\n{m['content']}")

Output

--- system ---
You are the support assistant for Acme Bank.
- Answer only questions about Acme Bank accounts and cards.
- If you are not sure, say so and suggest contacting support.
- Reply in at most 3 short sentences.
--- user ---
Customer tier: premium
<question>
How do I freeze my card?
</question>

To send it: with Anthropic, pass SYSTEM as system= and the user message in messages=; with OpenAI's Responses API, pass SYSTEM as instructions= and the user text as input=.

A real example

Example

A one-line change that broke a bank's chatbot

A bank's assistant had the rule "Reply in at most 3 short sentences." A product manager added "Always be thorough and explain fully" to improve answers about mortgages. Mortgage answers got better, but card-freeze answers became five paragraphs long and customers on mobile complained.

The two instructions conflicted, and the model resolved it differently per question. The team rewrote it as one clear rule ("Reply in at most 3 sentences, unless the user asks about mortgages, then up to 8 bullet points"), ran their 150-question eval set, and only then shipped. They now treat every prompt edit like a code change: reviewed, tested, versioned.

The easy way

Start here

Use this six-line skeleton for any prompt, then fill it in:

Context: who you are helping and why.
Task: one verb.
Input: inside tags.
Rules: 3–5 specific bullet points.
Format: exactly what the output looks like.
If unsure: what to do instead of guessing.

When you grow: the production setup

Teams keep prompts as versioned templates in the repo (or a prompt-management tool like Langfuse), review prompt changes like code, run them against a golden dataset in CI (eval gates), A/B test big changes, and log which prompt version produced each answer. Long-lived assistants also manage everything else the model sees, which is context engineering.

Trade-offs

What you gain

The fastest, cheapest way to change model behaviour: no training, instant deploys.
Works with any model, including closed APIs.
System prompts give a stable place for rules, persona and format.

What it costs you

Behaviour can shift in surprising ways from small wording changes.
Long prompts cost tokens on every request (caching helps).
Prompts don't guarantee compliance; users can try to override them.

When not to use it

You need guaranteed formats or exact values: enforce them in code with structured outputs and validation, not just words.
You need hundreds of rules or large knowledge: move knowledge into retrieval and rules into code, keeping the prompt focused.

Common mistakes

Vague instructions like "be concise" or "be professional" with nothing measurable.
Mixing instructions and data without delimiters.
Testing a prompt on two or three hand-picked examples, then shipping.
Copy-pasting slightly different versions of the same prompt across the codebase.
Putting secrets or private data in the system prompt.

Interview questions

Try answering out loud first, then open the model answer.

What makes a good prompt?

Model answer

Clear context and audience, a single explicit task, inputs separated from instructions with delimiters, specific constraints, an exact output format, guidance for uncertain cases, and examples when the judgement is subtle. Then it's validated empirically on a representative eval set rather than a few demos, and maintained as a versioned template.

What's the role of the system prompt, and what shouldn't go in it?

Model answer

It sets persistent behaviour for the conversation: role, scope, rules, tone, tools and format, with higher priority than user turns, and it's ideal for caching because it's stable. It shouldn't contain secrets, credentials or per-user private data, because it can leak through prompt extraction, and it shouldn't be relied on as the only security control.

How do you manage prompts in a production codebase?

Model answer

Store them as templates in version control (or a prompt registry) with IDs and versions; keep variable parts as typed slots filled by code; review changes like code; run eval suites in CI and block regressions; roll out major changes behind flags or A/B tests; and log the prompt version with each request so issues can be traced and rolled back.

Quiz

Pick an answer to see whether it’s right and why.

0 of 6 answered

Key terms

Prompt: The input you give a language model, including instructions, context and the question.
System prompt: Standing instructions set by the app (role, rules, tone, format) that frame a whole conversation and take priority over user messages.
Prompt template: A fixed prompt with named slots (like a question or a customer tier) that code fills in for each request.
Delimiter: Markers such as XML tags or triple quotes that clearly separate pasted data from instructions in a prompt.
Prompt engineering: Writing and testing the instructions, context, examples and format you give a model so it behaves reliably.

Before and after

Read first

010 Calling an LLM API for the first time