Area 03 // AI AgentsLesson 52 / 14015 min read
Context engineering and compaction
Context engineering is deciding exactly what goes into the model's context window at each step (instructions, tools, retrieved knowledge, memory, history, tool results) within a budget, and context compaction is shrinking a long history into a summary so long-running agents keep working without running out of room.
Think of it like this
Analogy
Before a big meeting, a good assistant prepares a briefing pack for their boss: the agenda, the three documents that matter, notes from last time, and the key numbers. Not the entire filing cabinet. During a long day, they replace the morning's pile of notes with a one-page summary so the boss can keep up.
Context engineering is preparing the briefing pack for every step. Compaction is replacing the pile with the one-page summary.
The problem it solves
Long-running agents fill their context fast: tool definitions, search results, file contents, logs, earlier reasoning. Soon the window is full, or so cluttered that the model loses track of the goal, ignores early instructions (lost in the middle), and costs climb with every turn because the whole history is resent.
Prompt engineering (lesson 20) is about writing good instructions. Agents need more: actively managing everything in the window as the task unfolds.
How it works: context engineering
- List every source that could enter the context: system prompt, tool definitions, retrieved knowledge, long-term memory, the plan, recent conversation, tool results, files.
- Set a token budget per step (well under the window) and give each source a priority.
- Include in priority order until the budget is used: rules and goal first, then the latest request, then the most relevant knowledge and results.
- Shape each piece: concise tool results, retrieved chunks with titles, structured state instead of long prose.
- Order for caching: stable content first (system prompt, tools), changing content last, so prompt caching works across turns.
- Fetch just in time: give the agent tools to look things up (search, read file) instead of loading everything up front.
- Offload: write big artefacts (logs, data) to files and keep only paths or summaries in context.
- Isolate: hand noisy sub-tasks (reading 50 files) to a sub-agent that returns a short result.
How it works: context compaction
- Watch the size of the context each turn.
- When it nears a threshold, summarise older turns into a compact note: the goal, constraints, decisions made, what's done, what's left, open questions, important facts and IDs.
- Keep recent turns verbatim, plus anything pinned (the original request, the plan).
- Replace the old turns with the summary and continue.
- Clear stale tool results separately: once a result has been used, its full text rarely needs to stay.
- Use provider features where available. As of November 2026, Anthropic's API offers server-side compaction (a beta that summarises automatically near a threshold) and context editing that clears old tool results; frameworks offer similar helpers.
- Test it: run long tasks and check the agent still respects early constraints after compaction.
See it in code
- Needs
- Nothing: plain Python 3.
Assembling context by priority within a budget, and compacting history into a summary plus recent turns.
def tokens(text: str) -> int:
return max(1, len(text) // 4) # rough estimate: ~4 characters per token
def build_context(parts: list[tuple[int, str, str]], budget: int) -> list[str]:
"""Context engineering: include the most important parts first, stop at the budget."""
chosen, used = [], 0
for priority, name, text in sorted(parts): # 1 = most important
if used + tokens(text) <= budget:
chosen.append(name)
used += tokens(text)
return chosen + [f"({used}/{budget} tokens)"]
parts = [
(1, "system rules", "x" * 400),
(2, "user's latest message", "x" * 200),
(3, "retrieved docs (top 3)", "x" * 1600),
(4, "tool results so far", "x" * 2400),
(5, "old chat history", "x" * 6000),
]
print("fits:", build_context(parts, budget=1500))
def compact(history: list[str], keep_last: int = 2) -> list[str]:
"""Context compaction: replace old turns with a short summary, keep recent turns word for word."""
old, recent = history[:-keep_last], history[-keep_last:]
summary = f"[summary of {len(old)} earlier turns: user is planning a Lisbon trip, budget 1500 GBP]"
return [summary] + recent # a real system asks an LLM to write the summary
history = [f"turn {i}" for i in range(1, 9)]
print("compacted:", compact(history))fits: ['system rules', "user's latest message", 'retrieved docs (top 3)', 'tool results so far', '(1150/1500 tokens)']
compacted: ['[summary of 6 earlier turns: user is planning a Lisbon trip, budget 1500 GBP]', 'turn 7', 'turn 8']The 1,500-token old history didn't fit, so it would be compacted rather than dropped. Real systems count tokens with the provider's tokenizer and have an LLM write the summary with a prompt listing what it must preserve.
A real example
Example
A coding agent on a three-hour refactor
A coding agent refactoring a large codebase hit its context limit after about 40 minutes: file contents and test logs had piled up. After that it started repeating changes it had already made.
The team engineered its context: file reads returned only the relevant functions, test runs returned only failures, old tool results were cleared after use, the plan lived in a TODO.md file the agent updated, and at 70% of the window the history was compacted into a summary of completed steps, decisions and remaining work. The agent finished three-hour tasks reliably, and cost per task fell by about 40% thanks to smaller contexts and better caching.
The easy way
Start here
Three habits cover most of it: keep tool results short; put stable instructions first; and when the history gets long, summarise everything except the last few turns, making sure the summary keeps the goal, constraints and decisions.
When you grow: the production setup
Production harnesses track context composition per step (what took the tokens), use provider compaction and context-editing features, persist plans and notes to files or state, use sub-agents for exploration, test long-horizon tasks specifically for "forgetting", and treat context strategy as a key part of harness engineering.
Trade-offs
What you gain
What it costs you
When not to use it
- Short tasks that comfortably fit in the window: keep it simple and send everything.
Common mistakes
- Treating the context window as free storage and dumping everything in.
- Putting a timestamp or changing data at the top, breaking caching.
- Summaries that lose constraints ("budget 1,500") or decisions.
- Keeping huge old tool outputs long after they were used.
- Never testing behaviour after compaction.
Interview questions
Try answering out loud first, then open the model answer.
What is context engineering?
Model answer
The discipline of curating what enters the model's context at each step of an LLM application or agent: system instructions, tool definitions, retrieved knowledge, memory, plan, recent history and tool results, with priorities and a token budget, in an order that supports caching, using just-in-time retrieval, offloading to files and sub-agent isolation. It's broader than prompt engineering because it manages dynamic state over long tasks.
How does context compaction work and what can go wrong?
Model answer
When the context approaches a threshold, older turns are summarised by an LLM into a compact state description (goal, constraints, decisions, progress, open items) and replaced, keeping recent turns verbatim. Risks: losing critical details or constraints, summarising errors as facts, repeated compaction degrading quality, and cache invalidation. Mitigate with explicit summary templates, pinned content, files for durable notes, and long-horizon evals.
How do sub-agents help with context management?
Model answer
They isolate exploratory or verbose work (searching codebases, reading many documents) in a separate context window, returning only a concise result to the main agent. The main context stays focused and small, which improves quality and cost, at the price of extra calls and the risk that the sub-agent's summary omits something important.
Quiz
Pick an answer to see whether it’s right and why.
0 of 6 answered
Key terms
- Context engineering
- Deciding exactly what goes into the model's context at each step (instructions, tools, knowledge, memory, history, results) within a budget.
- Context compaction
- Replacing a long conversation history with a faithful summary plus recent turns, so a long-running agent stays within its context window.
- Context window
- The maximum number of tokens a model can consider in one request, counting both the input and the generated output.
- Token budget
- A limit on how many tokens a request, step or whole task may use, to control cost, latency and context size.
- Prompt caching
- A provider feature that reuses the processed beginning of a prompt when it repeats exactly, making those tokens much cheaper and faster.
Before and after
Progress is saved in this browser only.