Area 03 // AI AgentsLesson 63 / 14016 min read
Coding agents and computer-use agents
Coding agents read, edit and run code in a repository using file, search and shell tools, verifying their work with tests, while computer-use and browser agents operate graphical interfaces by looking at screenshots and clicking and typing like a person.
Think of it like this
Analogy
A coding agent is like a contractor who's been given the keys to your workshop: they read the plans (code), use the tools (editor, terminal), test their work (measure twice), and leave a note of what they changed (a pull request) for you to inspect.
A computer-use agent is like a remote assistant who can only see your screen through a video call and control your mouse and keyboard. Slower and clumsier than someone with direct access, but they can operate any app a person can.
The problem it solves
Some of the most valuable agent work is in software itself (fixing bugs, writing tests, migrations, upgrades) and in apps that have no API (old internal systems, third-party websites, desktop software). Both need agents with powerful tools: running shell commands, editing files, clicking buttons. Powerful tools mean real risk, and AI-written code that nobody checks can quietly pile up bugs and security holes.
How it works: coding agents
- Tools: read files, search the codebase (grep, glob), edit files (exact string replacement or patches), run shell commands (build, tests, linters), and git. Many coding agents also use subagents for exploration and review.
- Instructions: the task plus the project's instruction file (
AGENTS.md,CLAUDE.md) with build and test commands and rules (harness engineering). - The loop: explore the relevant code → plan → make a small edit → run the tests → read failures → fix → repeat → summarise.
- Verification: tests, type checks and linters are the agent's ground truth; hooks can enforce "tests must pass before done".
- Output: a branch and pull request with a clear description, reviewed by a human and checked by CI.
- Run it safely: in a container or VM, with only the repository mounted, limited network, no production secrets (tool sandboxing).
- Products (as of November 2026) include terminal and IDE agents (Claude Code, OpenAI Codex, GitHub Copilot's agent mode, Cursor and others) and cloud agents that work on issues and open PRs.
Vibe coding vs agentic engineering
- Vibe coding (a term popularised in 2025): describing what you want, accepting whatever the AI writes without really reading it, and iterating on whether it seems to work. Great for throwaway prototypes and personal scripts.
- The risk: code nobody understands, missing edge cases, security holes, and growing technical debt.
- Agentic engineering: using agents seriously for production code. You write a clear spec, give the agent a good harness, insist on tests (often written first), keep changes small, review every diff, and let CI and evals gate merges.
- The engineer's role shifts from typing every line to specifying, reviewing, testing and designing the loops, but responsibility for the code stays with people.
How it works: computer-use and browser agents
- A computer-use agent gets a screenshot of the screen (in a virtual machine or container).
- The model decides an action: click at (x, y), type text, press keys, scroll, or take another screenshot. Providers offer dedicated computer-use tools for this (as of November 2026, for example Anthropic's computer-use toolset and OpenAI's computer-use support).
- The harness performs the action in the VM and returns a new screenshot.
- Repeat until the task is done or a limit is hit.
- Browser agents do the same in a web browser, often also reading the page structure (DOM or accessibility tree), which makes clicks more reliable than pixels alone; tools like Playwright drive the browser.
- Be careful: web pages can contain indirect prompt injection; keep agents away from logged-in accounts and payments unless a human approves each sensitive step.
- Prefer APIs when they exist: UI automation is slower, costlier and breaks when layouts change.
See it in code
- Install
pip install pytest- Needs
- Python 3.10+. The 'model' edit is scripted; the tests really run.
The core of a coding agent: run the tests, read the failure, edit, run again.
import re, subprocess, sys, tempfile, textwrap
from pathlib import Path
# The core loop of a coding agent: change code, run the tests, read the failure, try again.
with tempfile.TemporaryDirectory() as repo:
src, tests = Path(repo, "price.py"), Path(repo, "test_price.py")
src.write_text("def with_vat(net):\n return net * 1.02\n") # the bug: VAT is 20%
tests.write_text(textwrap.dedent("""
from price import with_vat
def test_vat():
assert round(with_vat(100), 2) == 120.0
"""))
def run_tests() -> tuple[bool, str]: # a tool the agent can call
r = subprocess.run([sys.executable, "-m", "pytest", "-q", "-p", "no:cacheprovider"],
cwd=repo, capture_output=True, text=True)
summary = re.search(r"\d+ (passed|failed)", r.stdout).group(0) # e.g. "1 failed"
return r.returncode == 0, summary
ok, summary = run_tests()
print("before fix:", summary)
# The model reads the failure ("assert 102.0 == 120.0") and edits the file (an edit tool call).
src.write_text(src.read_text().replace("1.02", "1.20"))
ok, summary = run_tests()
print("after fix :", summary, "-> ready for review" if ok else "")before fix: 1 failed
after fix : 1 passed -> ready for reviewA real coding agent wraps exactly this in the agent loop with file, search and shell tools, inside a sandbox, and stops at a pull request.
A real example
Example
Two ways to use the same coding agent
Two developers at the same company used the same coding agent.
The first vibe-coded an internal dashboard in an afternoon: described it, accepted every change, shipped it. It worked, until a security review found it exposed salary data to every employee because the agent had skipped an authorisation check nobody had specified or tested.
The second worked agentically on a payments service: wrote a short spec and acceptance tests first, let the agent implement in small steps in a sandbox with the repo's AGENTS.md, reviewed each diff, and let CI (tests, linters, security scan) gate the merge. Slower than vibing, but the change shipped without incidents, and was still about three times faster than writing it by hand.
The easy way
Start here
Try a coding agent on a small, well-tested repo: ask it to fix one failing test or add one small feature, read every change it makes, and run the tests yourself. For computer use, try a browser agent on a harmless public site (filling a demo form) inside a sandboxed browser, never with your real accounts.
When you grow: the production setup
Teams run coding agents in disposable containers with least-privilege credentials, enforce tests and linters through hooks and CI, require human review on every PR, measure merge rates and defect rates of agent PRs, and grow into loops for routine work. Computer-use agents run in isolated VMs with allow-listed sites, no stored credentials, human approval for sensitive actions, and full screen recordings for audit.
Trade-offs
What you gain
- Coding agents dramatically speed up well-specified, testable work.
- Computer use automates apps with no API.
- Tests and CI give coding agents an objective way to check themselves.
What it costs you
- Powerful tools (shell, clicks) carry real security risk.
- Unreviewed agent code accumulates bugs and debt (vibe coding).
- Computer use is slow, costly and brittle compared with APIs.
When not to use it
- No tests or way to verify changes: the agent can't check itself and you can't trust it.
- A stable API exists: don't automate the UI.
- Production systems without a sandbox, review and rollback.
Common mistakes
- Running coding agents on your laptop with all your credentials and full network access.
- Shipping AI-written code nobody has read.
- Letting agents work for hours without running the tests.
- Using computer use where an API would do.
- Logging browser agents into real accounts with payment details.
Interview questions
Try answering out loud first, then open the model answer.
How do coding agents work and how do you make them reliable?
Model answer
They run the agent loop with file read/search, edit, shell and git tools, guided by the task and repo instruction files: explore, plan, make small edits, run tests and linters, read failures, iterate, and produce a PR. Reliability comes from the harness: clear specs and AGENTS.md, good tests as ground truth, hooks enforcing verification, sandboxed execution, small changes, context management for long tasks, and human review plus CI gating.
What are the risks of computer-use agents and how do you mitigate them?
Model answer
They can click and type anything, so mistakes or prompt injection from page content can trigger harmful actions, leak data or make purchases; they're also slow and brittle. Mitigate by running in isolated VMs or containers, restricting network to allow-listed sites, avoiding stored credentials, requiring human approval for sensitive steps, capping steps, recording sessions, and preferring APIs whenever available.
Vibe coding vs agentic engineering: when is each appropriate?
Model answer
Vibe coding (accepting AI output without careful review) is fine for prototypes, demos and personal throwaway scripts where failure is cheap. Production work calls for agentic engineering: explicit specs and constraints, tests first, small reviewed diffs, sandboxed agents with good harnesses, and CI and eval gates, keeping humans accountable for what ships.
Quiz
Pick an answer to see whether it’s right and why.
0 of 6 answered
Key terms
- Coding agent
- An agent that reads, edits and runs code in a repository using file, search and shell tools, verifying its work with tests.
- Computer-use agent
- An agent that operates a computer's graphical interface by looking at screenshots and choosing clicks, typing and scrolling.
- Browser agent
- An agent that operates a web browser, often using the page structure as well as screenshots.
- Vibe coding
- Building software by accepting AI-written code without really reading or testing it. Fine for throwaway prototypes, risky in production.
- Agentic engineering
- Using AI agents for production software with engineering discipline, including clear specs, tests, constraints, sandboxes and human review.
- Sandbox
- An isolated environment (container, VM, restricted process) where code or agents run with limited access, so mistakes and attacks stay contained.
Before and after
Progress is saved in this browser only.