Skip to content
← All cheat sheets

Cheat sheet · Area 08

Backend

HTTP and status codes

HTTP is the request-and-response language that browsers, apps and AI services use to talk to your backend, where each request has a method, path, headers and body, and each response carries a status code that says at a glance whether it worked and, if not, whose fault it was.

  • Request = method (GET, POST, PUT, PATCH, DELETE) + path + headers + optional body.
  • 2xx success (200 OK, 201 Created, 204 No Content). 3xx redirect.
  • 4xx client's fault: 400 bad request, 401 not authenticated, 403 not allowed, 404 not found, 422 invalid data, 429 too many requests.
  • 5xx server's fault: 500 bug, 502/503/504 upstream or overloaded. Retry 429 and 5xx (with backoff), not 4xx.
  • LLM APIs use the same codes: 429 rate limit, 529/503 overloaded, 400 bad request.

REST APIs with FastAPI

A REST API exposes your app's data and actions as resources at URLs that clients use with standard HTTP methods, and FastAPI is a popular Python framework for building them quickly, with request validation and interactive OpenAPI documentation generated from your code.

  • REST: resources as nouns in URLs (/documents/42), actions as HTTP methods (GET, POST, PATCH, DELETE).
  • CRUD = Create (POST), Read (GET), Update (PUT/PATCH), Delete (DELETE).
  • FastAPI: decorate functions with @app.get/@app.post; type hints = validation + docs.
  • OpenAPI spec at /openapi.json, interactive docs at /docs, generated automatically.
  • Version your API (/v1/...) and keep responses consistent.

Input validation with Pydantic, and config

Pydantic lets you describe the exact shape and rules of data with Python classes, so every request is checked and cleaned before your code runs, and Pydantic Settings applies the same idea to configuration loaded from environment variables.

  • class ChatRequest(BaseModel): message: str = Field(min_length=1, max_length=4000)
  • FastAPI validates request bodies with your model; failures return 422 with a list of errors.
  • Use Literal for allowed values, Field(ge=, le=) for ranges, field_validator for custom rules.
  • Validate at the boundary: requests, LLM outputs, tool arguments, config.
  • BaseSettings reads typed config from env vars (and .env) and fails fast on bad values.

Async and await

Async code lets one Python process handle many slow network operations at once, such as LLM calls, database queries and web requests, by switching to other work while each one waits, which makes it ideal for AI backends that spend most of their time waiting on APIs.

  • async def makes a coroutine; await pauses it until the result is ready, letting other tasks run.
  • asyncio.gather runs many awaits at once: 5 × 0.5s calls take ~0.5s, not 2.5s.
  • Limit concurrency with asyncio.Semaphore to respect rate limits.
  • Never call blocking code (time.sleep, requests, sync SDK calls) inside async def; use async clients or asyncio.to_thread.
  • Async helps I/O-bound work (waiting); it doesn't speed up CPU-heavy work.

Streaming with SSE, and WebSockets

Server-sent events (SSE) keep one HTTP response open so the server can push tokens to the browser as the model writes them, which is how most chat apps stream answers, while WebSockets open a two-way connection for when the client also needs to send messages during the stream.

  • SSE: one HTTP response, Content-Type text/event-stream, messages as 'data: ...' followed by a blank line.
  • SSE is one-way (server → client), works through most proxies, auto-reconnects in browsers (EventSource).
  • WebSocket: a two-way, long-lived connection; good for voice, live collaboration, 'stop' and interrupts.
  • Most LLM chat UIs: POST the question, stream the answer over SSE.
  • Set proxy timeouts and disable buffering; handle disconnects (cancel the LLM call).

Authentication with JWT and OAuth 2.0

Authentication proves who is calling your API; a JWT is a signed token that carries the user's identity and permissions so any server can verify it without a database lookup, and OAuth 2.0 is the standard way to let users sign in with another provider or grant an app limited access to their accounts.

  • Authentication = who are you? Authorization = what may you do (checked per action)?
  • JWT = header.payload.signature (base64). Signed, NOT encrypted: never put secrets in it.
  • Always verify signature, expiry (exp), issuer and audience; pin the algorithm.
  • Short-lived access tokens (minutes) + refresh tokens; send it in the Authorization header as a Bearer token.
  • OAuth 2.0 authorization code flow + PKCE for 'Sign in with...' and delegated access with scopes.

CORS

CORS (cross-origin resource sharing) is the browser rule that decides whether a web page from one origin may read responses from an API on another origin, and your API opts in by sending headers that list exactly which origins, methods and headers it allows.

  • Origin = scheme + host + port. https://app.example.com and https://api.example.com are different origins.
  • Browsers block pages from reading cross-origin responses unless the API sends Access-Control-Allow-Origin.
  • Non-simple requests (JSON bodies, Authorization header) send an OPTIONS preflight first.
  • Allow exact origins; never '*' with credentials. CORS is not authentication.
  • CORS only affects browsers; scripts and servers ignore it.

SQL vs NoSQL, indexing and transactions

Databases store your app's lasting data (users, conversations, documents, usage), where SQL databases keep structured tables with relations and strong guarantees, NoSQL stores trade some of that for flexible or massive-scale access, indexes make lookups fast, and transactions make groups of changes all-or-nothing.

  • SQL (Postgres, MySQL, SQLite): tables, joins, constraints, transactions. Default choice.
  • NoSQL: document (MongoDB), key-value (Redis, DynamoDB), wide-column, graph. Pick for a specific access pattern.
  • Postgres + pgvector covers most AI apps: users, chats, usage AND vectors in one place.
  • Index the columns you filter and sort by; check with EXPLAIN. Indexes speed reads, slow writes.
  • Transactions (ACID): several writes succeed together or not at all.

Caching with Redis

Redis is a very fast in-memory data store, and caching with it means saving the results of slow or expensive work (LLM answers, embeddings, database queries) under a key with an expiry time, so repeated requests are served in milliseconds without redoing the work.

  • Cache-aside: look in the cache → hit: return it → miss: do the work, store it with a TTL, return it.
  • Key = everything that changes the result: model + prompt version + normalised input (+ tenant).
  • Always set a TTL; invalidate when the source data changes.
  • Great to cache in AI apps: embeddings, exact repeated prompts, retrieval results, tool and API responses.
  • Redis also does rate-limit counters, sessions, queues and locks.

Message queues and background jobs

A message queue holds tasks that one part of your system produces and another part processes later, so slow or failure-prone work (document ingestion, long agent runs, emails) moves out of web requests into background workers that retry failures and scale independently.

  • Producer puts a job on a queue → returns quickly (202 + job ID) → worker picks it up → saves result → status/notify.
  • Tools: Redis (RQ, Celery, arq), RabbitMQ, SQS, Kafka (event streams), workflow engines (Temporal) for long multi-step jobs.
  • Retries with backoff for transient errors; dead-letter queue after N failures.
  • Most queues deliver at least once: make jobs idempotent.
  • Scale workers on queue depth; cap concurrency to respect LLM rate limits.

Rate limiting

Rate limiting caps how many requests (or tokens) each user, API key or tenant can make in a time window, protecting your API, your budget and your upstream LLM quota, with algorithms such as fixed windows, sliding windows and token buckets, and a clear 429 response when the limit is hit.

  • Limit per user / API key / tenant (not just per IP), on requests AND tokens or cost.
  • Fixed window: simple, but allows double bursts at window edges. Sliding window: smoother. Token bucket: allows short bursts.
  • Store counters in Redis so all servers share them.
  • Over the limit → 429 Too Many Requests + Retry-After (+ RateLimit headers).
  • Layer it: edge/CDN limits, API gateway, per-route limits, per-tenant token budgets.

Retries with backoff and idempotency keys

Retries with exponential backoff and jitter let a client recover from temporary failures such as rate limits and overloaded servers without making things worse, and idempotency keys make it safe to retry operations with side effects (payments, emails, tool actions) because repeats return the original result instead of doing the work again.

  • Retry only transient errors: timeouts, 429, 5xx/overloaded. Never 400/401/403/422.
  • Exponential backoff: wait 1s, 2s, 4s, 8s... capped; add random jitter; respect Retry-After.
  • Limit attempts and total time; then fail over or fail clearly.
  • Idempotency key: client sends a unique key; server stores the result; repeats get the stored result.
  • Agents: make tool actions idempotent so a retried step can't refund or email twice.

File uploads to S3

AI apps receive documents, images and audio from users, and the robust way to handle them is to validate the request, let the browser upload straight to object storage such as Amazon S3 using a short-lived presigned URL, and then process the file in a background job triggered by the upload.

  • Object storage (S3, GCS, Azure Blob, R2): cheap, durable storage for files, addressed by bucket + key.
  • Presigned URL: a short-lived signed URL that lets the browser upload one file directly, no keys exposed.
  • Flow: API validates (type, size, user) → presigned URL → browser uploads → event → background job ingests.
  • Private buckets, user-scoped keys, size and type limits, malware scanning, never trust the filename.
  • Large files: multipart uploads. Keep originals; store derived data (text, chunks) separately.

Pagination

Pagination returns long lists (conversations, documents, search results, logs) a page at a time, either with page numbers and offsets, which are simple but slow and unstable on large or changing data, or with cursors that say "continue after this item", which are fast and consistent.

  • Never return unbounded lists; set a default and maximum page size.
  • Offset (LIMIT 10 OFFSET 20): simple, supports 'jump to page 5', but slow deep down and shifts when rows are added.
  • Cursor/keyset (WHERE id < last_id ORDER BY id DESC LIMIT 10): fast and stable; return next_cursor.
  • Sort by a unique, indexed key (or a tie-breaker like (created_at, id)).
  • Make cursors opaque (encode them) so clients don't depend on their contents.

Logging and monitoring

Logging records what your backend did, ideally as structured JSON lines with a request ID that ties together everything one request touched, and monitoring turns logs, metrics and traces into dashboards and alerts so you notice slow, failing or expensive behaviour before users report it.

  • Structured logs: one JSON object per line with level, message, request_id and useful fields.
  • Levels: DEBUG (dev only), INFO (normal events), WARNING (odd but handled), ERROR (failed), CRITICAL.
  • One request ID per request, passed to every log line, downstream call and trace.
  • Log for AI: model, prompt version, tokens, latency, cost, cache hits, tool calls, guardrail results.
  • Never log secrets; redact personal data; set retention.

Testing APIs with pytest

Testing an AI backend with pytest means calling your endpoints in-process with a test client, replacing the real LLM with a fake through dependency injection so tests are fast, free and repeatable, and checking status codes, response shapes, prompts sent and edge cases, while evals cover answer quality separately.

  • pytest: functions named test_*, plain assert, fixtures for setup, parametrize for many cases.
  • FastAPI TestClient calls the app in-process: no server needed.
  • Inject the LLM client with Depends; override it in tests with a fake (app.dependency_overrides).
  • Test behaviour: status codes, response shape, validation errors, prompts sent, tool calls, error paths.
  • Unit tests (fast, fake LLM) on every push; evals (real model, quality) as a separate gate.