Area 08 of 08

Backend

The backend skills every AI app sits on: HTTP, FastAPI, validation, async, streaming, auth, databases, caching, queues, reliability and testing.

Printable cheat sheet for this area

1. 125

   HTTP and status codes

   HTTP is the request-and-response language that browsers, apps and AI services use to talk to your backend, where each request has a method, path, headers and body, and each response carries a status code that says at a glance whether it worked and, if not, whose fault it was.

2. 126

   REST APIs with FastAPI

   A REST API exposes your app's data and actions as resources at URLs that clients use with standard HTTP methods, and FastAPI is a popular Python framework for building them quickly, with request validation and interactive OpenAPI documentation generated from your code.

   CoversREST APIs · Building an API with FastAPI · API documentation (OpenAPI)

3. 127

   Input validation with Pydantic, and config

   Pydantic lets you describe the exact shape and rules of data with Python classes, so every request is checked and cleaned before your code runs, and Pydantic Settings applies the same idea to configuration loaded from environment variables.

   CoversInput Validation with Pydantic · Environment variables and config

4. 128

   Async and await

   Async code lets one Python process handle many slow network operations at once, such as LLM calls, database queries and web requests, by switching to other work while each one waits, which makes it ideal for AI backends that spend most of their time waiting on APIs.

5. 129

   Streaming with SSE, and WebSockets

   Server-sent events (SSE) keep one HTTP response open so the server can push tokens to the browser as the model writes them, which is how most chat apps stream answers, while WebSockets open a two-way connection for when the client also needs to send messages during the stream.

   CoversStreaming with SSE · WebSockets

6. 130

   Authentication with JWT and OAuth 2.0

   Authentication proves who is calling your API; a JWT is a signed token that carries the user's identity and permissions so any server can verify it without a database lookup, and OAuth 2.0 is the standard way to let users sign in with another provider or grant an app limited access to their accounts.

   CoversAuthentication with JWT · OAuth 2.0

7. 131

   CORS

   CORS (cross-origin resource sharing) is the browser rule that decides whether a web page from one origin may read responses from an API on another origin, and your API opts in by sending headers that list exactly which origins, methods and headers it allows.

8. 132

   SQL vs NoSQL, indexing and transactions

   Databases store your app's lasting data (users, conversations, documents, usage), where SQL databases keep structured tables with relations and strong guarantees, NoSQL stores trade some of that for flexible or massive-scale access, indexes make lookups fast, and transactions make groups of changes all-or-nothing.

   CoversSQL vs NoSQL · Database Indexing · Transactions and ACID

9. 133

   Caching with Redis

   Redis is a very fast in-memory data store, and caching with it means saving the results of slow or expensive work (LLM answers, embeddings, database queries) under a key with an expiry time, so repeated requests are served in milliseconds without redoing the work.

10. 134

    Message queues and background jobs

    A message queue holds tasks that one part of your system produces and another part processes later, so slow or failure-prone work (document ingestion, long agent runs, emails) moves out of web requests into background workers that retry failures and scale independently.

    CoversMessage Queues · Background Jobs

11. 135

    Rate limiting

    Rate limiting caps how many requests (or tokens) each user, API key or tenant can make in a time window, protecting your API, your budget and your upstream LLM quota, with algorithms such as fixed windows, sliding windows and token buckets, and a clear 429 response when the limit is hit.

12. 136

    Retries with backoff and idempotency keys

    Retries with exponential backoff and jitter let a client recover from temporary failures such as rate limits and overloaded servers without making things worse, and idempotency keys make it safe to retry operations with side effects (payments, emails, tool actions) because repeats return the original result instead of doing the work again.

    CoversRetries with Backoff · Idempotency Keys

13. 137

    File uploads to S3

    AI apps receive documents, images and audio from users, and the robust way to handle them is to validate the request, let the browser upload straight to object storage such as Amazon S3 using a short-lived presigned URL, and then process the file in a background job triggered by the upload.

14. 138

    Pagination

    Pagination returns long lists (conversations, documents, search results, logs) a page at a time, either with page numbers and offsets, which are simple but slow and unstable on large or changing data, or with cursors that say "continue after this item", which are fast and consistent.

15. 139

    Logging and monitoring

    Logging records what your backend did, ideally as structured JSON lines with a request ID that ties together everything one request touched, and monitoring turns logs, metrics and traces into dashboards and alerts so you notice slow, failing or expensive behaviour before users report it.

16. 140

    Testing APIs with pytest

    Testing an AI backend with pytest means calling your endpoints in-process with a test client, replacing the real LLM with a fake through dependency injection so tests are fast, free and repeatable, and checking status codes, response shapes, prompts sent and edge cases, while evals cover answer quality separately.