Serverless functions and the easy hosting route

How it works: serverless functions

1. You write a handler: a function that receives an event (an HTTP request, a queue message, a file upload) and returns a response.
2. The platform runs it on demand: it starts instances when requests arrive, runs many in parallel, and scales to zero when idle. You pay per invocation and duration (examples: AWS Lambda, Google Cloud Run functions, Azure Functions, Vercel and Cloudflare functions).
3. Cold starts: a new instance needs time to start and import libraries; heavy imports (like ML libraries) make it worse. Keep packages small and create clients outside the handler so warm instances reuse them.
4. Limits: maximum run time (minutes, not hours), memory, payload size, concurrent instances, and sometimes limited streaming support; check each platform's current limits.
5. No local state: keep conversation history, files and caches in external services (databases, Redis, object storage).
6. Long work goes async: the function enqueues a job and returns an ID (batch and async).

How it works: Hugging Face Spaces and managed platforms

1. Managed container platforms (Google Cloud Run, AWS App Runner, Azure Container Apps, Fly.io, Render, Railway and others) take your Docker image or repo, give you HTTPS, autoscaling (often to zero), logs and secrets settings.
2. Hugging Face Spaces host ML demos from a git repo: pick Gradio, Streamlit or Docker, push, and get a public URL; a free CPU tier suits demos, with paid upgrades for GPUs and persistent storage.
3. Static and frontend hosts (Vercel, Netlify, Cloudflare Pages) serve your web UI and often include serverless functions for the API.
4. Configure secrets in the platform's settings, never in the repo (secrets).
5. Know when to graduate: very high steady traffic, GPUs for self-hosted models (GPU instances), special networking or long-running workers may justify Kubernetes or dedicated servers.

import json
 
# A serverless function is just a handler: the platform calls it with an event and returns its response.
# This shape (event in, dict with statusCode/body out) is used by AWS Lambda behind an HTTP endpoint;
# other platforms are similar. Keep clients outside the handler so warm invocations reuse them.
INVOCATIONS = {"cold_starts": 0}
CLIENT = None
 
def get_client():
    global CLIENT
    if CLIENT is None:                     # only on a cold start (a fresh container)
        INVOCATIONS["cold_starts"] += 1
        CLIENT = lambda q: f"(model answer to: {q})"   # e.g. anthropic.Anthropic() in real code
    return CLIENT
 
def handler(event, context=None):
    body = json.loads(event.get("body") or "{}")
    question = body.get("question", "").strip()
    if len(question) < 3:
        return {"statusCode": 400, "body": json.dumps({"error": "question too short"})}
    answer = get_client()(question)
    return {"statusCode": 200, "headers": {"Content-Type": "application/json"},
            "body": json.dumps({"answer": answer})}
 
for q in ["What is RAG?", "hi", "What is an embedding?"]:
    r = handler({"body": json.dumps({"question": q})})
    print(r["statusCode"], r["body"])
print("cold starts:", INVOCATIONS["cold_starts"], "for 3 invocations")

200 {"answer": "(model answer to: What is RAG?)"}
400 {"error": "question too short"}
200 {"answer": "(model answer to: What is an embedding?)"}
cold starts: 1 for 3 invocations