Langfuse and observability dashboards

How it works: Langfuse

1. Get a project: sign up for Langfuse Cloud, or self-host (open source; MIT licence except the enterprise ee folders; a Docker Compose setup for small deployments). Create API keys.
2. Set environment variables: LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_BASE_URL.
3. Instrument with the Python SDK (v5 as of November 2026, built on OpenTelemetry): @observe() on functions creates spans; @observe(as_type="generation") marks model calls; integrations for OpenAI, Anthropic, LangChain/LangGraph and others capture model calls automatically.
4. Add context: propagate_attributes(user_id=..., session_id=..., metadata=...) so you can filter by user, conversation and version.
5. Attach scores: eval results and user feedback via score_current_trace(...) or create_score(...) (numeric, categorical or boolean).
6. Use the platform:
   • Tracing: view each trace as a tree, with inputs, outputs, tokens, cost and latency per step; group by sessions.
   • Prompt management: store versioned prompts, fetch them at runtime, and see which version produced each trace (prompt versioning).
   • Datasets and experiments: build golden datasets from traces and run evals against them.
   • LLM-as-a-judge evaluators: configured in the UI to score traces automatically.
   • Annotation queues: for human labeling.
   • Dashboards: cost, latency, volume and scores over time.
7. Flush at the end of short scripts (get_client().flush()) so buffered data is sent.

How it works: observability dashboards

Observability dashboards turn traces into the few numbers you watch daily:

1. Traffic: requests per minute or day, per feature.
2. Reliability: error rate, timeouts, retries, fallback rate, guardrail blocks.
3. Latency: p50, p95, p99 end to end, time to first token, per step (P95 latency).
4. Cost: tokens and money per day, per model, per feature, per user or tenant (cost per request).
5. Quality: judge scores, user feedback rate, escalation rate, per prompt version.
6. Slices: filter all of the above by version, model, language, customer.
7. Alerts: thresholds on error rate, p95, cost per hour and quality scores (alerts and dashboards).

import anthropic
from langfuse import get_client, observe, propagate_attributes
 
client = anthropic.Anthropic()
 
@observe()                                         # a span for the retrieval step
def retrieve(question: str) -> list[str]:
    return ["Full-time employees get 25 days of annual leave."]
 
@observe(as_type="generation")                     # marks this span as a model call
def answer(question: str, sources: list[str]) -> str:
    msg = client.messages.create(
        model="claude-haiku-5-5", max_tokens=300,
        system="Answer only from the sources.",
        messages=[{"role": "user", "content": f"{sources}\n\nQuestion: {question}"}])
    return msg.content[0].text
 
@observe()                                         # the root span = the whole request
def handle(question: str, user_id: str, session_id: str) -> str:
    with propagate_attributes(user_id=user_id, session_id=session_id, metadata={"prompt": "v12"}):
        text = answer(question, retrieve(question))
        get_client().score_current_trace(name="has_citation", value=0)   # e.g. from a quick check
        return text
 
print(handle("How many leave days do I get?", user_id="u42", session_id="s-1001"))
get_client().flush()                               # send buffered spans before the script exits

Full-time employees get 25 days of annual leave.

On the real platform

What this looks like in Langfuse · See the real screen in the official docs

1234

1. 1Trace summary: latency, cost, tokens, user and prompt version.
2. 2The span tree: rewrite, retrieve, rerank, generate, guardrail.
3. 3The selected generation: model, tokens, cost, full input and output.
4. 4Scores attached to the trace: evals and user feedback.

The Langfuse web app (cloud or self-hosted) is organised the same way in both:

1. Tracing → Traces: a table of requests with latency, cost, tokens, user and scores. Filter by user, session, tag, version or score.
2. Click a trace: the span tree on the left (root request, retrieval, generation), the selected span's input, output, model, tokens and cost on the right.
3. Sessions: all traces of one conversation in order.
4. Prompts: versioned prompts with labels such as production; each trace links to the version used.
5. Datasets / Experiments: golden sets built from traces and the results of eval runs.
6. Dashboards: cost, latency and score charts over time, sliceable by model, user and version.