Prompt versioning, model versioning and registries

How it works: prompt versioning

1. Treat prompts as artifacts: the template, the model it's meant for, parameters (max_tokens, tools, output schema) and few-shot examples together form one version.
2. Store them in git (reviewed in pull requests: Git and GitHub) or in a prompt management tool (e.g. Langfuse) that keeps every version.
3. Make versions immutable: a version number or content hash; changes create new versions (see the code).
4. Use labels (production, staging) that point at versions; the app fetches by label (with caching and a fallback), so releasing or rolling back is moving a label.
5. Evaluate before promoting: an eval gate runs on the new version; record its scores with the version.
6. Log the version on every trace (tracing).

How it works: model versioning

1. Pin exact model IDs or dated snapshots in config; treat aliases that move (like "latest") as risky in production.
2. Track deprecations: providers retire models; plan migrations with evals before deadlines.
3. Version your own models (fine-tunes, embedding models, rerankers): weights, training data version, code commit, hyperparameters and eval results (LoRA).
4. Embedding models are special: changing one means re-embedding the whole index; version the index together with the model.
5. Release model changes like code: eval gate, canary, monitoring (release strategies).

How it works: model registries and MLOps

1. A model registry stores model versions with metadata and lineage, plus stage labels or aliases (e.g. MLflow Model Registry, cloud ML platforms, or private Hugging Face Hub repos with revisions).
2. MLOps is the practice around it: reproducible training pipelines, data and model versioning, automated evaluation, deployment, monitoring for drift, and retraining.
3. The flow: train or fine-tune → register version with metrics → evaluate → promote label to staging → canary → production; keep previous versions for rollback.
4. For API-model apps, the "registry" may simply be versioned config: prompt versions, model IDs and parameters in git, with releases tagged.

import hashlib, json
 
# A tiny prompt + model registry: every version is immutable and identified by a content hash;
# "labels" (production, staging) are movable pointers, so rollback = move the label back.
REGISTRY = {"versions": {}, "labels": {}}
 
def register(name: str, template: str, model: str, params: dict) -> str:
    body = {"template": template, "model": model, "params": params}
    vid = hashlib.sha256(json.dumps(body, sort_keys=True).encode()).hexdigest()[:8]
    REGISTRY["versions"][f"{name}@{vid}"] = body
    return f"{name}@{vid}"
 
def promote(label: str, version: str):
    REGISTRY["labels"][label] = version
    print(f"{label:10} -> {version}")
 
def get(label: str) -> dict:
    return REGISTRY["versions"][REGISTRY["labels"][label]]
 
v1 = register("support-answer", "Answer using the sources. Cite like [1].\n{sources}\nQ: {question}",
              "claude-haiku-5-5", {"max_tokens": 500})
v2 = register("support-answer", "Answer ONLY from the sources, cite every sentence like [1]. "
              "If unsure, say so.\n{sources}\nQ: {question}", "claude-sonnet-5-5", {"max_tokens": 600})
promote("production", v1)
promote("staging", v2)
promote("production", v2)          # after the eval gate passed on staging
promote("production", v1)          # rollback: just move the label, nothing is redeployed
print("serving:", get("production")["model"], "| versions stored:", len(REGISTRY["versions"]))

production -> support-answer@4cb6248e
staging    -> support-answer@a78a4bef
production -> support-answer@a78a4bef
production -> support-answer@4cb6248e
serving: claude-haiku-5-5 | versions stored: 2