Prompt caching and token pricing

How it works: token pricing

1. Providers price per million tokens, separately for input tokens (everything you send) and output tokens (everything generated, including hidden reasoning tokens).
2. Output costs more, typically 4–5× input, because generation is sequential.
3. Cost per request = input tokens ÷ 1M × input price + output tokens ÷ 1M × output price.
4. Prices vary 20–100× across model tiers. As of November 2026 for example: Claude Sonnet 5.5 and GPT-6.1 Sol are $2 / $10 per 1M; Claude Haiku 5.5 and GPT-6 Luna are $0.10 / $0.50; top models cost more (Claude Opus 5.5 $4 / $20, GPT-6 Astra $10 / $50). Check the providers’ pricing pages before you budget.
5. Other price modifiers: long-context surcharges on some models, cached-input discounts, batch API discounts (~50%), and premium "fast" modes.
6. Read your real usage from each response (usage.input_tokens, usage.output_tokens, cached token counts) and log it per feature.

How it works: prompt caching

1. Many requests start the same way: system prompt, tool definitions, a long document, the earlier part of a conversation.
2. The provider stores the processed prefix (its KV cache) for a short time after a request.
3. A later request with an identical prefix is a cache hit: those tokens are billed at a discount (roughly 10% of the input price at OpenAI and Anthropic as of November 2026) and the response starts faster.
4. It's an exact prefix match. One changed character early on (a timestamp, a user name, reordered tools) breaks everything after it. Put stable content first and changing content last.
5. Provider differences (as of November 2026): OpenAI caches long prefixes automatically. Anthropic uses cache_control markers (or a top-level automatic setting); writing to the cache costs a little more than normal input (about 1.25×) and the default lifetime is about 5 minutes, with a longer option.
6. Check it works: look at cached-token counts in the usage data (cache_read_input_tokens for Anthropic, cached_tokens details for OpenAI). Zero hits usually means something in your prefix changes.

# Cost of a support bot per month, with and without prompt caching.
# Prices: $ per 1M tokens, example mid-tier model (as of Nov 2026: $2 in, $10 out;
# cached reads ~10% of the input price, cache writes ~125%). Check your provider.
PRICE_IN, PRICE_OUT = 2.00, 10.00
CACHE_READ, CACHE_WRITE = PRICE_IN * 0.10, PRICE_IN * 1.25
 
requests_per_month = 300_000
shared_prefix = 6_000      # system prompt + tool definitions + policy docs, same every time
unique_input = 400         # the user's message and recent history
output = 250
cache_hit_rate = 0.95      # share of requests that find the prefix already cached
 
def per_million(tokens, price):
    return tokens / 1e6 * price
 
no_cache = requests_per_month * (
    per_million(shared_prefix + unique_input, PRICE_IN) + per_million(output, PRICE_OUT))
 
hits = requests_per_month * cache_hit_rate
misses = requests_per_month - hits
with_cache = (hits * per_million(shared_prefix, CACHE_READ)
              + misses * per_million(shared_prefix, CACHE_WRITE)
              + requests_per_month * (per_million(unique_input, PRICE_IN) + per_million(output, PRICE_OUT)))
 
print(f"without caching: ${no_cache:,.0f}/month")
print(f"with caching   : ${with_cache:,.0f}/month")
print(f"saved          : {1 - with_cache / no_cache:.0%}")

without caching: $4,590/month
with caching   : $1,557/month
saved          : 66%

On the real platform

What this looks like in provider usage dashboards · See the real screen in the official docs

1234

1. 1Total cost for the period.
2. 2Cache read tokens: prompt-cache hits, billed at a fraction of the input price.
3. 3Daily cost: look for spikes after deploys or traffic changes.
4. 4Per-model breakdown, including cache writes and reads.

Both consoles have a usage or cost page. Use it like this:

1. Filter by API key or project so you see one app at a time.
2. Split by model: is an expensive model doing work a cheap one could do?
3. Compare input vs output tokens: input-heavy bills point to caching and shorter prompts; output-heavy bills point to max_tokens, shorter formats or lower reasoning effort.
4. Look for cached input tokens: if they're near zero for a chatbot with a long system prompt, caching isn't working.
5. Set budget alerts and hard limits before launch.