Prompt Caching

Every provider the toolkit supports can serve a repeated prompt prefix from a cache at a steep discount — typically 0.1x the normal input rate. What differs is who asks for it.

Provider	Mechanism	Opt-in needed?
Anthropic	Explicit cache_control breakpoints	Yes — Agent(prompt_caching=True); nothing is cached otherwise
OpenAI	Automatic on prompts above ~1024 tokens	No
Google	Implicit caching, on by default for Gemini 2.5+	No
Fireworks	Automatic	No

Only Anthropic requires action from you. On the other three the discount arrives on its own; your job is to structure prompts so the cache can hit, and to read the resulting counters.

Anthropic: opt-in caching

prompt_caching accepts True (5-minute TTL) or a dict {"ttl": "5m" | "1h"}.

ConstructorBuilderYAML

agent = Agent(model="claude-sonnet-4-5", prompt_caching=True)             # 5-minute TTL
agent = Agent(model="claude-sonnet-4-5", prompt_caching={"ttl": "1h"})    # 1-hour TTL

agent = (
    AgentBuilder()
    .model("claude-sonnet-4-5")
    .prompt_caching({"ttl": "1h"})
    .build()
)

model: claude-sonnet-4-5
prompt_caching:
  ttl: 1h

The parameter is ignored for non-Anthropic providers, which cache automatically.

Where the breakpoints go

Each request carries two cache_control breakpoints, placed at the stability boundaries Anthropic's documentation prescribes:

1. The system block — which caches the tool definitions and system prompt together, since tools are serialized ahead of the system prompt in the request.
2. The last content block of the most recent message — so the next roundtrip reads everything before it as a cache hit.

The second breakpoint moves forward on every roundtrip, and the previous message-level markers are cleared before the new one is placed. That matters because Anthropic allows at most four markers per request; a marker per iteration would exhaust the budget in a four-tool run.

Caching pays off within a single run

Because the breakpoint moves, a multi-iteration tool-calling run reads its own earlier turns from cache. A verification run on claude-haiku-4-5 with an ~8.7k-token prefix wrote 4,394 tokens and already read 4,322 in the same run; an identical second run read 8,716 cached tokens with zero writes, at roughly one-fifth the cost.

Assistant messages that the toolkit appends as SDK objects (to preserve thinking signatures) are never marked — in the tool loop the most recent message before each call is always a dict-shaped user message, so the breakpoint always lands somewhere valid.

Minimum cacheable length

Anthropic will not cache a prefix shorter than a per-model minimum. The registry records it as cache_min_prompt_tokens:

Models	Minimum
claude-fable-5, claude-fable-5-1, claude-opus-5-5, claude-opus-5, claude-sonnet-5-5, claude-haiku-5-5	512 tokens
claude-sonnet-5, claude-sonnet-4-6, claude-sonnet-4-5, claude-opus-4-8	1,024 tokens
claude-opus-4-7	2,048 tokens
claude-opus-4-6, claude-opus-4-5, claude-haiku-4-5	4,096 tokens

These values were re-read from Anthropic's prompt-caching page for 0.3.0. claude-haiku-4-5 moved from 2,048 to 4,096, so a prefix in the 2k-4k range that looked cacheable under the old figure never actually cached.

The minimum is metadata, not a gate

The toolkit does not check cache_min_prompt_tokens before placing breakpoints — it cannot know the served prompt size before the request is sent, so the figure is published for callers and tooling that want to reason about whether caching is worth requesting. A below-minimum prompt still ships both breakpoints and is simply not cached by the API; nothing errors, and no write is billed. Enabling caching on a small prompt is harmless, just pointless.

One unmarkable message costs you both breakpoints

The message breakpoint is placed on the last content block of the newest message. When that message cannot carry one — an empty content list, or a trailing block that is not a dict — the placement stops rather than falling back to the message before it, so the request ships one breakpoint instead of two and the whole message history re-reads at the full input rate. It is logged at debug only. This is a known limitation, pinned by tests so that changing it is a deliberate decision; it is not reachable through ordinary Agent use, where the newest message is always a well-formed user turn.

The automatic providers

OpenAI, Google, and Fireworks cache server-side without being asked. There is no opt-in parameter, and prompt_caching does nothing on those providers.

• OpenAI caches prompts above roughly 1,024 tokens automatically.
• Google uses implicit caching, enabled by default for Gemini 2.5 and newer. It is best-effort and depends on temporal proximity between requests.
• Fireworks caches automatically on the models with published cached rates.

Google's explicit CachedContent API is deliberately not wired up

Implicit caching already delivers the same per-token discount with no storage charge for the toolkit's dominant pattern — multi-turn agent loops resending a stable prefix within a short window. Explicit caching wins only in narrower shapes: when you need a guaranteed rather than best-effort discount, when reuse spans long time gaps, or when many independent sessions share one large context. Those cases would have to clear the storage charge to break even.

What caching costs

Cache reads are cheap; on Anthropic, cache writes carry a premium that the TTL choice controls, and OpenAI's GPT-5.6+ rows charge a smaller one.

Charge	Rate vs. standard input	Registry key
Cache read	0.1x on most models	cached_input
Anthropic cache write, 5m TTL	1.25x	cache_write_5m
Anthropic cache write, 1h TTL	2.0x	cache_write_1h
OpenAI cache write (GPT-5.6+ rows)	1.25x of the served tier's uncached rate, on the written tokens only	cache_write_input_multiplier

The two providers bill a write differently. An Anthropic write is charged in addition to the prompt. An OpenAI write is a subset of the prompt's input tokens, re-priced at the multiple, so only the difference above the uncached rate is added. See OpenAI cache writes.

0.1x is the common ratio, not a rule — OpenAI's 4o-generation rows discount to 0.5x, and two Anthropic rows go deeper than 0.1x: claude-opus-5-5 reads cache at $0.20/M (0.05x) and claude-fable-5-1 at $0.25/M (0.025x). One OpenAI row does too: gpt-6.1-sol reads cache at $0.10/M (0.05x), where gpt-6-sol reads at 0.1x. Read the model's own cached_input rather than deriving it from the input price.

Google and Fireworks charge for reads alone; Anthropic and OpenAI bill a write as above. Every cached rate in the registry is verified against the provider's own published pricing page: 81 of the 107 model rows carry cached_input, including every Anthropic and every Fireworks row. The 26 without it publish no cached rate upstream — OpenAI's pro, TTS, transcription and realtime-transcription / translation / GPT-Live rows, Google's image, TTS, Live and transcription rows. A model without the key prices its cached tokens at the full input rate and logs a warning — a silent 0.0 would make cached tokens look free.

Tiered models can carry a per-tier cached rate

Rows on the tiered_200k / tiered_272k / tiered_100k schemas may publish cached_input_less_than_<N>_prompt and cached_input_greater_than_<N>_prompt; the cost engine prefers the key for the tier the request landed in. Google tiers context-cache reads at 200k and OpenAI doubles cached input above 272k, so a long-prompt cache hit on those models used to be under-billed by half. See Cost Tracking.

claude-haiku-5-5 (0.5.4) tiers its writes as well: cache_write_5m / cache_write_1h have _less_than_100k_prompt / _greater_than_100k_prompt forms, and the tier is decided on the prompt length Anthropic counts, cache reads and writes included. A 60k-token prompt that writes 50k tokens to the cache is a 110k-token prompt and bills the high tier on the input and on the write (Anthropic prompt-length tiers).

Cache writes bill at the TTL Anthropic served

cache_write_ttl is a request, not a guarantee. A response's usage.cache_creation splits the write across ephemeral_5m_input_tokens and ephemeral_1h_input_tokens, and each share is priced at its own rate — the same "bill what was served, not what was requested" rule the service tiers follow. A request that asked for a 1h TTL and was partly served at 5m is billed n x $1.25/M + m x $2.00/M on claude-haiku-4-5, not (n + m) x $2.00/M. On claude-haiku-5-5 each share is further priced at the tier the request landed in.

A response carrying no breakdown falls back to the requested TTL, which is the previous behaviour exactly. Before this release the split was never read, so a mixed-TTL write was mis-billed in whichever direction the two disagreed.

Retried attempts keep their cached share

RetryAttemptCost carries cached_input_tokens, and RetryAccumulator exposes total_cached_input_tokens; the failed attempts' cached share rolls into AgentResult.cached_input_tokens together with their input, so a failed attempt on a long cached prefix is no longer billed entirely at the full input rate. The count is a subset of the attempt's input_tokens, on the same convention as everywhere else. See Cost Tracking.

Choosing a TTL

The 1-hour TTL costs 2x on write instead of 1.25x, so it pays off only when the same prefix is reread beyond the 5-minute window. A rough guide:

• 5m (default) — interactive sessions, tool-calling loops, anything where turns arrive back to back. This is the right default for almost every agent workload.
• 1h — a large stable context (a long document, a big system prompt) queried repeatedly across a session with gaps longer than five minutes.

See Cost Tracking for how the discount is applied — by subtraction from the full-rate prompt, never as an addition.

Observing cache activity

Two fields on AgentResult report what the cache did:

result = await agent.run("Analyze this contract")

print(result.cached_input_tokens)   # prompt tokens served from cache
print(result.cache_write_tokens)    # tokens written to cache (Anthropic, and OpenAI GPT-5.6+ rows)

Per-request detail lives in result.request_cost_records, where each record carries its own cached_input_tokens and cache_write_tokens.

Both fields are reported on every run shape, including Anthropic runs that take the structured-output path (an output_schema with no local tools). Those runs reported 0 for both before 0.3.0 — the cost was priced from the real counts, but they never reached the result.

The counters are read from each provider's native usage object:

Provider	Cache read field	Cache write field
Anthropic	usage.cache_read_input_tokens	usage.cache_creation_input_tokens
OpenAI	usage.input_tokens_details.cached_tokens	usage.input_tokens_details.cache_write_tokens (a subset of input, reported by GPT-5.6+ models)
Google	usage_metadata.cached_content_token_count	—
Fireworks	usage.prompt_tokens_details.cached_tokens, falling back to a flat usage.cached_prompt_tokens	—

A usage object may arrive as a typed model or as a raw mapping

An SDK's stream decoder types only the fields the response class declares and leaves the rest as raw JSON. On the Fireworks Responses path that meant the Chat-Completions spelling prompt_tokens_details — which the path accepts for the token totals — arrived as a plain dict, and the cached share read as 0 and billed at the full input rate. Every cached-token read now accepts both shapes.

Both Gemini paths report the cached share. A run that took the structured-output path (an output_schema with no local tools) used to price its implicit-cache hits correctly and then return cached_input_tokens: 0, because the count never reached the result dict — a live probe recorded an 8x cost drop across three identical prompts while all three reported zero cached tokens. The cost was always right; only the reporting was missing.

Cached tokens are a subset of input_tokens on all four providers

Anthropic's API is the odd one out: it reports cache reads outside usage.input_tokens, where the other three report them inside the prompt total. The Anthropic provider folds the reads back in before anything is reported, so across all four providers input_tokens is the full prompt size, cached_input_tokens is its cached share, and the uncached figure is input_tokens - cached_input_tokens. That holds for AgentResult, every entry in request_cost_records, DetailedCostReport per-model totals, and the usage a hook sees.

Reported Anthropic input tokens grew after 0.2.0

Until 0.2.0 the toolkit surfaced Anthropic's raw counts, so input_tokens was the uncached share and the prompt total was input_tokens + cached_input_tokens. A cached Anthropic run now reports a larger input_tokens than the same run did on 0.2.0. Cost is unaffected — it was always computed from the folded total — but any dashboard, assertion, or budget check that adds the two fields together on Anthropic now double-counts the cached share.

Structuring prompts for cache hits

Caching keys on an exact prefix match, so the same discipline helps on every provider:

• Put stable content first. System prompt, tool definitions, and long reference documents belong at the front; per-request specifics go at the end.
• Do not interpolate volatile values into the prefix. A timestamp, request ID, or shuffled tool order in the system prompt invalidates the cache on every call.
• Keep tool definitions stable across a session. On Anthropic they are cached together with the system prompt, so changing the tool list rewrites that block.
• Reuse a session. AgentSession resends a growing but stable history, which is exactly the shape caching rewards.

A large tool surface can be deferred instead of cached

Anthropic caches the tools array together with the system prompt, as one prefix. That makes the tools array part of your cache key: change it between turns and every later request of the run pays full price. Two things protect that prefix.

A session freezes the tools array it starts with, so an MCP server that stops answering list_tools() mid-session cannot shrink it. And deferred tools (Agent(defer_tools=...)) are stamped once, before the first request, in that frozen order — so the serialised system + tools prefix is byte-identical across every iteration of a run and across two identical runs. Which tools you defer never changes their wire position.

A deferred tool never carries a cache breakpoint. Anthropic rejects defer_loading together with cache_control on the same entry, and a breakpoint there would cache nothing anyway, because deferred tools are excluded from the prefix by definition. The only place you can attach one is cache_control on a remote MCP toolset; it is removed with a warning. Put the breakpoint on a tool that loads eagerly.

Changing defer_tools between turns of a session is a configuration change, not server drift: the stamp is a client-side overlay re-derived per request and deliberately kept out of AgentSession.tool_surface_drift, which stays a statement about what the MCP servers produced. What was actually deferred is on AgentResult.deferred_tools.

Compaction and caching interact

Compaction rewrites earlier turns, so the bytes before the message breakpoint no longer match anything the provider holds and the next request pays a cache write rather than a read. That is usually still a win — compaction exists to cut a context that has grown expensive — but expect the write.

The system breakpoint is unaffected: system and tools are frozen for the life of a session, so the expensive, stable half of the prefix keeps reading from cache straight through a compaction pass. That is precisely why the toolkit places two breakpoints rather than one.

Related pages

• Cost Tracking — how cached tokens and writes are priced
• Service Tiers — the other lever on per-token cost
• Sessions — multi-turn history, the pattern caching rewards
• Compaction — history rewriting and its effect on cached prefixes
• Providers — provider-specific capabilities