Cost Tracking

The toolkit surfaces token and cost metrics through AgentResult and optional detailed run reports.

Default metrics in AgentResult

• input_tokens
• output_tokens
• total_tokens — input_tokens + output_tokens; the reasoning_tokens and cached_input_tokens subsets are deliberately not added
• reasoning_tokens — the thinking share of output_tokens, reported on every provider and every run shape
• cached_input_tokens — prompt tokens served from the provider's cache (see Prompt caching)
• cache_write_tokens — prompt-cache writes: Anthropic cache-creation tokens, billed on top of input_tokens; OpenAI GPT-5.6+ written tokens, a subset of input_tokens (see OpenAI cache writes)
• request_cost_records — one record per API request (see Per-request cost records)
• service_tier_served / priority_request_count — the run-level tier answer (see Service Tiers)
• cost — the request records, the built-in tool charges and the spend the run's tools reported
• cost_estimated
• tool_reported_cost / tool_reported_costs — what the run's own tools reported spending, already in cost (see Tool-reported spend)
• iteration_metrics (when available) — one entry per provider request, each saying why it was made (see One iteration entry per request)

Detailed cost reports

Enable detailed reports when constructing Agent:

agent = Agent(
    model="claude-sonnet-4-5",
    instructions="You are concise.",
    detailed_costs=True,
    cost_output_dir="agent_costs",
)

When enabled, reports are persisted as JSON and result.detailed_cost_file points to the file.

Report aggregates

Alongside the token and retry totals, each report carries the run's cache aggregates and its per-request records:

Field	Description
total_cached_input_tokens	Cached prompt tokens across the run
total_cache_write_tokens	Cache-write tokens across the run: Anthropic cache-creation tokens, plus OpenAI written tokens on the rows that report them (see OpenAI cache writes)
request_cost_records	The full per-request record list (see Per-request cost records)
total_tool_reported_cost / tool_reported_costs	What the run's tools reported spending, and one record per call (see Tool-reported spend)
schema_version	Persisted-shape marker — see below

total_cost in the report is aligned to AgentResult.cost, which is the authoritative billed figure. Summing the per-iteration cost values gives the run's token cost — the same figure as summing request_cost_records — and a smaller number than cost only by the built-in tool charges (search, code containers), which are per run, not per request, and by the spend the run's tools reported.

Individual iteration entries also carry cached_input_tokens, cache_write_tokens and purpose.

One iteration entry per request

Every provider request of a tool-calling run has its own iteration_metrics entry, so the entries sum to request_cost_records — and so do the report's total_input_tokens / total_output_tokens, which are built from them. Until this release all four providers built a tool round's entry from its follow-up request alone: the round's first request, and every re-request (a max_tokens continuation, Google's empty-turn retry, Anthropic's pause_turn continuations), had no entry, so summing iteration_metrics understated a tool run — by roughly half on a one-round run — while request_cost_records and AgentResult.cost were always complete.

Each entry's purpose says why its request was made, in the same vocabulary as the opt-in request_start event (agent.hooks.REQUEST_PURPOSES):

purpose	The request was
generation	the run's first request (or a fresh turn)
tool_followup	the answer to a tool round
max_tokens_continuation	a re-request with a higher output ceiling after a cut inside a tool call
truncation_retry	an Anthropic or OpenAI re-request with a higher ceiling after a cut outside a tool call
empty_turn_retry	Google's one re-request of an empty final turn
pause_turn_continuation	an Anthropic pause_turn continuation

A round's first request and its follow-up share an iteration_number (the request_start numbering) and are told apart by purpose, so a host can count, say, answers that needed the empty-turn retry without turning on request_start. The round's tool calls and compaction fields stay on the follow-up's entry, as before; the first request's entry carries none. Durations no longer overlap: the follow-up's duration_ms covers the tool execution and the follow-up request, the first request's entry the request before it, and duration_ms on a pause_turn_continuation entry is 0 (the paused request's entry carries the wall time). Compaction summarisation calls stay out of iteration_metrics; they are reported in the compaction_* fields.

Each entry's thinking_enabled says whether that request carried thinking — a thinking object on Claude (an always-on row with no config included; an honoured off request, {"type": "disabled"} where the row allows it, not), a reasoning object other than effort none on OpenAI, a thinking_config other than a zero budget on Gemini, a reasoning_effort other than none on Fireworks — and the same flag is on each iterations[] entry of the detailed cost report. It is what the request carried, not what the model does on its own: a Gemini, OpenAI or Fireworks row that reasons by default with no setting given reports False, while a Claude always-on row reports True because the toolkit sends the adaptive object for it. Until 0.5.3 no loop set it, so every per-request entry said False, on a thinking run too. The summarize compaction call has no entry of its own (its cost is in the compaction_* fields), so it carries no flag.

The run-level DetailedCostReport.thinking_enabled is any() of the entries whenever a run has them (0.5.3), so it cannot contradict iterations[]. The single-request path has no entries, and there the flag is what the configuration given resolves to: on Claude the provider's own resolution (an always-on row thinks with no config, an honoured off request does not, and the no-tools path's registry gate is applied with whether a built-in tool is enabled); on the other providers whether a config was given, with effort none on OpenAI and Fireworks read as off and a Gemini budget_tokens read after the row's validation — 0 is off on a row that can turn thinking off, and on where the row runs it as dynamic thinking instead (gemini-2.5-pro); a level the row does not take, or a dict with neither key ({"type": "disabled"}), sends nothing and reads as off. An OpenAI or Fireworks effort the row's validation drops still reads as given there, and raw provider keywords (reasoning_config=, thinking_budget=) are not read on those providers.

Report schema versioning

Persisted reports carry a schema_version marker so consumers can tell the shapes apart:

Version	Shape
1	Reports written before the marker existed. Loading one back yields schema_version == 1, with the cache fields defaulting to 0 and request_cost_records to [].
2	Current shape — adds total_cached_input_tokens, total_cache_write_tokens, request_cost_records, and per-iteration cache fields.

DetailedCostReport.from_dict() reads legacy reports without error, so existing report archives stay loadable.

Interpreting cost fields

• cost_estimated=True means one or more provider/tool values were estimated.
• OpenAI speech is exact since 0.5.1 on a supports_sse_usage row (gpt-4o-mini-tts). synthesize_speech bills the provider's own token counts from the SSE response, so cost_estimated is False. DPT_OPENAI_TTS_SSE_USAGE=off restores the earlier estimate from the text and the clip's duration. See Exact OpenAI TTS cost.
• tool_costs_estimated provides per-tool estimation flags where available.
• Reasoning token behavior and accounting differ by provider/model.
• A stream closed before its complete event is still reported. Its spend is an AgentResult dict marked "abandoned": True on Agent.last_abandoned_stream, is banked into an AgentSession's totals, and reaches hooks. See Stopping a stream early.

tool_costs_estimated['code_execution'] means observed but unmeasurable

An estimate flag should mark a number the toolkit had to guess, not a number it knows exactly. Code execution is billed by wall-clock session duration on Anthropic and by container count on OpenAI, so the flag is set only on an Anthropic run that observed at least one session but could not measure its duration. Container-count billing is exact by construction, and a code-execution tool that was configured but never invoked is correctly billed at $0.00.

Before 0.3.0 the flag was set whenever a run recorded zero sessions and zero duration — i.e. exactly the idle case — so simply enabling code_execution and never triggering it marked the whole run's cost as estimated.

Grounded searches are billed per query, and the evidence arrives mid-stream

Google reports a grounded search only through grounding_metadata on the streamed response, and does not guarantee it arrives on the last chunk. Before this release the toolkit read that metadata from the final chunk alone, so a search that really ran could be reported as not having run and billed as zero. The metadata is now accumulated across the whole turn and merged, de-duplicated, so each distinct query is counted exactly once — see the provider page. result.built_in_tools_invoked is the observed-evidence list, and result.distinct_urls now carries the grounding URLs on the tool-calling path as well as the structured-output path. A search Gemini returns only as a circulated invocation (a tool_call part with GOOGLE_SEARCH_WEB, seen under tool combination) is counted and billed the same way; a run that enabled search and looks grounded but carries nothing countable bills $0 with tool_costs_estimated["google_search_grounding"] and cost_estimated set, never a confident zero.

A duration-billed request never reports a confident $0.00

Pricing a row on the per_minute schema without an audio_minutes value used to return (0.0, False) — a confident zero for a request the provider certainly billed. Such a call now bills core.MINIMUM_BILLABLE_MINUTES (0.02, the same floor multimedia uses for an unmeasurable clip) and returns cost_estimated True. A duration the caller did measure is billed as measured, even below the floor, because flooring a real measurement would over-bill a genuinely short clip.

The floor is row data now, not a constant: a pricing row may publish its own minimum_billable_minutes, and a row that publishes 0 opts out of both the floor and the flag. Only a session row does — see Sessions billed by the second. Every other per-minute row omits the key and is billed exactly as described above.

Not every speech model is billed by duration

per_minute is one schema among several, and a speech model may be priced on the ordinary standard schema with an input_audio rate instead. gemini-3.5-transcribe is the first such row: $2.00/M audio-input tokens and $12.00/M text output, billed from the response's usage metadata, with duration_minutes reported but never priced. Its live responses report no output token count, so the transcript's output tokens are estimated and the result is flagged cost_estimated — see Multimedia.

A prompt can split three ways

Where a price row publishes an audio rate and an image rate — every realtime 2.x row and every Gemini Live row does — the input side is billed as audio at input_audio, image at input_image and whatever is left at the text input rate, and a cached share spanning all three is discounted against the rate each modality was actually charged at. Before this release the split was audio-versus-everything-else and the audio arm always won, so an image token on a realtime row was billed at the text rate. See Realtime Voice.

Per-component costs

calculate_cost accepts an optional cost_components_out dict. When one is supplied it is cleared and filled with this call's per-component costs in USD:

components: dict[str, float] = {}
cost, estimated, tool_flags = calculate_cost(
    model, provider, input_tokens, output_tokens, cost_components_out=components
)
components["input_image_usd"], components["cache_discount_usd"], components["total_usd"]

Key	Covers
input_text_usd / input_image_usd / input_audio_usd	The prompt, split by modality where the row prices them separately
cache_discount_usd	The cached re-pricing — negative, because the cached share is priced at full rate and then moved down
cache_write_usd	Anthropic cache creation; on an OpenAI row with cache_write_input_multiplier, the extra cost of re-pricing the written tokens at that multiple (only the difference above the uncached rate)
output_text_usd / output_image_usd / output_audio_usd	The completion, split the same way
duration_usd	A per_minute row's charge: billed minutes × price_per_minute, including a minimum-billable floor. 0.0 on every token-priced row
tool_usd	Built-in tool costs
total_usd	The returned total

Every entry except total_usd sums to total_usd on every pricing schema, so a client-facing report can show named cost lines without re-deriving a single rate from the registry. Image results expose the same arithmetic ready-made — see Multimedia. A realtime session keeps the same dict live as session.cost_components — see Realtime Voice.

core.COST_COMPONENT_KEYS names the eleven keys once, in the order above. A call that prices nothing — negative token counts, an unknown model, an unusable price row — fills every key with 0.0, so a dict you reuse across calls never shows the previous call's lines beside a $0.00 total. On a TTS (audio_output) row the charge lands in output_audio_usd; it used to be labelled output_text_usd.

Duration-billed rows now report their charge (duration_usd)

Before duration_usd existed, a per_minute row (speech-to-text, live transcription, live translation, GPT-Live) left the dict untouched: none of the token and tool components could carry a duration, so the components summed to 0.0 (or to a reused dict's previous numbers) beside a non-zero total. The row's charge is now duration_usd, every token component is 0.0, and the sum-to-total_usd rule holds on every row the registry prices — a unit test checks it row by row, at every published service tier. The key is additive: a reader that sums the components gets the right total, and one that reads named keys sees nothing move on a token-priced row.

token_source has one vocabulary

token_source says where a run's token counts came from:

Value	Meaning
"api_response"	The provider reported the counts
"stream_partial"	Accumulated from a stream that then failed
"rejected_before_stream"	The provider rejected the request before it streamed anything, so nothing was billed: cost 0.0, zero tokens, cost_estimated False (see below)
"estimated" and the "*_metadata_missing" variants	Counted locally, because the response carried no usage

The single-call provider paths historically emitted "api_metadata" for the same fact the loop paths called "api_response", so an assertion on either passed or failed depending on which path a run happened to take. AgentResult now always reports "api_response", and the legacy spelling is still accepted wherever token_source is read. Use agent.cost.normalize_token_source() rather than comparing strings yourself; agent.cost.TOKEN_SOURCE_API_RESPONSE and TOKEN_SOURCE_ALIASES are the vocabulary.

Where a token count comes from

Pre-flight counts — count_tokens(), Agent.estimate_tokens(), and the Level-1 compaction trigger — prefer the provider's own counting API (OpenAI's Token Counting API, Google's count_tokens, Anthropic's messages.count_tokens). Only when that is unavailable does the toolkit fall back to a local tiktoken estimate, and then to a ~4-characters-per-token approximation.

Which of those answered is not guesswork: core.count_tokens_with_source() (and core.async_count_tokens_openai_with_source() on the async OpenAI path) returns the count together with a label — openai_token_counting_api, openai_tiktoken_fallback_tiktoken_builtin, openai_tiktoken_fallback_registry, openai_tiktoken_fallback_assumed_default, or text_approximation_*. count_tokens() and async_count_tokens_openai() are unchanged count-only wrappers over them.

The tiktoken encoding itself now comes from the registry (descriptions.<model>.tokenizer_encoding), so teaching the toolkit a new model family is a registry edit rather than a code change. A row that omits the key is counted against an assumed o200k_base and logs a warning naming the exact JSON path to add. Before this release the fallback special-cased names beginning gpt-5; any other model tiktoken did not recognise silently degraded to the 4-chars-per-token approximation. Billing was never affected — this path feeds estimates only.

Gemini tool-use tokens are input

Gemini reports the server-side tool output it feeds the model in usage_metadata.tool_use_prompt_token_count, outside prompt_token_count. That output is the pages a URL-context call retrieves, and code execution's generated code and results. Google bills it as input. The URL context guide says the retrieved content "is counted as part of the input tokens", and code execution's intermediate tokens "are billed as input tokens". total_token_count includes them.

Since 0.5.1 the toolkit bills them too. Both Google paths add them to each request's input_tokens, and their modality split joins the input breakdown, so a request's cost record, the run's input_tokens and cost all include them. Measured 2026-09-28 on gemini-3.5-flash-lite:

Request	Prompt	Tool use	Total
One URL (URL context)	17	1,323	1,427
One code-execution run	23	163	327

Before 0.5.1 only the prompt was billed. A W10 integration run with several built-in tools had 9,394 tool-use tokens beside a 301-token prompt, and its token cost was reported as $0.00099 instead of $0.0038 (its $0.042 of search queries was billed correctly).

• Only on a response that shows URL context or code execution ran. Google Search is priced per query, and Google says its tokens "are not double-charged". It reported no tool-use tokens when measured. A response whose only tool evidence is a search, or which shows none, keeps its tool-use tokens unbilled, as before.
• The long-context tier counts them. On a tiered_200k row the tier is judged on the billed input, tool-use tokens included, because they are input to the model. Google's pricing page does not say either way, so this is an assumption: a 199k-token prompt that retrieves 2k tokens of pages is priced at the >200k rates.
• The in-loop compaction check reads the prompt alone. It measures the context the request sent. The tool output is the server's own for that request, and what of it comes back (code parts, circulated tool responses) is counted in the next request's prompt.
• The per-iteration tool_use_prompt_token_details reports the same tokens by modality. They are already inside input_tokens and input_token_breakdown, so do not add them again.

DPT_GOOGLE_BILL_TOOL_USE_TOKENS=off restores the previous billing (see Configuration).

Sessions billed by the second

Three realtime rows are billed by the clock rather than by tokens, and a session is not a recorded clip: its duration is always measurable, which changes what an honest number looks like.

Row	Rate	Which duration is billed
gpt-live-1	$0.05 / minute of active session time	the provider's own cumulative usage.seconds, whenever it has reported one — including a reported 0
gpt-realtime-translate	$0.034 / minute	the input audio the toolkit appended, falling back to connected wall clock. Always cost_estimated
gpt-transcribe, gpt-live-transcribe, gpt-realtime-whisper	$0.0045 / $0.017 / $0.017 per minute	the seconds the provider reported per completed item, falling back to connected wall clock — and that fallback is flagged cost_estimated; see Live transcription

gpt-live-1 has no token price of any kind. Its minute includes the silences and the time a backend spends thinking, is billed per second and is never rounded up to a whole minute, so a 90-second conversation is $0.075 rather than $0.10. The provider sends a running total in session.usage.updated and a final one in session.closed, and both are cumulative snapshots: the toolkit replaces its figure rather than adding to it. (Adding them up is the expensive mistake here — three updates on a 31.5-second session would bill 61.5 seconds.) A reported zero is a report, not a missing one: an idle session legitimately bills nothing, and reading that zero as "nothing reported" billed the socket's wall clock instead. Only a session the provider never reported on falls back to its own wall clock, and only that one is flagged estimated.

A session says what its cost leaves out

session.cost on a GPT-Live session is the voice session and nothing else. A session that delegates to a Responses backend also spends that model's tokens and its tools' charges, on a rate card this session never reads. Rather than quietly omitting them, such a session sets cost_estimated from the moment it connects and spells the reason out in words in session.cost_excludes, while session.delegated_usage lists every backend response it saw, one entry per response id, so you can price them yourself. cost_excludes is non-empty exactly when the number is incomplete, and cost_estimated is True whenever it is non-empty. A session that delegates to your application instead — the default — has no invisible money and reports a confident total.

A price row can state its own minimum billable duration. minimum_billable_minutes in the pricing row overrides core.MINIMUM_BILLABLE_MINUTES; absent means the constant, which is what every file-transcription row uses. gpt-live-1 publishes 0, so a session that never got past its handshake costs exactly $0.00 with no floor and no warning — the floor exists for a duration the caller could not measure, and a session's duration is always measurable, so a zero there is measured rather than missing.

A token-priced session that is told no tokens is still not free. Where a realtime row publishes the vendor's own audio-tokens-per-second conversion (input_audio_tokens_per_second / output_audio_tokens_per_second — 25 each on gemini-3.5-live-translate-preview, Google's published figure), a session with no usage report derives its usage from the audio it sent and received and sets session.usage_estimated and cost_estimated. On the translation row that estimate is provisional and silent while the session runs, because the provider's first usage_metadata lands three to four seconds into the stream: it warns once only at close(), and only if no report ever came — which a sub-four-second stream will produce. A real report replaces the estimate outright rather than adding to it, and reported usage that trails the audio by a second or two at close is billed as reported rather than topped up. A row that publishes no conversion invents nothing.

One reconciliation detail on that row: its reports also list a growing text share inside prompt_tokens_details that is not part of prompt_token_count. The toolkit bills from the totals, so that share is not charged — which matches Google's own audio-only billing footnote. The row still publishes text rates mirroring its audio rates, as a guard against the day such a token does enter the totals.

Prompt caching and cached-token pricing

Providers charge a reduced rate for prompt tokens served from their prompt cache. The toolkit reads the cached counts out of each provider's usage object and re-prices them against the model's cached_input rate in model_information.json.

Enabling caching

This section covers how cached tokens are priced. For how to turn caching on — Anthropic needs an explicit Agent(prompt_caching=True), the other three providers cache automatically — and how to structure prompts so the cache hits, see Prompt Caching.

How cached tokens are priced

Pricing works by subtraction, never addition: the whole prompt is priced at the full input rate, then the cached share is moved down to the model's cached rate. Internally calculate_cost requires that the cached count be a subset of the input count it receives, so a cached token is discounted exactly once.

One convention on all four providers: cached ⊆ input

cached_input_tokens is always the cached share of input_tokens, so the uncached figure is input_tokens - cached_input_tokens no matter which provider served the run — in AgentResult, in every request_cost_records entry, in DetailedCostReport per-model totals, and in the usage a hook sees.

Anthropic's API does not report it that way: cache reads arrive outside usage.input_tokens. The Anthropic provider folds them in at the point of reporting, which is what makes the single convention hold.

Reported Anthropic input tokens grew after 0.2.0

On 0.2.0 the toolkit surfaced Anthropic's raw counts, so its input_tokens was the uncached share and the prompt total was input_tokens + cached_input_tokens. A cached Anthropic run now reports a larger input_tokens. The cost did not change — it was always computed from the folded total — but code that adds the two fields together on Anthropic now double-counts.

Situation	Behavior
Model has a cached_input price	Cached share re-priced at that rate; an INFO log records the discount
Model has no cached_input price	Cached share stays at the full input rate and a warning is logged — a silent 0.0 default would make cached tokens free
cached_input_tokens > input_tokens	Clamped to input_tokens with a warning (indicates a provider/convention mismatch)

Cached audio is re-priced at the published audio cached rate

A model that prices audio input separately (input_audio) can also publish a separate cached audio rate, cached_input_audio — on gemini-2.5-flash and gemini-2.5-flash-lite that rate is several times the text cached rate (0.10 vs 0.03, and 0.03 vs 0.01). When Google reports a modality breakdown of the cached tokens (usage_metadata.cache_tokens_details, surfaced to calculate_cost as cached_token_breakdown), the cached AUDIO share is discounted against the audio input rate and the remainder against the text cached rate.

The split applies only when the input side was itself billed with an audio split and the row publishes cached_input_audio. Without the breakdown, or on a model with no published cached audio rate, the whole cached share uses the flat cached_input rate exactly as before. Pricing a cached audio prompt at the text cached rate underbills it.

Tiered models can price cached reads per tier

A row on the tiered_200k / tiered_272k / tiered_100k schema may publish cached_input_less_than_<N>_prompt and cached_input_greater_than_<N>_prompt alongside the flat cached_input. calculate_cost prefers the key for the tier the request actually landed in and falls back to cached_input when the tier-specific key is absent. Google tiers its context-cache reads at 200k, and OpenAI bills a >272k request's cached input at 2x as well — before this, a long-prompt cache hit was under-billed by half.

Anthropic prompt-length tiers

claude-haiku-5-5 (0.5.4) is the first Claude row priced by prompt length, and it needed a third tiered schema, tiered_100k, because Anthropic's rule differs from Google's and OpenAI's in two ways. The prompt length that picks the tier is every input token, cache reads and cache writes included, and the cache-write prices are tiered too: cache_write_5m / cache_write_1h have _less_than_100k_prompt / _greater_than_100k_prompt forms beside the flat keys. "Prompts up to 100,000 tokens" bill the low tier, so exactly 100,000 is the low tier, and each request is tiered on its own.

The Anthropic call sites fold cache reads into input_tokens and pass cache writes separately, so calculate_cost decides the tier on input_tokens + cache_write_tokens for this schema only (the Google and OpenAI tiers still read input_tokens alone), then prices every component at that tier:

Request	Prompt length	Tier	Cost
50,000 in, 10,000 out	50,000	low	50,000 x $0.10/M + 10,000 x $0.50/M = $0.0100
60,000 in (50,000 of them cache reads), 0 out	60,000	low	10,000 x $0.10/M + 50,000 x $0.01/M = $0.0015
60,000 in, 50,000 written (5m), 0 out	110,000	high	60,000 x $0.50/M + 50,000 x $0.625/M = $0.0613
200,000 in (150,000 cache reads), 0 out	200,000	high	50,000 x $0.50/M + 150,000 x $0.05/M = $0.0325

A fallback hop that is all cache reads (input_tokens equal to the reads) decides its tier on the reads alone, which is the same rule. A cache write Anthropic served across both TTLs is priced share by share (Cache writes bill at the TTL Anthropic served), and every share lands in the tier of the whole request: the Anthropic paths pass the request's prompt length to each call as calculate_cost(tier_prompt_tokens=...), so a 60k prompt writing 30k at 5m and 30k at 1h is a 120k prompt on both shares. The 9 Oct 2026 check on the real row billed a 6,797-token cache write at the low-tier write price and a 264,555-token prompt at the high tier.

Both cache fields are populated on every run shape. Anthropic runs that take the structured-output path (an output_schema with no local tools) reported cached_input_tokens == 0 and cache_write_tokens == 0 before 0.3.0 even though the cost was already priced correctly — the counts simply never reached the result dict.

Anthropic cache writes

Writing to the cache is billed in addition to the prompt, at a TTL-specific rate. The toolkit reads usage.cache_creation_input_tokens and bills it against cache_write_5m or cache_write_1h — at the TTL Anthropic served, not the one requested, which is the same rule the service tiers follow. A response's usage.cache_creation splits the write across ephemeral_5m_input_tokens and ephemeral_1h_input_tokens, and each share is priced at its own rate; a response carrying no breakdown falls back to the requested cache_write_ttl ("5m" by default), which is the previous behaviour exactly. A model missing the relevant write price falls back to the full input rate with a warning. See Prompt Caching.

OpenAI cache writes

On GPT-5.6 and later families OpenAI bills a cache write at 1.25x the uncached input rate, and the Responses usage reports it: usage.input_tokens_details.cache_write_tokens (typed in openai 3.16), beside cached_tokens. Unlike Anthropic's cache creation, the write is not an extra charge on top of the prompt — OpenAI's prompt-caching guide says each input token bills at exactly one of the uncached, cached-input or cache-write rates, so the written tokens are a subset of input_tokens, disjoint from the cached share.

The toolkit reads that count on the OpenAI tool loop and structured path, reports it as cache_write_tokens (on results and per-request records, as on Anthropic) and prices it the way OpenAI bills it: a row that carries cache_write_input_multiplier (1.25 on gpt-6-astra / -sol / -luna, gpt-6.1-sol and gpt-5.6-sol / -terra / -luna, matching the pricing page's cache-write column) re-prices the written tokens at that multiple of the uncached input rate of the tier the request landed in — so on these tiered_272k rows the write rate doubles above 272k exactly as the input rate does, and a priority request writes at 1.25x the priority input rate. Only the difference is added, as cache_write_usd, so the components still sum to the total. gpt-5.6-luna, 100k prompt with 20k cache reads and 30k written:

Share	Tokens	Rate ($/M)	Cost
Uncached	50,000	0.20	$0.0100
Cached read	20,000	0.02	$0.0004
Cache write	30,000	0.25 (1.25 x 0.20)	$0.0075

Before this, the written tokens were billed at the plain input rate ($0.0060 for that share) and cache_write_tokens stayed 0 on OpenAI results — a writing request under-reported by 0.25x the input rate per written token. A row that publishes no write price (OpenAI models before GPT-5.6: "no additional cache-write charge") bills reported writes at the plain input rate, never as an addition. It is a billing correction, with no switch: the figure moves only up, and only on a request that wrote to the cache.

Registry pricing keys

These keys are optional in model_information.json — a model without them simply prices at the full input rate.

Key	Applies to
cached_input	Every model with a published cached rate — 81 of the 107 rows, on all four providers. The 26 without it (OpenAI pro / TTS / transcription / translation / GPT-Live rows, Google image / preview TTS / Live / transcription rows) publish no cached price upstream
cached_input_less_than_<N>_prompt / cached_input_greater_than_<N>_prompt	Tiered rows (tiered_200k, tiered_272k, tiered_100k) whose cached rate differs by tier
cached_input_audio	Rows that also publish input_audio and charge a distinct cached rate for audio tokens — gemini-2.5-flash and gemini-2.5-flash-lite (standard and priority tiers), and the OpenAI realtime rows gpt-realtime, gpt-realtime-1.5, gpt-realtime-2, gpt-realtime-2.1 and gpt-realtime-2.1-mini
cache_write_5m / cache_write_1h	Anthropic models only — a write billed in addition to the prompt. On claude-haiku-5-5 the _less_than_100k_prompt / _greater_than_100k_prompt forms carry the tier's price and the flat key is the fallback
cache_write_input_multiplier	OpenAI GPT-5.6+ rows (1.25) — written tokens are a subset of the prompt, re-priced at this multiple of the served tier's uncached input rate (see OpenAI cache writes)

Per-request cost records

A run's token cost is the sum of its per-request costs, not a single pricing pass over summed token totals. Every provider loop appends one record per API request to request_cost_records, and the run total is sum(record["cost"] for record in records) plus one per-run built-in tool cost pass, plus the spend the run's tools reported.

Why per-request summing matters

Tiered pricing is nonlinear: f(sum of requests) != sum of f(request). Pricing summed totals could push a run across a 200k or 272k tier threshold that no individual request ever crossed, billing the whole run at the higher tier. Summing per-request costs prices each request at the tier it was actually served under.

Each record contains:

Field	Description
input_tokens / output_tokens / reasoning_tokens	Tokens for that single request
cached_input_tokens	Cached prompt tokens for that request — always a subset of input_tokens, on every provider
cache_write_tokens	Cache-write tokens — Anthropic records (billed on top of input) and OpenAI tool-loop records (a subset of input)
served_model	Model that actually served the request — differs from the requested model after an Anthropic server-side fallback
requested_tier / served_tier	Service-tier attribution — populated on OpenAI and Google when a tier was requested, None otherwise. See Service Tiers
served_tier_source	Google only: "body", "header" or "none" — how the served tier was established, so an unreadable signal is distinguishable from a reported downgrade. Records on the other providers, including a synthesised one, omit the key rather than carrying None
cost	Token cost of that request alone (built-in tool costs are per-run and excluded)
synthesised_from_run_totals	Present and True only on a structured-output run's single synthesised record (see below)
estimated	Present and True only on the record of an OpenAI tool-loop request that a failure event ended on its first attempt after part of its reply had streamed (0.5.2). No usage arrived, so output_tokens is a count of the streamed text, input_tokens is 0, and the price is at standard rates. See Providers

reasoning_tokens is a share of output_tokens, not an extra charge

On the run, AgentResult.reasoning_tokens is the thinking share of AgentResult.output_tokens on all four providers — the cost already includes it. Summing input_tokens + output_tokens + reasoning_tokens overstates a run. For the same reason AgentResult.total_tokens is input_tokens + output_tokens; it used to add the reasoning subset a second time. See Thinking and Reasoning.

Google reports thought tokens outside candidates_token_count, and both Google entry points fold them into the run's output_tokens before reporting it — the tool-calling path did not until this release, so a thinking tool run under-reported output_tokens and total_tokens by the whole thought count although those tokens were billed. The per-request records, the iteration_metrics entries and the iteration_complete event follow the same rule: on every provider, a record's output_tokens is reasoning-inclusive and reasoning_tokens is its thought share, so a record's tokens are exactly the tokens its cost priced, and the records' output_tokens sum to the run's output_tokens. Until this release a Google record and iteration entry carried the candidates count alone with reasoning_tokens beside it, so its billed output was output_tokens + reasoning_tokens and DetailedCostReport.total_output_tokens understated a Google thinking run. A reader that added the two fields on Google records now double-counts the thoughts and should read output_tokens alone.

AgentResult collapses the per-request tier attribution into service_tier_served and priority_request_count; see the run-level answer.

Anthropic pause_turn continuations are separate API requests and get one record each, carrying pause_turn_continuations: 1. As with any other request record, a continuation's input_tokens folds in its cache reads, its cached_input_tokens / cache_write_tokens are the counts Anthropic reported for that continuation, its usage.iterations are billed on top, and its served_model is resolved from the response. Before this release a run's continuations shared one record carrying the total count, with zero cache counts and the requested model, and a continuation that read a 100k-token prefix from cache contributed roughly nothing to the bill; its cache writes were never billed at all.

Continuation records on every Anthropic path report the reasoning_tokens the response carries (a subset of output_tokens, never priced a second time); they used to report 0, and the tool loop's run-level thinking total now includes them. Continuations are also sent through the same request helper as every other Anthropic request, so DPT_PREFIX_AUDIT audits them and a prefix-binding 400 on a continuation runs the one-shot drop_block recovery. The request body is unchanged.

Structured-output runs get one synthesised record

A run with an output_schema and no local tools makes a single API request and takes a path that never built per-request records. Since 0.3.0 that run's record is synthesised from the run-level totals and flagged synthesised_from_run_totals: True, so request_cost_records and DetailedCostReport are populated for every run rather than only tool-calling ones.

Because it is built from run totals, retries and pause_turn continuations on that path are folded into the one record — its cost is the run's whole token cost, not a per-attempt figure. A call that failed before any request completed still yields an empty list. The one exception is an Anthropic structured / skills run that paused: it returns one record per request (the first request, then each continuation, all but the first carrying pause_turn_continuations: 1), each priced from its own usage, and the run totals are their sums. The final record carries the run's built-in tool cost, as the synthesised record does. Before this release that path priced only the final response, so every earlier response of a paused run — input, output, cache reads and writes — was billed at $0.

That folding now also covers Gemini's MAX_TOKENS truncation retry, which re-issues the request with a higher ceiling and reports only its own usage. The truncated attempt was a complete, billed request that never appeared in the run's totals; it is now folded in like any other failed-and-retried attempt, named in retry_details, and the retry carries the requested service tier forward (see Service Tiers).

An image session records per request too

An ImageSession turn carries its own cost_records, one per API request, and turn.cost is their sum; the session's total_cost is the sum over turns. The same rule, for the same reason. An OpenAI persistence="server" step produces two records — the mainline response_model and the image model, both billed. The image record is priced from the image model's own token counts in response.tool_usage["image_gen"], with cost_estimated=False. Only when that block is missing does it fall back to the per-quality price table, flagged cost_estimated=True.

An iteration's cost in iteration_metrics is the same figure as its request record's cost — it is carried over, not recomputed. That matters for tiered models: a second, independent pricing pass would not know which tier the request was actually served under, and would also emit a duplicate cost line into the log for every request.

Early-exit runs now record their iteration too

A run can leave a provider loop before it reaches the normal end-of-iteration bookkeeping — a forced-tool or allowed_tools exit, a truncation or content-filter finish reason, the structured-output Path-B success, or an error return. Those paths used to return iteration_metrics: [] while request_cost_records was populated, so a successful forced-tool run persisted as a zero-iteration, zero-total DetailedCostReport even though a request had been made and billed.

All four providers now record the in-flight iteration before every such exit, using the values already in scope and the same key set as the normal path — calculate_cost is never called a second time, and a per-pass watermark makes double-counting impossible (it replaced an iteration-number guard once a round's two requests came to share a number — see One iteration entry per request). AgentResult.cost and request_cost_records were always correct and are unchanged; what changed is that per-iteration duration, tool names, and stop_reason telemetry no longer vanishes on those paths.

Tool-reported spend

Some tools spend money of their own: a speech or image tool calls a model, a sub-agent runs a whole agent. Since 0.5.1 such a tool can report what it spent, and the calling agent adds it to its own cost. The marker is one key at the top level of the tool's result:

from doc_processing_toolkit import tool
from doc_processing_toolkit.utils import tool_spend_marker


@tool
def translate(text: str) -> dict:
    result = my_translation_call(text)   # your own paid API
    return {"text": result.text, **tool_spend_marker(result.cost, estimated=False)}
    # -> {"text": ..., "_dpt_spend": {"usd": 0.0123, "estimated": False}}

• Only _dpt_spend is read. A cost field in a tool result is never taken as spend: the agent factory's estimate_task_cost returns a projection and get_agent_session_status a running total. A malformed value (not a non-negative number) is ignored with a WARNING.
• It travels over every transport. The marker is part of the result's JSON, so local @tool functions, in-process FastMCP, stdio servers and client-side remote servers (mcp_remote_client) all carry it; a provider-run {"type": "mcp"} server's results never reach the toolkit, so they cannot. All four tool loops read it with utils.read_tool_reported_spend before max_tool_result_chars cuts the text the model sees, so truncation cannot hide it.
• Tools that emit it: the multimedia server's synthesize_speech, transcribe_audio, generate_image and edit_image, and the agent factory's create_agent (its child run's cost; a child run that failed reports cost: None and no marker, because its spend is unknown) and suggest_agent_config (both of its sub-agent runs when the first reply was unusable). send_to_agent_session does not: the turn it starts finishes in the background and is billed on that session's own totals. A third-party MCP server has no standard way to report spend, so its calls add nothing.
• Where it shows. AgentResult.tool_reported_cost is the sum, tool_reported_costs has one record per reporting call (tool_name, tool_call_id, usd, estimated, iteration), and both are already inside cost, so session totals include them. An estimated figure sets cost_estimated and tool_costs_estimated[tool_name]. The detailed cost report carries total_tool_reported_cost and tool_reported_costs. Each tool_call_result stream and hook event of a reporting call carries tool_reported_cost and tool_reported_cost_estimated, so an abandoned stream's record and a hook abort's partial include the tools that had already run.
• A failed or retried run keeps it. A tool that ran before the run failed was still paid for, so the spend travels with the error. A retried attempt's records are added to the final result's tool_reported_costs, and a run whose retries ran out keeps its last attempt's records. A stream's error event adds the spend to its cost and names it in tool_reported_cost / tool_reported_costs; those two keys are absent when no tool reported anything. A hook abort counts it once: the partial already includes it, and the error payload does not add it again.
• Realtime sessions do not count it. A realtime session runs tools through its own dispatcher, and a _dpt_spend key in their results is passed to the model unread.
• The request records are unchanged. They stay token-only, so cost is now the sum of request_cost_records, the built-in tool charges and tool_reported_cost.
• Counting it yourself already? If your app adds the multimedia or agent-factory tool costs to the agent's cost itself, it now counts them twice. Stop adding them, or set DPT_AGENT_TOOL_REPORTED_COST=off, which restores the totals from before 0.5.1 exactly.

Per-model cost attribution (Anthropic server-side fallback)

On Claude Fable 5, Fable 5.1, Opus 5.5, Opus 5 and Sonnet 5.5, a request the model's safety classifiers decline can be retried on a fallback model inside the same API call — Claude Opus 4.8, or Claude Opus 5 where the row permits it (Fable 5.1 lists both; Opus 5.5 lists Opus 5 first, then Opus 4.8), and Claude Sonnet 5 for Sonnet 5.5. The fallback is billed at its own rates, not the requested model's: an Opus 5.5 request that falls back to Opus 5 bills Opus 5's cached reads at $0.50/M rather than Opus 5.5's $0.20/M. Each billed attempt is priced at the rates of the model that actually ran it, so a single run's tokens can span more than one model.

Field	Description
AgentResult.token_totals_by_model	Billed tokens attributed to the model that ran them (a billed attempt that declined included), e.g. {"claude-opus-4-8": {"input_tokens": 412, "output_tokens": 264}}. Contains only the requested model unless a fallback ran. Empty for providers/results that do not report it.
IterationMetrics.model	The model that served that iteration's final API call. Differs from the requested model only after a fallback. None where not reported.

An attempt that was declined before producing any output is billed only when its refusal category is one Anthropic bills (Claude refusals are billed by category); an unbilled attempt is excluded from the attribution. cost already reflects per-model rates — these fields exist for transparency and drill-down, not for recomputing the total.

Caching across a fallback

Each attempt's usage.iterations entry (BetaMessageIterationUsage for a declined hop, BetaFallbackMessageIterationUsage for the serving one, anthropic 0.109.1) carries its own cache_read_input_tokens / cache_creation_input_tokens / cache_creation. Every billed attempt is priced from those: its reads at its own model's cached_input rate, its writes at its own model's cache_write_* rate and the TTLs it was served, and its reads added to its own row in token_totals_by_model; the reported cached_input_tokens / cache_write_tokens are the sums over the billed attempts. Earlier builds priced the call's top-level cache figures once, at the served model's rates, which mis-billed two attempts that cached differently. Only an entry that carries no cache fields at all (an older API shape) still falls back to that top-level pricing.

Claude refusals are billed by category

A Claude request that a safety classifier declines comes back as HTTP 200 with stop_reason: "refusal" and a stop_details.category. Whether Anthropic bills it depends on when the refusal arrives and on that category, and the toolkit prices it the same way (0.5.2):

The refusal	Billed by Anthropic	What the result reports
Arrives before any output, category bio, frontier_llm or reasoning_extraction	Yes, like any request, at the rates of the model that ran it	The input tokens, the cache reads and writes, and their cost
Arrives before any output, any other category (cyber, general_harms) or no category	No	The input tokens as the API reported them, and a cost of 0.0. cached_input_tokens and cache_write_tokens read 0: no read was discounted and no write charged. (A server compaction pass on the same response is still billed, and its own cache counts are reported)
Arrives after some output (mid-stream)	Yes: the input and the output already produced	Tokens and cost, as for any request

usage carries the request's tokens in every row, so the usage alone cannot tell a billed refusal from a free one; the category does. The billed categories are registry data (pricing.anthropic.refusal_billing.billed_before_output_categories), because Anthropic says the list may change: when it does, the fix is one line of JSON.

The same rule applies to each attempt of a server-side fallback. An attempt that declined before any output is billed when its category is a billed one, at the rates of the model that declined. An attempt that produced output is billed whole, whatever happened after: if it ran a server tool and then declined, its last, output-less step is billed with the rest. The category of a declined attempt is read from the response's fallback block (trigger.category); for the last attempt of a chain in which every model declined, it is the response's own stop_details.category. An attempt whose category the response does not state is left unbilled. A last attempt that reported no output and did not decline (it stopped on the context window, say) is priced as before 0.5.2.

The request's tokens follow its bill. The API's top-level usage describes only the attempt that produced the returned message. When an earlier attempt is billed too, the request's cost record, AgentResult.input_tokens / output_tokens, the iteration_metrics entry and token_totals_by_model all count every billed attempt, so a record's tokens match its cost and its cached_input_tokens stay a subset of its input_tokens. A request that sent 540 tokens to the model that answered, after a billed decline of 474, reports 1,014 input tokens. When no attempt was billed, the tokens the API reported stay on the result at a cost of 0.0.

This holds for every response on which a fallback ran, a refusal or not, so on such a response a server compaction pass is in the request's tokens as well as in its cost. Without a fallback, a tool-loop request still counts the pass in its cost and its cache counts only, not in input_tokens (the single-request path counts it in both).

Two live cases from 2026-10-02 show what changed:

• A claude-sonnet-5-5 request declined for reasoning_extraction and answered by claude-sonnet-5: the 489 input tokens of the declined attempt ($0.000978) are now part of the run's cost and of its token counts. They were priced at zero and not counted.
• A claude-opus-5-5 request that claude-opus-5 declined as well: the run now reports $0.004653 for the two attempts (517 input tokens each). It reported $0.00.

Before 0.5.2 the toolkit never billed a fallback attempt that declined before any output, always billed a refusal that had no fallback, and took a request's tokens from the top-level usage alone. An attempt that declined partway was billed for the steps that produced output and not for an output-less last step (after a server tool); that step is now billed too, so such a request costs more than it did. DPT_ANTHROPIC_REFUSAL_BILLING=legacy restores all four exactly.

Anthropic server-side compaction is billed

When Anthropic compacts a conversation server-side, the response reports two separate pieces of work. The top-level usage covers the pass that produced your answer; the compaction pass — the prompt it read and the summary it wrote, both of which Anthropic charges for — appears only as a compaction entry inside usage.iterations.

The toolkit prices that entry as its own request, folding its cache reads into its input at the cached rate and charging its cache writes at the 5-minute or 1-hour rate the response says was actually served. Iterations the top-level total already aggregates — the server-side tool loop's own message passes — are deliberately not counted again (providers.anthropic.ANTHROPIC_SEPARATELY_BILLED_ITERATIONS and ANTHROPIC_AGGREGATED_ITERATIONS are the two sets). An iteration type the toolkit does not recognise is left unbilled and named in a warning, so the reported cost can under-state the invoice but never invents a charge.

Server-compacted responses used to be billed at $0

The usage.iterations array was read only when a server-side fallback entry was also present, so a compacted response — which carries a compaction entry and no fallback_message — had the whole array ignored. On a 1M-context model at $10/M input that is $1.50 of silent under-report for a single 150k-token compaction.

The same read now copes with both shapes the SDK produces: the beta Usage type declares iterations as typed objects, while the non-beta one hands them back through model_extra as plain dicts, where the served cache-TTL split used to be missed.

This applies to both Anthropic entry points. A run with no local tools — the structured-output and plain-text path, which is where a context_editing run with compaction switched on and no function tools lands — priced only the top-level usage until this release, so its compaction pass was billed at $0: one measured request reported $0.002 against $0.172 owed. That path now bills the iteration and also reports its tokens in the result's input_tokens, output_tokens and cached_input_tokens, so its cost and its token counts agree.

Retry Cost Tracking

API calls that fail and are retried can silently consume tokens that never appear in the final result. The toolkit now tracks costs across all retry attempts so you can see the full picture.

How it works

When a provider API call fails and tenacity retries it, a RetryAccumulator captures cost data from each failed-and-retried attempt via tenacity's before_sleep hook. Data from the final attempt (the successful one, or the one that exhausted retries) stays in the provider result's top-level cost/token fields. At the end of the call the accumulator's totals are rolled up — failed-attempt cost is added into the top-level totals, and the retry share is surfaced as separate diagnostic fields on AgentResult. IterationMetrics carries the same field names, but they are reserved and always zero.

Each failed-and-retried attempt is priced, with the model's registry rates, from the tokens recovered from its broken stream — the served tier and the cached share honoured on the same conventions as a successful request. When the stream reported no input count, the pre-flight estimate is used and the attempt (retry_details[*].cost_estimated) and the run (cost_estimated) are marked estimated. retry_cost is the sum of those prices and cost includes it; the final attempt of a run that exhausts its retries is priced the same way. Before this release the hook recorded the attempts' tokens but never a price, so retry_cost was always 0.0 and cost silently omitted every retried attempt — a run that retried now reports a higher, correct cost.

A failed tool-loop attempt is billed per request

Tenacity retries the whole provider call, so an attempt that broke on its third request had already been billed for its first two. Each tool loop now attaches those requests' already-priced per-request records to the failed attempt (request_cost_records in _partial_stream_data, on all four providers), and the attempt is billed as their sum plus the in-flight request's partial output — the same rule as a successful run. It used to be priced from the loop's run-to-date token sum as one request, which on a tiered_200k / tiered_272k row crossed a tier threshold no single request crossed: two completed 150k/160k requests on gemini-3.1-pro-preview were billed $1.24 instead of $0.62. The completed requests' output tokens, which the attempt used to drop, now count in retry_output_tokens and the attempt's cost, so on a standard-priced row a retried multi-request attempt can report slightly more than before; on a tiered row that crosses the threshold it reports less. A single-request attempt is priced exactly as before.

One retry layer

Tenacity is the toolkit's only retry layer, so every retried request is one of the attempts counted above. The Anthropic and OpenAI SDK clients are built with max_retries=0 (Fireworks already was; google-genai has no SDK retry layer to turn off) — before this, each of tenacity's five attempts could hide up to two silent SDK retries, a worst case of 15 requests of which only 5 were logged or billed in retry_count / retry_cost. So that nothing the SDKs used to retry stops being retried, the retry predicate retries HTTP 408, 409, 429 and every 5xx except 501 (so 500, 502–504, 529 and gateway statuses such as Cloudflare's 520–524), connection and timeout errors from every SDK, and the rate-limit exception classes. Gemini's google.genai ClientError / ServerError are classified by the same status rules, read from their .code, so a Gemini 429 or 503 is retried like any other provider's. A server x-should-retry response header overrides those rules the way both SDKs treat it: true retries whatever the status, false stops a retry even on a retryable status. The wait between attempts honours a server retry-after-ms / retry-after header (seconds or an HTTP date, capped at 60 s) and otherwise keeps the exponential back-off of 4 s, 4 s, 4 s, 8 s. Model outputs are unaffected; a one-off 429 now waits tenacity's first back-off instead of the SDK's half-second. DPT_SDK_RETRIES=default restores the SDKs' own retries.

AgentResult retry fields

Every AgentResult exposes the run-level retry aggregate at the top level:

Field	Description
retry_count	Total number of failed-and-retried API attempts across the run. Zero on a clean run.
retry_cost	USD cost of the failed retry attempts only. cost always includes this value; subtract retry_cost from cost to get the cost of the successful (or final-exhaustion) attempt alone.
retry_input_tokens	Input tokens consumed by the failed-and-retried attempts only. Already included in input_tokens. Zero on a clean run.
retry_output_tokens	Output tokens produced by the failed-and-retried attempts only. Already included in output_tokens. Zero on a clean run.

These run-level fields are the authoritative retry figures. Read retry spend from here, not by summing the per-iteration fields below.

IterationMetrics retry fields

Each iteration in the result's iteration_metrics list carries the same field names:

Field	Description
retry_count	Reserved — always 0.
retry_cost	Reserved — always 0.0.
retry_input_tokens	Reserved — always 0.
retry_output_tokens	Reserved — always 0.
retry_details	Reserved — always []. Shape when populated: per-attempt breakdowns with attempt_number, input_tokens, cached_input_tokens, output_tokens, cost, cost_estimated, token_source, error, duration_ms.

A failed attempt keeps its cached share, and is billed by the same rules

A stream that fails carries the cached share of its prompt — cached_input_tokens, a subset of that attempt's input_tokens on the same convention as everywhere else. A run that ends on a failed request is priced by utils.extract_partial_stream_cost with the same inputs a successful request gets: the cached prefix at the cache-read rate instead of as fresh input, the tier the provider actually served where the attempt carries one, its modality breakdowns, and cost_estimated rather than an estimate presented as exact — a 100k-token prompt with a 90k cached prefix on claude-sonnet-5 used to be reported at $0.205 and is $0.043. Every provider's error result forwards that attempt's cached_input_tokens and cost_estimated to AgentResult. Attempts that were retried carry the field on RetryAttemptCost, and their cached share rolls into AgentResult.cached_input_tokens together with their input tokens, so input_tokens - cached_input_tokens stays the uncached figure on a retried run.

Where the cached count comes from: an Anthropic stream reports the cache reads announced on its message_start event (0 if it died before that event), and the Google, OpenAI and Fireworks tool loops report their run-to-date cached count. The single-request structured-output paths on Google, OpenAI and Fireworks claim no cached share — the attempt's usage was never read, so its input is a pre-flight estimate, and the whole estimate is priced as uncached.

Per-iteration retry fields are reserved and always zero

Retries happen inside a single provider call, below the iteration boundary, so no provider attributes them to an iteration. All four providers emit these keys hardcoded to zero. They are kept for shape stability and possible future per-iteration attribution.

Before 0.3.0, AgentSession derived its total_retry_* metadata by summing them — which made every session retry total permanently zero even on runs that retried. Sessions now read the run-level fields instead, so total_retry_cost and friends reflect real retry spend.

Error billing classification

Not all errors consume tokens. is_billable_error() classifies errors to determine whether cost estimation is attempted:

Classification	Error Types	Tokens Charged?
Billable	Timeouts, 500 Internal Server Error, 529 Overloaded	Yes — generation ran before failure
Non-billable	400 Bad Request, 401 Unauthorized, 403 Forbidden, 404 Not Found, 429 Rate Limit	No — request rejected before processing

A rejected request costs nothing

A request the provider rejects before it streams anything is billed nothing, and the toolkit now reports it that way on every provider. The test is read from the exception alone (utils._rejected_before_stream): the SDK error carries an HTTP response with an error status — a streamed response is sent with a 2xx status before its first chunk, and a mid-stream error arrives with that 2xx response or with none — or the connection never reached the server (ConnectError / ConnectTimeout). Such a failure's record carries cost 0.0, zero tokens, cost_estimated False, and token_source "rejected_before_stream" (the helper's record also names it in reason; utils.REJECTED_BEFORE_STREAM is the constant). It used to be a "stream_partial" record billing the pre-flight input estimate — 13 tokens, $0.00013 on one live 400 — a confident non-zero for unbilled work.

The rule wins over the status class: a 5xx or 529 that came back as an HTTP error response is not charged its input estimate either, although the table above calls those statuses billable — that classification still applies when the failure carries no response to read. A stream that failed after its first chunk keeps the estimated partial billing. In a tool loop, a request rejected after earlier requests completed bills exactly those completed requests. It is a reporting correction, with no switch: a figure that could only have been too high goes to zero.

DetailedCostReport retry summaries

When detailed_costs=True, the cost report carries the run's retry aggregates:

• total_retry_cost — USD cost of the failed-and-retried attempts
• total_retry_input_tokens / total_retry_output_tokens — tokens spent on those attempts
• total_retry_count — number of failed-and-retried API attempts
• total_cost_inclusive — a derived key with no backing attribute that now simply equals total_cost, kept for backward compatibility

Each iteration entry also includes a per-attempt retry_details list.

Report retry totals became real in 0.3.0

These four totals used to be summed from the per-iteration retry fields, which are reserved and always zero — so a run that retried heavily still persisted total_retry_cost: 0.0. The report now takes them from the run-level AgentResult fields, the same source AgentSession uses, so they reflect actual retry spend.

The per-iteration retry_* fields and retry_details are still reserved: every iteration entry reports zeros and an empty list. Only the run-level totals are populated.

total_cost_inclusive no longer double-counts retry spend

total_cost is set from AgentResult.cost, and that value already includes retry_cost, so the old total_cost + total_retry_cost derivation over-reported: a run billed $0.0500 of which $0.0100 was retries persisted "total_cost": 0.05 and "total_cost_inclusive": 0.06, though $0.06 was never charged. The key now simply equals total_cost, and is retained only so existing readers keep working. The same correction applies to AgentSession.metadata["total_cost_inclusive"].

Read total_cost as the billed figure and total_retry_cost as the retry share within it. If total_retry_cost is a significant fraction of total_cost, investigate timeout configuration or provider stability. See Understanding Retry Costs.

Related recipes

• Cost Tracking Report
• Thinking + Reasoning