Providers

The toolkit routes model calls across Anthropic, OpenAI, Google, and Fireworks.

Provider routing

• Model-to-provider detection is metadata-driven (data/model_information.json).
• Agent runtime selects provider function-calling vs structured-output APIs.

model_information.json is the canonical metadata source for:

• model limits
• feature support flags (tooling/reasoning/media capabilities)
• pricing and cost metadata, including the optional prompt-cache keys cached_input, cache_write_5m, and cache_write_1h (see Cost Tracking)
• the tiktoken encoding used for local token estimates (descriptions.<model>.tokenizer_encoding)
• the image dimension rule, aspect ratios and output formats a row accepts (size_constraints, supported_aspect_ratios, supported_output_formats, supports_input_fidelity) — see Multimedia
• descriptions.<model>.supports_forced_tool_choice — see Forced tool choice

This file should be treated as configuration data, not narrative API docs.

Non-text rows may publish no capacities

limits.<provider>.models.<model>.context_window_tokens and its output cap may be null on a row whose descriptions.<model>.modality is non-text — an image, TTS, STT or realtime model whose vendor publishes no such numbers — rather than being given invented text capacities. Text rows stay strict and must publish both. Such a row is loud, not silent, if it ever reaches resolve_context_window: the usual warning fires, names the exact JSON path, and floors at 128,000.

Rate limits are one account's figures

A limits row also carries rpm, tpm and rpd. Nothing in the toolkit throttles on them; get_model_info reports rpm and tpm. They describe the account the registry was measured on, and limits.<provider>.notes says where each provider's figures come from. Since 0.5.1 the Google rows carry this organisation's Tier 3 figures from AI Studio's per-project limits page (2026-09-27), where null records "Unlimited" (most rows' daily cap, every Live row's RPM). Check your own account's limits before sizing a workload by them.

Which session families a model serves

utils.resolve_session_kinds(model) returns the session families a model can serve, drawn from utils.SESSION_KINDS — text, realtime, live, transcription, translation, speech, image — and utils.get_session_kind_capabilities() returns the whole (provider, kind) -> models record. They are derived from descriptions.<model>.modality and the capability flags already on the row, so most rows need no maintenance of their own; a row may also state session_kinds outright, which is read verbatim, for the rows derivation cannot reach.

They exist because a flat provider -> adapter map cannot express a dual-surface model: gpt-realtime and gemini-3.1-flash-live-preview serve both realtime and transcription on one socket, and whisper-1 serves both transcription and translation. Realtime session routing now consumes the record — open_realtime_session(model, session_kind=...) resolves (provider, session_kind) to an adapter and refuses an unserved kind before the socket opens (see Which kind of session) — and it is published for callers routing work to a model.

Derived and declared are not interchangeable when you open a socket. A conversational realtime row derives transcription because its own session transcribes what it hears, and that derived statement stays exactly as it is in discovery. Opening such a row as a dedicated session of another kind is refused, because the dedicated adapters configure a session a conversational model rejects: open_realtime_session(model, session_kind=...) accepts a non-conversational kind on a row that also serves the conversational kind only when the row declares that kind in session_kinds.

Row provenance

Any registry row may carry _-prefixed metadata keys — _source for where a number came from, _verified for when it was last checked, _tier for which account tier a limits row describes. They exist so a reader can tell a published figure from a floor, a ceiling or a guess.

Nothing in the toolkit reads them and nothing emits them. Readers select row keys by name, and the two places that hand a whole row onward — the get_model_info discovery tool and the model catalogue suggest_agent_config builds — filter them out first with utils.strip_registry_metadata. detect_provider rejects a _-prefixed name outright, and pricing rows are validated against a closed key vocabulary, so a provenance note that loses its underscore fails the test suite instead of becoming a silent unpriced field.

Before this release a _-prefixed key inside pricing.<provider>.models took the whole list_available_models tool down with 'str' object has no attribute 'get' — not just the offending row — which is why row-level provenance was previously unsafe.

Models that cannot take a forced tool choice

A model that does not declare supports_function_calling_and_structured_outputs gets structured output through a synthetic submit_final_response tool, which requires a forced tool_choice. A row may declare supports_forced_tool_choice: false for a model that ignores or rejects one; absent, the key means true.

On Fireworks the flag has always been warn-only. On Anthropic it is now a gate: combining output_schema with tools on such a model raises a clear compatibility error before any request is sent rather than returning a provider 400 mid-session, and a caller-supplied any/tool choice is downgraded to auto with a WARNING naming the model and supports_forced_tool_choice.

claude-fable-5-1 was the first row to carry it, claude-opus-5-5 the second and claude-sonnet-5-5 the third. The two keys travel together by design: each also declares supports_function_calling_and_structured_outputs, which keeps structured output on the native output_config.format path — without that, every structured run would take the forced-tool path and 400 unconditionally. See Steering tool use with tool_choice.

Common provider capabilities

• Function calling with tool execution
• Structured output via response schema
• Token and cost metadata normalization, including cached-prompt-token extraction and one per-request cost record per API call

Provider-specific parameter examples

• Anthropic: effort_config, skills, max_pause_turn_iterations
• OpenAI: verbosity_config, parallel_tool_calls, image_detail, openai_transport
• Google: media_resolution, media_resolution_list, video_metadata, file_active_timeout_s, file_poll_interval_s, file_poll_backoff
• Fireworks: OpenAI-compatible function/structured API shape (with feature differences)

Fireworks model name format

Fireworks model names use the full path format: accounts/fireworks/models/<model-name> (e.g., accounts/fireworks/models/kimi-k3, or accounts/fireworks/models/ember-1, Fireworks' own Kimi K3 derivative, added in 0.5.1). Check model_information.json or use list_available_models() for the full list of supported models.

Fireworks Fast serving path (routers/ namespace)

Fireworks selects its Fast serving path (~100+ tokens/sec, identical model quality, higher per-token price) via a distinct model ID under the routers/ namespace — accounts/fireworks/routers/<model>-fast (e.g., accounts/fireworks/routers/kimi-k3-fast, accounts/fireworks/routers/glm-5p3-fast) — not via a request parameter. Fast tiers are first-class entries in model_information.json with their own pricing and limits; pass the verbatim router ID as the model name to use them.

Built-in tools by provider (high-level)

• Anthropic: built-in web/code tooling plus provider-specific skill support
• OpenAI: built-in tools + reasoning/verbosity controls
• Google: built-in tools and rich media controls with model-specific constraints
• Fireworks: OpenAI-compatible paths with feature-specific differences

Always validate model-level support using model_information.json and integration tests.

One remote-MCP marker on every provider

result.built_in_tools_invoked names the built-in tools with evidence they ran. A provider-executed remote MCP call ({"type": "mcp", ...}) is reported as the neutral "remote_mcp" on Anthropic, OpenAI and Fireworks, so "remote_mcp" in result.built_in_tools_invoked answers "did a remote MCP tool run?" without a per-provider list. Anthropic keeps the MCP tool's own name beside it (for example ["ask_question", "remote_mcp"]), because its mcp_tool_use blocks carry it; before this release Anthropic reported the tool name alone. Google has no provider-executed remote MCP (use mcp_remote_client, which runs client-side and shows up in tools_used instead).

Combining built-in tools with custom tools

Whether a model can run provider built-in tools and your own @tool/MCP tools in the same request is per-model, driven by the registry flag supports_concurrent_built_in_and_custom_tools. When it is false, supplying both raises a ValueError from validate_builtin_tools_concurrency.

Provider	Concurrent built-in + custom tools	Notes
Anthropic	Supported	Both halves of every server_tool_use / *_tool_result pair are echoed back automatically
OpenAI	Supported	—
Google	Preview — Gemini 3 text models only	Every Gemini 3 text row is flagged true; Gemini 3 image rows and all Gemini 2.5 rows are false. The flag, not a fixed list, is the contract
Fireworks	Not supported	false for every Fireworks model

Google's combination path is Preview, and its required request shape changed in 0.3.0 — split Tool objects plus tool_config.include_server_side_tool_invocations, where the older merged shape is still accepted but silently skips the built-in. A Preview caveat around file artifacts also applies. See Built-in tools recipe for the request shape, the affected models, and the caveat.

Anthropic model notes

Claude Fable 5.1

claude-fable-5-1 is the registry's first preserved-thinking model: the API checks that every thinking block you replay still belongs to the conversation prefix — the system prompt, the tools array and the earlier messages — that produced it. The toolkit answers that check automatically for this row; see Preserved thinking and block binding.

Four things about the row are worth knowing before you point production at it.

• Adaptive thinking is always on and cannot be disabled. Every run produces and bills reasoning tokens; thinking={"type": "disabled"} is coerced to adaptive with a warning (why).
• A forced tool_choice is rejected by the API, so the toolkit downgrades one to auto and structured output takes the native path (Models that cannot take a forced tool choice).
• It is not offered on Priority Tier. service_tier="priority" bills at standard rates with a warning (Service Tiers).
• Cache reads are $0.25 per million — 2.5% of the $10 input rate rather than the usual 10% — which makes prompt caching unusually worthwhile on long sessions. Output is $50/M, the context window is 1M tokens and the per-request output cap is 128k.

It may also fall back server-side to Claude Opus 4.8 or Opus 5, and its rate-limit bucket is null in limits because none is published — deliberately, rather than inheriting another model's ceiling, which a consumer would size batches against.

Deploying it requires 30-day data retention

There is no zero-data-retention option for this model without express authorization from Anthropic. Clear that before a regulated workload lands on it.

Claude Opus 5.5

claude-opus-5-5 succeeds Claude Opus 5 for long-running agentic coding and knowledge work, at a lower price: $4 input / $20 output per million tokens, with cache reads at $0.20 (5% of the input rate — deeper than Opus 5's 10%) and 5-minute / 1-hour cache writes at $5 / $8. The context window is 1M tokens and the per-request output cap is 128k. It is the second preserved-thinking row after Fable 5.1, and it shares three of Fable 5.1's four breaking changes; the toolkit handles each from the registry row alone.

Breaking change	What the API does	Registry key that handles it here
Thinking cannot be turned off	thinking: {"type": "disabled"} and {"type": "enabled", "budget_tokens": N} both 400, at every effort level	adaptive_thinking_always_on — a disabled or absent config is coerced to adaptive with a warning naming the model; a budget config is converted to adaptive with a warning
No forced tool choice	tool_choice any / tool 400 on Messages, Batches and token counting	supports_forced_tool_choice: false — a forced choice is downgraded to auto with a WARNING naming the model and the key; supports_function_calling_and_structured_outputs: true keeps structured output on the native path
Preserved-thinking prefix check	A replayed thinking block whose prefix changed 400s (enforced by default for accounts created on or after 2026-08-31)	The three binding keys — every request carries thinking.block_binding.prefix_mismatch_behavior: "drop_block" and the binding beta, exactly as on Fable 5.1 (Preserved thinking)
New computer-use tool version	On the Claude API only computer_toolset_20260801 is accepted; computer_20251124 400s	None needed — the toolkit has no computer-use built-in tool

Assistant prefill is rejected as on every always-on row; the toolkit never sends one (the compaction seam and the max_tokens recovery included), so nothing row-specific is required. Five more things are worth knowing before you migrate from Opus 5.

• The default effort is medium, not high. The toolkit sends no output_config.effort unless you set one, so an Opus 5.5 run that omits it thinks one level shallower than the same run on Opus 5. Pass effort_config explicitly ("low" / "medium" / "high" / "xhigh" / "max") when you switch models (Thinking).
• Text between tool calls can arrive as thinking. A response can begin with a thinking block, and what an earlier model wrote as text between tool calls can come back as a progress-update thinking block (at most one before each tool call) — empty at the API's default display (omitted); the toolkit asks for summarized, which returns its text. In the runs of 2026-10-03 the model wrote those remarks as ordinary text instead (Progress notes).
• Safeguard classifiers cover cyber, bio and reasoning_extraction. A decline is HTTP 200 with stop_reason: "refusal", and server-side fallback passes it to the next model like any other decline. That model has classifiers of its own: a reasoning_extraction decline on Opus 5.5 was declined again by Claude Opus 5 (2 of 2 on 2026-10-02), so the run failed with the refusal and served_model: "claude-opus-5", and both declined attempts are billed (Cost Tracking). (An earlier version of this page said fallback does not retry this category. It does; the retry met the same answer.) The row's allowed_fallback_models are claude-opus-5 (the default) and claude-opus-4-8; both drop Opus 5.5's thinking blocks, unbilled — on the Claude API only Fable 5.1 and Mythos 5.1 read them. A fallback request is built for Opus 5.5 and so carries the binding beta, which means the fallback model's drop is expected to appear in input_transformations and model_binding_mismatches; that has not been observed live yet (who reads whose blocks).
• It is not offered on Priority Tier. service_tier="priority" bills at standard rates with a warning, and its rate-limit bucket is null in limits: the model has its own pool, which is not published.
• Fast mode and batch pricing are not modelled. Fast mode ($8 / $40, research preview) needs a speed parameter the toolkit does not have, and the toolkit has no batch path.

Unlike Fable 5.1, Opus 5.5 documents no data-retention prerequisite. Native tool_search was verified live on 2026-09-23 — both the regex and bm25 variants returned a tool_search_tool_result and ran the deferred tool, with cache hits (Tool search and deferred tools). Programmatic tool calling, the skills keys and effort at xhigh / max are still carried on the row by inheritance from Opus 5, pending a live check.

Claude Sonnet 5.5

claude-sonnet-5-5 (released 2026-09-28) succeeds Claude Sonnet 5 at the same price: $2 / $10 per million tokens, cache reads $0.20, a 1,000,000-token context window and a 128,000-token output cap. The minimum cacheable prompt is 512 tokens, half of Sonnet 5's. Its only server-side fallback target is claude-sonnet-5, and the toolkit turns that fallback on by default.

What differs from Sonnet 5, and what the toolkit does about each:

Change	What the API does	What the toolkit does
Thinking cannot be set to disabled	400, naming between_tools as the replacement	thinking={"type": "disabled"} sends between_tools on this row, up to effort "high"; above it the run is adaptive with a warning (Turning thinking off)
between_tools takes no other key	display and block_binding beside it are a 400	A thinking-off run sends the value alone, leaves earlier turns' thinking blocks out, and gives the fallback entry thinking: {"type": "disabled"} (why)
No forced tool choice	tool_choice any / tool is a 400	supports_forced_tool_choice: false: a forced choice is downgraded to auto with a WARNING, and structured output takes the native path
Thinking blocks are bound to the conversation before them	A replay after an edit to earlier history is a 400, on accounts created from 2026-08-31	The third row with enforces_thinking_prefix_check: a thinking run sends block_binding: drop_block and its beta header
No sampling parameters, no thinking budget	temperature, top_p, top_k and {"type": "enabled", "budget_tokens": N} are a 400	temperature is dropped with a WARNING (hyperparameters: ["max_tokens"]); a budget is converted to adaptive
Thinking on by default, text omitted	A request with no thinking field thinks, and returns empty thinking blocks	adaptive_thinking_always_on: no config means {"type": "adaptive", "display": "summarized"}, so the thinking is readable

Four things to know before pointing a workload at it.

• Text between tool calls can arrive as thinking, not text. The short updates the model writes before a tool call can come back as thinking blocks: in reasoning_content and thinking_delta events, not in output or text_delta. With thinking turned off they can still arrive that way, and are billed as output. It is the model's choice: in the runs of 2026-10-03 the same kind of remark arrived as ordinary text (Progress notes), so an interface that shows them should read both.
• A prompt that asks the model to narrate its reasoning is refused under the default thinking setting. A system prompt reading "Before each tool call, write a short paragraph on what you just learned and what you will check next" was declined with stop_details.category: "reasoning_extraction" on every attempt (3 of 3 on 2026-10-02 with fallback off, 6 of 6 the day before). With thinking turned off the same prompt ran on Sonnet 5.5 every time (3 of 3) and returned the notes as thinking. Turning thinking off is not a general remedy, though: a system prompt asking for the reasoning verbatim inside <thinking> tags, and one asking for a reasoning field in JSON output, were refused with thinking on and with thinking off (2 of 2 each way), while "give a short explanation of how you reached the answer" ran both ways. Ask for a short explanation or a summary of actions.
• With the default fallback, that refusal is answered by Claude Sonnet 5, and the conversation stays there. The run succeeds and request_cost_records[i]["served_model"] reads claude-sonnet-5 (4 of 4). Anthropic then routes later requests of that conversation straight to the fallback model for about an hour, as long as they carry fallbacks, which every run on this row does by default: 10 of 10 thinking-off runs made after a refused one were served by Sonnet 5, where the same run with fallbacks=[] was served by Sonnet 5.5. Read served_model rather than assuming, and pass fallbacks=[] to keep a workload on Sonnet 5.5 and see the refusal. Anthropic bills the declined attempt as well as the answer, because reasoning_extraction is a billed refusal category, and the run's cost includes it (Cost Tracking).
• The default effort is "high", and the levels are recalibrated. A level does not produce the same amount of thinking as on Sonnet 5. Anthropic suggests "medium" for well-specified agentic work and for latency-sensitive chat. Adaptive thinking also decides per request whether to think at all, and Anthropic notes that on a reasoning task with JSON output the model often answers without thinking first. A short combinatorics question sent with an output format, which Sonnet 5 and Haiku 4.5 think about, was answered with no thinking on 4 of 4 runs at the default effort, so reasoning_content can be empty on a thinking run.

Sonnet 5.5 reads thinking blocks from Sonnet 5, Opus 4.8, Haiku 4.5 and earlier models, and not from Opus 5, Opus 5.5 or any Fable or Mythos model. Its own blocks are read by Opus 5.5 and by no other model, and only in the account that produced them (Who reads whose blocks). It is not offered on Priority Tier. Checked live through Agent on 2026-10-02: web search, web fetch (including from inside code execution), code execution, tool search (regex and bm25), built-in skills, remote MCP, structured output with tools, and prompt caching.

Claude Haiku 5.5

claude-haiku-5-5 (released 2026-10-07) is the fast, low-cost Claude 5 row for classification, routing, extraction and subagent work: a 1,000,000-token context window, a 128,000-token output cap, a 512-token minimum cacheable prompt, the Scale rate-limit bucket (10,000 requests, 10M input and 2M output tokens a minute) and no server-side fallback — a refusal on this row is final, and the registry publishes an empty allowed_fallback_models list, under which a caller-supplied target is dropped with a warning rather than sent.

It is the first Claude row priced by prompt length. A request whose prompt is up to 100,000 tokens bills $0.10 / $0.50 per million; one over 100,000 bills $0.50 / $2.50, on every key: cache reads are $0.01 and $0.05, 5-minute cache writes $0.125 and $0.625, 1-hour writes $0.20 and $1.00. The prompt length counts every input token, cache reads and cache writes included, and each request is tiered on its own, so a request over the threshold pays the higher prices even when most of its prompt is a cache hit. The registry carries it as the tiered_100k schema, and the cost engine picks the tier on input_tokens + cache_write_tokens — exactly 100,000 is the low tier (Cost Tracking). Its tokenizer is the newer one introduced with Opus 4.7, so the same text counts about 30% more tokens than on Haiku 4.5: a prompt that fitted under the threshold by that margin on Haiku 4.5 may not here.

What differs from Haiku 4.5, and what the toolkit does about each:

Change	What the API does	What the toolkit does
Adaptive thinking on by default, medium default effort	A request with no thinking field thinks and returns empty thinking blocks; {"type": "enabled", "budget_tokens": N} is a 400	adaptive_thinking_always_on: no config means {"type": "adaptive", "display": "summarized"}; a budget is converted to adaptive. The toolkit sends no default effort, so the API's medium applies unless you pass one
Thinking can be turned off, up to high effort	{"type": "disabled"} is a 200 at effort high or below and a 400 above it	thinking_off_type: "disabled", thinking_off_max_effort: "high": the value is sent as asked up to high; above it the run is adaptive with a warning (Turning thinking off). As on Sonnet 5.5, a thinking-off run sends no block_binding and leaves earlier turns' thinking blocks out of its requests
Forced tool choice is accepted, with or without thinking	tool_choice any / tool is a 200; the reply is the tool call with no thinking block	supports_forced_tool_choice: true: with thinking off the choice is sent as asked. With thinking on the toolkit still downgrades it to auto under its general thinking rule, although this row would accept it (recorded as technical debt)
No sampling parameters, no prefill	temperature, top_p, top_k and an assistant-final message are a 400	temperature is dropped with a WARNING (hyperparameters: ["max_tokens"]); the toolkit never sends a prefill
Thinking blocks are bound to the conversation before them	A replay after an edit to earlier history is a 400, on accounts created from 2026-08-31; blocks work only in the account that produced them	The fourth row with enforces_thinking_prefix_check: a thinking run sends block_binding: drop_block and its beta header (Preserved thinking and block binding)
A reply that reaches the context window is cut	stop_reason: "model_context_window_exceeded", a 200 carrying the partial output (earlier rows return a 400 before generating)	Since 0.5.4 the tool loop returns success: False with an error naming the stop reason and the two remedies (shorten the input, enable compaction), without a retry — a larger max_tokens cannot help — and the structured path returns the same error instead of parsing the partial reply. The request is billed as served
Computer use needs the toolset	computer_20250124 is a 400; computer_toolset_20260801 replaces it	Not modelled

Checked live through Agent on 2026-10-09: thinking off at medium (200) and at xhigh (400), a forced choice with thinking on (accepted by the API), a thinking budget and a prefill (both 400), a tool run and a structured run at the low tier, a 6,797-token cache write billed at the low-tier write price and its read at $0.01 per million, tool search (regex and bm25) with a deferred tool, and a 264,555-token prompt billed at the high tier. It is not offered on Priority Tier.

A row decides which sampling knobs are sent

descriptions.<model>.hyperparameters lists the generation knobs a model accepts. A row that omits one no longer has it sent: utils._model_accepts_param is the single reader, shared by OpenAI, Google and Anthropic, so a row's declaration means the same thing whichever provider serves it. A dropped knob produces one warning naming it, never a failed request.

Absent means permissive: a row with no hyperparameters list has nothing dropped, which is why adding the key to a row is a real wire change for that row alone.

Provider	Rows that no longer receive temperature	Why
Google	gemini-3.6-flash, gemini-3.7-flash, gemini-3.5-flash-lite	Google's own instruction is to strip temperature, top_p and top_k from Gemini 3 generation configs. Honoured on both Google paths
Anthropic	claude-fable-5, claude-fable-5-1, claude-opus-5-5, claude-opus-5, claude-sonnet-5-5, claude-sonnet-5, claude-haiku-5-5, claude-opus-4-8, claude-opus-4-7	Each answers a non-default temperature with a 400 ("temperature is deprecated for this model"), so their rows list max_tokens as their only hyperparameter
OpenAI	The reasoning rows, which list reasoning and max_output_tokens only	The o-series and GPT-5.x answer temperature with Unsupported parameter

Derive the answer from the registry rather than from a model name — get_model_info(model) reports each row's hyperparameters list.

temperature is also never sent alongside extended thinking

The API rejects a modified temperature when thinking is enabled, so the Anthropic tool loop has always dropped it there; the structured-output and plain-text path now follows the same rule. The two gates are independent: a row may accept temperature and still not receive it on a thinking request.

Google request handling

Waiting for an uploaded file

A Google request carrying media uploads it through the Files API and waits for the file to reach ACTIVE before sending. That wait used to be five inline copies of the same loop with fixed ceilings, so a long video failed with a timeout nothing could raise. It is one shared policy now, configurable per call through file_active_timeout_s, file_poll_interval_s and file_poll_backoff on both Google entry points — and therefore as Agent(..., file_active_timeout_s=600) — with the process defaults DPT_GOOGLE_FILE_ACTIVE_TIMEOUT_S, DPT_GOOGLE_FILE_POLL_INTERVAL_S and DPT_GOOGLE_FILE_POLL_BACKOFF (configuration).

Pass nothing and set nothing and the wait behaves exactly as before. When it does time out, the error names the file, the state it was last seen in, how many checks were made and which knob to turn — with the state normalised first, because FileState is a str-mixin enum whose str() renders FileState.PROCESSING rather than the wire value.

Tool schemas are filtered to what types.Schema accepts

google.genai.types.Schema is extra='forbid': a tool input schema carrying any keyword it does not define — the JSON Schema meta-keywords $schema, $id and $comment, or const, examples, exclusiveMaximum and the like — raised a pydantic ValidationError while the tool loop built the FunctionDeclaration, and the loop then dropped every tool of that MCP server, not only the offending one. After the $ref inlining and additionalProperties / anyOf-with-null clean-up every provider shares, the Google tool loop now also drops each keyword the installed SDK's Schema.model_fields does not list (utils._strip_non_google_schema_keys, the same helper the Gemini Live path uses). Property names are never filtered, and the field list is read from the installed google-genai, so a keyword a future release adds is passed through automatically. A schema that carried none of these keywords produces exactly the declaration it did before; the other three providers receive the schema unchanged.

Grounding evidence is merged across the whole stream

Every Google request streams. grounding_metadata and url_context_metadata used to be copied from the final chunk alone, so a turn whose search metadata arrived earlier reported no built-in tool, counted no searches and was therefore billed nothing for a search that really ran. Both objects are now accumulated across every chunk and merged, with list fields unioned and de-duplicated in first-seen order — de-duplication matters in both directions, because the search count is the length of the query list and a stream that repeats its cumulative metadata must still bill once. The Google tool-calling path also reports distinct_urls now; only the structured-output path extracted grounding URLs before, so a tool-calling run could bill a grounded search and hand the caller no sources.

Under tool combination Gemini may also return a grounded search only as a circulated server-side invocation — a tool_call part with tool_type GOOGLE_SEARCH_WEB and args.queries, answered by a tool_response part — with no grounding_metadata at all. Such a run used to report built_in_tools_invoked == [], zero searches and an exact $0 search line. The toolkit now reads those parts too: invocation queries are counted together with web_search_queries, each query once, and billed at the model's search rate, and built_in_tools_invoked lists google_search_grounding only when there is search evidence — a query or an invocation. URL-context grounding alone is not a search and no longer reports one. If search was enabled and the answer looks grounded (a grounding-api-redirect link in the text, or tool_use_prompt_token_count > 0 with no other built-in evidence) but no query or invocation came back, no count is guessed: the search line stays $0, tool_costs_estimated["google_search_grounding"] and cost_estimated are set, and a WARNING is logged.

Google response handling

Finish reasons

Gemini reports why it stopped in candidates[0].finishReason. The toolkit acts on six of the documented values:

Reason	What the toolkit does
MAX_TOKENS	Raise max_output_tokens by 50% and, when a truncated function call is detected, re-request (billed). At the model's own ceiling, or after GOOGLE_MAX_TOKENS_RECOVERY_MAX_PASSES (3) raises in one run, fail.
SAFETY, RECITATION	Fail with Content filtered: <reason>
MALFORMED_FUNCTION_CALL, TOO_MANY_TOOL_CALLS, MISSING_THOUGHT_SIGNATURE	Fail, naming the cause

Any other value — Google documents 21, the Python SDK ships 18, and the API is extended faster than the SDK tracks it — is logged once at WARNING with its raw value and the response is processed normally. providers.google.GOOGLE_KNOWN_FINISH_REASONS is the set the catch-all stays quiet about. The reason reaches you as the bare wire name in IterationMetrics.stop_reason.

The MAX_TOKENS recovery is capped at three raises

A re-request after a truncated function call does not complete an iteration, so max_iterations never bounded the recovery: a run that kept finishing MAX_TOKENS climbed by 50% a pass until the model's own cap — from max_tokens=100 to a 65,536 cap that was 18 billed requests. The tool-calling loop now raises the ceiling at most providers.google.GOOGLE_MAX_TOKENS_RECOVERY_MAX_PASSES (3) times per run; the next MAX_TOKENS finish returns the same failed result as a run at the model's ceiling (success=False, data=None), with an error that begins Response truncated (MAX_TOKENS) - recovery limit reached. A run already at the ceiling keeps the ... - at model limit wording. The constant is read at call time, so a host can patch it; 0 turns the recovery off. The structured-output path is unaffected: it has always retried a truncation exactly once.

This is a behaviour change

google-genai's FinishReason is a str-mixin Enum whose str() renders 'FinishReason.STOP', never 'STOP' — and the loop compared str(candidate.finish_reason) against the bare values. All six branches were therefore dead in production: a truncated turn, a filtered candidate, a malformed function call, too many consecutive tool calls and a missing thought signature all fell through to the natural exit and returned success=True with partial or empty text, while iteration_metrics[*]["stop_reason"] (and so DetailedCostReport) carried "FinishReason.STOP".

Those five value classes now return success=False where clients previously received success=True with partial text. The same fix applies to prompt_feedback.block_reason on a blocked prompt, which had been reported as "BlockedReason.SAFETY".

Empty final turns

After a tool result Gemini can return a turn with finishReason: STOP, no content and zero output tokens — a run that came back with an empty answer and no error, indistinguishable from a model that had nothing to say. The toolkit re-requests once (billed, logged at WARNING, and accounted as its own request cost record) and fails the run with "Empty model response" if the turn is still empty.

The re-request is not identical. It appends one short synthetic user turn — providers.google.GOOGLE_EMPTY_TURN_NUDGE_TEXT, "Use the tool results above to answer the question." — sent on the wire only and never stored in history, including when compaction fires later in the same run (the post-compaction history resync used to copy it into the messages a session stores, so a user turn nobody wrote was replayed on every later turn). Live, a byte-for-byte re-send came back empty again in 8 of 14 occurrences, each in under a second; the nudged form recovered 44 of 44, every one with the correct answer. It is still exactly one re-request.

A final turn carrying only a thought summary is not treated as empty: the model did generate, the tokens were billed, and the summary is returned in AgentResult.reasoning_content. Re-sending identical contents after real generation would be a re-roll, not a recovery.

DPT_GOOGLE_EMPTY_TURN_RETRY=identical keeps the byte-for-byte re-send and =off restores the behaviour before any recovery existed — see Configuration and Environment Variables.

OpenAI response handling

Incomplete and failed streams

The Responses API can end a stream with response.incomplete or response.failed in place of response.completed, or send an error event. The toolkit decides what each one means before the retry layer sees it. The two columns are the toolkit's two code paths. A run with local tools (functions or MCP servers the toolkit executes) goes through the tool loop. Every other run sends one request, including a run whose only tools are built-in ones such as web search, the code interpreter or a remote MCP server.

Event	The tool loop	One request
response.incomplete, reason max_output_tokens	The limit is raised by 50% and the same request is sent again, at most three times a run. After that, or once the limit is the row's max_output_tokens_request (which it is when no max_tokens was set), the run fails with Response truncated ...	Fails on the first attempt with Response truncated: the output limit was reached before the reply was complete ...
response.incomplete, reason content_filter	Fails with Response blocked by content filter	The same
response.incomplete, any other reason (max_messages, steered, one added later, or none)	Fails with Response incomplete (reason: ...). A function call the response ends in is not executed	Fails with the same message
response.failed with code server_error, rate_limit_exceeded or vector_store_timeout	Retried by the retry layer, unless its message reports a validation failure, which fails on the first attempt	The same
response.failed with any other code the SDK lists (invalid_prompt, the image codes, the policy codes)	Fails on the first attempt, with the server's message	The same
error with status 408, 409, 429 or any 5xx except 501	Retried	The same
error with any other status, or a validation failure	Fails on the first attempt	The same
error with no status and code server_error, rate_limit_exceeded or vector_store_timeout	Retried	The same

A response.failed event carries a code and no HTTP status, so the code decides: image_too_large fails once whatever its text says. The one thing read from the message of a retryable code is a validation failure (a schema error the server reports about its own event, which every attempt would meet again); that fails once. A code outside the SDK's list, or an error event that states no status, is judged next by the type of its error object, where it has one: invalid_request_error, authentication_error, permission_error and not_found_error fail once (context_length_exceeded is an invalid_request_error), rate_limit_error and server_error are retried. Only when the type does not decide is the wording read: rate-limit, overload, timeout and server-error wording is retried, wording that names an invalid request, a missing resource or a permission fails once, and anything else is retried.

Four things follow from the first row:

• Every truncated request is billed and reported. It has its own entry in request_cost_records and in iteration_metrics, priced from the usage the event carries, at the tier it was served (the event states the tier; checked live on a priority request). In iteration_metrics the cut request keeps the purpose it was sent under and has stop_reason: "incomplete" (the cut that ends a run has "max_output_tokens"). The request that replaces it has purpose truncation_retry, or max_tokens_continuation when the cut fell inside a function call.
• The partial reply is not kept. The request is sent again as it was, so the conversation never contains half an answer, and the cut reply's reasoning summary is left out of reasoning_content. A reasoning model often spends a small limit before it writes any text, and there is then nothing to keep.
• A re-request is not a retry, and a run has three of them. retry_count stays 0, and the five-attempt limit of the retry layer does not apply. Each raise is a billed request that sends the whole input again, so the loop raises the limit at most providers.openai.OPENAI_MAX_TOKENS_RECOVERY_MAX_PASSES (3) times a run, the bound the Gemini loop has. Three raises multiply the limit by about 3.4: a run that starts at 48 tokens ends at 162, and a reply that needs more fails after four billed requests with Response truncated due to max_output_tokens: recovery limit reached (...). The count starts again on every run and on every retried attempt of a run, so each turn of a session climbs from max_tokens. The constant is read at call time, so a host can patch it; 0, or anything that is not a count, turns the recovery off. Set max_tokens to what the reply needs, which on a reasoning model includes its reasoning.
• A stream consumer sees the cut reply and then the whole one. The text and thinking deltas of the cut request have been delivered by the time the cut is known, and the request that replaces it streams from its start. With Agent(emit_request_start=True) a request_start event whose purpose is truncation_retry or max_tokens_continuation marks the restart (Streaming).

A reply that was partly streamed before a failure event is reported by estimate, because no usage arrives with the event: the streamed text is counted and priced at standard rates, whichever service tier was requested. That holds for an event that is retried (each attempt's estimate is in retry_cost) and for one that fails on the first attempt. In the tool loop a first-attempt failure gets a cost record of its own, marked estimated: True, with no input tokens (the request's input is not known), and the run reports cost_estimated: True. The single-request path adds its pre-flight input estimate and does not set cost_estimated, as on every provider's failure path today.

Until 0.5.2 each of these events was raised as a retryable error. A reply cut off at the limit was sent five times and failed with Retry exhausted after 5 attempts, reporting a cost of zero for requests that were billed, and a rejected request was sent five times before it failed. An error event also lost its own code and message. No environment variable restores that behaviour.

Over HTTP an error event usually does not reach the toolkit as an event. The SDK's own stream reader raises it as a bare openai.APIError (no HTTP status, the event's code on .code), which the retry layer had no rule for: a server_error in mid-stream, whose message says "You can retry your request", failed the run on its first attempt (seen live on 2026-10-03). From 0.5.2 that exception is classified by the rules of the table, so a transient code is retried, a fault in the request fails once, and the partial reply is reported the same way in both cases.

One case is outside this handling: a response.failed event, or an error event the SDK does not raise itself, that arrives before response.created. The SDK's own stream state raises a RuntimeError for it before the toolkit sees the event, so the run fails once and the server's message is lost.

Provider SDK versions

Provider SDKs are declared in setup.py and requirements.txt. All four provider SDKs are declared with a floor and no ceiling: anthropic>=0.72.0, openai>=3.14.0, google-genai>=2.25.0, fireworks-ai>=1.2.9. Each floor names the lowest version actually exercised.

A ceiling is imposed on every application that installs the toolkit, and a resolver satisfies it by downgrading their SDK — so one is added only with a tested constraint, a named incompatibility and a maintained line beneath it. mcp<2 is the only provider-adjacent cap that qualifies today (mcp 2.0 removed mcp.server.fastmcp, which the toolkit's own MCP servers depend on). The only other cap in the package is httpx<1, which imposes nothing new: google-genai and fireworks-ai each cap httpx below 1 themselves (openai 3.x and anthropic 1.x use httpx2). aiohttp>=3.13.5 and requests>=2.33.1 are declared too, as floors only, because the package imports both. The old fireworks-ai<2, google-genai<2, anthropic<1 and tenacity<10 caps were removed; tests/unit/test_sdk_canary.py fails the gate on an untested major of any of them instead, and tested_constraints.txt records the exact versions a release was verified against — which is not the same as the supported range.

The dual-era Fireworks import guard below stays in place, so an application still pinned to the legacy line degrades rather than breaks.

Fireworks: both SDK eras are supported

fireworks-ai 1.x moved the client classes to the package top level and removed the fireworks.client.error module. The toolkit imports Fireworks through a dual-path guard in doc_processing_toolkit/__init__.py that tries the modern top-level surface first and falls back to the legacy fireworks.client modules, so either era works with no change to calling code.

Package-root export	Meaning
FIREWORKS_SDK_AVAILABLE	True when either era imported successfully
FIREWORKS_SDK_MAJOR	1 for fireworks-ai >= 1.x, 0 for < 1.x, None when the SDK is absent

Fireworks and AsyncFireworks are re-exported from the package root. Import them from doc_processing_toolkit rather than from fireworks.client, which survives on 1.x only as a deprecation shim that is slated for removal.

Two behaviors are gated on FIREWORKS_SDK_MAJOR:

• Client retries — clients on >= 1.x are constructed with max_retries=0 so the SDK's built-in retries do not stack multiplicatively with the toolkit's own retry policies. The legacy constructor has no such parameter and is not given one.
• Streaming usage — the include_usage request that guarantees a final usage chunk is sent as a stream_options keyword on < 1.x and through extra_body on >= 1.x, which rejects the keyword. The wire payload is identical either way, so token and cost capture stay accurate on both eras.

Application code does not need to branch on FIREWORKS_SDK_MAJOR; it is exported for diagnostics and for downstream code that constructs Fireworks clients itself.

Anthropic: both SDK lines are supported

anthropic 1.0 (2026-08-20) is a breaking major, and the 0.x line ended at 0.125.0 the day before. Since toolkit 0.5.2 both lines work with no change to calling code; 1.x is the verified one (tested_constraints.txt), and the unit gate is repeated on 0.x at each release. Toolkit 0.5.1 and earlier do not run on anthropic 1.x, so an application on an older toolkit must pin anthropic<1 itself.

Package-root export	Meaning
ANTHROPIC_SDK_AVAILABLE	True when the SDK imported successfully
ANTHROPIC_SDK_MAJOR	1 for anthropic >= 1.0, 0 for 0.x, None when the SDK is absent

Five behaviors are gated on ANTHROPIC_SDK_MAJOR, so that the toolkit puts the same request on the wire on both lines:

• Sampling parameters. 1.x removed temperature, top_p and top_k from the request method signatures, so there they are sent through extra_body. On 0.x they stay keywords.
• Client timeout. 1.x transports over httpx2 and must be given an httpx2.Timeout; 0.x gets the httpx.Timeout it always did. The values are the same (30 s connect, 7,200 s read).
• Custom skill titles. beta.skills.create takes display_title on 0.x. On 1.x the keyword became display_name, which the API ignores under the skills-2025-10-02 beta the toolkit uses, so the title is sent through extra_body.
• Files API header. 0.x adds the files-api-2025-04-14 beta header to beta.files uploads and deletes by itself; 1.x sends only the betas it is given, so the toolkit passes it there.
• input_transformations. 1.x delivers the entries as typed SDK objects; the toolkit returns them as dicts on both lines, as AgentResult.input_transformations documents.

One difference belongs to the SDK and is left alone: when an assistant turn is sent back on a later request of a tool loop, 1.x (and 0.125.0) omits two fields that older 0.x releases sent as null (citations on a text block, caller on a tool-use block). The API reads both forms the same way.

anthropic 1.x also verifies TLS against the operating system trust store, like openai 3.x; see TLS trust store.

Application code does not need to branch on ANTHROPIC_SDK_MAJOR; it is exported for diagnostics and for downstream code that constructs Anthropic clients itself.

Fireworks remote-MCP calls are driven with the OpenAI SDK

fireworks-ai >= 1.x ships no binding for the Fireworks Response API, so the toolkit's Fireworks remote-MCP path drives that API with the OpenAI SDK pointed at https://api.fireworks.ai/inference/v1 — the platform endpoint is OpenAI-compatible. This uses the openai package (already a dependency) and FIREWORKS_API_KEY; no Fireworks SDK class is involved. See MCP. That path returns reasoning_content like every other Fireworks path; it used to report the reasoning token count and drop the text.

Fireworks reasoning counts that were estimated reach the run total

A Fireworks deployment that omits completion_tokens_details gets its reasoning tokens estimated from the reasoning text. That estimate now reaches the run-level AgentResult.reasoning_tokens; before this release the estimate was made and the run total still read 0. A count the provider did report is used as reported and nothing is added to it.

Sync and async initialization

• Sync: initialize_clients(...)
• Async: initialize_async_clients(...)

See Async Clients for when to use each.

Retries

Tenacity is the toolkit's only retry layer, on all four providers: up to five attempts per provider call, each exactly one HTTP request, each logged and priced into retry_count / retry_cost. The Anthropic and OpenAI SDK clients are built with max_retries=0, Fireworks 1.x clients always were, and google-genai has no SDK retry layer. DPT_SDK_RETRIES=default gives the Anthropic and OpenAI SDKs their own retries back (configuration).

The retry predicate retries HTTP 408, 409, 429 and every 5xx except 501, plus connection and timeout errors. A server x-should-retry header overrides the status rule in either direction. Gemini's ClientError / ServerError are classified by the same rules from their .code, so a Gemini 429 or 503 is now retried like any other provider's. The wait honours retry-after-ms / retry-after (capped at utils.RETRY_AFTER_MAX_SECONDS, 60 s) and otherwise backs off exponentially from 4 s. See Cost Tracking for how each attempt is billed.

A failure that arrives as a stream event rather than as an HTTP error is classified before it is raised, because the retry layer retries every stream error it is handed. On OpenAI the rule is the status rule above, with a response.failed code standing in for the status (OpenAI: incomplete and failed streams). Fireworks classifies its stream events against its own, shorter list (408, 429 and the common 5xx statuses). On both, an event that names a fault in the request fails on the first attempt, with one gap on Fireworks: its fail-once error is a plain ValueError, so a message that happens to contain "try again later" or "rate limit" is still retried by the retry layer's wording check.

Anthropic server-side compaction support

Every Anthropic row carries descriptions.<model>.supports_server_compaction. On a row where it is false, a context_editing built-in with compaction=True sends its other edits but leaves out the compact_20260112 edit and its beta, with one WARNING naming the key, instead of receiving a 400. See Compaction.

Streaming Transport

All providers now use streaming internally for API calls. This is transparent to callers using Agent.run() or Agent.run_sync() — the returned AgentResult is identical.

Streaming provides several benefits:

• Timeout resilience — each streamed chunk resets the idle timer, eliminating most timeout failures for long generations.
• Partial output capture — if a request fails mid-generation, token usage from the partial stream is captured for cost tracking.
• Consistent behavior — all four providers (OpenAI, Anthropic, Google, Fireworks) use the same streaming approach.

Note

No code changes are required. Existing Agent.run() and Agent.run_sync() calls work as before; only the underlying transport has changed.

OpenAI: a WebSocket for the tool loop

The OpenAI tool loop can send its requests over one WebSocket to the Responses API instead of one HTTP request each (0.5.2, opt-in): Agent(openai_transport="websocket"), or DPT_OPENAI_TRANSPORT=websocket as the process default. A loop of several requests gets faster and a run of one or two gets slower; results, billing and retries are the same. HTTP stays the default. See OpenAI WebSocket Transport.

Provider update maintenance workflow

Provider update procedures from the legacy guide are now captured in:

• Provider Updates

For source module coverage choices (full API vs brief overview), see:

• Toolkit Modules