Thinking and Reasoning¶

The toolkit provides a unified thinking parameter on Agent that enables extended thinking and reasoning capabilities across all four providers. Each provider implements thinking differently — the toolkit normalizes the configuration surface while preserving provider-specific semantics.

How thinking works¶

When thinking is enabled, the model performs internal reasoning before producing its final answer. This improves accuracy on complex tasks like multi-step math, logic puzzles, code generation, and nuanced analysis. The reasoning process consumes additional output tokens and may increase latency.

The Agent accepts a single thinking dict that is automatically transformed into the correct provider-specific format via ParameterValidator and transform_thinking_param() in the parameter mapping layer.

Agent(thinking={...})
  → ParameterValidator.validate()
    → transform_thinking_param()
      → provider-specific API parameter

Provider support matrix¶

Capability	Anthropic	OpenAI	Google	Fireworks
Budget-based thinking	✅	❌	✅	❌
Effort/level control	✅	✅	✅	✅
Adaptive thinking	✅	❌	❌	❌
Interleaved thinking (with tools)	✅	❌	❌	❌
Works with function calling	✅	✅	✅	✅
Works with structured output	✅	✅	✅	✅

The universal `thinking` parameter¶

The thinking parameter is a dict whose keys are interpreted per provider:

from doc_processing_toolkit import Agent

# Anthropic — budget-based
agent = Agent(model="claude-sonnet-4-5", thinking={"budget_tokens": 10000})

# Anthropic — adaptive (recommended for supported models)
agent = Agent(model="claude-opus-4-7", thinking={"type": "adaptive"})

# Anthropic — off (see "Turning thinking off" below)
agent = Agent(model="claude-sonnet-5", thinking={"type": "disabled"})

# OpenAI — effort-based
agent = Agent(model="gpt-5.2-pro", thinking={"effort": "high"})

# OpenAI — pro mode (GPT-5.6 models only)
agent = Agent(model="gpt-5.6-sol", thinking={"effort": "high", "mode": "pro"})

# Google — budget-based
agent = Agent(model="gemini-2.5-pro", thinking={"budget_tokens": 8192})

# Google — level-based
agent = Agent(model="gemini-3-flash-preview", thinking={"level": "high"})

# Fireworks — effort-based
agent = Agent(model="accounts/fireworks/models/glm-5p3", thinking={"effort": "high"})

Providers also accept provider-specific parameters directly (see sections below).

Anthropic: Extended and adaptive thinking¶

Anthropic supports two thinking modes.

Adaptive thinking¶

Adaptive thinking lets the model decide how much reasoning to apply. This is the recommended mode for models that support it (those with supports_adaptive_thinking: true in model_information.json — currently Claude Fable 5, Claude Fable 5.1, Claude Opus 5.5, Claude Opus 5, Claude Opus 4.8, Claude Opus 4.7, Claude Opus 4.6, Claude Sonnet 5.5, Claude Sonnet 5, Claude Sonnet 4.6 and Claude Haiku 5.5). Consult model_information.json for the authoritative, up-to-date list.

On some models the toolkit treats thinking as always on. These are flagged with adaptive_thinking_always_on: true (currently Claude Fable 5, Claude Fable 5.1, Claude Opus 5.5, Claude Opus 5, Claude Sonnet 5.5 and Claude Haiku 5.5): when no thinking config is supplied for such a model, the toolkit defaults it to {"type": "adaptive"} so request handling matches what the model actually does — on both paths, including a no-tools run, which used to send no thinking object at all and therefore got neither summaries nor block binding. The detailed cost report's thinking_enabled is True for such a run. Claude Fable 5 rejects thinking: {"type": "disabled"} outright, and so do Claude Fable 5.1 and Claude Opus 5.5 (which also reject {"type": "enabled", "budget_tokens": N} at every effort level — effort is the only thinking control left on Opus 5.5, and its API default is "medium", not Opus 5's "high"). Claude Opus 5 accepts disabled at effort "high" or below and rejects it at "xhigh" and "max". Claude Sonnet 5.5 rejects disabled as well, and takes between_tools in its place, up to effort "high". Claude Haiku 5.5 behaves like Opus 5: disabled up to "high", adaptive with a WARNING above it. Turning thinking off says what the toolkit sends in each case.

Claude Sonnet 5 also thinks when a request carries no thinking field, but its row is not flagged. With no thinking config the toolkit sends no field: the model may think, its thinking is not returned, and the run is handled as a run without thinking. Pass {"type": "adaptive"} to read the thinking, or {"type": "disabled"} to stop it.

On Claude Fable 5, Claude Fable 5.1 and Claude Opus 5.5, {"type": "disabled"} is coerced, not obeyed

The server runs adaptive thinking on those models regardless, so refusing the request locally would only cost you a working call, and thinking=None already coerces on every always-on row. A disabled config is therefore rewritten to adaptive with a warning naming the model and the billing consequence — because you are billed for reasoning tokens at the output rate that you explicitly asked not to have. Other thinking.* keys you set are preserved, so the request body is otherwise unchanged. A legacy {"type": "enabled", "budget_tokens": N} config is converted to adaptive the same way (the budget is dropped, with a warning naming the model).

agent = Agent(
    model="claude-opus-4-7",
    thinking={"type": "adaptive"},
)

Tip

When a model supports adaptive thinking, the toolkit automatically converts legacy budget-based configs to adaptive mode during validation.

Budget-based (legacy) thinking¶

Explicitly control the thinking token budget. The interleaved flag enables reasoning between tool calls during multi-turn tool use.

agent = Agent(
    model="claude-sonnet-4-5",
    thinking={"budget_tokens": 10000, "interleaved": True},
)

budget_tokens — positive integer; defaults to 10,000 if omitted.
interleaved — boolean; enables interleaved thinking with tools (adds the required beta header automatically). interleaved_thinking is accepted as an alias — before this release only interleaved was read, so a caller using the other spelling got no interleaved thinking, no beta header and no warning.

Turning thinking off¶

thinking={"type": "disabled"} asks for a run with no thinking:

agent = Agent(model="claude-sonnet-5", thinking={"type": "disabled"})

{"type": "between_tools"} is accepted as a second spelling of the same request. It is what Claude Sonnet 5.5 takes in place of disabled. You can write either on any Claude model: the toolkit treats both as "off" and sends what the model's registry row calls for.

Model	What the toolkit sends for "off"	Result
Claude Sonnet 5.5, effort `"high"` or below (or no effort set)	`thinking: {"type": "between_tools"}`	No thinking before the response. The short updates the model writes between tool calls still come back as thinking blocks
Claude Sonnet 5.5, effort `"xhigh"` or `"max"`	`thinking: {"type": "adaptive"}`, with a WARNING	The model thinks. The API rejects `between_tools` above `"high"`, and the toolkit keeps the effort you asked for
Claude Sonnet 5	`thinking: {"type": "disabled"}`	No thinking, at every effort level
Claude Opus 5, effort `"high"` or below (or no effort set)	`thinking: {"type": "disabled"}`	No thinking
Claude Opus 5, effort `"xhigh"` or `"max"`	`thinking: {"type": "adaptive"}`, with a WARNING	The model thinks. The API rejects `disabled` above `"high"`, and the toolkit keeps the effort you asked for rather than lowering it
Claude Haiku 5.5, effort `"high"` or below (or no effort set)	`thinking: {"type": "disabled"}`	No thinking
Claude Haiku 5.5, effort `"xhigh"` or `"max"`	`thinking: {"type": "adaptive"}`, with a WARNING	The model thinks. The API rejects `disabled` above `"high"`, and the toolkit keeps the effort you asked for
Claude Opus 4.8 and older, Claude Sonnet 4.6 and older, Claude Haiku 4.5	no `thinking` field	No thinking: these models think only when asked to
Claude Fable 5, Claude Fable 5.1, Claude Opus 5.5	`thinking: {"type": "adaptive"}`, with a WARNING	The model thinks. These models cannot turn thinking off (see the warning above)

Two registry keys carry the difference, so nothing here is keyed on a model name:

thinking_off_type — the type to send for "off". claude-sonnet-5, claude-opus-5 and claude-haiku-5-5 set it to "disabled", because each thinks when the request has no thinking field, and claude-sonnet-5-5 sets "between_tools". A row without the key gets no thinking field.
thinking_off_max_effort — the highest effort at which the API accepts that value. claude-opus-5, claude-sonnet-5-5 and claude-haiku-5-5 set "high". Above it the run falls back to adaptive thinking with a WARNING that names the model, the cap and the effort.

With thinking off the run is handled as a run without thinking in every other respect:

temperature is sent on a model whose row accepts one (Claude Haiku 4.5, Claude Opus 4.6 and older, Claude Sonnet 4.6 and older). Claude Sonnet 5.5, Claude Sonnet 5 and Claude Opus 5 accept no sampling parameter, with thinking on or off.
A forced tool_choice ("any", or a named tool) is sent as given. Claude Sonnet 5.5 is the exception: its row accepts no forced choice at all, so it is downgraded to auto with a WARNING, as on a thinking run.
No block_binding is attached, and no other key of your thinking dict (display, block_binding, ...) is sent: an off value takes none. One WARNING names the keys that were left out.
AgentResult.reasoning_tokens is 0, and the cost report's thinking_enabled is False, as is every per-request iteration_metrics[i].thinking_enabled (populated since 0.5.3). Claude Sonnet 5.5 is the exception to the first half: the updates it writes between tool calls are thinking blocks, so a tool run reports their tokens in reasoning_tokens (they are billed as output), their text in reasoning_content, and thinking_delta events as they stream. A run with no tool call reports 0.

When the model cannot turn thinking off, or the effort is above the cap, the run thinks and is a thinking run in every one of these respects: the other keys of your dict are sent with {"type": "adaptive"}, exactly as if you had asked for adaptive thinking with them, and thinking_enabled is True — on the report and on every per-request entry.

One request of a run is outside this: the summary request that summarize compaction makes. It is a separate call with no thinking setting of its own, so on Claude Sonnet 5 and Claude Opus 5 the model may think while it writes the summary, and on Claude Sonnet 5.5 it is sent as a thinking request (adaptive, with block_binding).

The agent-factory MCP server takes the same request: create_agent(thinking_config_json='{"type": "disabled"}'). Before 0.5.2 it dropped the key and the run thought.

On OpenAI and Fireworks models the same dict is not \"off\"

{"type": "disabled"} is read on Claude rows only. On an OpenAI or Fireworks model the type is not read: the request carries the effort in the same dict, medium when there is none, where the row lists it, and the model reasons. Since 0.5.2 one WARNING says so on a run whose dict has no effort of its own. To reason less on those rows, pass {"effort": ...} with an effort the row lists (supported_reasoning_values in the registry); the GPT-5.1 to GPT-6 rows list none, the GPT-5 rows minimal, and gpt-6.1-sol nothing below low. {"type": "disabled", "effort": "none"} is therefore a portable spelling (Claude reads type, a GPT-5.1+ row reads effort) and is not warned about. A per-row "off" for these providers is deferred. On Google the key is ignored with a warning and no thinking setting is sent.

Before 0.5.2, an "off" request turned thinking on

Until this release no Claude request could be switched off through the toolkit. The type was dropped and the dict was read as a budget config with the default budget_tokens: 10000: a 10,000-token thinking budget on Claude Haiku 4.5 and the older budget models, adaptive thinking on every model that supports it. Claude Opus 5 was coerced to adaptive with a warning although the API accepts disabled there. A run that passes {"type": "disabled"} now produces no reasoning tokens on Claude Sonnet 5, on Claude Opus 5 up to effort "high", and on every model that thinks only when asked (the last row of the table). There is no switch back to the old behaviour. Requests that pass None, {"type": "adaptive"} or a budget are unchanged.

Live on 2026-10-02 through Agent: disabled on Claude Sonnet 5, Claude Opus 5 and Claude Haiku 4.5 returned reasoning_tokens: 0 on plain, tool, forced-tool-choice and structured-output runs, and on Claude Sonnet 5 at each of the five effort levels. A session whose first turn thought continued with thinking off on the second, including one whose first turn ran on Claude Opus 5.5 and whose signed thinking blocks were replayed into a thinking-off Claude Sonnet 5 request. On Claude Sonnet 5.5 the same day: between_tools on plain, tool, structured-output and web-search runs; a three-step tool run whose system prompt asked for a note before each tool call returned three notes (188 to 208 reasoning_tokens, about 450 characters in reasoning_content, 28 to 33 thinking_delta events); the same run with no such instruction returned none; and effort "xhigh" ran adaptive with the warning.

Thinking off on a model that checks the thinking prefix¶

Some models check that a replayed thinking block still belongs to the conversation before it (see Preserved thinking and block binding). On a thinking-on run the toolkit sends thinking.block_binding, and the server drops a block whose prefix has moved. A thinking-off run cannot do that when the off value is between_tools: the API rejects any other key beside it, block_binding included. Anthropic's alternative is to keep the history append-only, or to remove the thinking blocks. The toolkit cannot promise the first across turns (a history mode, a media placeholder, compaction, an edited system prompt or a changed tool list all rewrite what earlier blocks were bound to), so on a row that sets both thinking_off_type and enforces_thinking_prefix_check (claude-sonnet-5-5, and claude-haiku-5-5 since 0.5.4) a thinking-off run does the second:

Where	What a thinking-on run does	What a thinking-off run does
Every request	Sends `block_binding` and its beta header	Sends neither
History from earlier turns	Replays its thinking blocks; the server drops the stale ones and reports them	Leaves every thinking block of earlier turns out of the request. The stored session history is not changed, unless the run compacts (next row but one)
Inside one run	Replays the run's thinking blocks	The same: the blocks the model writes between tool calls are replayed while the tool loop only appends
Client-side compaction inside the tool loop	Keeps the blocks for the server to judge	Removes the run's thinking blocks from the compacted history, with a WARNING naming the strategy. The session stores that history, so it no longer holds the thinking blocks of earlier turns either
A prefix-mismatch 400	Retries once with `drop_block`	Retries once with every thinking block removed from the request's history, and builds the rest of the run from that history
Server-side fallback	Sends the fallback entries as given	Gives each entry the thinking value its own model takes, for example `{"model": "claude-sonnet-5", "thinking": {"type": "disabled"}}`

Details worth knowing:

The blocks a model writes between tool calls with thinking off are its progress notes. They reach reasoning_content and the thinking_delta stream events, and they are billed as output tokens like any thinking. What is not replayed is the notes of earlier turns: a later turn sees the earlier questions, tool calls, tool results and answers, without them. One INFO line says so.
Compacting with thinking off costs the session its earlier thinking blocks. When client-side compaction fires during a thinking-off run, the session takes the loop's compacted history as its own, as it does after any compaction, and that history was built without the earlier turns' blocks. A later thinking turn, or a move to Claude Opus 5.5, then starts without that reasoning. The text, tool calls and tool results are kept as the strategy leaves them. Compaction rewrites what most of those blocks were bound to, so the server would have dropped them on the next thinking request anyway; the ones lost for no reason are those that sat before the first message the strategy changed. Without compaction the stored history keeps every block, and the next thinking turn replays them.
Because earlier turns' blocks are never sent, the retry in the fifth row is a backstop. It sets AgentResult.prefix_binding_retry, as the drop_block retry does, fires once per run, and never in strict mode (strict_prefix_binding=True, anthropic_block_binding="error", DPT_ANTHROPIC_BLOCK_BINDING=error, or a block_binding of "error" in your own thinking dict), where the 400 is raised with its cause.
A fallback entry that already carries its own thinking is left as you wrote it. Otherwise the entry is given what a direct request to that model would send for "off": the model's own off value, adaptive if the model cannot turn thinking off at this run's effort, or disabled on a model that is off without a field.
A thinking-off request above the row's thinking_off_max_effort runs adaptive thinking, so it is a thinking-on run in every row of this table.

No request of a run on a row that does not check the prefix changes: disabled on Claude Sonnet 5 and Claude Opus 5 replays its history untouched and sends its fallback entries as before.

The 400 itself is not reproducible on every account. The API enforces the prefix check for organisations created from 2026-08-31, and between_tools offers no way to opt in on an older one. What was checked live on 2026-10-02 is the remedy: under between_tools, a replay with every thinking block removed, the open tool round's included, returned 200, and so did a fallback entry carrying thinking: {"type": "disabled"} where the plain entry was a 400.

Unknown thinking keys pass through¶

The toolkit normalises only what it interprets — type, display, budget_tokens and the interleaved flag. Every other key you put in thinking reaches the Anthropic API unchanged, so you can use a thinking.* field months before the toolkit supports it natively:

agent = Agent(
    model="claude-sonnet-5",
    thinking={
        "type": "adaptive",
        "block_binding": {"prefix_mismatch_behavior": "drop_block"},  # forwarded verbatim
    },
)

A key the toolkit does not recognise is logged at WARNING — the commonest cause is a misspelling like interleved — and forwarded anyway. Nothing is sent that you did not ask for: with no extra keys the request body is byte-identical to previous releases.

The one exception is a request to turn thinking off that the model honours: an off value takes no other key, so the rest of the dict is not sent, and a WARNING names what was left out.

Two toolkit-side details worth knowing. budget_tokens and the interleaved flag are dropped under {"type": "adaptive"}, because the model decides there. And interleaved_thinking is a toolkit flag that selects a beta header, not an API field, so it never appears in the request body.

Effort configuration¶

Anthropic also supports a separate effort_config parameter (independent of thinking) that controls thinking depth without an explicit token budget:

agent = Agent(
    model="claude-opus-4-7",
    thinking={"type": "adaptive"},
    effort_config="high",
)

Valid values depend on the model (supports_effort_configuration in model_information.json): "low", "medium", and "high" on all effort-supporting models, with "xhigh" and "max" additionally available on newer models (Claude Fable 5, Fable 5.1, Opus 5.5, Opus 5, Opus 4.8, Opus 4.7, Sonnet 5.5, Sonnet 5) and "max" (without "xhigh") on Opus 4.6 and Sonnet 4.6.

When effort_config is omitted the toolkit sends no effort and the API default applies — "high" on Claude Opus 5, but "medium" on Claude Opus 5.5. A profile moved from Opus 5 to Opus 5.5 without setting effort_config therefore thinks one level shallower; set it explicitly when you switch. Claude Sonnet 5.5 defaults to "high", and Anthropic recalibrated its levels: the same level does not produce the same amount of thinking as on Claude Sonnet 5, so re-run an effort comparison rather than carrying a setting across.

Constraints¶

Temperature and top_k are fixed when thinking is enabled.
Tool choice must be "auto" or "none" with thinking active.
Neither applies to a run whose thinking is turned off.
Thinking blocks (including redacted blocks) must be preserved unmodified in multi-turn conversations — the toolkit handles this automatically via _preserve_thinking_blocks(), which round-trips every field the Anthropic SDK returned (signatures, citations, cache_control, server-attached metadata) by dumping each block with model_dump(exclude_unset=True) rather than hand-picking known fields. The same round-trip applies to text, tool_use, server_tool_use, and redacted_thinking blocks so extended-thinking-with-tools flows don't trip Anthropic's "modified-block" validation.
Server-tool blocks are echoed as pairs: when the model invokes an Anthropic built-in tool (e.g. web search) mid-turn, the server_tool_use block and its *_tool_result block (web_search_tool_result, web_fetch_tool_result, code_execution_tool_result, etc.) both round-trip in the echoed assistant message, as do remote-MCP mcp_tool_use/mcp_tool_result pairs — Anthropic rejects an unpaired server_tool_use with a non-retryable 400. This applies to all three echo sites (the custom-tool loop, the pause_turn handler, and max_tokens continuation), so built-in tools compose safely with custom tools in the same turn.

Extra beta headers: `anthropic_betas`¶

The toolkit computes anthropic-beta headers from the request configuration — skills, code execution, context management, MCP client, strict tools, interleaved thinking, server-side fallback. anthropic_betas adds your own on top:

agent = Agent(
    model="claude-sonnet-5",
    anthropic_betas=["mid-conversation-tool-changes-2026-07-01"],
)

The computed values come first and the merge is an order-preserving dedupe, so your additions never reorder the toolkit's own list and a duplicate is a no-op. None and [] both mean "no change".

A non-empty list moves the request to the beta endpoint

Any beta header routes the request through client.beta.messages instead of client.messages. That is not free on a model that would otherwise send none, so pass anthropic_betas only when you actually need the beta.

Some models already use the beta endpoint regardless: claude-fable-5, claude-fable-5-1, claude-opus-5-5, claude-opus-5 and claude-sonnet-5-5 enable server-side fallback by default (they publish allowed_fallback_models in the registry), which carries its own beta header.

Progress notes: `display: "updates"`¶

On Claude Fable 5, Fable 5.1, Opus 5.5 and Sonnet 5.5 the model can write a short progress note before a tool call ("Found 412 upstream timeouts from auth. I'll check the auth log next."). A note arrives as a thinking block. thinking.display decides what text thinking blocks carry:

`display`	Reasoning blocks	Progress notes
`omitted` (the API's default)	empty	empty
`summarized` (what the toolkit sends when you give no `display`)	summary text	text, not told apart from reasoning
`updates` (beta `thinking-display-updates-2026-08-18`)	empty	text

updates is for an interface that hides reasoning and shows a status line for each step: any thinking text that arrives is a progress note. The toolkit needs no option of its own for it, because both parts already pass through:

agent = Agent(
    model="claude-opus-5-5",
    tools=[read_log],
    thinking={"type": "adaptive", "display": "updates"},
    anthropic_betas=["thinking-display-updates-2026-08-18"],
)

async for event in agent.stream("Users report login failures since 10:00. Find the root cause."):
    if event.event_type == "thinking_delta":
        print(f"[note] {event.content}", end="", flush=True)   # a progress note
    elif event.event_type == "text_delta":
        print(event.content, end="", flush=True)                # the model's own text

thinking is passed through. The toolkit fills in display: "summarized" only when you give no display, so "updates" reaches the request unchanged.
The beta header is yours to add. The display value needs it; without it the API answers with a 400. anthropic_betas appends it to the headers the toolkit computes.
Notes come out where thinking text always does: thinking_delta events on Agent.stream(), and AgentResult.reasoning_content.

What it returned live (2026-10-03). The run was a five-step tool chain: read one service's log, follow the trail to the next, name the root cause. Nine runs on Claude Sonnet 5.5, Claude Opus 5.5 and Claude Fable 5.1, with and without a system prompt asking for a progress update before each tool call:

`display`	`thinking_delta` events	`reasoning_content`	Where the model's remarks between steps went
`updates` (9 runs, three models)	0	empty	Ordinary text (`text_delta`)
`summarized` (same prompt, Sonnet 5.5)	0	empty	Ordinary text (`text_delta`)

So updates hid the reasoning, as it is meant to: Opus 5.5 and Fable 5.1 reported 8 and 40 reasoning_tokens and no reasoning text. In these runs the models wrote their remarks as text, not as notes. This is the stream of one updates run on Sonnet 5.5, with the prompt that asked for updates:

[text]  Starting with the gateway log to see where login requests are failing.
[tool]  read_log(service="gateway")
[text]  Gateway shows 412 timeouts from auth, so I'm checking the auth log next.
[tool]  read_log(service="auth")
[text]  Auth's token refresh is failing because sessions returns 503, so I'm checking sessions next.
[tool]  read_log(service="sessions")
...
[text]  **Root cause:** Deploy 4417 at 09:40 disabled log rotation on host ks-2. ...

Whether a remark arrives as a note or as text is the model's choice, so an interface that shows a status line should show the text between tool calls as well.

The Claude Sonnet 5.5 alternative: thinking off. With thinking={"type": "disabled"}, Sonnet 5.5 runs between_tools (Turning thinking off): no reasoning, no beta header of your own, and what the model writes between tool calls can arrive as thinking blocks. Captured the same day, with a system prompt asking for a short paragraph before each tool call and fallbacks=[]:

[text]  I'm starting the investigation with the gateway log, since that is the entry point for user
        login requests. ...
[tool]  read_log(service="gateway")
[note]  Found 412 upstream timeouts from `auth` starting at 10:02, matching the reported failures —
        the gateway looks healthy, so the issue is likely in `auth` or its dependencies. I'll check
        the `auth` log next to see why it's slow to respond.
[tool]  read_log(service="auth")
[note]  Found that auth token refreshes have failed since 10:01 due to 503s from the sessions
        dependency, which explains the gateway timeouts but isn't the root cause yet. I'll check
        the sessions log next to find why it's returning 503s.
[tool]  read_log(service="sessions")
...

Four notes, 414 reasoning_tokens: notes are billed as output tokens, like any thinking. Two things to know before relying on this:

The same system prompt under adaptive thinking is refused on Sonnet 5.5 (reasoning_extraction; see Claude Sonnet 5.5). A shorter prompt ("post a one-sentence progress update before each tool call") was not refused, and produced text, not notes, in both modes.
With the default fallback, a conversation that was refused recently is answered by Claude Sonnet 5 for about an hour, and Sonnet 5 writes no notes. Check served_model on the request's cost record, or pass fallbacks=[].

Preserved thinking and block binding¶

Some Anthropic models check that a replayed thinking block still belongs to the conversation it was produced in: the system prompt, the tools array and the earlier messages that preceded it. Client-side compaction rewrites exactly those, so on a model that runs the check a sliding_window session would otherwise fail outright.

The toolkit answers with thinking.block_binding.prefix_mismatch_behavior, sent together with the thinking-binding-controls-2026-08-01 beta header on every request of a run whose model's registry row sets enforces_thinking_prefix_check. "drop_block" has the server drop the stale blocks and report them; "error" fails the request instead. Nothing is stripped locally: inside a tool round the latest assistant message must be echoed back untouched, so the removal has to be the server's. (The runs that cannot send the field, a thinking-off run on claude-sonnet-5-5 or claude-haiku-5-5, leave earlier turns' blocks out by themselves; see Thinking off on a model that checks the thinking prefix.)

Four rows turn this on: claude-fable-5-1, the first, claude-opus-5-5, the second, claude-sonnet-5-5, the third, and claude-haiku-5-5, the fourth. Every request of a thinking run on any of them carries thinking.block_binding.prefix_mismatch_behavior="drop_block" and the binding beta, which also moves the request to the beta endpoint. (A run on claude-sonnet-5-5 or claude-haiku-5-5 with thinking turned off cannot carry the field; see Thinking off on a model that checks the thinking prefix.) Every other row is byte-identical to before, which the golden request-body suite proves cell by cell, and DPT_ANTHROPIC_BLOCK_BINDING=off (or Agent(anthropic_block_binding="off")) sends the pre-Fable-5.1 body — a frozen golden cell per row proves that rollback is byte-identical to the ungated twin.

Who reads whose blocks. Thinking blocks are tied to the model that produced them as well as to the conversation:

Claude Fable 5.1 and Claude Mythos 5.1 read each other's blocks and those of every earlier Claude model. No earlier model reads theirs.
Claude Opus 5.5 reads blocks produced by Claude Opus 5 and earlier Opus, Sonnet and Haiku models, but not blocks from Fable or Mythos models. On the Claude API only Fable 5.1 and Mythos 5.1 read Opus 5.5's blocks; any other model drops them.
Claude Sonnet 5.5 reads blocks produced by Claude Sonnet 5, Claude Opus 4.8, Claude Haiku 4.5 and earlier models. It does not read blocks from Claude Opus 5, Claude Opus 5.5, or any Fable or Mythos model. On the Claude API its own blocks are read by Claude Opus 5.5 and by no other model (Fable 5.1 predates it, so the first bullet does not cover it): a session moved up to Opus 5.5 keeps the earlier reasoning, and any other move drops it, a server-side fallback to Claude Sonnet 5 included. Checked live on 2026-10-02 with a session whose first turn ran on Sonnet 5.5: Opus 5.5 reported no drop, Fable 5.1 reported model_binding_mismatch for the block on each request, and Sonnet 5 and Haiku 4.5 answered and reported nothing, for the reason given below.
Claude Sonnet 5.5's blocks also work only in the account that produced them, or one linked to it. A session restored under another account loses them: the request succeeds, and a row that sends the binding beta lists the drop in input_transformations with reason organization_binding_mismatch. Neither counter on the result counts that reason; read the list.

A block the current model cannot read is always dropped, never rejected: the request succeeds, the block is not billed and does not count toward input_tokens, and prefix_mismatch_behavior has no effect on it — so strict_prefix_binding=True does not turn a model-binding drop into an error, and a strict CI run will not catch one. Switch back to a model that can read the block and it is read again, because the API never edits your history.

The drop is reported only when the request carries the thinking-binding-controls-2026-08-01 beta — in input_transformations with reason: "model_binding_mismatch", counted in AgentResult.model_binding_mismatches rather than prefix_binding_mismatches. The toolkit sends that beta on every thinking run of the four gated rows, or on any Anthropic row with thinking enabled when you opt in with Agent(anthropic_block_binding="drop_block") (or strict_prefix_binding=True, which sends it with "error"). A client-side switch from Sonnet 5.5, Opus 5.5 or Fable 5.1 to an ungated row — Opus 5, Opus 4.8, Sonnet 5, Haiku 4.5 — therefore drops the earlier model's blocks silently and reports 0 today unless you opt in on the new agent. A server-side fallback is different: the request is built for the gated primary and carries the beta, so the fallback model's drop of the primary's blocks is expected to be reported — that has not been observed live yet. On any of these, a prefix mismatch is still the alarm; a model mismatch is expected.

Keep sending the full history, thinking blocks included, and let the API drop what the current model cannot read. The toolkit does exactly that: it never strips or rebuilds thinking because the model changed. The only things that remove blocks are ones you choose, each documented — prefix_strip_fallback, the no_reasoning and messages_only history modes, compaction, and turning thinking off on claude-sonnet-5-5 or claude-haiku-5-5, which leaves earlier turns' blocks out of that run's requests (why).

The final answer keeps its thinking on the preserved-thinking rows. On claude-fable-5-1, claude-opus-5-5, claude-sonnet-5-5 and claude-haiku-5-5 (the rows whose registry entry sets produces_preserved_thinking_blocks) an AgentSession stores each turn's final answer as the block list the model returned — the signed thinking / redacted_thinking blocks plus the text, byte for byte — on the tool loop and on a run with no local tools alike, so the next request replays that thinking as well as the tool rounds'. Earlier builds stored every final answer as plain text, which discarded that turn's reasoning on the client, the one thing Anthropic's guidance says makes the loss permanent. The rule is a storage-time gate:

every other Anthropic row stores the answer as plain text exactly as before (their request bodies are unchanged), and so does a gated turn that carried no thinking block;
DPT_ANTHROPIC_RETAIN_FINAL_TURN=off restores plain-text storage on the gated rows too (see Configuration and Environment Variables);
a plain-text AgentSession.stream() turn keeps its lightweight plain-text record; a streamed turn with output_schema is stored exactly as run() stores it, retained final turn included, and the same switch turns it back into plain text.

Cost. Replayed thinking is input: each retained answer adds its thinking tokens to the input of every later request of the session on the gated rows, which compounds across a long session and brings compaction forward. A session later moved to an ungated model still sends the blocks; the API drops the ones that model cannot read (unbilled, as described above). When the history is prepared for a non-Anthropic provider, or flattened by the text_summary history mode, a retained answer is rendered as its plain text and its thinking is left out — the same pattern DPT_GOOGLE_RETAIN_FINAL_TURN follows for Gemini.

agent = Agent(model="claude-sonnet-5", anthropic_block_binding="drop_block")
ci_agent = Agent(model="claude-sonnet-5", strict_prefix_binding=True)   # "error"

Precedence runs from the most specific to the most general: a thinking={"block_binding": ...} you pass yourself, then anthropic_block_binding, then strict_prefix_binding, then DPT_ANTHROPIC_BLOCK_BINDING, then the registry. An unrecognised environment value is ignored with one warning per process; an unrecognised parameter value raises. A row that declares supports_thinking_block_binding: false is never sent the field, even on an explicit opt-in.

Three registry keys drive all of it — enforces_thinking_prefix_check (send the field), supports_thinking_block_binding (absent means yes), and produces_preserved_thinking_blocks, which is diagnostic only today. claude-fable-5-1, claude-opus-5-5, claude-sonnet-5-5 and claude-haiku-5-5 carry all three; no other row carries any of them.

What has been proved live, and what has not

The binding is live-accepted on both Anthropic paths — the tool-calling loop and the structured-output path — and a strict-mode ("error") tool loop carrying a real thinking block passes clean, which is the check working rather than merely being tolerated. A reported drop_block transformation has not been observed live yet: on the rewrite scenarios that would provoke one, the model did not produce thinking blocks to drop. Treat the counters below as correct plumbing whose "mismatch" arm is still waiting on a live sighting.

What a run reports. AgentResult.prefix_binding_mismatches counts blocks dropped because the prefix moved — the alarm — and .model_binding_mismatches counts the ones a different model could not read, which is expected after a model change and not a bug in your integration. .prefix_binding_retry is True when a prefix-binding 400 was recovered from: the toolkit retries once, and only when the retry body genuinely differs (a mismatch, no binding already present, a thinking object to attach one to, not strict mode, not an opted-out row). All three are mirrored on DetailedCostReport and are 0 / 0 / False on every other provider. Under sliding_window a mismatch on every window slide is the design working: the cost is reasoning quality on the retained turns, not a failed request, and the run logs one warning naming the compaction strategy in force.

A 400 about a thinking block now says which of three failures it is — a prefix-binding mismatch, a signature that does not verify, or a modified latest assistant message — and raises a ValueError naming the cause and the remedy, quoting the provider's original error verbatim. Only the first is ever recovered from.

Two switches that need no beta header

Agent(prefix_audit=True) (or DPT_PREFIX_AUDIT=1) fingerprints the system prompt, the tools array and every message before each Anthropic request and logs what moved since the previous one, without changing a single request byte. It works on every model and is the only mechanism that catches an MCP server whose tool list changes mid-session.

Agent(prefix_strip_fallback=True) is for a client that cannot send the beta header at all: it removes every replayed thinking / redacted_thinking block at the completed-turn boundary instead, dropping a turn left with no content. It never runs between roundtrips of an open tool round.

Both have AgentBuilder methods and YAML config keys, and a typo in the YAML fails at load time rather than on the first request.

Reading `input_transformations`¶

When Anthropic changes something about a request's input before running it — for example dropping a preserved thinking block whose binding no longer matches the conversation prefix — it reports what it did. The toolkit reads that array off message_start, the final message_delta and the assembled response body, so an SDK that stops replaying extras on events cannot make the report disappear, and returns it:

result = await agent.run("Continue the analysis")

for entry in result.input_transformations:
    entry["type"]           # e.g. "thinking_dropped"
    entry["reason"]         # e.g. "prefix_binding_mismatch"
    entry["request_index"]  # which API request of the run reported it (1-based)

Entries whose type or reason the toolkit does not recognise are kept verbatim — Anthropic adds values over time, and "ignore what you don't recognise" means do not act on it, not discard it. An unrecognised reason is counted as neither kind of mismatch and kept as it arrived. The list lets you dashboard "reasoning dropped per session" from the result rather than by parsing logs, and the three counters above are derived from it.

It is empty on every row but claude-fable-5-1, claude-opus-5-5, claude-sonnet-5-5 and claude-haiku-5-5, the four that run the thinking prefix check.

OpenAI: Reasoning effort¶

OpenAI reasoning models (o-series, GPT-5 family) use an effort/summary configuration:

agent = Agent(
    model="o3",
    thinking={"effort": "high"},
)

effort — valid values depend on the model. O-series models support "low", "medium", "high". GPT-5 models additionally support "none" and "xhigh"; GPT-5.6 models add "max". The GPT-6 rows take "low" to "max"; gpt-6-sol and gpt-6-luna also accept "none", gpt-6-astra and gpt-6.1-sol do not. On those two a thinking dict whose effort is "none" fails validation with a logged error and the whole reasoning setting is left out of the request, summary included: the model reasons at its default, "medium", and no reasoning summary comes back.
summary — optional; controls reasoning summary output. Valid values: "auto", "concise", "detailed". Defaults to "auto".
mode — optional; GPT-5.6 models only. "standard" (default) or "pro" for pro-grade reasoning.

Reasoning mode (GPT-5.6 pro mode)¶

GPT-5.6 models (gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna) accept a reasoning mode. Pro mode runs deeper, pro-grade reasoning at standard token rates (it simply spends more reasoning tokens):

agent = Agent(
    model="gpt-5.6-sol",
    thinking={"effort": "high", "mode": "pro"},
)

Mode support is registry-driven: a model must list the mode in supported_reasoning_modes in model_information.json, and the key is dropped (with a warning) for models that don't. In pro mode, the allowed effort values come from the model's supported_reasoning_values_pro_mode list — pro mode requires an effort of at least "medium", so a lower effort (e.g. "low") is dropped and the API default for pro mode ("medium") applies. The mode key is only sent to the API when explicitly set.

Reasoning tokens are extracted from output_tokens_details.reasoning_tokens in the API response and reported in the result's reasoning_tokens field.

Note

Reasoning tokens are counted as part of output tokens for billing. Enable thinking only when the task benefits from deeper reasoning.

Google: Thinking budget and thinking level¶

Google supports two mutually exclusive thinking modes, split across model generations.

Thinking budget (Gemini 2.5 models)¶

Token-budget control for Gemini 2.5 Pro, Flash, and Flash Lite:

agent = Agent(
    model="gemini-2.5-pro",
    thinking={"budget_tokens": 8192},
)

Special values:

Value	Meaning
`-1`	Dynamic — model decides allocation
`0`	Disable thinking (where supported)

Model-specific budget ranges are registry-driven. Each model's rules live in descriptions.<model>.thinking_budget_rules in model_information.json, with the keys min, max, default, and allow_zero:

Model	Min	Max	Default (when unset)	Allows zero
gemini-2.5-pro	128	32,768	`-1` (dynamic)	No
gemini-2.5-flash	0	24,576	`-1` (dynamic)	Yes
gemini-2.5-flash-lite	512	24,576	unset (no thinking field sent)	Yes

Because the rules are data, adding budget support for a new Gemini model is a registry edit and needs no provider code. Two consequences follow at runtime:

A model whose registry row has no thinking_budget_rules does not take budget_tokens. The parameter is dropped with a warning. For the Gemini 3.x family this is deliberate — those models take a thinking level instead, and the warning names thinking_level and the model's supported values.
A budget outside [min, max] is not an error: it is logged as a warning and replaced with -1 (dynamic), so a mis-sized budget degrades to model-chosen allocation rather than failing the run.

Thinking level (Gemini 3.x models)¶

Discrete thinking levels for newer Gemini 3.x models:

agent = Agent(
    model="gemini-3-flash-preview",
    thinking={"level": "high"},
)

This can also be passed as the provider-specific thinking_level parameter:

agent = Agent(
    model="gemini-3-flash-preview",
    thinking_level="high",
)

Supported levels vary by model:

Model	Supported levels
gemini-3.1-pro-preview	low, medium, high
gemini-3-pro-image	low, high
gemini-3.1-flash-image	minimal, low, medium, high
gemini-3.1-flash-lite-image	minimal, high
gemini-3.1-flash-lite	minimal, low, medium, high
gemini-3-flash-preview	minimal, low, medium, high
gemini-3.5-flash	minimal, low, medium, high
gemini-3.5-flash-lite	minimal, low, medium, high
gemini-3.6-flash	minimal, low, medium, high
gemini-3.7-flash	low, medium, high

Warning

gemini-3.7-flash does not support the minimal level, unlike every earlier Gemini 3.x Flash model. Code that hardcodes thinking={"level": "minimal"} and then switches to 3.7 will have the level dropped — validate_thinking_level_google() logs an error and returns None, so the run proceeds at the model's own default thinking level.

Warning

You cannot specify both budget_tokens and level in the same thinking dict for Google — the toolkit raises a ValueError if both are present.

Google reads two keys; any other is named in a WARNING

The Google thinking dict takes exactly level and budget_tokens. Thought summaries are always requested, so there is no include_thoughts key, and the SDK's own spellings (thinking_level, thinking_budget) are not accepted here. Any other key is ignored, and since this release it is named in one WARNING from doc_processing_toolkit.agent.parameter_mapping together with the accepted keys; when neither accepted key is present the warning adds that no thinking setting is sent. thinking={"thinking_level": "high"} used to run at the model's default thinking with no word at all. The request itself is unchanged.

Thinking tokens are reported via usage_metadata.thoughts_token_count in the API response.

Thought summaries used to leak into the answer

With thinking enabled, Gemini's final response can carry thought-summary parts next to the answer. The tool-calling loop summed the text of every part, so the model's reasoning became — or was prefixed to — AgentResult.output, and the session stored and replayed that concatenation as the assistant message. Answers now exclude thought parts and the summary is returned in reasoning_content, which was previously always empty on Google (and, when it was populated at all, held the literal string "True" — the part.thought flag rather than the text). A thought-only final turn therefore yields an empty output with the summary in reasoning_content; no answer is invented.

Fireworks: Reasoning effort¶

Fireworks reasoning models accept an effort level:

agent = Agent(
    model="accounts/fireworks/models/glm-5p3",
    thinking={"effort": "high"},
)

Valid values are per-model, read from each model's supported_reasoning_values list in model_information.json. Most Fireworks models accept "none", "low", "medium" and "high"; several also accept "xhigh" and "max", and a few reject "none" outright:

Model	Supported values
`minimax-m3`	none, low, medium, high
`inkling`	none, low, medium, high, xhigh
`kimi-k3`, `routers/kimi-k3-fast`, `ember-1`	none, low, medium, high, xhigh, max
`qwen3p8-max`, `deepseek-v4p1-flash`	none, low, medium, high, xhigh, max
`glm-5p3`, `glm-5p3-flash`, `routers/glm-5p3-fast`	low, medium, high, xhigh, max (no `none`)
`gpt-oss-120b`	low, medium, high (no `none`)
`nemotron-lightning-3p5-30b-a3b`, `nemotron-3-ultra-nvfp4`	(no list — see below)

Kimi K3's list reached xhigh and max in 0.5.1 (probed 2026-09-27; it stopped at high before, from its docs). Moonshot documents K3's native levels as low, high and max, with max the model's default, so an agent that sets no effort gets the model's default and one that asks for max now reaches it; Fireworks may map medium and xhigh onto the native levels. none does switch reasoning off on Fireworks for K3 and ember-1. ember-1, Fireworks' own K3 derivative, takes the same list; its model page says its reasoning traces are about 40% shorter than the base model's.

On 2026-09-28 both rows answered a counting problem correctly at every level (three samples each of low, high and max). low and high reasoned for about the same number of tokens. max sometimes reasoned several times longer: up to 1,499 tokens against about 200, and one run took 36 seconds. So max is worth paying for only on problems that need it.

Two Fireworks families reject none, and two ignore effort entirely

The GLM 5.3 rows are thinking-only: Fireworks returns a 400 for reasoning_effort="none" ("GLM-5.3 is a thinking-only model"), so "none" is absent from their registry list and validate_reasoning_effort_fireworks() drops it locally rather than letting the request fail at the provider. gpt-oss-120b behaves the same way ("Invalid reasoning effort: none"). The two NVIDIA Nemotron rows carry no supported_reasoning_values at all because they ignore the parameter — any effort you pass is dropped with a WARNING and the model's own default applies.

accounts/fireworks/models/qwen3p8-max does accept "none" (confirmed by a live probe on 2026-09-20), but its vendor default effort is xhigh, so a dropped reasoning_effort falls back to an expensive setting rather than the cheapest. validate_reasoning_effort_fireworks() logs a prominent WARNING whenever it drops an unsupported value for exactly this reason — the request still succeeds, but cost and latency change.

The parameter maps differently depending on the API path:

Response API (MCP tools): nested as {"reasoning": {"effort": value}}
Chat Completions API (function calling): top-level reasoning_effort

Fireworks reasoning content may be returned as a reasoning_content field or within <think>...</think> tags in the message content. The toolkit extracts and combines both automatically.

Thinking tokens in results¶

After a run, thinking/reasoning tokens and the reasoning text are available on the AgentResult:

result = await agent.run("Explain quantum entanglement.")
print(result.reasoning_tokens)   # int — thinking tokens, a SUBSET of output_tokens
print(result.reasoning_content)  # str — the thinking text or summary, kept out of `output`

Both fields are reported on every path — with or without local tools, streaming or not — and on every provider:

Provider	`reasoning_tokens` source	`reasoning_content`
Anthropic	`usage.output_tokens_details.thinking_tokens`	Full `thinking` blocks
OpenAI	`usage.output_tokens_details.reasoning_tokens`	Reasoning summary only (the raw chain of thought is never returned)
Google	`usage_metadata.thoughts_token_count`	Thought summary only
Fireworks	`usage.completion_tokens_details.reasoning_tokens`, estimated from the text when a build omits it	`reasoning_content` field and/or `<think>` tags

Before this release only three of the eight provider paths reported anything: Anthropic's thinking tokens and blocks were never read, Fireworks' reported count was never surfaced on any path, Google's no-tools path reported neither the thought tokens it had already billed nor the summary, and OpenAI's reasoning summary was extracted and then dropped. The reported count now also reaches request_cost_records[*].reasoning_tokens and iteration_metrics, and the create_agent MCP result.

Reasoning tokens are already billed

reasoning_tokens is a subset of output_tokens on all four providers — Anthropic, OpenAI and Fireworks report it inside their output/completion total, and the toolkit folds Google's thoughts_token_count into output_tokens before pricing. It is reported for observability and never priced a second time, which is why result.total_tokens is input_tokens + output_tokens and does not add it. That formula changed in this release; it used to add the reasoning subset a second time, which was invisible only because most paths reported zero.

reasoning_content is the run's reasoning, not the last turn's

On a tool run the model does its thinking on the turn that chooses the tool and answers in a short final turn, so reporting the final turn's text alone produced reasoning_tokens > 0 next to an empty reasoning_content. reasoning_content is now the whole run's reasoning on all four providers — tool-choosing turns first, the final turn last.

A reasoning-named schema field also receives the reasoning

With an output_schema that has a field named internal_reasoning, thoughts or reasoning, every provider appends reasoning_content to that field of output as well. DPT_MERGE_REASONING_INTO_SCHEMA=off stops the copy, and the merge is scheduled to be retired in 0.5. See Reasoning in a schema field.

Summaries are requested for you

Google requests thought summaries on every thinking-enabled run (ThinkingConfig(include_thoughts=True), on both the tool loop and the no-tools path), and OpenAI reasoning models default to summary: "auto". Neither provider returns the raw chain of thought, and either may return no summary for a short turn — an empty reasoning_content there is a provider choice, not a toolkit failure. Nothing extra is requested and no extra tokens are spent to populate these fields.

Anthropic's streamed thinking breakdown

On a streamed response usage.output_tokens_details.thinking_tokens arrives on the final message_delta event only, and the SDK's accumulated final message keeps output_tokens_details=None. Every toolkit request streams, so a no-tools Anthropic run reported reasoning_tokens: 0 next to a populated reasoning_content, and the tool loop fell back to a count_tokens API request per thinking block to estimate a number Anthropic had already sent. Both stream consumers now restore the breakdown onto the final message, and the estimate survives only as a fallback for a response with no breakdown at all — one fewer API round trip per thinking block, and reporting only.

OpenAI structured-output runs report reasoning tokens too

Before 0.3.0, an OpenAI run that took the structured-output path (an output_schema with no local tools) returned reasoning_tokens: 0 and cached_input_tokens: 0 even on a reasoning model — the counts were read but never reached the result dict. Both are reported now. Billing is unchanged either way: reasoning tokens are already billed inside output_tokens, and the cached share was always priced correctly.

Validation¶

The toolkit validates thinking parameters at multiple layers:

Parameter mapping — transform_thinking_param() normalizes the thinking dict to provider format with sensible defaults.
Provider validation — dedicated validators check model-level support and constraints:
- validate_extended_thinking_config_anthropic()
- validate_reasoning_config_openai()
- validate_thinking_budget_google() / validate_thinking_level_google()
- validate_reasoning_effort_fireworks()
Parameter filtering — _filter_params_for_function() ensures only parameters accepted by each provider function are passed through.

The source of truth for whether a model supports thinking — and which configuration values are valid — is data/model_information.json. The validation functions listed above read feature flags and supported values from this file (e.g., supports_extended_thinking, supports_adaptive_thinking, supported_reasoning_values, supported_thinking_levels) to validate the config passed to the Agent for a given model.

Invalid configurations are logged and gracefully degraded (typically by returning None to skip thinking) rather than raising exceptions.

Because degrading is silent from the caller's point of view, these drops are logged loudly. In particular, validate_reasoning_effort_fireworks() emits a WARNING naming the dropped value, the model, and its supported_reasoning_values, and states that the request proceeds without the parameter. That matters because the model's own default effort then applies — on a model whose default is xhigh, a dropped "low" means paying for full-effort reasoning.

Tip

Watch for DROPPING reasoning_effort= in your logs when a run costs more than expected.

Interaction with other features¶

Tools: Thinking works alongside both function-calling and built-in tool paths. Anthropic's interleaved thinking enables reasoning between tool calls.
Structured output: Thinking is supported in structured-output mode across all providers.
Sessions: Thinking configuration is set at the Agent level and applied to every session turn. Anthropic thinking blocks are automatically preserved across turns.
Cost tracking: Reasoning tokens are included in cost calculations. For Anthropic, thinking tokens count toward output token billing; for OpenAI, reasoning tokens are part of output_tokens.