Skip to content

Recipe: Structured Output with Schemas

Goal

Return machine-validated structured output via output_schema.

Example

from pydantic import BaseModel, Field
from doc_processing_toolkit import Agent

class SimpleAnswer(BaseModel):
    answer: str = Field(description="Answer text")
    confidence: float = Field(ge=0, le=1)

agent = Agent(model="gemini-2.5-flash", instructions="Respond in schema.")
result = await agent.run("What is the capital of Japan?", output_schema=SimpleAnswer)
print(result.output)

Custom tools + schema:

from doc_processing_toolkit import tool

@tool
def add(a: int, b: int) -> int:
    return a + b

agent = Agent(model="gpt-4o", instructions="Use tools then return schema.", tools=[add])
result = await agent.run("Add 4 and 9 and return result.", output_schema=SimpleAnswer)

Built-in tools + schema:

from doc_processing_toolkit.schemas import BuiltInToolConfig

agent = Agent(
    model="claude-sonnet-4-5",
    instructions="Use web search and return schema output.",
    built_in_tools={"web_search": BuiltInToolConfig(enabled=True)},
)
result = await agent.run("Find Tokyo population and summarize.", output_schema=SimpleAnswer)

What to validate

  • output is schema-shaped object/dict
  • required keys and value constraints are satisfied
  • provider returns no schema validation error

Notes

  • Keep schemas concise and deterministic for better reliability.
  • Use explicit descriptions on fields that require semantic constraints.

Reasoning goes to reasoning_content, not a schema field

Read the model's reasoning from result.reasoning_content — the canonical location on every provider and every run shape. A schema field named internal_reasoning, thoughts or reasoning (case-insensitive) additionally receives a copy today: all four providers append the reasoning text after whatever the model wrote there, under a ***Internal Reasoning:*** heading.

class Answer(BaseModel):
    answer: str
    reasoning: str  # the model's own words, plus the merged reasoning by default

result = await agent.run("...", output_schema=Answer)
result.reasoning_content      # the reasoning: read it here
result.output["reasoning"]    # model text + "***Internal Reasoning:***" + the same reasoning

DPT_MERGE_REASONING_INTO_SCHEMA=off stops the copy, so the field keeps exactly what the model wrote (see Configuration). The merge is scheduled to be retired in 0.5 — the default becomes off, as a breaking change — so new code should not rely on it.

Anthropic: the model's final answer wins when a turn emits several JSON blocks

An Anthropic response is an ordered list of content blocks, and any non-text block — a server_tool_use, a search result — ends the current text run. A web-search turn therefore commonly carries two text blocks: a pre-search draft, then the model's final, post-search answer.

Concatenating them produces {...}{...}, which is not parseable JSON; brace repair then isolates the first object, so the draft was silently returned as data with error: None while the refined answer was discarded.

The toolkit now walks the text blocks in reverse and takes the first one that both parses and validates against your output_schema. Schema validation is load-bearing here — a trailing bare scalar like "done" is valid JSON and must not hijack the answer. A warning is logged when more than one block validates, naming how many were found.

Every other shape is unchanged byte-for-byte: a single text block, one JSON object split across blocks, prose followed by JSON, and a run with no schema all fall back to the previous concatenate-then-parse behavior, including its error semantics.

Google: output truncation is reported distinctly

When a Gemini structured-output response is cut off at the output-token limit, the JSON is incomplete and cannot be repaired — a mid-string cut is not a brace-balancing problem. The toolkit detects this from finish_reason=MAX_TOKENS and handles it in two steps instead of raising a misleading "invalid JSON" error:

  1. Below the model's cap — max_output_tokens is automatically raised to min(max_tokens × 1.5, model cap) and the call is retried once, logged as a warning.
  2. At the model's cap — a ValueError is raised naming the actual numbers: Google response truncated at max_output_tokens (N of MAX) — output too large for this page/schema.

The second case is deliberately not retried: an identical retry costs money and produces the same truncation. Fix it by narrowing the schema, splitting the input into smaller pages, or asking for less output per call.

Fireworks: an empty response names truncation as the cause

When a Fireworks structured-output call comes back with no content at all, the raised ValueError distinguishes the two ways that happens:

  • finish_reason == "length" — Fireworks response content is empty: the response was truncated by max_tokens=N before any usable content was produced. Raise max_tokens.
  • anything else — Fireworks response content is empty.

Previously the generic message was raised first, before the truncation warning could be logged, so a max_tokens that was simply too small for the schema looked like an unexplained empty response. The truncation warning still fires on a truncated-but-non-empty response, where the JSON is present yet incomplete.

Fireworks: the schema instruction reaches the model

Beside response_format, a Fireworks structured-output request carries one extra user message: "Output MUST be pure JSON matching the ... schema below", followed by the schema, as the Fireworks JSON-mode documentation recommends. Since 0.5.1 it is sent when a system prompt is set too. Before, the request was built from a copy of the message list and the instruction was appended to the original afterwards, so with a system prompt it never reached the model (the case since 0.4.0). A plain-text run without MCP tools takes the same path, because the toolkit answers it through a SimpleTextResponse schema. response_format still constrained the output, but a row whose chat template does not render the schema (glm-5p3-flash) answered without seeing it. There is no switch.

With tools, every Fireworks row keeps the tools-first fallback (supports_function_calling_and_structured_outputs: false). Sending tools and a json_schema response_format in the same request was measured on all 13 rows on 2026-09-28. Twelve answered in one request with valid JSON and never called a tool the prompt required, even when told to. inkling called it only when the prompt forbade computing the answer itself; with two other phrasings it answered at once (0 of 5 each), while the fallback called the tool 5 of 5. The measurements are in pricing.fireworks._comments in the registry.

Test sources

  • tests/e2e/test_agent_e2e.py (test_thinking_with_structured_output, test_custom_tools_with_structured_output, test_built_in_tools_with_structured_output)
  • tests/integration/test_sync_wrappers.py (provider structured-output tests)