Recipe: Detailed Cost Tracking Report

Goal

Enable detailed per-run cost persistence.

Example

from doc_processing_toolkit import Agent

agent = Agent(
    model="gpt-4o",
    instructions="You are concise.",
    detailed_costs=True,
    cost_output_dir="agent_costs",
)

result = await agent.run("Explain quantum computing in one sentence.")
print(result.cost)
print(result.detailed_cost_file)

Example report shape

This sample came from a run with thinking off

Every reasoning_tokens below is 0 because the run that produced it did no thinking — not because the field is always zero. On a thinking run it is populated on every provider and every run shape, both per iteration and per request record; see Thinking and Reasoning.

{
    "run_id": "f47ac10b-58cc-4372-a567-0e02b2c3d479",
    "agent_name": null,
    "timestamp": "2026-02-16T12:34:56.789012",
    "provider": "anthropic",
    "model_name": "claude-sonnet-4-5",
    "total_input_tokens": 2600,
    "total_output_tokens": 210,
    "total_reasoning_tokens": 0,
    "total_cost": 0.012,
    "cost_estimated": false,
    "total_duration_ms": 3200,
    "thinking_enabled": false,
    "thinking_config": null,
    "max_iterations": 10,
    "temperature": 1.0,
    "iterations": [
        {
            "iteration_number": 1,
            "input_tokens": 1200,
            "output_tokens": 90,
            "reasoning_tokens": 0,
            "cost": 0.0087,
            "tool_calls": [],
            "timestamp": "2026-02-16T12:34:57.123456",
            "duration_ms": 1800,
            "thinking_enabled": false,
            "stop_reason": "tool_use",
            "model": "claude-sonnet-4-5",
            "cached_input_tokens": 0,
            "cache_write_tokens": 1000
        },
        {
            "iteration_number": 2,
            "input_tokens": 1400,
            "output_tokens": 120,
            "reasoning_tokens": 0,
            "cost": 0.0033,
            "tool_calls": [],
            "timestamp": "2026-02-16T12:34:58.623456",
            "duration_ms": 1400,
            "thinking_enabled": false,
            "stop_reason": "end_turn",
            "model": "claude-sonnet-4-5",
            "cached_input_tokens": 1000,
            "cache_write_tokens": 0
        }
    ],
    "total_iterations": 2,
    "tools_available": ["calculator"],
    "tools_used": ["calculator"],
    "tool_call_count": 1,
    "tool_success_rate": 1.0,
    "tool_usage_summary": {
        "calculator": {
            "calls": 1,
            "total_tokens": 0,
            "total_cost": 0.0,
            "total_duration_ms": 0,
            "success_count": 1,
            "failure_count": 0,
            "avg_duration_ms": 0,
            "success_rate": 1.0
        }
    },
    "pricing_info": {},
    "start_time": "2026-02-16T12:34:56.789012",
    "end_time": "2026-02-16T12:34:59.989012",
    "total_retry_cost": 0.0,
    "total_retry_input_tokens": 0,
    "total_retry_output_tokens": 0,
    "total_retry_count": 0,
    "total_cost_inclusive": 0.012,
    "total_cached_input_tokens": 1000,
    "total_cache_write_tokens": 1000,
    "request_cost_records": [
        {
            "input_tokens": 1200,
            "output_tokens": 90,
            "reasoning_tokens": 0,
            "cached_input_tokens": 0,
            "cache_write_tokens": 1000,
            "served_model": "claude-sonnet-4-5",
            "requested_tier": null,
            "served_tier": null,
            "cost": 0.0087
        },
        {
            "input_tokens": 1400,
            "output_tokens": 120,
            "reasoning_tokens": 0,
            "cached_input_tokens": 1000,
            "cache_write_tokens": 0,
            "served_model": "claude-sonnet-4-5",
            "requested_tier": null,
            "served_tier": null,
            "cost": 0.0033
        }
    ],
    "schema_version": 2
}

In this run the first request wrote 1000 tokens into the prompt cache (billed on top at the cache_write_5m rate) and the second read them back at the cached_input rate. total_cost (0.012) is the sum of the two per-request costs — 0.0087 + 0.0033 — plus a per-run built-in tool pass, which is 0 here because calculator is a local tool. Each iterations[] entry's thinking_enabled is false because this run sent no thinking; on a thinking run it is true on every entry whose request carried thinking (populated since 0.5.3 — before, it was always false).

Cached tokens are inside input_tokens

Request 2's prompt was 1400 tokens, 1000 of them served from cache, so its uncached share is input_tokens - cached_input_tokens = 400. That subset convention holds on all four providers — Anthropic reports cache reads outside its input_tokens, and the toolkit folds them in before reporting (a change since 0.2.0, where this example's request 2 would have shown input_tokens: 400). See Prompt caching.

What to validate

• report JSON is written to cost_output_dir
• result.detailed_cost_file points to an existing file
• result.cost_estimated/result.tool_costs_estimated align with report contents
• total_tool_reported_cost equals result.tool_reported_cost: the spend the run's tools reported, which total_cost already includes (see Tool-reported spend)
• schema_version is 2; reports written before this field existed load back as version 1

Reading per-request records

request_cost_records holds one entry per API request, and the run's token cost is their sum. This matters on tiered-pricing models: pricing a run's summed tokens can cross a 200k/272k tier threshold that no single request crossed, which overbills the whole run.

import json

with open(result.detailed_cost_file) as f:
    report = json.load(f)

records = report["request_cost_records"]
token_cost = sum(r["cost"] for r in records)
tool_cost = report["total_cost"] - token_cost

print(f"{len(records)} requests, ${token_cost:.6f} tokens + ${tool_cost:.6f} tools")

cached = report["total_cached_input_tokens"]
if cached:
    print(f"{cached} prompt tokens served from cache")

Records also carry served_model, which differs from the requested model when an Anthropic server-side fallback ran. requested_tier and served_tier carry service-tier attribution: they are populated on OpenAI and Google when a tier was requested, and None otherwise (including on every Anthropic record). Google records add served_tier_source ("body", "header" or "none"), which distinguishes a tier the provider reported from one the toolkit could not read at all. Billing follows the tier the provider actually served, not the one requested — see Service Tiers.

result.service_tier_served and result.priority_request_count collapse those records into the run-level answer, and the persisted report carries the same two fields.

Do not add reasoning_tokens to the totals

A record's reasoning_tokens is the thinking share of that record's own output_tokens on every provider, and its cost already includes it. input_tokens + output_tokens is the run's token total — which is exactly what AgentResult.total_tokens reports.

Service tiers

service_tier="priority" asks for the faster lane and service_tier="flex" for the cheaper best-effort one ("ultrafast", on gpt-6-astra only, is requested and billed the same way). The toolkit bills the tier the provider served, request by request, and tells you when that was not the tier you asked for:

import asyncio
from doc_processing_toolkit import Agent

async def main():
    for model, tier in (
        ("gpt-5.6-luna", "priority"),
        ("gemini-3.5-flash-lite", "priority"),
        ("gpt-5.6-luna", "flex"),
    ):
        agent = Agent(model=model, instructions="Answer in one word.", service_tier=tier)
        result = await agent.run("What colour is the sky on a clear day?")

        print(model, tier, result.service_tier_served, result.tier_downgraded,
              result.priority_request_count, result.cost)
        for record in result.request_cost_records:
            print("  ", record.get("requested_tier"), record.get("served_tier"),
                  record.get("served_tier_source"), record.get("cost"))

asyncio.run(main())

What the verifying run printed, and how to read it:

Request	service_tier_served	tier_downgraded	Per record: requested_tier / served_tier / served_tier_source	Meaning
gpt-5.6-luna, "priority"	"priority"	False	priority / priority / key absent	Served and billed at the priority row — $0.000138, twice the $0.000069 standard price of the same answer
gemini-3.5-flash-lite, "priority"	"unverifiable"	True	priority / unverifiable / none	Google's streaming endpoint does not expose the served tier, so the request is billed at standard and marked. The recorded cost may be under-stated if Google did serve priority; only the invoice settles it
gpt-5.6-luna, "flex"	None	False	None / None / key absent	This row has no flex price, so the tier was dropped with a DROPPING service_tier='flex' warning and the request ran at standard — as if no tier had been asked for

served_tier_source exists only on Google records ("body", "header" or "none"); read it with .get(). tier_downgraded means "at least one request was not served the tier it asked for" in either direction — for flex, being served standard costs more — so read requested_tier / served_tier for the direction. priority_request_count counts the requests actually billed at the priority row. The persisted report carries service_tier_served and priority_request_count too.

Which rows carry a tier table. A tier is valid exactly when the model's registry row has a service_tiers.<tier> price; anything else is dropped with a warning and runs standard. Priority: the four GPT-6 rows, gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna (2x) and nine Gemini rows from gemini-2.5-flash-lite to gemini-3.8-flash (1.8x). Flex: gemini-3.8-flash and the four GPT-6 rows (0.5x). Ultrafast: gpt-6-astra (6x). Anthropic and Fireworks rows carry none. The current list is in Service Tiers; the registry is the source of truth.

Publishing named cost lines

A client-facing report usually wants the cost broken down, not just totalled. Two surfaces give you that without re-deriving a rate from the registry:

• calculate_cost(..., cost_components_out=components) fills the dict you pass with input_text_usd, input_image_usd, input_audio_usd, cache_discount_usd (negative), cache_write_usd, output_text_usd, output_image_usd, output_audio_usd, tool_usd and total_usd. Every entry but the last sums to total_usd. See Cost Tracking.
• generate_image and edit_image results already carry a cost_breakdown (cost_input_text_usd, cost_input_image_usd, cost_text_usd, cost_image_usd, cost_cache_discount_usd) that sums to cost, alongside input_tokens and output_tokens. See Multimedia. When an agent calls these tools through the multimedia MCP server, their spend is already in the run's cost, with one entry per tool call in tool_reported_costs (0.5.1), so do not add it again. See Tool-reported spend.

Understanding Retry Costs

When API calls fail and are retried, each attempt may consume tokens. Read that hidden spend off AgentResult, not off the persisted report.

Reading retry spend from the result

result = await agent.run("Summarize this document", files=["./sample.pdf"])

if result.retry_count:
    clean_cost = result.cost - result.retry_cost
    print(f"{result.retry_count} retried attempts cost ${result.retry_cost:.6f}")
    print(f"the successful attempt alone cost ${clean_cost:.6f}")
    print(f"retried attempts burned {result.retry_input_tokens} in / {result.retry_output_tokens} out")

if result.cost and result.retry_cost / result.cost > 0.1:
    print("more than 10% of cost was retries — review timeout config or provider stability")

retry_cost is included in cost, and retry_input_tokens / retry_output_tokens are included in input_tokens / output_tokens — subtract to isolate the successful attempt. All four are zero on a clean run.

The report's total_retry_* fields are real as of 0.3.0

DetailedCostReport used to derive its retry totals by summing the per-iteration retry_* fields, which are reserved and hardcoded to zero on every provider (retries happen inside a single provider call, below the iteration boundary), so a run that genuinely retried still wrote "total_retry_cost": 0.0. The four run-level totals are now copied straight from the AgentResult fields above.

The per-iteration retry_* fields and retry_details remain reserved, so the per-iteration block in the JSON below still shows the shape rather than values you will observe. See Cost Tracking.

total_cost_inclusive equals total_cost

total_cost is taken from AgentResult.cost, which already includes retry_cost, so the old total_cost + total_retry_cost derivation added the retry spend a second time. The key now simply mirrors total_cost and is kept only so existing readers do not break. Read total_cost as the billed figure.

The retry field shape in a report

{
    "total_cost": 0.0523,
    "total_retry_cost": 0.0112,
    "total_retry_count": 2,
    "total_retry_input_tokens": 1800,
    "total_retry_output_tokens": 45,
    "iterations": [
        {
            "iteration_number": 1,
            "cost": 0.0523,
            "retry_count": 2,
            "retry_cost": 0.0112,
            "retry_input_tokens": 1800,
            "retry_output_tokens": 45,
            "retry_details": [
                {
                    "attempt_number": 1,
                    "input_tokens": 900,
                    "output_tokens": 20,
                    "cost": 0.0056,
                    "token_source": "stream_partial",
                    "error": "APITimeoutError: Request timed out"
                },
                {
                    "attempt_number": 2,
                    "input_tokens": 900,
                    "output_tokens": 25,
                    "cost": 0.0056,
                    "token_source": "stream_partial",
                    "error": "APITimeoutError: Request timed out"
                },
                {
                    "attempt_number": 3,
                    "input_tokens": 900,
                    "output_tokens": 140,
                    "cost": 0.0411,
                    "token_source": "api_usage",
                    "error": null
                }
            ]
        }
    ]
}

Read as a shape: two timeout failures consuming ~$0.011 in tokens before the third attempt succeeded. The four total_retry_* keys at the top of a real report carry figures like these; the per-iteration retry_* fields and retry_details nested under iterations still come back as zeros and an empty list, because retries happen below the iteration boundary.

Note

The token_source field in each retry detail indicates how token counts were determined. "stream_partial" means tokens were estimated from partial streaming output before the error. "billable_error_input_only" means only input tokens were estimated (no output was received).

Test sources

• tests/e2e/test_agent_e2e.py (test_detailed_cost_tracking)
• tests/unit/test_service_tier.py, tests/unit/test_flex_service_tier.py, tests/integration/test_service_tier_live.py (service tiers)

Related pages

• Service Tiers — priority and flex lanes, and why billing follows the served tier
• Prompt Caching — cut the input-token bill this report measures (enable, verify hits via cached_input_tokens)
• Cost Tracking — how every metric in the report is computed