Skip to content

Recipe: Compaction Strategies

Goal

Enable automatic context compaction to keep long-running agent sessions within the model's context window.

Prerequisites

  • An agent with compaction=True
  • Tools or sessions that produce enough conversation history to approach the context window limit

Basic compaction setup

from doc_processing_toolkit import Agent

agent = Agent(
    model="claude-sonnet-5",
    instructions="You are a thorough research assistant.",
    tools=[
        {
            "type": "mcp_script",
            "path": "research_tools.py",
            "name": "research",
        }
    ],
    compaction=True,
)

result = await agent.run("Analyze all files in the data/ directory and summarize findings.")
print(result.output)
print(f"Compaction events: {result.compaction_count}")

With defaults, the agent uses the summarize strategy and triggers compaction when token usage exceeds 80% of the model's context window.

Strategy examples

Summarize (default)

Calls an LLM to generate a structured summary of the conversation, replacing old messages:

agent = Agent(
    model="claude-sonnet-5",
    instructions="You are a research assistant.",
    tools=[...],
    compaction=True,
    compaction_config={
        "strategy": "summarize",
    },
)

Sliding window

Keeps the last N messages and discards everything before. Fast but loses older context:

agent = Agent(
    model="claude-sonnet-5",
    instructions="You are a research assistant.",
    tools=[...],
    compaction=True,
    compaction_config={
        "strategy": "sliding_window",
        "keep_recent_messages": 10,
    },
)

Setting keep_recent_messages

When using tools, set keep_recent_messages to at least 4–6. The sliding window includes an orphan fix that advances past incomplete tool-call sequences at the cut point, which can reduce effective retention below the configured value. Values of 2–3 are risky when tool calls are present.

A low value is no longer unsafe, only ineffective: when the whole retained window would be tool traffic the cut point is walked backward instead, so the request stays valid and may keep more than keep_recent_messages. sliding_window guarantees validity, not size — see What sliding_window guarantees. A turn whose own tool loop fills the context window needs tool_result_clearing alongside it.

To add a truncation notice so the model knows context was removed:

agent = Agent(
    model="claude-sonnet-5",
    instructions="You are a research assistant.",
    tools=[...],
    compaction=True,
    compaction_config={
        "strategy": "sliding_window",
        "keep_recent_messages": 10,
        "sliding_window_include_truncation_notice": True,  # default
    },
)

The notice prepends a message like "[Earlier conversation context (N messages) has been truncated by sliding window compaction.]" to the retained history.

Tool result clearing

Replaces old tool results with placeholder text, keeping message structure intact. Good for conversations with large tool outputs:

agent = Agent(
    model="claude-sonnet-5",
    instructions="You are a research assistant.",
    tools=[...],
    compaction=True,
    compaction_config={
        "strategy": "tool_result_clearing",
        "keep_recent_tool_results": 5,
        "clear_tool_inputs": True,
    },
)

Note

tool_result_clearing works at both Level 1 (pre-call) and Level 2 (intra-run). At Level 1, it operates on provider-native session history messages. At Level 2, it operates on in-run provider messages during the tool loop.

Combined

Applies strategies in sequence: tool_result_clearing → sliding_window → summarize. Each stage reduces tokens; summarization only runs if still above threshold after prior stages:

agent = Agent(
    model="claude-sonnet-5",
    instructions="You are a research assistant.",
    tools=[...],
    compaction=True,
    compaction_config={
        "strategy": "combined",
        "keep_recent_messages": 8,
        "keep_recent_tool_results": 3,
    },
)

You can customize the order of strategies:

compaction_config={
    "strategy": "combined",
    "combined_order": ["sliding_window", "summarize"],
}

Custom configuration

agent = Agent(
    model="claude-sonnet-5",
    instructions="You are a research assistant.",
    tools=[...],
    compaction=True,
    compaction_config={
        "strategy": "combined",
        "trigger_threshold": 0.75,
        "keep_recent_messages": 10,
        "keep_recent_tool_results": 5,
        "archive_dir": "./my_archives",
        "summary_model": "gpt-4o-mini",
        "min_messages_before_compaction": 6,
        "max_compactions_per_session": 3,
    },
)

Session usage with compaction

Compaction works seamlessly with sessions. The session tracks cumulative compaction counts and archive paths:

from doc_processing_toolkit import Agent

agent = Agent(
    model="claude-sonnet-5",
    instructions="You are a research assistant.",
    tools=[...],
    compaction=True,
)

async with agent.session() as session:
    await session.run("Analyze the first batch of data files.")
    await session.run("Now analyze the second batch.")
    await session.run("Compare both batches and summarize.")

    print(f"Total compactions: {session.metadata['compaction_count']}")
    print(f"Archives: {session.metadata['compaction_archives']}")

Reading compaction results

Compaction metadata is available on AgentResult:

result = await agent.run("Process all documents in the directory.")

print(f"Compaction events: {result.compaction_count}")
print(f"Compaction input tokens: {result.compaction_input_tokens}")
print(f"Compaction output tokens: {result.compaction_output_tokens}")
print(f"Compaction cost: ${result.compaction_cost:.6f}")
print(f"Archives: {result.compaction_archives}")

Per-iteration metrics also include compaction details:

for m in result.iteration_metrics:
    if m.compaction_triggered:
        print(f"Iteration {m.iteration_number}: {m.compaction_strategy}, "
              f"cleared {m.compaction_tokens_cleared} tokens")

compaction_tokens_cleared is an estimate — the pre-compaction token count scaled by the local shrink ratio, not a fresh API recount. Treat it as a magnitude, not a billing figure. Earlier versions reported the entire pre-compaction count here regardless of strategy, so numbers from this field are not comparable across toolkit versions. See How the reported token figures are computed.

YAML configuration

model: claude-sonnet-5
instructions: You are a research assistant.
compaction: true
compaction_config:
  strategy: summarize
  trigger_threshold: 0.8
  archive_dir: ./compaction_archives
  keep_recent_messages: 5
  min_messages_before_compaction: 4

Builder pattern

from doc_processing_toolkit import Agent

agent = (
    Agent.builder()
    .model("claude-sonnet-5")
    .instructions("You are a research assistant.")
    .with_compaction(
        strategy="sliding_window",
        keep_recent_messages=10,
        trigger_threshold=0.75,
    )
    .build()
)

Or using AgentBuilder directly:

from doc_processing_toolkit import AgentBuilder

agent = (
    AgentBuilder()
    .model("claude-sonnet-5")
    .instructions("You are a research assistant.")
    .with_compaction(
        strategy="sliding_window",
        keep_recent_messages=10,
        trigger_threshold=0.75,
    )
    .build()
)

Archive files

When compaction triggers, the original conversation is archived to a JSON file before being replaced:

  • Path format: {archive_dir}/compaction_{session_id}_{timestamp}_{compaction_number}.json
  • Default directory: ./compaction_archives
  • Multimodal content (base64 images) is replaced with lightweight placeholders in the archive.

Disable archival if not needed:

compaction_config={
    "strategy": "sliding_window",
    "archive_history": False,
}