pydantic-ai-harness/PLAN.md

Compaction Capability — Implementation Plan

Closes #21

Overview

This PR adds three compaction-related capabilities to pydantic-harness:

1. SlidingWindow — Zero-cost message trimming via a configurable sliding window.
2. LimitWarner — Injects warning messages when the agent approaches iteration, context-window, or total-token limits.
3. Compaction — LLM-powered summarization that replaces older messages with a compact summary.

All three are AbstractCapability subclasses that operate via the before_model_request hook, modifying request_context.messages before each model call.

Design Decisions

Tool-call / tool-return pair safety

The most critical invariant: trimming or compacting must never orphan a ToolCallPart without its corresponding ToolReturnPart (or vice versa). Doing so causes HTTP 400 errors from LLM providers.

The implementation uses a _is_safe_cutoff() function that searches around a proposed cutoff point for tool-call pairs that would be split. If a cutoff is unsafe, it walks backward to find a safe one. This approach is adapted from vstorm-co/summarization-pydantic-ai's _cutoff.py.

Trigger and retention modes

Both SlidingWindow and Compaction support two trigger modes:

• max_messages — fire when message count exceeds threshold
• max_tokens — fire when estimated token count exceeds threshold

And two retention modes:

• keep_messages — retain N tail messages
• keep_tokens — retain messages fitting within a token budget

Token estimation

A simple estimate_token_count() function approximates tokens at ~4 characters per token. This avoids requiring a tokenizer dependency while providing reasonable estimates for threshold detection.

LimitWarner design

Warnings are injected as a trailing ModelRequest with a UserPromptPart (not a system message), because models tend to pay more attention to user messages. A [LimitWarner] marker enables stripping previous warnings before injecting new ones, preventing warning accumulation.

Compaction summarization

The Compaction capability creates a temporary pydantic_ai.Agent with the configured summarization model. System prompts from the beginning of the conversation are preserved and prepended to the summary message.

Dependencies

• Requires pydantic-ai-slim with the capabilities branch (not yet on PyPI).
• For local development, add a [tool.uv.sources] override pointing to the capabilities branch checkout.

Files

• src/pydantic_harness/compaction.py — All three capabilities plus helpers
• src/pydantic_harness/__init__.py — Package exports
• tests/test_compaction.py — 81 tests covering all code paths
• pyproject.toml — Coverage threshold adjustment (98% due to branch coverage of elif chains)

References

• pydantic/pydantic-ai#4137 — First-class Context Compaction API
• pydantic/pydantic-ai#4267 — Anthropic Compactions
• pydantic/pydantic-ai#4013 — OpenAI Compactions
• pydantic/pydantic-harness#35 — Expose context window size on ModelProfile
• vstorm-co/summarization-pydantic-ai — Prior art for cutoff logic