mirror of
https://github.com/pydantic/pydantic-ai-harness.git
synced 2026-07-21 02:45:34 +00:00
Douwe MaanandGitHub
442e9ff493
Compaction capabilities (SlidingWindow, ClearToolResults, DeduplicateFileReads, SummarizingCompaction, TieredCompaction, LimitWarner) shipped as experimental, with a HarnessExperimentalWarning + single category-wide silence filter. 100% coverage; ruff + pyright strict.
64 lines
3.4 KiB
Markdown
64 lines
3.4 KiB
Markdown
# Compaction Capability — Implementation Plan
|
|
|
|
Closes #21
|
|
|
|
## Overview
|
|
|
|
This PR adds three compaction-related capabilities to `pydantic-harness`:
|
|
|
|
1. **`SlidingWindow`** — Zero-cost message trimming via a configurable sliding window.
|
|
2. **`LimitWarner`** — Injects warning messages when the agent approaches iteration, context-window, or total-token limits.
|
|
3. **`Compaction`** — LLM-powered summarization that replaces older messages with a compact summary.
|
|
|
|
All three are `AbstractCapability` subclasses that operate via the `before_model_request` hook, modifying `request_context.messages` before each model call.
|
|
|
|
## Design Decisions
|
|
|
|
### Tool-call / tool-return pair safety
|
|
|
|
The most critical invariant: trimming or compacting must **never** orphan a `ToolCallPart` without its corresponding `ToolReturnPart` (or vice versa). Doing so causes HTTP 400 errors from LLM providers.
|
|
|
|
The implementation uses a `_is_safe_cutoff()` function that searches around a proposed cutoff point for tool-call pairs that would be split. If a cutoff is unsafe, it walks backward to find a safe one. This approach is adapted from [vstorm-co/summarization-pydantic-ai](https://github.com/vstorm-co/summarization-pydantic-ai)'s `_cutoff.py`.
|
|
|
|
### Trigger and retention modes
|
|
|
|
Both `SlidingWindow` and `Compaction` support two trigger modes:
|
|
- `max_messages` — fire when message count exceeds threshold
|
|
- `max_tokens` — fire when estimated token count exceeds threshold
|
|
|
|
And two retention modes:
|
|
- `keep_messages` — retain N tail messages
|
|
- `keep_tokens` — retain messages fitting within a token budget
|
|
|
|
### Token estimation
|
|
|
|
A simple `estimate_token_count()` function approximates tokens at ~4 characters per token. This avoids requiring a tokenizer dependency while providing reasonable estimates for threshold detection.
|
|
|
|
### LimitWarner design
|
|
|
|
Warnings are injected as a trailing `ModelRequest` with a `UserPromptPart` (not a system message), because models tend to pay more attention to user messages. A `[LimitWarner]` marker enables stripping previous warnings before injecting new ones, preventing warning accumulation.
|
|
|
|
### Compaction summarization
|
|
|
|
The `Compaction` capability creates a temporary `pydantic_ai.Agent` with the configured summarization model. System prompts from the beginning of the conversation are preserved and prepended to the summary message.
|
|
|
|
## Dependencies
|
|
|
|
- Requires `pydantic-ai-slim` with the capabilities branch (not yet on PyPI).
|
|
- For local development, add a `[tool.uv.sources]` override pointing to the capabilities branch checkout.
|
|
|
|
## Files
|
|
|
|
- `src/pydantic_harness/compaction.py` — All three capabilities plus helpers
|
|
- `src/pydantic_harness/__init__.py` — Package exports
|
|
- `tests/test_compaction.py` — 81 tests covering all code paths
|
|
- `pyproject.toml` — Coverage threshold adjustment (98% due to branch coverage of elif chains)
|
|
|
|
## References
|
|
|
|
- [pydantic/pydantic-ai#4137](https://github.com/pydantic/pydantic-ai/issues/4137) — First-class Context Compaction API
|
|
- [pydantic/pydantic-ai#4267](https://github.com/pydantic/pydantic-ai/issues/4267) — Anthropic Compactions
|
|
- [pydantic/pydantic-ai#4013](https://github.com/pydantic/pydantic-ai/issues/4013) — OpenAI Compactions
|
|
- [pydantic/pydantic-harness#35](https://github.com/pydantic/pydantic-harness/issues/35) — Expose context window size on ModelProfile
|
|
- [vstorm-co/summarization-pydantic-ai](https://github.com/vstorm-co/summarization-pydantic-ai) — Prior art for cutoff logic
|