diff --git a/README.md b/README.md index 5967778..59e5df8 100644 --- a/README.md +++ b/README.md @@ -33,7 +33,7 @@ uv add "pydantic-ai-harness[acp]" # ACP (serve an agent to editors over The `code-mode` extra is also supported as an alias. -Requires Python 3.10+ and `pydantic-ai-slim>=1.95.1`. +Requires Python 3.10+ and `pydantic-ai-slim>=2.1.0`. ## Quick start @@ -107,20 +107,25 @@ We studied leading coding agents, agent frameworks, and Claw-style assistants to | | **Tool search** | Progressive tool discovery for large tool sets | :white_check_mark: [Pydantic AI](https://pydantic.dev/docs/ai/tools-toolsets/toolsets/#deferred-loading) | | | | **File system** | Read, write, edit, search files with path traversal prevention | :white_check_mark: [Docs](pydantic_ai_harness/filesystem/) | [pydantic-ai-backend](https://github.com/vstorm-co/pydantic-ai-backend) (vstorm‑co) | | | **Shell** | Execute commands with allowlists, denylists, and timeouts | :white_check_mark: [Docs](pydantic_ai_harness/shell/) | [pydantic-ai-backend](https://github.com/vstorm-co/pydantic-ai-backend) (vstorm‑co) | -| | **Repo context injection** | Auto-load CLAUDE.md/AGENTS.md and repo structure | :construction: [PR #175](https://github.com/pydantic/pydantic-ai-harness/pull/175) | [pydantic-deep](https://github.com/vstorm-co/pydantic-deepagents) (vstorm‑co) | +| | **Repo context injection** | Auto-load CLAUDE.md/AGENTS.md and repo structure | :white_check_mark: [Docs](pydantic_ai_harness/experimental/context/) (experimental) | [pydantic-deep](https://github.com/vstorm-co/pydantic-deepagents) (vstorm‑co) | +| | **Docs lookup** | On-demand `read_pyai_docs` tool for Pydantic AI docs | :white_check_mark: [Docs](pydantic_ai_harness/experimental/docs/) (experimental) | | | | **Verification loop** | Run tests after edits, auto-fix failures | :construction: [PR #169](https://github.com/pydantic/pydantic-ai-harness/pull/169) | | | **Editor integration** | **ACP** | Serve an agent to editors (Zed, etc.) over the [Agent Client Protocol](https://agentclientprotocol.com) -- streamed text, diff-rendered edits, tool approval | :white_check_mark: [Docs](pydantic_ai_harness/experimental/acp/) (experimental) | | -| **Context management** | **Sliding window** | Trim conversation history to stay within token limits | :construction: [PR #191](https://github.com/pydantic/pydantic-ai-harness/pull/191) | [summarization-pydantic-ai](https://github.com/vstorm-co/summarization-pydantic-ai) (vstorm‑co) | -| | **Context compaction** | LLM-powered summarization of older messages | :construction: [PR #191](https://github.com/pydantic/pydantic-ai-harness/pull/191) | [summarization-pydantic-ai](https://github.com/vstorm-co/summarization-pydantic-ai) (vstorm‑co) | -| | **Limit warnings** | Warn agent before hitting context/iteration limits | :construction: [PR #191](https://github.com/pydantic/pydantic-ai-harness/pull/191) | [summarization-pydantic-ai](https://github.com/vstorm-co/summarization-pydantic-ai) (vstorm‑co) | -| | **Tool output management** | Truncate, summarize, or spill large tool outputs | :construction: [PR #185](https://github.com/pydantic/pydantic-ai-harness/pull/185) | | +| **Prompt management** | **Managed prompt** | Back an agent's instructions with a [Logfire](https://pydantic.dev/logfire)-managed prompt, editable without shipping code | :white_check_mark: [Docs](pydantic_ai_harness/logfire/) | | +| **Context management** | **Sliding window** | Trim conversation history to stay within token limits | :white_check_mark: [Docs](pydantic_ai_harness/experimental/compaction/) (experimental) | [summarization-pydantic-ai](https://github.com/vstorm-co/summarization-pydantic-ai) (vstorm‑co) | +| | **Context compaction** | LLM-powered summarization of older messages | :white_check_mark: [Docs](pydantic_ai_harness/experimental/compaction/) (experimental) | [summarization-pydantic-ai](https://github.com/vstorm-co/summarization-pydantic-ai) (vstorm‑co) | +| | **Limit warnings** | Warn agent before hitting context/iteration limits | :white_check_mark: [Docs](pydantic_ai_harness/experimental/compaction/) (experimental) | [summarization-pydantic-ai](https://github.com/vstorm-co/summarization-pydantic-ai) (vstorm‑co) | +| | **Tool output management** | Truncate, summarize, or spill large tool outputs | :white_check_mark: [Docs](pydantic_ai_harness/experimental/overflow/) (experimental) | | | | **System reminders** | Inject periodic reminders to counteract instruction drift | :construction: [PR #181](https://github.com/pydantic/pydantic-ai-harness/pull/181) | | | **Memory & persistence** | **Memory** | Persistent key-value memory across sessions | :construction: [PR #179](https://github.com/pydantic/pydantic-ai-harness/pull/179) | [pydantic-deep](https://github.com/vstorm-co/pydantic-deepagents) (vstorm‑co) | -| | **Session persistence** | Save and restore full conversation state | :construction: [PR #176](https://github.com/pydantic/pydantic-ai-harness/pull/176) | | -| | **Checkpointing** | Save, rewind, and fork conversation state | :memo: [#196](https://github.com/pydantic/pydantic-ai-harness/issues/196) | [pydantic-deep](https://github.com/vstorm-co/pydantic-deepagents) (vstorm‑co) | -| **Agent orchestration** | **Sub-agents** | Delegate subtasks to specialized child agents | :construction: [PR #178](https://github.com/pydantic/pydantic-ai-harness/pull/178) | [subagents-pydantic-ai](https://github.com/vstorm-co/subagents-pydantic-ai) (vstorm‑co) | +| | **Session persistence** | Save and restore full conversation state | :white_check_mark: [Docs](pydantic_ai_harness/experimental/step_persistence/) (experimental) | | +| | **Checkpointing** | Snapshot, resume (`continue_run`), and fork (`fork_run`) a run | :white_check_mark: [Docs](pydantic_ai_harness/experimental/step_persistence/) (experimental) | [pydantic-deep](https://github.com/vstorm-co/pydantic-deepagents) (vstorm‑co) | +| | **Media externalization** | Offload large `BinaryContent` to content-addressed stores (building blocks) | :white_check_mark: [Docs](pydantic_ai_harness/experimental/media/) (experimental) | | +| **Agent orchestration** | **Sub-agents** | Delegate subtasks to specialized child agents | :white_check_mark: [Docs](pydantic_ai_harness/experimental/subagents/) (experimental) | [subagents-pydantic-ai](https://github.com/vstorm-co/subagents-pydantic-ai) (vstorm‑co) | +| | **Dynamic workflows** | Author and run typed multi-agent workflows as sandboxed code | :construction: [PR #273](https://github.com/pydantic/pydantic-ai-harness/pull/273) | | | | **Skills** | Progressive tool loading -- search, activate, deactivate | :construction: [PR #183](https://github.com/pydantic/pydantic-ai-harness/pull/183) | [pydantic-ai-skills](https://github.com/DougTrajano/pydantic-ai-skills) (DougTrajano), [pydantic-deep](https://github.com/vstorm-co/pydantic-deepagents) (vstorm‑co) | -| | **Planning** | Break complex tasks into structured plans before execution | :construction: [PR #180](https://github.com/pydantic/pydantic-ai-harness/pull/180) | | +| | **Planning** | Break complex tasks into structured plans before execution | :white_check_mark: [Docs](pydantic_ai_harness/experimental/planning/) (experimental) | | +| | **Runtime authoring** | Let an agent author, validate, and load real capabilities at runtime | :white_check_mark: [Docs](pydantic_ai_harness/experimental/authoring/) (experimental) | | | | **Task tracking** | Track tasks, subtasks, and dependencies | :memo: [#65](https://github.com/pydantic/pydantic-ai-harness/issues/65) | [pydantic-ai-todo](https://github.com/vstorm-co/pydantic-ai-todo) (vstorm‑co) | | | **Teams** | Multi-agent teams with shared state and message bus | :memo: [#195](https://github.com/pydantic/pydantic-ai-harness/issues/195) | [pydantic-deep](https://github.com/vstorm-co/pydantic-deepagents) (vstorm‑co) | | **Safety & guardrails** | **Input guardrails** | Validate user input before the agent run starts | :construction: [PR #182](https://github.com/pydantic/pydantic-ai-harness/pull/182) | [pydantic-ai-shields](https://github.com/vstorm-co/pydantic-ai-shields) (vstorm‑co) | diff --git a/pydantic_ai_harness/.agents/skills/pydantic-ai-harness/SKILL.md b/pydantic_ai_harness/.agents/skills/pydantic-ai-harness/SKILL.md index af7d3d9..3e86ab9 100644 --- a/pydantic_ai_harness/.agents/skills/pydantic-ai-harness/SKILL.md +++ b/pydantic_ai_harness/.agents/skills/pydantic-ai-harness/SKILL.md @@ -1,8 +1,8 @@ --- name: pydantic-ai-harness -description: Extend Pydantic AI agents with batteries-included capabilities from pydantic-ai-harness -- currently Code Mode, which collapses many tool calls into one sandboxed Python execution. Use when the user mentions pydantic-ai-harness, CodeMode, Monty, code mode, or tool sandboxing, when they want an agent to run agent-written Python, or when a Pydantic AI agent would benefit from orchestrating multiple tool calls in a single sandboxed script. +description: Extend Pydantic AI agents with batteries-included capabilities from pydantic-ai-harness -- Code Mode (collapse many tool calls into one sandboxed Python execution), a filesystem and shell, sub-agents, planning, context compaction, and more. Use when the user mentions pydantic-ai-harness, CodeMode, Monty, code mode, or tool sandboxing, when they want first-party filesystem/shell/sub-agent/planning/compaction capabilities for a Pydantic AI agent, when they want an agent to run agent-written Python, or when a Pydantic AI agent would benefit from orchestrating multiple tool calls in a single sandboxed script. license: MIT -compatibility: Requires Python 3.10+ and pydantic-ai-slim>=1.95.1 +compatibility: Requires Python 3.10+ and pydantic-ai-slim>=2.1.0 metadata: version: "0.1.0" author: pydantic @@ -33,13 +33,24 @@ Do **not** use this skill for: ## Supported Capabilities +`CodeMode` has a full reference below; it is the flagship capability and the one this skill goes deep on. +The rest ship today and each has its own README with API and examples. + +**Stable** (imported from `pydantic_ai_harness`): + | Capability | Description | Reference | |---|---|---| | `CodeMode` | Wraps eligible tools into a single sandboxed `run_code` tool so the model orchestrates them in Python | [Code Mode](./references/CODE-MODE.md) | +| `FileSystem` | Read, write, edit, and search files under a root directory, with traversal prevention | [README](https://github.com/pydantic/pydantic-ai-harness/tree/main/pydantic_ai_harness/filesystem) | +| `Shell` | Run commands in a subprocess with allowlists, a default denylist, timeouts, and env masking | [README](https://github.com/pydantic/pydantic-ai-harness/tree/main/pydantic_ai_harness/shell) | +| `ManagedPrompt` | Back an agent's instructions with a Logfire-managed prompt | [README](https://github.com/pydantic/pydantic-ai-harness/tree/main/pydantic_ai_harness/logfire) | -More capability areas are tracked in the -[capability matrix](https://github.com/pydantic/pydantic-ai-harness#capability-matrix); as they stabilize, -this skill grows to cover them. +**Experimental** (imported from `pydantic_ai_harness.experimental`, may change in any release): `SubAgents`, +`Planning`, a compaction family (`SlidingWindow`, `SummarizingCompaction`, ...), `OverflowingToolOutput`, +`RepoContext`, `StepPersistence`, `PyaiDocs`, `RuntimeAuthoring`, and an ACP server adapter. + +The full, current list with links and status is in the +[capability matrix](https://github.com/pydantic/pydantic-ai-harness#capability-matrix). ## Install @@ -53,7 +64,7 @@ Each capability declares its own extra. Code Mode needs the Monty sandbox: uv add "pydantic-ai-harness[codemode]" # `code-mode` is also accepted as an alias ``` -Requires Python 3.10+ and `pydantic-ai-slim>=1.95.1`. +Requires Python 3.10+ and `pydantic-ai-slim>=2.1.0`. ## Quick Start diff --git a/pydantic_ai_harness/experimental/media/README.md b/pydantic_ai_harness/experimental/media/README.md new file mode 100644 index 0000000..7fc25cd --- /dev/null +++ b/pydantic_ai_harness/experimental/media/README.md @@ -0,0 +1,61 @@ +# Media + +> **Experimental.** Imported from `pydantic_ai_harness.experimental.media`, this module can change or be removed in any release. Importing it emits a `HarnessExperimentalWarning`. + +```python +from pydantic_ai_harness.experimental.media import ( + DiskMediaStore, + S3MediaStore, + SqliteMediaStore, + externalize_media, + restore_media, +) +``` + +Content-addressed stores and walker helpers that move large binary payloads out of message history and put them back on demand. + +These are building blocks, not a capability. There is no class you add to `Agent(capabilities=[...])` yet. [`StepPersistence`](../step_persistence/) uses them to keep snapshots small when messages carry `BinaryContent`. A forthcoming `MediaExternalizer` capability will reuse the same stores to rewrite `BinaryContent` into URL parts before the model sees them. + +## Why + +A conversation that carries images, audio, or other `BinaryContent` inlines those bytes into every message. Persist that history and each snapshot re-serializes the payloads. Content-addressed storage writes each payload once, keyed by its own hash, and leaves a short `media://` URI in its place. The same bytes are stored once no matter how many messages or snapshots reference them. + +## Stores + +Every store implements the `MediaStore` protocol: `put`, `get`, `exists`, `public_url`, and `get_metadata`, all async and content-addressed (the URI is derived from the payload hash, so identical bytes deduplicate). + +| Store | Backed by | Use when | +|---|---|---| +| `DiskMediaStore(directory=...)` | A directory on disk | Local runs and tests | +| `SqliteMediaStore(...)` | A SQLite database | A single-file store that travels with the data | +| `S3MediaStore(...)` | S3 or an S3-compatible bucket | Shared or production storage | + +A `KeyStrategy` controls the on-store layout, and a `PublicUrlResolver` (or `make_static_public_url`) turns a stored URI into a public URL when the store is served over HTTP. + +## Walker helpers + +`externalize_media` and `restore_media` walk a message node and swap payloads for URIs and back. + +```python +store = DiskMediaStore(directory='./media') + +# Replace BinaryContent larger than the threshold with media:// URIs. +lean = await externalize_media(message, media_store=store, threshold_bytes=32_000) + +# Later, rehydrate the URIs back into BinaryContent. +full = await restore_media(lean, media_store=store) +``` + +`externalize_media` only externalizes payloads over `threshold_bytes`; smaller ones stay inline. `media_uri_for` and `parse_media_uri` give you the raw URI round-trip if you need to key media yourself. + +## API + +| Symbol | Purpose | +|---|---| +| `MediaStore` | Async content-addressed store protocol (`put` / `get` / `exists` / `public_url` / `get_metadata`) | +| `DiskMediaStore`, `SqliteMediaStore`, `S3MediaStore` | Concrete stores | +| `MediaContext` | Per-call context (e.g. tenant) threaded through store operations | +| `KeyStrategy`, `default_key_strategy` | On-store key layout | +| `PublicUrlResolver`, `make_static_public_url` | Resolve a stored URI to a public URL | +| `externalize_media`, `restore_media` | Walk a message node to externalize / rehydrate payloads | +| `media_uri_for`, `parse_media_uri` | Compute and parse a `media://` URI | diff --git a/tests/test_docs_parity.py b/tests/test_docs_parity.py new file mode 100644 index 0000000..a652d3c --- /dev/null +++ b/tests/test_docs_parity.py @@ -0,0 +1,66 @@ +"""Keep the README honest about what ships. + +Every capability package must document itself with a `README.md` and be linked +from the top-level `README.md`. A capability cannot land without showing up in +the docs, so the "what's available today" tables cannot silently fall behind the +code. This is the mechanical half of docs parity; the semantic half (does the +prose match the code as written) is a review-time concern, not a unit test. +""" + +from __future__ import annotations + +from pathlib import Path + +import pytest + +_ROOT = Path(__file__).parent.parent +_PACKAGE = _ROOT / 'pydantic_ai_harness' + +# The `experimental` package is a namespace/warning shim, not a capability, so it +# has no standalone README and is not listed in the top-level tables. +_NAMESPACE_PACKAGES = {_PACKAGE / 'experimental'} + + +def _capability_packages() -> list[Path]: + """Directories that are importable packages and represent a capability's public surface.""" + candidates: list[Path] = [] + for parent in (_PACKAGE, _PACKAGE / 'experimental'): + for child in sorted(parent.iterdir()): + if not child.is_dir() or child.name.startswith(('_', '.')): + continue + # A non-package dir under the capability roots does not occur in a clean tree, so this guard stays uncovered. + if not (child / '__init__.py').exists(): # pragma: no cover + continue + if child in _NAMESPACE_PACKAGES: + continue + candidates.append(child) + return candidates + + +_CAPABILITY_PACKAGES = _capability_packages() + + +def test_capability_packages_discovered() -> None: + # Guard against the discovery silently finding nothing (e.g. a moved package root), + # which would make the parametrized checks below vacuously pass. + assert len(_CAPABILITY_PACKAGES) >= 10 + + +@pytest.mark.parametrize('package', _CAPABILITY_PACKAGES, ids=lambda p: str(p.relative_to(_ROOT))) +def test_capability_has_readme(package: Path) -> None: + readme = package / 'README.md' + assert readme.exists(), ( + f'{package.relative_to(_ROOT)} is an importable capability package but has no README.md. ' + 'Add one (start from an existing capability README) so its public surface is documented.' + ) + + +@pytest.mark.parametrize('package', _CAPABILITY_PACKAGES, ids=lambda p: str(p.relative_to(_ROOT))) +def test_capability_linked_from_top_readme(package: Path) -> None: + top_readme = (_ROOT / 'README.md').read_text(encoding='utf-8') + link_target = f'{package.relative_to(_ROOT).as_posix()}/' + assert link_target in top_readme, ( + f'{package.relative_to(_ROOT)} is not linked from the top-level README.md. ' + f'Add a row for it (linking `{link_target}`) to the "What\'s available today" or "Roadmap" tables ' + 'so the README stays in step with the code.' + )