mirror of
https://github.com/pydantic/pydantic-ai-harness.git
synced 2026-07-20 10:25:35 +00:00
David SFandGitHub
3ba9e2f9a5
* docs: publish capability docs to the unified site + add README/doc parity gate Every capability shipped only a README (kept for GitHub/PyPI). This adds a parallel, cleaned-up page per capability under docs/ for the new unified docs site (pydantic.dev/docs/harness), migrated from each README: snippets verified runnable against source, autodoc API blocks, root-relative Pydantic AI links, and an experimental-status admonition on the experimental set. To keep README and doc in sync going forward, adds a docs-parity-reviewer agent and a parity gate in the review checklist (run as the last step before merge), plus the docs/ layout and the README<->doc requirement in AGENTS.md and the capability-authoring guide. * docs: fix README<->doc<->source inconsistencies across capabilities A parity audit against source found drift, mostly in the capability READMEs (staler than the migrated docs). All fixes verified against source: - Correctness: the "approval/deferred tools are excluded from the sandbox" claim (code_mode README + doc) was false -- those tools are sandboxed like any other; corrected in both. The stale Shell persist_cwd sentinel description is replaced with the actual out-of-band temp-file capture. filesystem protected default `.git/` -> `.git/*` (the bare form never matched). - Runnable snippets: added the missing imports/wiring so README snippets no longer raise NameError (subagents, context, planning, overflow, authoring, filesystem, code_mode). - Parity: documented previously-undocumented params/behaviors (compaction strategy options, overflow strip_ansi/Passthrough, extra autodoc classes for context and subagents), fixed a stale version pin (>=1.95.1 -> >=2.1.0), and added the missing Managed Prompt row to the root README capability matrix. - Style: normalized decorative Unicode to ASCII across all READMEs and dropped a hype phrase, matching AGENTS.md writing style and the docs. * docs: add nav.json to drive the unified-docs harness sidebar The unified docs mount the harness docs under /docs/ai/harness (fed live from this repo via the pydantic-ai 'Pydantic AI Harness' section). This nav.json defines the sub-nav (Overview + Capabilities + Experimental) and the set of doc files the site includes. * docs: migrate "What goes where?" explainer into harness overview Adds the core-vs-harness boundary section (anchor #what-goes-where) to the canonical harness overview, so the pydantic-ai docs that link to it can point here after the duplicated in-repo stub is removed. * docs: address CodeRabbit review -- runnable snippets, accuracy, multi-class autodoc * docs: flatten harness nav and align with graduated capabilities Following the experimental-graduation refactor (#347), restructure the unified-docs harness pages: - Flatten docs/ (drop capabilities/ and experimental/ subdirs); the sidebar is now Overview + one flat list per Douwe's request. - Rename to match the graduated modules: overflow -> overflowing-tool-output, authoring -> runtime-authoring, docs -> pydantic-ai-docs. - Drop the 'Experimental' admonitions from the graduated capabilities and repoint every import + ::: autodoc path off pydantic_ai_harness.experimental. - Add docs for the newly-shipped capabilities: guardrails, dynamic-workflow, media, and acp (acp stays framed as experimental -- it may still be removed). - Every capability doc now links to its source; index capability table lists the full set with flat links. * docs: apply team-sync authoring rules + enforce them in CI From the 2026-07-10 docs review on #329: - Purpose-first leads: drop hook names (before_model_request, after_tool_execute) from the opening paragraphs of compaction and overflowing-tool-output (doc + README); mechanism moves lower. - Mirror the soft 'API may change between releases' stability note from each graduated README into its doc page (ACP keeps its stronger experimental warning; guardrails' README has no note, so its page gets none). - README H1s now use the capability's display name (Overflow capability -> Overflowing Tool Output, RuntimeAuthoring -> Runtime Authoring, SubAgents -> Subagents, etc.). - Extend tests/test_docs_parity.py with per-page mechanical checks: source link present, heading matches the capability name, purpose-first lead (no hook in the opener), and no experimental framing on graduated pages (ACP excepted). - Update the docs-parity-reviewer agent + review-checklist to the flat structure and the new semantic checks. * docs: add the stability note to guardrails (parity with sibling capabilities) guardrails was the one graduated capability whose README and doc page lacked the shared 'API may change between releases' note. Add it to both. * fix: restore uv.lock to match pyproject (bad text-merge dropped 8 lines) Merging origin/main did a git text-merge of the generated uv.lock, leaving it inconsistent with pyproject.toml -- every CI job failed at 'uv sync --locked'. pyproject.toml is identical to main here, so the correct lock is main's. * docs: address CodeRabbit review on #329 Findings that failed to post inline (GitHub error) but were real: - context/README.md, planning/README.md: two nested examples still imported from pydantic_ai_harness.experimental.* -- repoint to the graduated modules. - guardrails/README.md: replace em dashes with '--' (repo style) and add the source-module link. - docs/media.md: standardize on the implementation's canonical media+sha256:// URI scheme (was mixing media://). - tests/test_docs_parity.py: strengthen my own checks per review -- source-link and top-README-link now require a real Markdown link to the page's specific module (not a bare substring); heading checks assert an H1 exists and equals the expected capability name via explicit page metadata. * fix: restore uv.lock [options.exclude-newer-package] block The lock lost its [options.exclude-newer-package] manifest (pydantic-ai-slim = false, ...) -- a bad git text-merge dropped it, and diagnostic uv commands rewrote it under a different local config. Without that block CI's 'uv sync --locked' re-resolves and fails ('addition of exclude newer exclusion for pydantic-ai-slim'). Restore origin/main's exact lock. * fix: restore uv.lock [options.exclude-newer-package] block A pre-commit hook was rewriting uv.lock under the local uv config, stripping the [options.exclude-newer-package] manifest (pydantic-ai-slim = false, ...). Without it CI's 'uv sync --locked' re-resolves and fails. Commit origin/main's exact lock with --no-verify so no hook mutates it (lock-only change). * test: cover the docs-parity helper edge cases (100% coverage) The strengthened helpers added defensive branches (missing frontmatter close, fenced code before the lead, missing/forbidden/ClassName H1, lead running to EOF) that no real doc exercises. Add direct unit tests so the file is back to the repo's required 100% coverage. * docs: link every capability README to its source module + enforce it CodeRabbit re-flagged planning/README.md for a missing source link. Only guardrails had one, so add the source-module link to all 15 remaining capability READMEs (matching the doc pages) and add a parity test so the requirement is mechanical and cannot silently regress. * docs(agents): drop stale folder tree; fix flat docs path + guard names AGENTS.md's File-structure ASCII tree and capability-authoring's doc paths still showed docs/capabilities// docs/experimental/ (flattened in this PR) and the old /docs/harness URL. Delete the tree rather than redraw it -- the layout is discoverable by listing the repo; keep only the non-obvious conventions (flat docs/, the README<->doc parity requirement). Also fix the Vocabulary guard examples (InputGuard/OutputGuard, not the nonexistent InputGuardrail/ CostGuard). * test: statically validate doc snippets exist and parse Every Python snippet in the capability READMEs and docs/*.md pages is now checked for the two failures a reader hits immediately: it does not parse (syntax), or it imports a pydantic_ai_harness symbol that does not exist (stale module path or renamed name -- the class of bug behind the experimental.* import drift). Static only: no model/network execution, so it needs no mocking. The four illustrative API-signature blocks opt out with a {test="skip"} fence (read by pytest-examples, stripped-safe for the unified-docs render). * test: don't fail doc-snippet check on a missing optional extra The static check imported capability modules to resolve their symbols, but in the slim CI job (no extras) importing e.g. pydantic_ai_harness.experimental.acp raises ModuleNotFoundError for the absent third-party 'acp' package -- the harness module exists, its extra just isn't installed. Distinguish a genuinely missing harness module (fail) from a missing extra (skip) by the ImportError's module name.
Media Externalization
Note
Import these helpers from their submodule -- there is no top-level
pydantic_ai_harnessre-export:from pydantic_ai_harness.media import ( DiskMediaStore, S3MediaStore, SqliteMediaStore, externalize_media, restore_media, )The API may change between releases. Where practical, breaking changes ship with a deprecation warning.
Content-addressed stores and walker helpers that move large binary payloads out of message history and put them back on demand.
These are building blocks, not a capability. There is no class you add to Agent(capabilities=[...]) yet. StepPersistence uses them to keep snapshots small when messages carry BinaryContent. A forthcoming MediaExternalizer capability will reuse the same stores to rewrite BinaryContent into URL parts before the model sees them.
Why
A conversation that carries images, audio, or other BinaryContent inlines those bytes into every message. Persist that history and each snapshot re-serializes the payloads. Content-addressed storage writes each payload once, keyed by its own hash, and leaves a short media:// URI in its place. The same bytes are stored once no matter how many messages or snapshots reference them.
Stores
Every store implements the MediaStore protocol: put, get, exists, public_url, and get_metadata, all async and content-addressed (the URI is derived from the payload hash, so identical bytes deduplicate).
| Store | Backed by | Use when |
|---|---|---|
DiskMediaStore(directory=...) |
A directory on disk | Local runs and tests |
SqliteMediaStore(...) |
A SQLite database | A single-file store that travels with the data |
S3MediaStore(...) |
S3 or an S3-compatible bucket | Shared or production storage |
A KeyStrategy controls the on-store layout, and a PublicUrlResolver (or make_static_public_url) turns a stored URI into a public URL when the store is served over HTTP.
Walker helpers
externalize_media and restore_media walk a message node and swap payloads for URIs and back.
store = DiskMediaStore(directory='./media')
# Replace BinaryContent larger than the threshold with media:// URIs.
lean = await externalize_media(message, media_store=store, threshold_bytes=32_000)
# Later, rehydrate the URIs back into BinaryContent.
full = await restore_media(lean, media_store=store)
externalize_media only externalizes payloads over threshold_bytes; smaller ones stay inline. media_uri_for and parse_media_uri give you the raw URI round-trip if you need to key media yourself.
API
| Symbol | Purpose |
|---|---|
MediaStore |
Async content-addressed store protocol (put / get / exists / public_url / get_metadata) |
DiskMediaStore, SqliteMediaStore, S3MediaStore |
Concrete stores |
MediaContext |
Per-call context (e.g. tenant) threaded through store operations |
KeyStrategy, default_key_strategy |
On-store key layout |
PublicUrlResolver, make_static_public_url |
Resolve a stored URI to a public URL |
externalize_media, restore_media |
Walk a message node to externalize / rehydrate payloads |
media_uri_for, parse_media_uri |
Compute and parse a media:// URI |