Files
David SFandGitHub 3ba9e2f9a5 docs: capability pages for the unified docs site + README/doc parity gate (#329)
* docs: publish capability docs to the unified site + add README/doc parity gate

Every capability shipped only a README (kept for GitHub/PyPI). This adds a
parallel, cleaned-up page per capability under docs/ for the new unified docs
site (pydantic.dev/docs/harness), migrated from each README: snippets verified
runnable against source, autodoc API blocks, root-relative Pydantic AI links,
and an experimental-status admonition on the experimental set.

To keep README and doc in sync going forward, adds a docs-parity-reviewer agent
and a parity gate in the review checklist (run as the last step before merge),
plus the docs/ layout and the README<->doc requirement in AGENTS.md and the
capability-authoring guide.

* docs: fix README<->doc<->source inconsistencies across capabilities

A parity audit against source found drift, mostly in the capability READMEs
(staler than the migrated docs). All fixes verified against source:

- Correctness: the "approval/deferred tools are excluded from the sandbox" claim
  (code_mode README + doc) was false -- those tools are sandboxed like any
  other; corrected in both. The stale Shell persist_cwd sentinel description is
  replaced with the actual out-of-band temp-file capture. filesystem protected
  default `.git/` -> `.git/*` (the bare form never matched).
- Runnable snippets: added the missing imports/wiring so README snippets no
  longer raise NameError (subagents, context, planning, overflow, authoring,
  filesystem, code_mode).
- Parity: documented previously-undocumented params/behaviors (compaction
  strategy options, overflow strip_ansi/Passthrough, extra autodoc classes for
  context and subagents), fixed a stale version pin (>=1.95.1 -> >=2.1.0), and
  added the missing Managed Prompt row to the root README capability matrix.
- Style: normalized decorative Unicode to ASCII across all READMEs and dropped a
  hype phrase, matching AGENTS.md writing style and the docs.

* docs: add nav.json to drive the unified-docs harness sidebar

The unified docs mount the harness docs under /docs/ai/harness (fed live from
this repo via the pydantic-ai 'Pydantic AI Harness' section). This nav.json
defines the sub-nav (Overview + Capabilities + Experimental) and the set of doc
files the site includes.

* docs: migrate "What goes where?" explainer into harness overview

Adds the core-vs-harness boundary section (anchor #what-goes-where) to the
canonical harness overview, so the pydantic-ai docs that link to it can point
here after the duplicated in-repo stub is removed.

* docs: address CodeRabbit review -- runnable snippets, accuracy, multi-class autodoc

* docs: flatten harness nav and align with graduated capabilities

Following the experimental-graduation refactor (#347), restructure the
unified-docs harness pages:

- Flatten docs/ (drop capabilities/ and experimental/ subdirs); the sidebar
  is now Overview + one flat list per Douwe's request.
- Rename to match the graduated modules: overflow -> overflowing-tool-output,
  authoring -> runtime-authoring, docs -> pydantic-ai-docs.
- Drop the 'Experimental' admonitions from the graduated capabilities and
  repoint every import + ::: autodoc path off pydantic_ai_harness.experimental.
- Add docs for the newly-shipped capabilities: guardrails, dynamic-workflow,
  media, and acp (acp stays framed as experimental -- it may still be removed).
- Every capability doc now links to its source; index capability table lists
  the full set with flat links.

* docs: apply team-sync authoring rules + enforce them in CI

From the 2026-07-10 docs review on #329:

- Purpose-first leads: drop hook names (before_model_request,
  after_tool_execute) from the opening paragraphs of compaction and
  overflowing-tool-output (doc + README); mechanism moves lower.
- Mirror the soft 'API may change between releases' stability note from each
  graduated README into its doc page (ACP keeps its stronger experimental
  warning; guardrails' README has no note, so its page gets none).
- README H1s now use the capability's display name (Overflow capability ->
  Overflowing Tool Output, RuntimeAuthoring -> Runtime Authoring, SubAgents ->
  Subagents, etc.).
- Extend tests/test_docs_parity.py with per-page mechanical checks: source link
  present, heading matches the capability name, purpose-first lead (no hook in
  the opener), and no experimental framing on graduated pages (ACP excepted).
- Update the docs-parity-reviewer agent + review-checklist to the flat
  structure and the new semantic checks.

* docs: add the stability note to guardrails (parity with sibling capabilities)

guardrails was the one graduated capability whose README and doc page lacked
the shared 'API may change between releases' note. Add it to both.

* fix: restore uv.lock to match pyproject (bad text-merge dropped 8 lines)

Merging origin/main did a git text-merge of the generated uv.lock, leaving it
inconsistent with pyproject.toml -- every CI job failed at 'uv sync --locked'.
pyproject.toml is identical to main here, so the correct lock is main's.

* docs: address CodeRabbit review on #329

Findings that failed to post inline (GitHub error) but were real:
- context/README.md, planning/README.md: two nested examples still imported
  from pydantic_ai_harness.experimental.* -- repoint to the graduated modules.
- guardrails/README.md: replace em dashes with '--' (repo style) and add the
  source-module link.
- docs/media.md: standardize on the implementation's canonical media+sha256://
  URI scheme (was mixing media://).
- tests/test_docs_parity.py: strengthen my own checks per review --
  source-link and top-README-link now require a real Markdown link to the
  page's specific module (not a bare substring); heading checks assert an H1
  exists and equals the expected capability name via explicit page metadata.

* fix: restore uv.lock [options.exclude-newer-package] block

The lock lost its [options.exclude-newer-package] manifest (pydantic-ai-slim
= false, ...) -- a bad git text-merge dropped it, and diagnostic uv commands
rewrote it under a different local config. Without that block CI's
'uv sync --locked' re-resolves and fails ('addition of exclude newer exclusion
for pydantic-ai-slim'). Restore origin/main's exact lock.

* fix: restore uv.lock [options.exclude-newer-package] block

A pre-commit hook was rewriting uv.lock under the local uv config, stripping
the [options.exclude-newer-package] manifest (pydantic-ai-slim = false, ...).
Without it CI's 'uv sync --locked' re-resolves and fails. Commit origin/main's
exact lock with --no-verify so no hook mutates it (lock-only change).

* test: cover the docs-parity helper edge cases (100% coverage)

The strengthened helpers added defensive branches (missing frontmatter close,
fenced code before the lead, missing/forbidden/ClassName H1, lead running to
EOF) that no real doc exercises. Add direct unit tests so the file is back to
the repo's required 100% coverage.

* docs: link every capability README to its source module + enforce it

CodeRabbit re-flagged planning/README.md for a missing source link. Only
guardrails had one, so add the source-module link to all 15 remaining
capability READMEs (matching the doc pages) and add a parity test so the
requirement is mechanical and cannot silently regress.

* docs(agents): drop stale folder tree; fix flat docs path + guard names

AGENTS.md's File-structure ASCII tree and capability-authoring's doc paths
still showed docs/capabilities// docs/experimental/ (flattened in this PR) and
the old /docs/harness URL. Delete the tree rather than redraw it -- the layout
is discoverable by listing the repo; keep only the non-obvious conventions
(flat docs/, the README<->doc parity requirement). Also fix the Vocabulary
guard examples (InputGuard/OutputGuard, not the nonexistent InputGuardrail/
CostGuard).

* test: statically validate doc snippets exist and parse

Every Python snippet in the capability READMEs and docs/*.md pages is now
checked for the two failures a reader hits immediately: it does not parse
(syntax), or it imports a pydantic_ai_harness symbol that does not exist (stale
module path or renamed name -- the class of bug behind the experimental.* import
drift). Static only: no model/network execution, so it needs no mocking. The
four illustrative API-signature blocks opt out with a {test="skip"} fence
(read by pytest-examples, stripped-safe for the unified-docs render).

* test: don't fail doc-snippet check on a missing optional extra

The static check imported capability modules to resolve their symbols, but in
the slim CI job (no extras) importing e.g. pydantic_ai_harness.experimental.acp
raises ModuleNotFoundError for the absent third-party 'acp' package -- the
harness module exists, its extra just isn't installed. Distinguish a genuinely
missing harness module (fail) from a missing extra (skip) by the ImportError's
module name.
2026-07-13 11:00:12 -05:00
..

Subagents

Note

Import this capability from its submodule -- there is no top-level pydantic_ai_harness re-export:

from pydantic_ai_harness.subagents import SubAgent, SubAgents

The API may change between releases. Where practical, breaking changes ship with a deprecation warning.

Let an agent delegate self-contained tasks to named child agents.

Source

The problem

A single agent that does everything accumulates a large tool set and a long context. Splitting the work across specialized sub-agents keeps each context focused, but wiring up delegation by hand means writing a tool per agent, forwarding deps, threading usage limits, and telling the model what it can delegate to.

The solution

SubAgents takes a sequence of SubAgent entries and exposes a single delegate_task(agent_name, task) tool. Each delegation runs the chosen sub-agent in its own run -- with its own message history, so it never sees the parent conversation -- and returns its output to the parent. The available sub-agents are listed in the system prompt as a static instruction, so the listing stays in the cached prefix.

from pydantic_ai import Agent
from pydantic_ai_harness.subagents import SubAgent, SubAgents

researcher = Agent('anthropic:claude-sonnet-4-6', name='researcher', description='Researches a topic and reports findings')
writer = Agent('anthropic:claude-sonnet-4-6', name='writer', description='Turns notes into polished prose')

orchestrator = Agent(
    'anthropic:claude-opus-4-7',
    capabilities=[SubAgents(agents=[SubAgent(researcher), SubAgent(writer)])],
)

result = orchestrator.run_sync('Research the history of TLS and write a one-paragraph summary.')
print(result.output)

A delegate's name -- how the parent model refers to it, and how it is listed in the prompt -- is the agent's own name, or a SubAgent(name=...) override. Two delegates resolving to the same name is an error, and an agent with no name and no override is rejected.

The tool

Tool Purpose
delegate_task(agent_name, task) Run the named sub-agent on a self-contained task and return its output.
  • The sub-agent runs with its own message history, so task must be self-contained.
  • An unknown agent_name raises ModelRetry, so the model can correct itself.
  • The result returned to the parent is str(result.output).

Deps, usage, tools, and capabilities

  • Deps are forwarded. The parent run's deps are passed to each sub-agent, so sub-agents share the parent's AgentDepsT (enforced by the type signature -- every sub-agent is an AbstractAgent[AgentDepsT, Any]).
  • Usage is shared by default. The parent's usage is passed to each sub-agent run, so token usage aggregates and a parent usage_limits applies across the whole agent tree. Set forward_usage=False to give each sub-agent run its own accounting.
  • Tools can be inherited. With inherit_tools=True, the parent agent's own tools (registered directly or via toolsets) are added to each sub-agent run, on top of the sub-agent's own. Tools contributed by the parent's capabilities are not inherited: they are bound to capability instances registered in the parent run, and would arrive without the hooks and instructions they depend on. Use shared_capabilities to give sub-agents a capability. This also excludes the delegate tool itself, so a sub-agent can't recurse into further delegation. Off by default.
  • Capabilities can be shared. shared_capabilities are applied to every sub-agent run -- e.g. give all sub-agents a common guardrail, memory, or planning capability without rebuilding each Agent.
  • Sub-agent events can be streamed. Pass an event_stream_handler and it's forwarded to each sub-agent run, so the sub-agent's model-streaming and tool events surface to the caller (the handler receives the sub-agent's own RunContext).

Per-delegate run controls

Each SubAgent carries its own budgets, so one delegate's controls do not touch the others. A SubAgent with no controls set runs with the SubAgents defaults.

from pydantic_ai import Agent
from pydantic_ai.usage import UsageLimits
from pydantic_ai_harness.subagents import SubAgent, SubAgents

reproducer = Agent('anthropic:claude-sonnet-4-6', instructions='Reproduce the reported bug from a minimal script.')
librarian = Agent('anthropic:claude-sonnet-4-6', instructions='Find relevant docs, issues, and prior art.')

orchestrator = Agent(
    'anthropic:claude-opus-4-7',
    capabilities=[
        SubAgents(
            agents=[
                SubAgent(reproducer, usage_limits=UsageLimits(request_limit=35), timeout_seconds=600, max_calls=1),
                SubAgent(librarian, usage_limits=UsageLimits(request_limit=18), timeout_seconds=300, max_calls=2),
            ]
        )
    ],
)
Field Effect
usage_limits A request/token budget for one delegation. The child runs with its own usage accounting, so the budget counts only that child's requests and tokens (not the parent's or siblings'), even when forward_usage=True. The tradeoff: that child's tokens no longer aggregate into the parent's usage. Reaching the budget is a soft outcome (see below), not a run-stopping UsageLimitExceeded.
timeout_seconds A wall-clock budget for one delegation. When the child exceeds it, its run is cancelled and the parent gets a soft steering message instead of hanging on the child. The cancelled child's event_stream_handler (if any) stops receiving events without a terminal event.
max_calls The maximum number of delegations to this sub-agent per parent run. Once reached, further delegations return a soft budget-exhausted message without running the child. Counts are scoped to one Agent.run (a run_id) and cleared when it ends, so each parent run and each level of a nested tree budgets independently.
on_failure A steering message returned to the parent for any soft degradation of this delegate, in place of the built-in default. Setting it also makes child failures soft (see below).
contain_errors Whether an unexpected crash in this delegate is caught and returned to the parent as a bounded ModelRetry instead of aborting the parent run (see below). Unset inherits the SubAgents(contain_errors=...) default (off).

Failure handling

A soft outcome returns a steering message to the parent as a normal tool result, so its model reads the message and decides what to do next (rather than immediately re-delegating, which a ModelRetry invites). A timeout, a reached usage_limits budget, and an exhausted max_calls budget are always soft. When on_failure is set, the message it carries replaces the built-in default for these outcomes.

A sub-agent run that fails with a soft model error (ModelRetry, UnexpectedModelBehavior, e.g. it exhausted its own retries) is, by default, converted into a ModelRetry for the parent -- so the parent's model sees Sub-agent '<name>' failed: ... and can react by re-delegating. The delegate tool defaults to tool_retries=2, so the parent aborts only after that many consecutive delegate failures; the counter resets after any successful delegation. Raise tool_retries to tolerate a flakier sub-agent, or set None to inherit the parent agent's default tool retries. Set on_failure for a delegate to make its failures soft instead: the child error returns the on_failure message as a normal tool result.

Hard errors propagate to stop the whole run. A UsageLimitExceeded from a child that has no per-delegate usage_limits (so it shares the parent's accounting) means the whole tree is out of budget and propagates; a child reaching its own usage_limits is soft, as above.

An unexpected crash -- any other exception the child raises, such as a provider ModelAPIError/FallbackExceptionGroup or a plain ValueError from a bad tool argument -- propagates by default and aborts the parent run. Set contain_errors=True (per delegate, or as the SubAgents default) to catch it and return it to the parent as a bounded ModelRetry instead, so one delegate crash cannot kill the whole run. Containment stays loud: the exception rides the retry message (Sub-agent '<name>' crashed: ...), it is logged via the standard logging module, and tool_retries still bounds consecutive crashes into an abort. This is orthogonal to on_failure -- a contained crash always raises the loud retry, never the soft on_failure return, so a genuine bug is never masked as success. Cancellation, a shared UsageLimitExceeded, pydantic-ai control-flow signals (CallDeferred, ApprovalRequired, the Skip* signals), and UserError always propagate regardless of contain_errors.

Discovery

The sub-agents are listed in the system prompt via get_instructions, using each agent's description (or a SubAgent(description=...) override). A sub-agent with no description is listed by name alone.

Loading sub-agents from disk

A repo's markdown agent definitions become delegates without writing any Agent code. By default every *.md file under the conventional folders is loaded as a sub-agent, alongside the explicitly-passed agents.

from pydantic_ai import Agent
from pydantic_ai_harness.subagents import SubAgents

orchestrator = Agent(
    'anthropic:claude-opus-4-7',
    capabilities=[SubAgents(inherit_tools=True)],  # auto-loads ./.agents/agents/ and ~/.agents/agents/
)

agent_folders controls where definitions come from. It defaults to 'agents', the conventional layout:

  • A folder-name str (the default 'agents'): for the project root (cwd) then the home root, load from <root>/.agents/<name>/, falling back to <root>/.claude/<name>/ when <root>/.agents/ is absent.
  • A sequence of paths loads from exactly those folders, in order.
  • None disables disk loading, exposing only the explicitly-passed agents.

Definition format

A definition is a markdown file with optional frontmatter:

---
name: researcher
description: Researches a topic and reports findings
tools: Read, Grep
---
You research topics. Report your findings, each with a source.
  • name is the delegate name (how the parent refers to it and how it is listed). It falls back to the filename stem when absent.
  • description drives the prompt listing.
  • The markdown body becomes the agent's instructions.
  • tools (or allowed-tools) is a comma-separated string or a YAML block list. See "Tools" below.
  • model and color are ignored: the model is inherited from the parent (see below), and color has no pyai equivalent.

Frontmatter is read by a small, dependency-free parser limited to those keys (pyyaml is not a harness dependency). Full YAML frontmatter is not supported.

Models and effort

Disk agents inherit the parent run's model by default. Per agent, the caller can override the model and set a thinking/effort level via agent_overrides, keyed by the agent's name:

from pydantic_ai_harness.subagents import AgentOverride, SubAgents

SubAgents(
    agent_folders='agents',
    agent_overrides={'researcher': AgentOverride(model='anthropic:claude-sonnet-4-6', effort='high')},
)

Every agent the capability builds runs at a minimum thinking-effort floor. MINIMUM_EFFORT_FLOOR and the clamp_effort(level, floor=...) helper are exported so an orchestrator can apply the same floor to its own agents (that orchestrator-side application is the caller's responsibility). clamp_effort maps None/False to the floor, leaves True (provider-default effort) unchanged, and raises a concrete level below the floor up to it. Effort is applied through pyai's ModelSettings.thinking.

Tools

A disk agent gets no tools by default (inherit_tools is False); set inherit_tools=True to expose the parent's tools to it through the inherit_tools mechanism, in which case its tools frontmatter is ignored. To map the frontmatter tool names to specific toolsets instead, pass a tool_resolver: it receives each tool name (so it can honor entries like Bash(git:*)) and returns the toolsets that provide it, or None for an unknown name, which is skipped with a warning.

from pydantic_ai_harness.experimental.subagents import SubAgents

def resolve(tool_name: str):
    return TOOLSETS.get(tool_name)  # -> Sequence[AgentToolset[object]] | None

SubAgents(agent_folders='agents', tool_resolver=resolve)

Precedence

When the same name appears in more than one source, the higher-precedence one wins and the others are skipped with a warning: explicitly-passed agents first, then the project folder, then the home folder (and, for an explicit path sequence, earlier paths before later ones). A duplicate name within the explicitly-passed agents list is still an error.

Configuration

SubAgents(
    agents=(),             # Sequence[SubAgent[AgentDepsT]] -- each pairs an agent with its run controls
    agent_folders='agents',# folder-name str (convention) | Sequence[Path] | None (disable)
    agent_overrides={},    # Mapping[str, AgentOverride] -- per-disk-agent model/effort override
    tool_resolver=None,    # Callable[[str], Sequence[AgentToolset[object]] | None] -- disk-agent tool mapping
    forward_usage=True,    # share the parent's usage with sub-agent runs
    inherit_tools=False,   # expose the parent's own tools to sub-agents (capability tools excluded)
    shared_capabilities=(),# capabilities applied to every sub-agent run
    event_stream_handler=None,  # forwarded to each sub-agent run to stream its events
    tool_name='delegate_task',
    tool_retries=2,        # extra delegate-tool attempts after a sub-agent error before aborting (None inherits the agent default)
    contain_errors=False,  # default for SubAgent.contain_errors: contain an unexpected crash as a bounded retry
)
SubAgent(
    agent,                 # AbstractAgent[AgentDepsT, Any] -- the child agent to run
    name=None,             # delegate name; defaults to the agent's own `name`
    description=None,      # prompt-listing description; defaults to the agent's own `description`
    usage_limits=None,     # per-delegation request/token budget (isolated accounting)
    timeout_seconds=None,  # per-delegation wall-clock budget
    max_calls=None,        # max delegations to this sub-agent per parent run
    on_failure=None,       # steering message for soft degradations of this delegate
    contain_errors=None,   # contain an unexpected crash as a bounded retry; None inherits the SubAgents default
)

SubAgents is not serializable via the agent spec (it holds live Agent instances), so get_serialization_name() returns None.

Notes

  • Sub-agents can themselves have SubAgents, forming a tree. Share usage (the default) and set a usage_limits on the top-level run to bound the whole tree.
  • Delegations the model issues in parallel run as independent sub-agent runs.

Further reading