mirror of
https://github.com/pydantic/pydantic-ai-harness.git
synced 2026-07-21 02:45:34 +00:00
main
19
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Aditya VardhanandGitHub
|
4ad83f8861 |
Clarify CodeMode final-expression returns (#370)
* Prevent empty CodeMode results through clearer guidance * Make CodeMode return guidance unambiguous |
||
|
Aditya VardhanandGitHub
|
67f8d870a5 | Prevent search result format from becoming a return type (#377) | ||
|
Aditya VardhanandGitHub
|
73513a60bc |
Remove third-party MCP endpoint from pydantic-ai-harness skill (#368)
* Use local tools in harness skill quick start * Align CodeMode skill security guidance |
||
|
Aditya VardhanandGitHub
|
4ee2b3291f |
ci: drop the pydantic-ai v2 beta early-warning job (#359)
The v2-beta job was a non-blocking visibility signal, not a required check. Removing it since it is no longer needed. |
||
|
Aditya VardhanandGitHub
|
1229858a57 |
Announce harness releases on Twitter (#341)
* Announce harness releases on Twitter Mirror pydantic-ai's post-release tweet so a PyPI publish also posts to the shared @pydantic account, keeping release visibility consistent across the workspace. Version is read from the tag ref since the release job emits no version output. Requires the TWITTER_* secrets to be provisioned on this repo. * Scope send-tweet secrets to the release environment zizmor's secrets-outside-env audit fails the lint job because the tweet job reads the TWITTER_* secrets without a dedicated environment. Bind it to the release environment, matching the release job's convention for secret-using jobs (the harness has no zizmor ignore config like pydantic-ai does). |
||
|
|
545a703227 |
feat: Adds Dynamic Workflows (#273)
* feat(dynamic-workflow): orchestrate sub-agents from a model-written script Turn-by-turn sub-agent delegation makes every composition step (chain, vote, loop) a separate model turn and pushes each intermediate result back through the orchestrator's context. DynamicWorkflow is Code Mode with sub-agents as the callables: the model writes one Monty-sandboxed script that composes the sub-agents with ordinary control flow, and only the final result returns to the model. Extracts the shared Monty execution loop into _monty_exec, reused by code_mode. The pydantic-ai bump the capability needs (defer_loading, ctx.enqueue) lands separately, per the repo policy on dependency changes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(deps): bump pydantic-ai 1.95.1 -> 1.105.0 DynamicWorkflow needs defer_loading (>=1.97) and ctx.enqueue (1.105) for on-demand loading and cache-stable runtime sub-agent reveal. The lock exempts the pydantic-ai family from its exclude-newer cutoff to pull the newer release. Collateral: xfail a code_mode ToolSearch test that relied on the old discovered-tool tracking, and regenerate the managed_prompt snapshot for the new tool-definition fields. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(examples): tidy example docs and keep them fully linted Drop the blanket 'examples/**' pydocstyle ignore — it masked a single missing docstring. Add the one-line docstring to fork_and_resume's main() instead, so examples stay under the same docstring rules as library code. Also drop the redundant "Runs / needs a key" column from the examples README. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(dynamic-workflow): render catalog via core FunctionSignature, reject keyword names Addresses PR #273 review. Render the sub-agent catalog through pydantic_ai's FunctionSignature (the same renderer code_mode uses) instead of a hand-rolled f-string. This is the real dedup with code_mode and fixes two latent bugs: the signature now forces keyword-only `task` (`async def name(*, task: str)`) to match dispatch, which reads kwargs['task']; and a description containing a newline or quote now renders as a safe docstring block instead of breaking the f-string. Reject sub-agent names that are Python keywords. `'class'.isidentifier()` is True, so a keyword name passed validation but the model could never call it (`await class(...)` is a syntax error) — a silently-uncallable agent. Validate non-keyword identifiers at construction, via a shared in-file helper. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(dynamic-workflow): harden public API ahead of release Adversarial review of the dynamic-workflow surface surfaced contracts that would be painful to change once released; fix them while cheap: - Correct the "usage_limits bounds the tree" claim — the parent's limit is never forwarded to sub-agents. Add `sub_agent_usage_limits` so a per-sub-agent limit plus `max_agent_calls` give a real tree-wide token ceiling, and document what `forward_usage` actually does. - Own `WorkflowResourceLimits` instead of leaking pydantic_monty's `ResourceLimits` onto the public field; add an explicit `'unlimited'` sentinel and merge a partial dict onto the backstop (no silent drop). - Narrow `agents` to `list` so the append-to-reveal contract is enforced by the type rather than explained in prose. - Drop the `description` -> `agent.name` fallback that rendered a useless self-referential docstring to the model. - Teach the model-facing tool description what the README already knew: keyword-only `task`, no `return_exceptions`/abort-on-raise, dict (not attribute) access for structured output, and the print envelope. Also refresh the runnable examples (audit + migrate) and keep the README and tests in sync. lint/typecheck clean, 100% branch coverage. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * removing time limit from monty * docs(examples): consolidate into one verified DynamicWorkflow showcase Replace the overlapping audit and migrate examples with a single example that exercises the full capability range — parallel fan-out, read/write confinement, an adversarial review, a feedback loop, typed fan-in, and a Logfire trace — so a reader sees what the capability is for without wading through two near-identical fan-out demos. Fixes issues an adversarial review surfaced: the retry loop now threads reviewer issues back to the migrator (so it can actually converge), the synthesizer's typed report survives to the caller via the orchestrator's output_type, and several claims (budget ceiling, confinement, run command portability) are corrected to match the code. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(deps): raise pydantic-ai-slim floor to 1.101.0 for RunContext.enqueue The dynamic_workflow reveal path calls ctx.enqueue to announce a sub-agent added mid-run, but RunContext.enqueue did not exist until pydantic-ai 1.101.0 (verified: absent in 1.100.0, present in 1.101.0). The declared floor was still >=1.95.1, so the test-floor job (--resolution lowest-direct) and any install at the floor would resolve a RunContext without enqueue and raise AttributeError when revealing an agent instead of announcing it. Bump the runtime dep, the logfire extra, and the dev group to >=1.101.0, and update the README requirement line to match. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(dynamic-workflow): move capability under experimental DynamicWorkflow ships as experimental so its API can change without a deprecation period, like the compaction menu. Importing it now emits a HarnessExperimentalWarning, silenceable category-wide. _monty_exec stays at the package root: it is shared with code_mode, which is stable, so it must not move into experimental. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(dynamic-workflow): correct stdlib import guidance for the sandbox The tool description and README presented `asyncio, math, json, re, typing` as the complete set of importable modules. In Monty, `datetime`/`os`/`sys` also import, but they are inert in DynamicWorkflow (no os/path hooks are registered, so their calls resolve to NameError). Reword to a curated, non-exhaustive list so the model-facing text is accurate without steering toward the stubbed dead ends. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(dynamic-workflow): harden sandbox boundary and reveal/config validation Adversarial review of the run_workflow path surfaced several silent-failure and crash modes, all reachable from model-authored scripts or host config: - A model script awaiting the same sub-agent call twice in one asyncio.gather made the Monty VM raise pyo3_runtime.PanicException (a BaseException), which escaped both Monty error handlers and tore down the whole agent run. Catch it by name and convert to a retry; non-panic BaseExceptions still propagate. - Sub-agent calls silently dropped extra keyword args and accepted a non-string task (a dict/list was smeared into message parts), running on wrong input with no signal. Reject both at dispatch, before the budget is touched. - A runtime reveal whose name was invalid or collided with an existing agent was silently dropped, keeping the old agent while the host believed it had swapped one in. Distinguish idempotent re-reveal (object identity) from a real conflict, warn once on the latter, never swap silently. - max_agent_calls < 1 produced a dead-on-arrival tool with a misleading "conclude with results gathered" message; a typo'd resource_limits key was silently ignored, disabling the only guard against a pure-CPU loop. Both now raise UserError at construction. Also pins the headline guarantee with a concurrent fan-out test asserting max_agent_calls admits exactly the budget under asyncio.gather (previously only covered sequentially). _toolset.py stays at 100% line+branch coverage. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: await cancelled dispatch tasks and survive bad mid-run reveals Cancelling a Monty execution previously fired task.cancel() and returned immediately, leaving in-flight tool calls and sub-agent runs unwinding in the background — they could mutate the shared usage accumulator after the tool call had already reported its result, and teardown errors were never retrieved. The executor now awaits the cancelled tasks so nothing outlives the call that spawned it. This changes code_mode's cancellation timing too (flagged in the PR description): teardown now waits for in-flight calls to unwind instead of abandoning them. Also: create the sequential dispatch coroutine after the pending-task barrier so cancellation at the barrier cannot leak a never-awaited coroutine, and make `for_run` index the agent list leniently — an invalid WorkflowAgent appended mid-run (which `_reveal_pending` deliberately tolerates) no longer hard-fails every subsequent run via `replace`'s re-validation. A new test pins that host-raised exceptions cannot be caught inside the sandbox: the budget-exhausted terminal result relies on it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(dynamic-workflow): document experimental status and actual return shapes The README never said the import warns or how to silence it, omitted the print()/None result shaping the tool actually performs, and the API block hid `description` and the WorkflowResourceLimits type. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(dynamic-workflow): cover deps forwarding, message isolation, and shared-counter limits An adversarial tests-vs-spec pass found three documented behaviors with no assertion behind them: parent deps reaching sub-agents, sub-agents never seeing the parent conversation, and sub_agent_usage_limits checked against the shared counter under forward_usage=True. Dropping any of them would have passed the suite. Also pins the documented backstop values and the served tool's max_retries, asserts the reveal announcement names the actual tool, and fixes two comments that described a duration backstop / a dataclasses.replace that don't exist. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(dynamic-workflow): record the core usage_limits TOCTOU race behind max_agent_calls The exact host-side counter exists because core's limit enforcement races under concurrent fan-out and RunContext can't forward the parent's limits; pin that context (and the upstream TODO) where the discipline lives so it isn't lost once the PR ships. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(examples): replace the generated example with a feature-led README Review feedback on #273: the example will be rewritten by hand, and the README should lead with the Anthropic dynamic-workflows post and the feature itself rather than narrating one example run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(dynamic-workflow): add a dedicated install extra Installing DynamicWorkflow's sandbox dependency via the code-mode extra conflated two capabilities; each now names its own extra (both resolve to pydantic-monty). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(code_mode): convert sandbox VM panics to retries A model-written script that awaits the same call twice in one asyncio.gather panics the Monty VM; pyo3 surfaces that as a BaseException that nothing caught, tearing down the whole agent run. DynamicWorkflow already guarded this; the guard now lives in _monty_exec and code_mode applies it too, dropping its REPL since the aborted VM's accumulated state can't be trusted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(dynamic-workflow): remove duplication that could drift Single-source the long-form field docs on the capability (the toolset docstrings now point there), derive the resource-limit key set from the TypedDict instead of a parallel frozenset, and freeze the rendered tool description itself rather than a catalog dict re-rendered every step -- making the cache-stability claim structural. Drops the for_run_step override that duplicated the base default, plus redundant tests and a stateless test fixture. No behavior or public API change; coverage stays at 100%. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: narrow the shared executor seam and drop dead type unions Name sanitization is a code_mode concern, so the mapping moves into its dispatch closure and MontyExecutor stops knowing about it. The empty pending_call_ids guard fell through to an identical resume call, so it goes. Concrete return types on get_toolset/for_run let tests drop four isinstance asserts that existed only to narrow types pyright can now see directly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: fix a comment stating a false invariant; unshadow a closure param _by_name is mutated during a run (reveals fold into it each step); the stable-registry invariant only holds while a script executes, which is what the executor relies on. The dispatch closure's name param shadowed call_tool's own, a future-edit hazard. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(dynamic-workflow): restructure README for faster comprehension Lead with a minimal runnable snippet and state the context-saving idea once instead of repeating it across Why/What/blockquote. Move Installation above the full example. Turn the sub_agent_usage_limits paragraph into a forward_usage True-vs-False table so the N*T hard ceiling and the best-effort concurrency caveat are scannable. No technical claims changed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(dynamic-workflow): address CodeRabbit review - Fix broken relative link to Code Mode README: the capability lives two levels deep under experimental/, so `../code_mode/` resolved to a non-existent experimental/code_mode/. Use `../../code_mode/`. - Add missing `-> None` return types to all four test_warnings methods (the bot flagged one; the whole file was missing them). - Narrow `_workflow_result`'s unconstrained passthrough from `Any` to `object`. Left the other flagged `Any`s in place: the `call` override and `dispatch` must match `AbstractToolset.call_tool` / `_monty_exec.DispatchFn`, and `AbstractAgent[..., Any]` carries a heterogeneous sub-agent output type that `object` would reject on assignment. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(dynamic-workflow): harden error surfaces, render output schemas, add reveal() API Findings from an end-to-end adversarial review (PR #273 branch), fixes authored with codex exec: - Serialize sub-agent output inside the dispatch error wrapper: a non-serializable output leaked host class paths into the retry text and failed identically on every retry. - Render each sub-agent's output schema in the catalog and reveal announcements (TypedDict-style, with cross-catalog type-name conflict handling). The model previously saw `-> Any` and had to be told the fields in prose. - Add DynamicWorkflow.reveal() as the supported runtime reveal API; list append remains the underlying mechanism. An aliased-list-only contract cannot survive the planned durable resume. - Return terminal {'error': ...} results for permanent conditions (nesting refusal, budget exhaustion) instead of ModelRetry, which burned retries into UnexpectedModelBehavior. The budget result now carries last_error and completed. - Salvage completed sub-agent results into retry messages and the budget-terminal dict so a retry can reuse them instead of re-spending the budget; the sandbox-panic path does the same. - Pass through model-safe exception messages (UsageLimitExceeded) so the model can react; keep the opaque form for everything else. - State the call budget and sub-agent statelessness in the tool description; warn instead of failing when a reveal has no pending message queue to announce into. - Docs: correct the N*T "hard ceiling" claim (core checks token limits after a response arrives), fix the examples README script (it subscripted str outputs), pin model-facing text with snapshots. The old "budget error cannot mask another failure" comment was disproven under asyncio.gather; the true invariant is documented. Refs: https://www.anthropic.com/engineering/code-execution-with-mcp, https://code.claude.com/docs/en/workflows Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(dynamic-workflow): align tests with pydantic-ai v2 deps typing Merging main raised the pydantic-ai-slim floor to >=2.1, whose Agent `deps_type` default became `object` (was `NoneType`). Constructing an `Agent(..., capabilities=[Cap[None](...)])` then fails overload matching: the capabilities pin `AgentDepsT=None` while the default supplies `object`. Switch the deps typevar to `object` throughout, matching the convention the code_mode and subagents tests already use. Also make test_max_agent_calls_exact_under_concurrent_fan_out order- independent. The completed list was asserted in exact order, but which three of eight concurrent calls win the budget and the order they finish are both scheduler-dependent (3.14 admits them in a different order than 3.13). Assert the invariant that matters: exactly the budget ran, each once, well-formed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: stop tracking .agents/scheduled_tasks.lock This is a machine-local runtime lock for Claude Code's scheduled-task runner (pid/session mutex), not project content. It was committed by accident. Remove it from the tree and gitignore it alongside the other .agents machine-local files. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(dynamic-workflow): accept raw agents, make reveal() the strict mutation API Pydantic AI 2.x agents carry their own name and description, so wrapping every entry in WorkflowAgent was boilerplate: agents=[reviewer, summarizer] now works, following the OpenAI-handoffs union pattern where a raw agent means "use the agent's own metadata" and the wrapper is a per-use-site override. WorkflowAgent gains the agent.description fallback and positional agent that the sibling SubAgent already had; the divergence had no rationale. Making reveal() the only post-construction channel (eager validation, UserError at the call site, internal catalog list) removes the reason the toolset needed lenient re-validation: the strict/lenient _rebuild duality, _reveal_warned, and the warn-and-skip paths are deleted, and an entry that reaches the toolset invalid now fails fast as a contract violation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(dynamic-workflow): rewrite README as a progressive tutorial The reference-style README front-loaded precise semantics before a reader had a mental model to hang them on. Reshape it into a learning arc -- smallest working example first, one concept per section, dense tables tucked into admonition boxes -- so the same facts land in the order a reader needs them. All technical claims are carried over unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(dynamic-workflow): use raw-agent shorthand in the examples README The migration example wrapped each sub-agent in WorkflowAgent solely to supply a catalog description, which read as pre-refactor mandatory wrapping and contradicted the prose above it ("documented by its description"). An agent's description is a property of the agent, not of one workflow, and the field has other consumers (agent spec, OTel gen_ai.agent.description), so its natural home is the Agent. Move the descriptions onto the agents and pass them raw; WorkflowAgent stays documented in the capability README's override section for the cases that need it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(dynamic-workflow): drop internal decisions log from the PR DYNAMIC_WORKFLOW_DECISIONS.md was an internal working-notes log (branch codenames, v1/maybe-v2 planning, open problems). It doesn't belong in the shipped repo and duplicates the capability README. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: fix pydantic-ai-slim version pin and tighten DynamicWorkflow docstrings Correct the required version in README.md to `pydantic-ai-slim>=2.1.0` (matches pyproject.toml; the previous `>=1.101.0` was wrong), and trim the verbose DynamicWorkflow capability/field docstrings to say the same thing more concisely. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dynamic-workflow): promote out of experimental DynamicWorkflow graduates to a released top-level capability after a hardening round driven by live model runs and adversarial review: - Live-exercised with a real Opus orchestrator across ten scenarios (fan-out, structured output, budget exhaustion, error salvage, runtime reveal, defer_loading, nesting refusal, open-ended research). The one model-facing trip point found -- reading the sub-agent call budget as per-script rather than per-run -- is fixed in the tool description. - Reveals on a deferred capability no longer leak: get_tools holds reveal folding and announcements until the capability is loaded. Ownership is resolved by identity through the run's capability registry, unwrapping wrapper chains, because ids cannot identify the owner (an id-less capability registers under a generated key, and a wrapper registers in place of what it wraps). - Public annotations erase sub-agent outputs as `object`, not `Any`. - README drops the experimental banner, documents the per-run budget and keyword-only construction, and loses a stray editing artifact that had been committed at the end of the file. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(dynamic-workflow): let sub-agents inherit the run model * test(dynamic-workflow): exclude never-run parent model fn from coverage The parent model callback in test_inherit_model_off_keeps_sub_agent_bound_model is asserted never to run, so its body is unreachable and broke fail-under=100. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dynamic-workflow): merge into CodeMode when both are on one agent With both capabilities registered, CodeMode used to fold run_workflow into the run_code sandbox as a plain function taking a code string, so the model had to write a script containing a second script as a string literal, run in a nested sandbox, with no return schema. The two now compose instead: when a CodeMode capability is present, DynamicWorkflow exposes each sub-agent as its own (task) tool carrying the agent's output schema, and CodeMode renders them as typed async functions inside run_code, callable alongside the agent's regular tools in one script. Budget, usage forwarding, and the nesting guard moved into a shared call path so both modes enforce the same rules; the sub-agent guidance run_workflow's description carried moves into instructions in merged mode. No new public API: the merge is automatic. Standalone run_workflow scripts are now statically type-checked against the sub-agent signatures before execution (every call is a fresh sandbox, so the check is always sound); a misspelled function or positional task costs a retry but no sub-agent budget. The runtime kwarg guards remain, because Any-typed values (e.g. json.loads results) evade the checker, and the tests now exercise them through exactly that route instead of reaching into private helpers. Also: budget-sharing across multiple run_workflow calls in one run gets a test (the tool description's headline claim was untested), the examples/ README folds into the capability README, and the unrelated .gitignore line is dropped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: explain DynamicWorkflow in the main README The main README showed CodeMode end to end but only gave DynamicWorkflow a matrix row, so a reader had no way to see the point of the capability without opening its package README. Add a compact section after the Quick start: the turn-by-turn delegation problem, the two-line setup, the script the model writes, and how it composes with CodeMode, budgets, and deferred loading. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(dynamic-workflow): move under experimental New capabilities start under pydantic_ai_harness.experimental per repo convention, and the planned extensions (structured sub-agent inputs, durable workflows) touch the sub-agent call contract, so the API needs room to move before names go stable. This reverses the earlier top-level promotion on top of everything built since: importing the capability now emits HarnessExperimentalWarning and there is no top-level export. The README additionally explains the experimental status, positions DynamicWorkflow against SubAgents (one-delegation-per-turn vs scripted choreography), and documents the merged-mode tool-name collision. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(dynamic-workflow): separate from CodeMode instead of merging The CodeMode merge coupled two independent capabilities, and when their composition did not line up -- a restrictive CodeMode `tool_selector`, or running out of the sub-agent budget mid-script -- it degraded in ways the merge could not recover cleanly. Drop the merge: DynamicWorkflow always exposes its `run_workflow` tool and no longer inspects the run's capabilities. `run_workflow` is itself a code-execution sandbox, so CodeMode must not fold it into `run_code` -- that would make the model pass a script as a string argument to a function inside another script. CodeMode now keeps any tool carrying `code_arg_name` metadata native; `run_workflow` already declares it, so with both capabilities the model sees `run_code` and `run_workflow` as two independent tools. The rule only affects code-execution tools, so CodeMode is unchanged for every other tool. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(dynamic-workflow): correct max_duration_secs semantics and refresh README The docs claimed `max_duration_secs` counts wall-clock, including time awaiting sub-agents fanned out with `asyncio.gather`, and justified having no default cap on that basis. Empirically that is false across every monty the capability supports: the timer is a per-bytecode-step check, so it measures in-sandbox execution time and excludes time the script spends suspended on the host awaiting sub-agents (sequential or gathered). Correct the claim in `_toolset.py`, `_capability.py`, and the README, and describe what the cap is actually for: a pure-CPU `while True` runaway, the one thing the sub-agent budgets cannot catch. Add a regression test that pins the corrected behavior so the stale claim cannot silently return: slow sub-agents under a small cap still complete. Rewrite the README prose in a plainer voice with no em-dashes, and cite Jarred Sumner's Bun Zig-to-Rust rewrite as a real-world example of the same pattern (at Claude Code's session scale, not this specific API). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
77c08faeb7 |
docs: reconcile README with shipped capabilities; enforce parity in CI (#342)
* docs: reconcile README with shipped capabilities; enforce parity in CI The README undersold the package: the "capability matrix" still listed capabilities as unmerged PRs after they shipped as experimental/*, the version floor was three minors stale, and the ecosystem example used an API (SubAgentConfig) that no longer exists. Readers could not tell what actually ships. Restructure around what ships today (4 stable + 10 experimental, each true to its exported classes) vs a roadmap of only unshipped work, and add a mechanical docs-parity test so a capability can no longer land without showing up in the README. Also add the missing media README and refresh the agent skill's capability list and version floor. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: mark unreachable non-package guard as no-cover The __init__.py guard in _capability_packages() never triggers in a clean tree (__pycache__ is already filtered by the underscore-prefix check), so the branch was uncovered and dropped total coverage below the 100% floor. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: keep README's capability matrix, add shipped rows Revert the README to main's structure and make the update additive: flip the newly-shipped capabilities to shipped in the existing capability matrix and add rows for the ones that were not listed (ManagedPrompt, PyaiDocs, RuntimeAuthoring, Media), rather than restructuring the page. Bump the pydantic-ai-slim floor to 2.1.0. Fix the SKILL.md anchors that pointed at the removed sections. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: add Dynamic Workflows (#273) roadmap row to matrix Reconcile against the 7/7 harness state sweep: #273 is complete but open (not merged, no package on main), so it belongs in the matrix as a tracked roadmap row. It was the one item from the sweep still missing after the shipped-row updates. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
b4365440b5 |
feat: serve a Pydantic AI agent to editors over ACP (experimental) (#274)
* feat: add ACP capability to serve agents to editors Editors and TUIs that speak the Agent Client Protocol (Zed and others) can drive an external coding agent, but plugging a Pydantic AI agent into one previously meant implementing the ACP server side by hand. run_acp_stdio serves any Agent over stdio: streamed text and thinking, file edits rendered as diffs, human-in-the-loop tool approval mapped to deferred-approval tools, per-workspace sessions via a session_config hook, model switching, cancellation, and optional session persistence. FileSystem.get_toolset/Shell.get_toolset now return their concrete toolset types so the ACP presenter can recognize their tool calls by name and annotate them with kinds, locations, and diffs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: skip ACP tests at collection when the acp extra is not installed The slim CI jobs sync without extras, so `agent-client-protocol` is absent and the ACP test modules failed at import during collection. Ignore them via conftest when `acp` can't be found; `test_packaging.py` stays collected since package metadata holds on base installs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(acp): enable unstable routing and bound streamed updates by byte size session/set_model and session/close are advertised at initialize, but the ACP SDK router rejects them as unstable unless run_agent is given use_unstable_protocol=True -- so the model picker and session-close affordance returned method_not_found over a real connection. Enable the flag. Streamed text was chunked by character count, but the SDK serializes with ensure_ascii=True, so a non-ASCII code point expands up to 12 bytes (a surrogate pair) inside the JSON string. A single agent_message_chunk of emoji/CJK could exceed the client's 64 KiB read buffer and drop the connection. Chunk by escaped byte length instead. Both gaps were invisible to the suite: the close/set_model tests called the adapter directly (bypassing the router) and the large-output test used only ASCII. Add a through-the-router stdio test and a non-ASCII chunking test as guards. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(acp): per-turn usage, read-resilient terminal cleanup, persistence + native-toolset tests Address review follow-ups on the ACP capability: - Report per-turn token usage on `PromptResponse.usage`, summed across approval passes (UNSTABLE ACP field; clients that don't support it ignore it). - Suppress client errors during the shielded terminal kill/release so a failing cleanup call can't mask the in-flight CancelledError (the spec requires the turn to end with a `cancelled` stop reason). - Cover the persistence guarantees that previously had no store-active test: a cancelled turn commits nothing, an approval-resume turn persists each update once (no duplicate tool-call start), and `StoredSession` round-trips through Pydantic across the full `SessionUpdate` union. - Add a through-the-wire stdio test that the editor-native fs/terminal toolsets route to the client when mounted per session. - Document the dynamic-`ApprovalRequired` partial-side-effect nuance, per-turn usage, and that session persistence composes with per-run step durability. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(acp): editor-native reads for read-only fs clients, writes delegated locally A client that advertises filesystem reads but not writes previously got no editor-native filesystem at all (acp_filesystem returned None, so callers fell back to a fully-local toolset and lost the editor's live view, e.g. unsaved buffers). It now returns a combined toolset: reads route through the editor, and writes are delegated to the local FileSystem capability rooted at the session cwd (reusing its path sandboxing rather than reimplementing writes). This is coherent only when the agent shares the workspace disk with the editor (same machine, or an agent inside the editor's container); for a remote editor the writes land on the agent's disk. Documented as such at the helper and README. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(acp): point _all_known_model_names at the now-merged known_model_names() pydantic-ai#5803 added `pydantic_ai.models.known_model_names()`, the public replacement for the `KnownModelName.__value__` introspection. It isn't in a released `pydantic-ai-slim` yet, so the swap (and the floor bump it requires) waits for that release; update the breadcrumb so it's actioned then. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(acp): replay user turns on session/load, advertise MCP capabilities Two spec-conformance gaps from PR review: - The transcript only ever recorded agent-direction updates, so a reopened session replayed a one-sided conversation. ACP requires session/load to replay the entire conversation, user turns included. The prompt's content blocks are now seeded into the turn's update list -- recorded for replay but never sent live (the client renders its own prompt; the prompt-turn spec sends no user echo) -- riding the existing commit path so a cancelled turn still rolls its user message back. - initialize left mcp_capabilities at the default (http/sse false), and the spec forbids clients from sending HTTP/SSE MCP servers that are not advertised -- making the documented session_config -> mcp_servers path unreachable for conforming clients over those transports (stdio is not gated). Advertising is now an explicit opt-in mirroring prompt_capabilities, since only the embedder knows which transports their session_config connects; setting it without a session_config fails at construction rather than inviting servers that session/new would reject. Also adds the guard test from review follow-up: _all_known_model_names fails loudly if pyai ever recomposes the KnownModelName alias. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(acp): make the collection guard coverable in a single environment The acp-extra collection guard was an `if` statement whose true arm only runs on slim installs, so local `make testcov` (always all-extras) could never reach 100% -- only CI's combined slim+all-extras matrix could. A conditional expression has no statement arc for branch coverage to miss, so the documented local gate works again without excluding a line CI genuinely covers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(acp): record why _all_known_model_names avoids the unreleased public API The breadcrumb read as if known_model_names() were already usable; make the deliberate avoidance explicit. The public enumerator is merged upstream but not in any released pydantic-ai-slim, and the test-floor CI job runs against the floor release, so calling it would break there. Move the rationale from the docstring to a code comment (docstring stays contract-only) and spell out the swap-and-delete trigger: a release that ships it plus a floor bump. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(acp): add spec-conformance suite driven over a real in-memory wire The existing tests invoke adapter methods directly, below the SDK's JSON-RPC router and serialization -- a boundary that cannot see two bug classes that already bit this adapter: a method the router gates before the adapter runs (the use_unstable_protocol reachability bug) and a frame whose serialized bytes overrun the client buffer (the ensure_ascii chunking bug). `tests/acp/_wire.py` closes that gap without a subprocess: a real ClientSideConnection talks to acp.run_agent across a socket.socketpair in one event loop, so every request crosses the router and codec and every update arrives as bytes the client re-parses, with asyncio's default 64 KiB reader limit standing in for the stdio buffer. `tests/acp/test_conformance.py` is organized by spec clause, not by adapter method, each with an oracle built from the spec/input rather than the adapter's own output: - version negotiation echoes the requested version (not a literal) - capabilities are advertised iff supported (load_session, mcp, auth_methods, session list/fork/resume, modes, config options) -- read back off the wire - unstable methods route through with the flag and are method_not_found without it (the load-bearing reason run_acp_stdio enables it) - error codes distinguish method_not_found from invalid_params - session/load replays the entire conversation including user turns - large non-ASCII output reassembles intact within the client's read buffer CONFORMANCE.md records the clause-to-test matrix, the open gaps (none a known bug), the two verified deviations, and the N/A ledger, derived from a spec-page sweep with adversarial verification of every candidate finding. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(acp): stop surfacing unconsumed additionalDirectories The adapter accepted the client's additionalDirectories and forwarded them to the session_config AcpSession, but never advertised sessionCapabilities.additionalDirectories -- so a conformant client never sent them, and nothing downstream consumed them (the FileSystem capability is single-root). The ACP capability is also still UNSTABLE. Drop the dead surface: remove the field from AcpSession and stop forwarding it. The acp.Agent base signature still carries the parameter, so the session methods accept and ignore it. Supporting it properly (multi-root filesystem + advertisement) is recorded as a deferred feature. Also trims CONFORMANCE.md to a public supported-capability summary. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(acp): keep inline image bytes and let run_acp_stdio advertise MCP An ACP image block always carries inline `data`; `uri` is only an optional source reference, so preferring the URL dropped the bytes the client actually sent in favour of a link the model may be unable to fetch. Prefer the inline data, matching the reference ACP agents. `run_acp_stdio`/`run_acp_stdio_sync` forwarded every adapter option except `mcp_capabilities`, so advertising MCP transports was only reachable by constructing `PydanticAIACPAgent` by hand. Forward it for parity. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(acp): reliable approval scope, mapped stop reasons, safe session reload "Always allow"/"always reject" keyed the scope on the raw tool arguments, which a streaming model (and OpenAI by default) delivers as a JSON string; re-ordered keys then produced a different key and silently re-prompted for a call already decided. Canonicalize via `args_as_dict()` so the same logical call shares one remembered decision. Completed turns always reported `end_turn`, hiding `max_tokens`/`refusal` from the client. Map the model's finish reason to the ACP stop reason. `session/load` overwrote a still-open session without cancelling its in-flight turn, leaking a task that could later persist stale state over the restored transcript. Tear it down first, sharing the close_session teardown path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(acp): correct the API reference and capability notes The README API block omitted `session_store`/`models` (and now `mcp_capabilities`), the limitations list contradicted the documented MCP-via-`session_config` support, and the overwrite-diff note misdescribed what the code emits. Also drop a dangling internal-doc reference from a source comment. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(acp): record the unhandled SessionStore failure contract A durable SessionStore can fail on save or return a corrupt payload on load, but the adapter currently propagates those exceptions rather than handling them. Document the gap on the Protocol where store implementers will see it, so the behaviour is a known boundary rather than a surprise -- without adding handling we have no consumer for yet. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(acp): correct the SessionStore failure note to match observed behaviour The previous note assumed store exceptions surface "raw". Verified against the SDK (connection.py `_run_request`) and over a real in-memory wire with a raising stub store: they are converted to JSON-RPC errors before reaching the client -- a `pydantic.ValidationError` to `invalid_params` (-32602), anything else to a generic internal error (-32603). Record what the client actually receives rather than what was guessed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(acp): handle SessionStore failures instead of leaking them `session_store` is a public, documented extension point, so a durable store that fails on save or returns a corrupt payload on load is reachable today -- not hypothetical. A failed save previously errored the very turn the user had already watched stream to completion; a corrupt load leaked raw pydantic/IO detail to the client. Make save failures non-fatal: log and swallow them so the turn (or session) that already committed in memory still succeeds, and the next save catches the store up. Make load failures fail `session/load` with a clear internal error rather than a leaked exception, since a session that cannot be read cannot be reopened. Verified over the in-memory wire with a failing stub store. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(acp): close out in-flight tool calls when a turn is cancelled A tool call announced as pending/in_progress was never driven to a terminal status when its turn was cancelled (via session/cancel or a dismissed permission dialog), so a client kept rendering it as running after the turn ended cancelled. On the cancel unwind, fail any tool call still lacking a terminal result -- sent live (a cancelled turn never commits its transcript) and shielded so the asyncio cancellation cannot abort the send. Verified over the in-memory wire. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(acp): reject prompts queued behind a session that closed or reloaded A prompt waiting on the session's turn lock held a reference to the old SessionState; close_session/load_session only cancel the *active* turn, so the queued prompt would run against the discarded state - invisible to session/cancel - and persist its orphaned history over the closed (or just restored) session. Re-checking liveness after acquiring the lock turns that zombie turn into the invalid_params the client already gets for a prompt sent after close. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(acp): never swallow a handler's own teardown cancellation prompt() and _cancel_active_turn awaited the turn task bare, so the CancelledError from connection teardown was indistinguishable from the turn's own cancellation and got converted into a response (or, via a dismissed permission dialog, replaced by the internal _TurnCancelled). The SDK cancels each handler task exactly once on shutdown and its sender is already closed by then, so a handler that survives that one cancel and tries to answer hangs Connection.close() forever. Awaiting through asyncio.shield keeps the two cancellation sources apart: the turn's end is read off the (done) task, while the handler's own cancellation propagates. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(acp): don't report rollback when a cancel lands in the post-commit save The store save after a turn commits was the one suspension point left between commit and return: a session/cancel delivered there escaped the turn task and the prompt answered 'cancelled' (the rollback signal, no user_message_id) while the session's in-memory history and transcript already contained the whole turn - the next prompt would build on history the client believed was discarded. The InMemorySessionStore never suspends, which is why the existing cancel-persistence test could not see this; any real (file/db) store can. A cancel that arrives after commit has simply lost the race, so the interrupted save is treated like the write failures _persist already swallows. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(acp): don't leak a terminal when run_command is cancelled mid-create The terminal/create call sat before the cleanup try-block, so a cancellation landing there unwound without a kill or release - but the request may already be on the wire (the SDK sender flushes queued payloads even when the awaiting future is cancelled), leaving the command running client-side with nobody holding its id. Running the create as a shielded task lets the cleanup await the response late, learn the id, and kill/release as usual. Folding the late-id case into one kill path and one release path also keeps the existing guarantee that a failing kill never skips the release. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(acp): end limit-hit turns with max_turn_requests, close out tool calls on errors pydantic-ai's default UsageLimits(request_limit=50) means a long tool loop routinely raises UsageLimitExceeded - which escaped the turn as a JSON-RPC internal_error, the exact condition ACP defines the max_turn_requests stop reason for (token limits map to max_tokens). The raising run's partial messages are not retrievable, so such a turn rolls back like a cancellation and says so (no user_message_id, no usage). Turns failing with any other exception now also close out their announced tool calls before the error reaches the client, matching the cancellation path, so the client does not render them as running next to the turn's failure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(acp): resolve workspace-relative paths before they reach fs/read|write ACP requires absolute paths on fs/read_text_file and fs/write_text_file, but models routinely emit workspace-relative ones - the local FileSystem tools document relative paths and share these tools' names, and the presenter layer already absolutizes for exactly that reason. The client-backed toolset forwarded them raw, producing non-conformant requests the client may reject. acp_filesystem now hands the toolset the session cwd so relative paths resolve before the wire; absolute paths and directly-constructed cwd-less toolsets are untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(acp): true up the conformance claims, comments, and doc links The stop-reason note predated the finish-reason mapping and said the opposite of what the code (and its tests) do; the prompt-capabilities bullet read as inbound enforcement when the spec puts that restriction on the client; the stdio MCP gap (the spec's unconditional MUST) was undisclosed; and two code comments claimed validation/rejection the SDK router does not actually perform - the adapter's own raises are the load-bearing behavior. Doc links now use the canonical pydantic.dev paths both old URLs redirect to. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(acp): shield the terminal create in place instead of via a side task The ensure_future + asyncio.shield shape from the previous commit hit Python 3.12+'s shield behavior: once the outer await is cancelled, a late failure of the inner create is reported to the loop exception handler even when the cleanup retrieves it, which spams production logs and fails under pytest-anyio. An anyio shield around the create await itself is simpler and leans on the same anyio-mediated cancellation the kill/release cleanup already depends on: the cancellation defers to the next await, by which point the terminal id is known and the existing cleanup kills and releases as usual. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(acp): small type and surface cleanups from review - ToolCallPermission.args was typed object although the only construction site passes args_as_dict(); Mapping[str, object] lets a custom policy scope by an argument without isinstance gymnastics. - default_permission_scope is referenced by the permission_policy docs as the fallback to compose with, so it must be importable: exported. - McpServer names the per-server union once instead of spelling it in two places; McpServers gains the TypeAlias marker its sibling had. - _model_state folds the None case so both call sites drop a repeated conditional; the load_session store guard's no-cover pragma was wrong (the SDK router routes session/load regardless of the advertisement, and test_persistence already executes the branch). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(acp): pin claimed behavior the suite asserted too weakly or not at all - The unsupported-method test asserted only RequestError despite name and CONFORMANCE.md promising method_not_found; codes now pinned (-32601, and -32602 for the unknown-session prompt). - The model-override test was tautological: both models answered successfully either way. It now asserts the override's distinct canned output (mutation-checked: dropping the per-run override fails it). - Usage summation across approval passes had no test; an approval turn's output tokens must exceed the resume pass alone (mutation-checked against 'usage = result.usage()'). - New protocol-contract coverage: cancel racing an unanswered permission dialog (turn ends cancelled, pending call driven to failed), double cancel idempotency, image block arriving in model history as decoded BinaryContent, the denial message reaching the failed update's raw_output, chunk_text's exact-budget boundary, and a set_model save failure being swallowed like every other persist failure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(acp): survive raw cancellation during terminal create; answer cancelled after a post-commit cancel An adversarial re-review of this branch's own fixes caught a regression: the anyio shield around terminal/create only blocks anyio-mediated cancellation, but the adapter and pydantic-ai deliver raw task.cancel() (turn.cancel, cancel_and_drain), which pierces it - reintroducing the leaked-terminal window end to end. The create now runs as its own task awaited via asyncio.wait: a raw cancel hits the waiter, not the create, and unlike asyncio.shield (3.12+) a late create failure is not reported to the loop exception handler when the cleanup retrieves it. The leak scenario is pinned at both the toolset level and through the real adapter/agent path. Also from the re-review: a cancel that lands after the turn committed (inside the store save) now answers 'cancelled' as the spec requires - while keeping the committed signals (user_message_id, usage, history) - and the one-tick teardown-vs-turn ambiguity that 3.10 cannot resolve is documented at both await sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(acp): record the post-commit-cancel semantics in CONFORMANCE Also drops an unbounded poll loop in the permission-dialog race test for a single deterministic tick (prompt sets active_turn before its first suspension point), which was the one partial branch left repo-wide. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(acp): let the turn build its own response and drop the commit-flag tuple _run_turn returned (usage, stop_reason, committed) so prompt() could reassemble a PromptResponse it had all the pieces for; building the response in the turn removes that protocol and the committed flag. Recording turn.updates is now unconditional (only the commit gate matters), and the presenter-fields/acp_filesystem branches collapse to single construction paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(acp): appease codespell (unparseable -> unparsable) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Support Pydantic AI 2.x in ACP * feat(acp): forward usage limits into agent runs Let hosts bound each ACP run segment with Pydantic AI usage ceilings while preserving the default run behavior when no explicit limits are configured. * feat(acp): resolve selected session models Let embedders map advertised ACP model ids to concrete Pydantic AI models and re-surface a resident session's current model state. * feat(acp): expose read-only session history Hosts can gauge context with the same estimate_token_count path the compaction tiers use by reading committed resident-session history. The returned list is a shallow snapshot, and callers must treat the shared ModelMessage objects as read-only. * Track ACP 0.11 session config models * chore(acp): fix coverage, trim redundancy, clarify module names Get the ACP PR to green CI and simpler to review, without changing behavior. - Coverage: the only uncovered line was a dead `return None` branch in the `_model_option` test helper (source was already 100%). Assert the option is present instead, dropping the untested branch and its now-redundant guards. - Remove CONFORMANCE.md: its supported/not-supported matrix duplicated the package README's feature sections and "Cancellation and limitations". Point the two references at the README. - Dedup the test ACP clients: RecordingClient, FakeClient, and WireClient each re-spelled the same ~13-method "unused capability" stub block to satisfy the SDK Client interface. Extract it once into RecordingClientBase; each client now subclasses it and overrides only what it exercises. Interface drift is one edit. - Rename two opaque modules for clarity (files only; public symbols unchanged): _present.py -> _presentation.py (matches its ToolCallPresentation exports) and _native.py -> _client_toolsets.py (client-routed filesystem/shell toolsets), with tests/acp/test_native.py -> test_client_toolsets.py to mirror the source. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(acp): move under experimental and shrink the public surface New capabilities start under pydantic_ai_harness.experimental, so ACP follows: importing warns HarnessExperimentalWarning and the API may change without deprecation. __all__ drops the typing-only aliases (25 to 18 names); default_permission_scope and McpServer stay because policies compose with the default scope and AcpSession.mcp_servers is typed by McpServer. Tests move to tests/experimental/acp to mirror the source tree; the experimental warning test lives in the SDK-gated test_acp.py so slim (no-extras) installs stay green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(acp): expose model_resolver and usage_limits on the stdio entry points PydanticAIACPAgent already accepted both, so wanting a per-run token ceiling or host-defined model ids forced users off run_acp_stdio onto the class and a hand-rolled acp.run_agent call, where forgetting use_unstable_protocol=True silently loses session/close. Forward the two params so the one-liner covers every adapter option. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: David SF <david.sanchez@pydantic.dev> |
||
|
Aditya VardhanandGitHub
|
3c5ec34f9e |
Bump vcrpy to 8.2.1 to fix YAML deserialization RCE (#300)
Resolves Dependabot alert GHSA-rpj2-4hq8-938g: vcrpy < 8.2.1 loads cassette files with PyYAML's unsafe loader, allowing arbitrary code execution on cassette load (e.g. in CI). 8.2.1 switches to SafeLoader. |
||
|
Aditya VardhanandGitHub
|
6eb91dfe86 | Bump pydantic-ai floor to 2.1 (#303) | ||
|
Aditya VardhanandGitHub
|
51ba5693bf |
Revert "fix(code_mode): tighten run_code prompt against phantom imports and…" (#317)
This reverts commit
|
||
|
|
4e21f7bcbf |
docs(agents): discourage private-helper imports in tests (#316)
Keep tests pinned to each capability's public surface so they keep passing across internal refactors of the `_`-prefixed modules, and point genuinely unreachable branches at `# pragma: no cover` instead of reaching into private helpers. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
e6dfd4c303 |
ci: add pydantic-ai v2 beta early-warning job and make code_mode v1/v2 compatible (#291)
* ci: add non-blocking pydantic-ai v2 beta test job The harness targets the pydantic-ai v1 line, but the `>=1.105.0` floor also admits v2 prereleases once prerelease resolution is enabled. Nothing in CI exercised that path, so v2-breaking changes were invisible until release. This adds a `test-v2-beta` job that resolves the latest v2 beta and runs the suite against it. It surfaces real breakage today: v2 dropped the `calls` argument from `ToolManager.get_parallel_execution_mode`, which CodeMode still calls with the v1 signature, so `code_mode` raises `TypeError` under v2. The job is intentionally kept out of the `check` gate so an expected red result on the unsupported v2 line never blocks a merge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: cap the v2-beta pytest step so a hang fails fast The v2 beta job is expected to surface breakage, but some v2 breaks hang instead of failing cleanly. When code mode raises an unhandled exception inside a DBOS workflow, DBOS's background recovery thread stays alive and the Python process never exits, so the step rode to the 20-minute job timeout and burned a full runner slot on every push. Wrap the pytest invocation in `timeout -k 30 300` so any such hang fails fast (exit 124). A per-test timeout plugin would not help: the hang happens during interpreter shutdown, after the test body completes, so the cap has to be on the process. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(code_mode): call get_parallel_execution_mode compatibly across v1/v2 pydantic-ai v2 (#5339, shipped in 2.0.0b1) dropped the `calls` argument from `ToolManager.get_parallel_execution_mode`: it now reads the run-scoped mode from a context var and applies per-tool `sequential` barriers separately. The harness passed `[]` specifically to isolate the context var from per-tool flags, which is exactly what the no-arg v2 call returns, so the two are equivalent. Inspect the method arity and call the matching shape. Inspecting rather than catching TypeError avoids swallowing a genuine TypeError raised inside the method. The `Callable[...]` annotation erases the bound signature so both call shapes typecheck whichever major's stubs pyright resolves. Without this, every code_mode run under v2 raises TypeError; inside a DBOS workflow that unhandled error also wedged the process (a non-daemon recovery thread blocked interpreter shutdown), which is what hung the v2-beta CI job. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(code_mode): cover both arities of the v1/v2 mode dispatch The arity-based dispatch added an `else` branch that the v1-pinned coverage gate never executes (the v2 no-arg call only runs under pydantic-ai v2), so total branch coverage fell to 99% and failed the `fail_under=100` gate. Extract the dispatch into `_global_mode_is_sequential` and unit-test both call shapes directly, so both branches are exercised whichever major is installed. This is honest coverage rather than a `# pragma: no cover` that would hide a branch that does run (in the v2-beta job). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: deselect v1-only OTel assertions from the v2-beta job Two instrumentation tests in test_managed_prompt.py assert pydantic-ai v1's OpenTelemetry attribute and span names. v2 deliberately renamed these (aggregated-usage attributes, GenAI-semconv span names), so the tests are expected-red on v2 and carry no signal in this job. Deselecting them keeps the v2-beta job a meaningful early-warning for capability breakage (code mode, durable execution) instead of going red on documented instrumentation drift. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: keep managed prompt v2 signal --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: David SF <david.sanchez@pydantic.dev> |
||
|
|
c56a4b42f8 |
Add host-backed OS and filesystem access to CodeMode sandbox (#262)
* feat(code_mode): expose host-backed OS access to the sandbox Sandboxed `run_code` had no way to reach the filesystem, environment, or wall clock: Monty supports it through an OS callback / `AbstractOS` and directory mounts, but `CodeMode` never threaded `os`/`mount` into `feed_start` or the snapshot resume loop, so callers couldn't enable it. Add `os` and `mount` options on `CodeMode`/`CodeModeToolset`, thread them through `feed_start` and every `resume` site (OS auto-dispatch stops the moment a resume omits them), and make the `run_code` description reflect whether host-backed access is configured. * test(code_mode): harden OS-access tests around the threading invariants Add edge cases that pin the behaviours most likely to regress: OS access surviving across REPL-persisted `run_code` calls, a raising `os` callback degrading to `ModelRetry` instead of crashing the loop, and `mount` accepting a `list[MountDir]`. Hoist the never-invoked callback used by the description/forwarding assertions into one shared helper. * docs(code_mode): tighten and verify the filesystem/OS access section Trim the host-access docs to the essentials and make the example self-contained (drop the undefined helper). The snippet and the documented `mount`/callback constructions are run end-to-end to confirm they work. * docs(code_mode): correct per-request scoping wording `os`/`mount` are static capability fields (no per-run resolver), so the "stateful AbstractOS rooted at a per-user directory" guidance over-claimed. Reword to: build CodeMode per request to scope access. Every other doc line was re-checked empirically against pydantic-monty 0.0.17. * fix(code_mode): don't advertise env/clock for mount-only sandboxes A `mount` only exposes filesystem paths; `os.getenv`/`os.environ` and `datetime.now()`/`date.today()` still require an `os` handler. The description used one host-access note for both, so mount-only agents were told env/clock were routed to the host and would emit calls that fail and burn run_code retries (verified against pydantic-monty 0.0.17). Split the description into three states (none / mount-only filesystem / os), and correct the README and docstrings that conflated the two. * docs(code_mode): correct two run_code description claims verified against monty Audited every statement in the run_code description, docstrings, and README against pydantic-monty 0.0.17. Two were imprecise: - "imported at the top of your snippet" -- mid-snippet imports work, so the rule is just "before use". - OS-enabled note said calls route "to the host environment", but an in-memory AbstractOS (e.g. OSAccess) handles them too -- it's the configured OS handler, not necessarily the host. * docs(code_mode): clarify overlay-mode write persistence and fix wording The mount docs implied writes reach the host, but MountDir defaults to copy-on-write overlay mode, so writes stay in the sandbox unless mode is 'read-write'. Also tighten two awkward/redundant doc lines. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(code_mode): rename public OS/mount surface to be backend-neutral The public type aliases leaked the Monty backend name into a surface we can't rename later. Rename them to match the existing CodeMode/CodeModeToolset convention, and rename the os= parameter to os_access= so it stops shadowing the stdlib os module that sandboxed code itself uses. - MontyOS -> CodeModeOS, MontyOSCallback -> CodeModeOSCallback, MontyMount -> CodeModeMount - CodeMode/CodeModeToolset param os= -> os_access= (mount unchanged) - internal resume()/feed_start() forwarding keeps Monty's literal os= kwarg Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(code_mode): stop shadowing the os module in the execution loop The OS/mount threading named its parameter `os`, shadowing the stdlib module inside the execution-loop helpers. Rename the variable to `os_access` (matching the public field) while keeping Monty's required `os=` keyword only at the resume/feed_start call sites. Also inline the single-use restriction-line helper into `_base_description`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(code_mode): make CodeMode config fields keyword-only The option list keeps growing; pin tools/max_retries as the only positional args and force os_access/mount (and future config) to be passed by name via a KW_ONLY sentinel, so adding options can't silently shift positional meaning. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(code_mode): make os_access/mount docs clear on first read Public docs should let a reader grasp the host-access surface without reverse-engineering it. Reframe the docstrings and README around when to reach for each primitive instead of what is switched off, drop the type-restating prose the annotations already carry, and lead with concrete tasks (share a dataset; inject just the secrets the agent needs). Tighten the os-access test sweep so each test asserts exactly its invariant: drop redundant negative description asserts (one note is interpolated, so the positive phrase alone proves selection), drop an assertion already owned by another test, and type the tmp_path fixtures. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(code_mode): clarify os_access callback return semantics The raw-callback example claimed non-allow-listed keys "stay hidden" by returning NOT_HANDLED. Verified against Monty: NOT_HANDLED *refuses* the call (raises in the sandbox -> model retry), it does not return None. A model probing for an optional secret would crash and burn retries. Distinguish the two return modes explicitly so users don't pick the wrong one: return a value (incl. None) to answer/hide, NOT_HANDLED to refuse a capability outright. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(code_mode): lock in os_access value-vs-NOT_HANDLED semantics Returning a value (including None) from an os_access callback answers the call -- a None reads back like an unset env var, so the sandbox keeps running. Returning NOT_HANDLED refuses the call, raising in the sandbox and surfacing as ModelRetry. These two paths are easy to confuse and silently regress, so pin both. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * code_mode: warn default run_code prompt about unavailable fs/env; drop Any from CodeModeOSCallback The PR's dynamic restriction note covered the mount and os-enabled states but not the default state, which still listed os/pathlib as importable without saying their I/O fails -- the exact wasted-retry case this change reduces. Extend the default note to state filesystem, env, and clock calls are unavailable without a mount or OS handler. Also type the public CodeModeOSCallback alias with object instead of Any: the harness only forwards the callback to Monty and never calls it, so object typechecks cleanly (pyright strict, 0 errors) and keeps Any out of the public API. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: David SF <david.sanchez@pydantic.dev> |
||
|
|
05ae737182 |
bumping pydantic ai (#280)
* bumping pydantic ai * Fix breakage from pydantic-ai 1.107: capability toolsets no longer cross runs All three failures came from pydantic-ai #5230 (on-demand capabilities): - SubAgents inherit_tools transplanted capability-contributed toolsets into sub-agent runs where the owning capability is not registered, which now fails CapabilityOwnedToolset's ownership resolution. Inherit only the parent's own toolsets; capability sharing is shared_capabilities' job. This also drops the delegate tool, replacing the name-based filter. - Tool search discovery moved from message-scanning to RunContext.discovered_tool_names, so the code_mode test now derives it via parse_discovered_tools like the agent graph does. - ToolDefinition gained capability_id, refreshing the logfire snapshot. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Test latest pydantic-ai in CI and set the floor to the verified minimum The lock-resolved test matrix never exercises pydantic-ai releases newer than the lock, and pydantic-ai's harness-compat job only covers harness code that exists when a pydantic-ai PR merges. Harness code merged after a core change (SubAgents, ManagedPrompt vs core #5230) was therefore never tested against it until a manual bump. The new test-latest job re-locks pydantic-ai-slim/pydantic-graph to the newest published versions on every run, so release breakage surfaces immediately. With latest covered by CI, the pydantic-ai-slim floor no longer needs to chase releases: set it to 1.105.0, the first release with capability ownership semantics (core #5230) that the harness now relies on, and verified against the full suite via the test-floor resolution. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Mirror pydantic-ai's lowest-versions matrix and unfreeze its resolution The test-floor job was not testing the floor: under the workflow-level UV_FROZEN=1, `uv sync --resolution lowest-direct` installs the locked (latest) versions, so the job duplicated the regular matrix while claiming lowest-version coverage. pydantic-ai's test-lowest-versions job sets UV_FROZEN=0 on the sync step for exactly this reason; do the same, and adopt the rest of its shape (full Python matrix, fail-fast off, job timeouts) so floor regressions that only reproduce on some Python version get caught. A genuinely-lowest resolution surfaced one over-coupled snapshot: the `logfire.metrics` span attribute only exists on logfire releases newer than the extra's 4.31.0 floor, so treat it as volatile in the ManagedPrompt span snapshots instead of raising the floor. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Replace UV_FROZEN with UV_LOCKED in CI Frozen mode installs the lockfile blindly: lock drift goes unnoticed and resolution flags become silent no-ops, which is how the lowest-versions job ended up testing the locked versions. Locked mode validates the lockfile against pyproject.toml and fails loudly on any mismatch, so a job that cannot do what it claims turns red instead of green. The two jobs that deviate from the lock on purpose (lowest-versions resolution, latest pydantic-ai upgrade) opt out per step, each with a comment. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Keep filtering the delegate tool when SubAgentToolset is used directly The capability-ownership filter only drops the delegate tool when the toolset arrives via the SubAgents capability, since that is the path that wraps it in CapabilityOwnedToolset. SubAgentToolset is publicly exported, and registered directly in Agent(toolsets=[...]) nothing wraps it, so inherit_tools=True forwarded delegate_task to sub-agents and re-enabled recursive delegation. Restore the name filter on top of the capability filter so both registration paths stay non-recursive. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Adapt to per-delegate run controls from main PR #283 made limits and call_counts required on SubAgentToolset, which broke the direct-construction test from the previous commit once CI merged the branch with main. The merge also stranded the CapabilityOwnedToolset import below the new SubAgentLimits dataclass; moved it back into the import block. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Aditya VardhanandGitHub
|
11189a5951 |
ci: remove guard-dependencies workflow (#245)
The hardcoded team-member allowlist auto-closed legitimate PRs from contributors not in the list (e.g. #244), which was too aggressive. |
||
|
Aditya VardhanandGitHub
|
fb62396335 |
Support codemode dependency group alias (#224)
|
||
|
|
9b1a121a92 |
Fix Monty's snapshot resume API (#216)
* fix(code-mode): match Monty's current snapshot resume API * bumping monty * single quotes * uv.lock * docstring: use single backticks per project convention Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * revert unrelated test docstring rewording Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Aditya VardhanandGitHub
|
8cb9941468 |
Adding Python 3.14 (#206)
* Adding Python 3.14 |