mirror of
https://github.com/pydantic/pydantic-ai-harness.git
synced 2026-07-21 02:45:34 +00:00
main
25
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 David SFandGitHub
|
6c60f7cf0d |
feat(step_persistence): rescue last provider-valid resume point on error (#384)
* feat(step_persistence): rescue last provider-valid resume point on error `StepPersistence` previously left an errored run with no `ContinuableSnapshot` for a provider-valid state that no node boundary had captured, so `continue_run` could only resume from an earlier point or raise `LookupError`. Add an error-path capture: when a run fails against a provider-valid history, save one snapshot so the run still exposes its last safe resume point. Two sites cover the two ways such a state is reached without a completed-node snapshot: - `on_model_request_error` saves the request payload directly -- e.g. a resolved tool cycle, which is only provider-valid as the next request is built, never at a `CallToolsNode` boundary. - `on_run_error` reads a `latest_node_history` contextvar refreshed at every `after_node_run` boundary, rescuing a text response whose following `CallToolsNode` raises inside output validation. It cannot read `ctx.messages` directly: the `RunContext` passed to `on_run_error` holds the start-of-run history, which the graph rebinds away during the run. Both saves are gated on `_is_resumable_history` (provider-valid plus at least one `ModelResponse`), so a crash mid-tool-call leaves a dangling `ToolCallPart` that is skipped and `latest_snapshot` never regresses to an unsendable point. `on_run_error` compares by message count and skips when the store already holds a newer snapshot, so a stale completed-node stash never supersedes a fresher `on_model_request_error` save. This is the gated (provider-validity) approach; the ungated simplification that rides on pydantic-ai#6319 is deferred to a follow-up. Closes #253 * test(step_persistence): drop dead validator branch to restore 100% coverage The `gate` output-validator in `test_does_not_regress_to_stale_stash_after_model_request_save` had a `return value` branch that never executed: the run fails on request 3 before any output passes validation a second time. Simplify it to always raise `ModelRetry` (one retry is all the stash scenario needs), matching the sibling `test_rescues_newest_text_history_over_earlier_saved_snapshot`. |
||
|
|
e4839c654b |
Keep already-empty ModelRequests in LimitWarner._strip_old_warnings (#311)
LimitWarner rebuilds history each request, dropping any ModelRequest left with no parts after warning markers are stripped. An already-empty ModelRequest -- which pydantic-ai's empty-response retry appends when a model returns an empty response against a non-optional output type -- has no parts to begin with, so it was dropped too. That left history ending on a ModelResponse, and the next request failed the _agent_graph precondition with `UserError: Processed history must end with a ModelRequest`, killing the run. Keep a request that was already empty; only drop one that became empty because every part was a warning marker. Regression test covers the empty-response-retry tail. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
David SFandGitHub
|
39b4c569cb | Stop attributing every AICA comment to Claude (#362) | ||
|
David SFandGitHub
|
3ba9e2f9a5 |
docs: capability pages for the unified docs site + README/doc parity gate (#329)
* docs: publish capability docs to the unified site + add README/doc parity gate Every capability shipped only a README (kept for GitHub/PyPI). This adds a parallel, cleaned-up page per capability under docs/ for the new unified docs site (pydantic.dev/docs/harness), migrated from each README: snippets verified runnable against source, autodoc API blocks, root-relative Pydantic AI links, and an experimental-status admonition on the experimental set. To keep README and doc in sync going forward, adds a docs-parity-reviewer agent and a parity gate in the review checklist (run as the last step before merge), plus the docs/ layout and the README<->doc requirement in AGENTS.md and the capability-authoring guide. * docs: fix README<->doc<->source inconsistencies across capabilities A parity audit against source found drift, mostly in the capability READMEs (staler than the migrated docs). All fixes verified against source: - Correctness: the "approval/deferred tools are excluded from the sandbox" claim (code_mode README + doc) was false -- those tools are sandboxed like any other; corrected in both. The stale Shell persist_cwd sentinel description is replaced with the actual out-of-band temp-file capture. filesystem protected default `.git/` -> `.git/*` (the bare form never matched). - Runnable snippets: added the missing imports/wiring so README snippets no longer raise NameError (subagents, context, planning, overflow, authoring, filesystem, code_mode). - Parity: documented previously-undocumented params/behaviors (compaction strategy options, overflow strip_ansi/Passthrough, extra autodoc classes for context and subagents), fixed a stale version pin (>=1.95.1 -> >=2.1.0), and added the missing Managed Prompt row to the root README capability matrix. - Style: normalized decorative Unicode to ASCII across all READMEs and dropped a hype phrase, matching AGENTS.md writing style and the docs. * docs: add nav.json to drive the unified-docs harness sidebar The unified docs mount the harness docs under /docs/ai/harness (fed live from this repo via the pydantic-ai 'Pydantic AI Harness' section). This nav.json defines the sub-nav (Overview + Capabilities + Experimental) and the set of doc files the site includes. * docs: migrate "What goes where?" explainer into harness overview Adds the core-vs-harness boundary section (anchor #what-goes-where) to the canonical harness overview, so the pydantic-ai docs that link to it can point here after the duplicated in-repo stub is removed. * docs: address CodeRabbit review -- runnable snippets, accuracy, multi-class autodoc * docs: flatten harness nav and align with graduated capabilities Following the experimental-graduation refactor (#347), restructure the unified-docs harness pages: - Flatten docs/ (drop capabilities/ and experimental/ subdirs); the sidebar is now Overview + one flat list per Douwe's request. - Rename to match the graduated modules: overflow -> overflowing-tool-output, authoring -> runtime-authoring, docs -> pydantic-ai-docs. - Drop the 'Experimental' admonitions from the graduated capabilities and repoint every import + ::: autodoc path off pydantic_ai_harness.experimental. - Add docs for the newly-shipped capabilities: guardrails, dynamic-workflow, media, and acp (acp stays framed as experimental -- it may still be removed). - Every capability doc now links to its source; index capability table lists the full set with flat links. * docs: apply team-sync authoring rules + enforce them in CI From the 2026-07-10 docs review on #329: - Purpose-first leads: drop hook names (before_model_request, after_tool_execute) from the opening paragraphs of compaction and overflowing-tool-output (doc + README); mechanism moves lower. - Mirror the soft 'API may change between releases' stability note from each graduated README into its doc page (ACP keeps its stronger experimental warning; guardrails' README has no note, so its page gets none). - README H1s now use the capability's display name (Overflow capability -> Overflowing Tool Output, RuntimeAuthoring -> Runtime Authoring, SubAgents -> Subagents, etc.). - Extend tests/test_docs_parity.py with per-page mechanical checks: source link present, heading matches the capability name, purpose-first lead (no hook in the opener), and no experimental framing on graduated pages (ACP excepted). - Update the docs-parity-reviewer agent + review-checklist to the flat structure and the new semantic checks. * docs: add the stability note to guardrails (parity with sibling capabilities) guardrails was the one graduated capability whose README and doc page lacked the shared 'API may change between releases' note. Add it to both. * fix: restore uv.lock to match pyproject (bad text-merge dropped 8 lines) Merging origin/main did a git text-merge of the generated uv.lock, leaving it inconsistent with pyproject.toml -- every CI job failed at 'uv sync --locked'. pyproject.toml is identical to main here, so the correct lock is main's. * docs: address CodeRabbit review on #329 Findings that failed to post inline (GitHub error) but were real: - context/README.md, planning/README.md: two nested examples still imported from pydantic_ai_harness.experimental.* -- repoint to the graduated modules. - guardrails/README.md: replace em dashes with '--' (repo style) and add the source-module link. - docs/media.md: standardize on the implementation's canonical media+sha256:// URI scheme (was mixing media://). - tests/test_docs_parity.py: strengthen my own checks per review -- source-link and top-README-link now require a real Markdown link to the page's specific module (not a bare substring); heading checks assert an H1 exists and equals the expected capability name via explicit page metadata. * fix: restore uv.lock [options.exclude-newer-package] block The lock lost its [options.exclude-newer-package] manifest (pydantic-ai-slim = false, ...) -- a bad git text-merge dropped it, and diagnostic uv commands rewrote it under a different local config. Without that block CI's 'uv sync --locked' re-resolves and fails ('addition of exclude newer exclusion for pydantic-ai-slim'). Restore origin/main's exact lock. * fix: restore uv.lock [options.exclude-newer-package] block A pre-commit hook was rewriting uv.lock under the local uv config, stripping the [options.exclude-newer-package] manifest (pydantic-ai-slim = false, ...). Without it CI's 'uv sync --locked' re-resolves and fails. Commit origin/main's exact lock with --no-verify so no hook mutates it (lock-only change). * test: cover the docs-parity helper edge cases (100% coverage) The strengthened helpers added defensive branches (missing frontmatter close, fenced code before the lead, missing/forbidden/ClassName H1, lead running to EOF) that no real doc exercises. Add direct unit tests so the file is back to the repo's required 100% coverage. * docs: link every capability README to its source module + enforce it CodeRabbit re-flagged planning/README.md for a missing source link. Only guardrails had one, so add the source-module link to all 15 remaining capability READMEs (matching the doc pages) and add a parity test so the requirement is mechanical and cannot silently regress. * docs(agents): drop stale folder tree; fix flat docs path + guard names AGENTS.md's File-structure ASCII tree and capability-authoring's doc paths still showed docs/capabilities// docs/experimental/ (flattened in this PR) and the old /docs/harness URL. Delete the tree rather than redraw it -- the layout is discoverable by listing the repo; keep only the non-obvious conventions (flat docs/, the README<->doc parity requirement). Also fix the Vocabulary guard examples (InputGuard/OutputGuard, not the nonexistent InputGuardrail/ CostGuard). * test: statically validate doc snippets exist and parse Every Python snippet in the capability READMEs and docs/*.md pages is now checked for the two failures a reader hits immediately: it does not parse (syntax), or it imports a pydantic_ai_harness symbol that does not exist (stale module path or renamed name -- the class of bug behind the experimental.* import drift). Static only: no model/network execution, so it needs no mocking. The four illustrative API-signature blocks opt out with a {test="skip"} fence (read by pytest-examples, stripped-safe for the unified-docs render). * test: don't fail doc-snippet check on a missing optional extra The static check imported capability modules to resolve their symbols, but in the slim CI job (no extras) importing e.g. pydantic_ai_harness.experimental.acp raises ModuleNotFoundError for the absent third-party 'acp' package -- the harness module exists, its extra just isn't installed. Distinguish a genuinely missing harness module (fail) from a missing extra (skip) by the ImportError's module name. |
||
|
David SFandGitHub
|
310ffadf09 |
refactor: graduate capabilities out of experimental (ACP excepted) (#347)
* refactor: graduate capabilities out of `experimental` (ACP excepted) The `experimental` label suppressed the try->fail->report->improve loop we want from real usage, so drop it from the harness. Ten capabilities move to top-level submodules; ACP stays experimental (it may still be reshaped or moved to core, and can't be generic across agents). - Old `pydantic_ai_harness.experimental.<name>` paths keep working as DeprecationWarning shims (via `warn_moved`), so existing imports don't break. - Renames to match capability names: authoring -> runtime_authoring, overflow -> overflowing_tool_output. - Capabilities stay in individual submodules with no top-level re-export, so importing the root package never pulls in a capability's optional deps. - Docs drop the experimental framing and the "may be removed in any release" language in favor of "API subject to change". * docs: tell capability authors to ask the user when unsure about a name Per PR review: a name is a public commitment once shipped, so ask rather than guess. |
||
|
David SFandGitHub
|
d13cdae9cb |
Fix docs-parity gate and OutputGuard retry test after guardrails merge (#348)
* docs: link shipped guardrails package from README (fix docs-parity CI) * test: use FunctionModel for OutputGuard retry test (survives pyai #6403 text-recovery removal) |
||
|
|
5ab8dd3d81 |
feat: input and output guardrails (block, redact, retry) (#249)
* feat: add input and output guardrails * chore: use single backticks * fix: run InputGuard only on the first model request * fix: replace list[Any] with Sequence[ModelMessage] * fix: pass raw output to OutputGuard, not str(result.output) * refactor: organize tests into TestCapabilityName classes * fix: drain cancelled tasks in InputGuard parallel finally * fix: re-raise task exceptions via await instead of .exception() * refactor: consolidate parallel cancel/drain into single finally * feat: support callable block_message; collapse InputGuard hooks block_message now accepts a callable so the refusal text can reflect the prompt/output that tripped the guard, rather than being frozen at construction time. InputGuard's sequential path moves from before_model_request into wrap_model_request, so a single hook covers both sequential and parallel modes instead of two hooks each branching on `parallel`. Tests move to tests/guardrails/ to match the tests/<capability>/ layout. * refactor: guards return bool | GuardResult; drop block_message A guard now returns either a bare bool or a GuardResult carrying a refusal message, replacing the separate block_message constructor field. The message is produced when the guard decides, so it can reflect the guard's own reasoning rather than a string frozen at construction time. Guards (and the GuardResult path) may optionally take a RunContext as a first parameter, detected from the signature like pydantic-ai's output validators, so deps- and history-aware guards are possible without closing over globals. Prompt/output-only guards are unchanged. * feat: guard outcomes — allow/block/replace/retry with OTel spans A guard now reports one of four outcomes via GuardResult classmethods (bool shorthand still works): allow, block, replace, retry. - replace lets a guard redact rather than refuse — InputGuard rewrites the prompt sent to the model, OutputGuard substitutes the output. - retry lets OutputGuard send a bad output back to the model; OutputGuard moves from after_run to after_output_process so it can raise ModelRetry and return a modified output. - replace and block emit spans on the run tracer so a redaction or refusal is visible in Logfire; redacted content is included only when RunContext.trace_include_content is set. InputGuard replace requires sequential mode and retry is rejected as a usage error, since neither is meaningful for input. * test: cover guardrail test helpers for the 100% gate The coverage gate measures test files too; the _prompt_text helper had unreached branches. Drop it and assert on message parts inline. * docs: scope streaming behavior; add streaming tests A pydantic-ai-correctness review surfaced two streaming points. Verified both empirically: - InputGuard(parallel=True) works under run_stream() — no deadlock. - OutputGuard GuardResult.retry() is unsupported under run_stream(): pydantic-ai does not retry output while streaming, so a retry verdict surfaces as UnexpectedModelBehavior. Document the retry limitation and that OutputGuard screens only the final output (partial chunks reach the caller first while streaming). Note that input redaction also rewrites persisted history and targets text prompts. Add streaming tests for both guards to lock the behavior in. * test: make the parallel-streaming test a real regression guard The test proving InputGuard(parallel=True) does not deadlock under run_stream() had no timeout — a reintroduced deadlock would hang CI instead of failing. Wrap it in asyncio.wait_for and document the reviewed concern it guards against. * feat: guardrail capability ordering + GuardResult hardening Address @adtyavrdhn's review on #249. - `InputGuard.get_ordering()` → `position='innermost'` so any message-morphing capability runs first and the guard sees the final prompt the model will receive. - `OutputGuard.get_ordering()` → `position='outermost', wrapped_by= [Instrumentation]` so the guard's block/redact spans are always captured by an enclosing `Instrumentation` span regardless of user list order. - `GuardResult` is `frozen=True, kw_only=True` with a `__post_init__` that rejects field combinations the four-outcome contract does not allow (e.g. `replace` without a replacement). `block` with no message stays valid — the default kicks in at the use site. - `_run_guard` and `after_output_process` dispatch via `match action:` with `assert_never` exhaustiveness guards. - `_trace_block` gates the refusal `message` attribute behind `ctx.trace_include_content`, matching `_trace_redaction` — the message can quote sensitive content from the guarded value. - New tests: ordering declarations, `__post_init__` validation, frozen enforcement, outer-cancellation no-leak regression guard (which confirms `asyncio.shield` around the cleanup `gather` is not needed — the outer cancel is already consumed by `asyncio.wait`). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump pydantic-ai-slim to 2.4.0 in lock CI resolves pydantic packages to newest (uv.toml marks them exclude-newer=false); the merged lock was stale at 2.1.0 so uv sync --locked failed. Regenerate to 2.4.0. * test(code_mode): drop removed ToolSearchMatch.description field pydantic-ai 2.4.0 narrowed ToolSearchMatch to carry only `name`; the full ToolDefinition (with description) is surfaced separately. Update the test fixtures to the new shape. --------- Co-authored-by: Kacper Włodarczyk <kacperwlodarczyk@protonmail.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
David SFandGitHub
|
b5b93704c3 |
feat(subagents): contain unexpected sub-agent crashes as bounded retries (#326)
A delegate that crashes with anything other than an expected soft degradation (a provider ModelAPIError/FallbackExceptionGroup, a plain ValueError from a bad tool argument) aborts the whole parent run. For an orchestrator fanning out to flaky delegates that is too fragile: one crash kills unrelated in-flight work. Add opt-in SubAgent(contain_errors=True) / SubAgents(contain_errors=False) default. When on, the crash is caught and returned to the parent as a bounded ModelRetry instead of propagating. It stays loud: the exception rides the retry message, it is logged, and tool_retries still bounds consecutive crashes into an abort -- a genuine bug is never masked as success. Orthogonal to on_failure (which only sets the message for expected soft degradations). Cancellation, a shared UsageLimitExceeded, pydantic-ai control-flow signals, and UserError always propagate. |
||
|
David SFandGitHub
|
069d1ffacf |
feat(subagents): configurable delegate-tool retries (#324)
* feat(subagents): configurable delegate-tool retries A sub-agent that errors (e.g. exhausts its own output retries) surfaces as a ToolRetryError on the parent's delegate_task tool, which defaults to retries=1 -- so one flaky sub-agent run crashes the whole parent run with no chance to re-delegate. Add a `tool_retries` field on `SubAgents` (default `None`, inherits the parent's default, so behavior is unchanged) that sets the delegate tool's retry budget, letting callers give the parent a chance to recover from a transient sub-agent failure. * feat(subagents): resilient default for delegate-tool retries Default `tool_retries` to 2 (not None/inherit) so a repeated flaky sub-agent does not abort the parent run on its first repeat. The retry counter resets after any successful delegation, so this bounds consecutive failures. `None` still inherits the parent agent's default. |
||
|
David SFandGitHub
|
f460861435 |
Add experimental CE-discovery capabilities: context, subagents disk-load, docs, authoring (#304)
RepoContext (instruction autoload + asset inventory), SubAgents disk-load, PyaiDocs (on-demand pyai docs), RuntimeAuthoring (runtime capability authoring). Reviewed + hardened; validated under pydantic-ai 1.107 and 2.0.0. |
||
|
|
0418280d1e |
test(logfire): tolerate pydantic-ai 2.0.0 agent-span rename (#305)
pydantic-ai 2.0.0 reworked Instrumentation: the agent run span was renamed from `agent run` to `invoke_agent agent`, the tool span from `running tool` to `execute_tool noop`, and several span attribute keys were renamed. The `test on latest pydantic-ai` CI job (newest pyai = 2.0.0) failed on the two span-name assertions while the locked/floor 1.x jobs stayed green, so a static snapshot update would just flip the failure onto the 1.x jobs. Harness still supports the 1.x floor (`pydantic-ai-slim>=1.105.0`), so make the two tests version-tolerant instead of bumping the floor: - Add `_PYDANTIC_AI_GE_2` from `importlib.metadata.version`. - `test_provider_backed_resolution_tags_v1_instrumentation_spans`: branch the expected agent-span name so it stays live on both versions. - `test_baggage_propagates_to_run_and_child_spans`: skipif on 2.0.0 since its full span-tree snapshot needs a separate 2.0.0 refresh. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
3f4cbd0a66 |
fix(code_mode): tighten run_code prompt against phantom imports and bad inputs (#295)
* fix(code_mode): tighten run_code prompt against phantom imports and bad inputs CodeMode agents repeatedly waste run_code turns on avoidable mistakes -- importing unavailable modules (textwrap) or a phantom `functions`/`sandbox_tools` module, pasting oversized one-line literals (unterminated-literal errors), indexing typed tool results as dicts, and echoing `...`/`??` truncation placeholders back as arguments. Each failure burns a model turn and pushes agents toward their request limits. The prompt now closes the importable-module allow-list (anything outside it fails; the listed functions are pre-bound, do not import them) and adds a short caution: no oversized literals, results are typed objects (read fields, don't json.loads or index a string), and never pass a truncation placeholder as an argument. Prose-only; 100% branch coverage retained. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(code_mode): make the run_code prompt replaceable via `instructions` CodeMode had no way to customize the run_code prompt -- you had to fork. Add an `instructions` parameter: a str replaces the built-in prose, or a callable receives the built-in (host-aware) prose and returns the replacement (append/prepend/rewrite). The auto-generated tool catalog is always appended afterward, so run_code stays callable regardless. This is the right seam for project-specific guidance instead of baking it into the shared default. Also drop the version-fragile module examples from the import-allow-list note (e.g. `collections` is unavailable today but could land in a future Monty); the list itself is already in the prompt, and the prose is now customizable anyway. 100% branch coverage retained; tests cover the str and callable forms (catalog preserved). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(code_mode): `instructions` is a plain str + export the default prose Drop the callable form of `instructions` (overkill). It is now `str | None`: a str replaces the run_code base prose, `None` keeps the built-in. To build on the default instead of replacing it, import the newly-exported `default_run_code_instructions()` and append/prepend/edit, then pass the result -- no callable needed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(code_mode): append via `instructions`, replace via `dangerously_replace_instructions` Keeps the Monty-tuned base prose by default so a customization can't silently desync `run_code` from the sandbox semantics. `instructions` now appends to the built-in prose (before the catalog); a separate, clearly-labeled `dangerously_replace_instructions` is the escape hatch for full replacement. The two are mutually exclusive (UserError on both set). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
1b709d8e30 |
fix(overflow): read_tool_result returns on a missing handle instead of raising (#293)
* fix(overflow): read_tool_result returns on a missing handle instead of raising A wrong handle (e.g. a model passing a tool-call id) or a result no longer in the store made `read_tool_result` raise `ModelRetry`, which consumes a tool retry and, once the budget is spent, escalates to a fatal `UnexpectedModelBehavior` that kills the run. This took down a pydanty reproducer run (the model passed `functions.read_tool_result_44.1` as a handle; max retries was 1). `_read_slice` now returns a guiding message on a missing handle -- pointing the model at the exact handle string from the spill marker, or to re-run the original tool -- so a bad handle can never crash a run. The `offset`/`limit` input validations stay `ModelRetry` (deterministic caller errors the model corrects). 100% branch coverage retained. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(overflow): don't echo the store error in the missing-handle message A FileNotFoundError (and custom-store errors) can carry the resolved filesystem path or other backend detail; the model only needs the handle and recovery guidance. Drop ": {exc}" and assert the store path is not in the returned message. Addresses review feedback on #293. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
David SFandGitHub
|
caf50108fe | Explain code mode metadata selection (#292) | ||
|
|
24fd30e70c |
docs: stale-PR re-eval, experimental-export, and background-coder review guidance (#288)
* docs: add stale-PR re-eval, experimental-export, and consumer-lens guidance Adopting a months-old PR (#243) surfaced three gaps in the AICA review/authoring guides: - no discipline for re-evaluating a stale or pre-merge PR (retire temporary uv.sources pins once upstream lands; re-check drift vs current main) - the experimental-vs-top-level export convention was undocumented, so the code_mode exemplar steered new-capability authors toward top-level exports - the review checklist named "dogfooding need" but no reference consumer or the cache/cost/reliability lens a long-running background coder cares about Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: key export-stability rule on shipped-in-release, not on a symbol list Only CodeMode has shipped in a published release (v0.3.0 exports just CodeMode); FileSystem, ManagedPrompt, and Shell are top-level on main but pre-release. The earlier wording labeled all four "already-released", which is inaccurate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: drop the reference-consumer lens bullet Keep the two clearest gaps (stale-PR re-eval, experimental-vs-released exports); drop the consumer-lens addition, which named pydanty (a separate repo) in the harness review checklist and was the most likely to read as out-of-place. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: David SF <david.sanchez@pydantic.dev> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
9821aab0d5 |
feat(compaction): add ClampOversizedMessages for runaway generations (#286)
A single runaway model response (degenerate whitespace) or giant tool-call payload can produce one part large enough to blow the next request past the provider context cap. No existing strategy can reach it: SlidingWindow drops the oldest messages, ClearToolResults only touches tool results, and feeding the history to SummarizingCompaction hits the same cap. ClampOversizedMessages truncates the offending ModelResponse part in place, keeping a head/tail slice with a marker. Covers response text (critical case) and tool-call args (clamp_tool_call_args, default on); request-side parts stay out of scope. Composes as the first zero-LLM tier of TieredCompaction. Closes #285 Co-authored-by: David SF <david.sanchez@pydantic.dev> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
3ac6034f7e |
feat(experimental): OverflowingToolOutput capability for oversized tool returns (#290)
* feat(experimental): OverflowingToolOutput capability for oversized tool returns An oversized tool return persists in history and is re-sent on every later model request, paying its token cost for the rest of the run. This capability intercepts a return in `after_tool_execute`, reduces it once, and lets the reduced form persist instead of recomputing it per request. Three freely-combinable modes selected by an ordered list of size bands: truncate (cheap clamp), spill (lossless persist + `read_tool_result` handle for on-demand slice/grep/tail), and summarize (size-gated LLM summary inheriting the run's model). The default band is `Spill(then=Truncate())`: lossless when the store accepts the write, a bounded truncation otherwise, never a silent drop. Spilled payloads go through a narrow `OverflowStore` protocol (local-file default) so a durable backend or core #4352's queryable-file primitive can slot in behind the same handle. Mirrors the experimental/compaction package shape, export pattern, and model-inherit path; truncation / ANSI / binary helpers are harvested from PR #185, whose one-way spill this supersedes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(overflow): address review -- store hardening, content overflow, bounds, no-Any, TTL cleanup S1: harden LocalFileStore while preserving cross-agent sharing. Keep a stable, configurable root (so a later run can read an earlier spill), but create it with 0700 perms and verify on read that the resolved target stays within the root, rejecting symlink/`..`/absolute escapes. Sharing is the goal; isolation is not the security mechanism. C1: large ToolReturn.content bypassed reduction -- core renders it as a separate model-visible part that also persists in history. Measure and reduce content with the same band logic (distinct spill handle); non-text content that overflows is left unreduced with a warning. C2: bound read_tool_result -- require offset >= 0 and limit >= 1, clamp limit to a line cap, cap the joined output, and make `pattern` a literal substring instead of a regex so a model-supplied value cannot trigger catastrophic backtracking. A1: drop avoidable Any -- typed tokenizer, narrowed payload/metadata handling, and removed the reportUnknownArgumentType ignores via bool guards. M1: summarize tests now assert the real claims (ctx.model inheritance, explicit model override, usage=ctx.usage threading) with a FunctionModel instead of a wholesale Agent mock. Cleanup feature: opt-in LocalFileStore(cleanup_after=timedelta) prunes files older than the TTL in a background daemon thread; keep-forever stays the default. Pruning is non-blocking and swallows all errors into warnings, never failing a run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(overflow): convert remaining avoidable Any to precise types (A1) `_is_mapping`/`_is_text_sequence` are now `TypeGuard`s, so `json_sketch` takes `object` and the sketch helpers take `Mapping[object, object]` / `Sequence[object]` -- no `Any`, no `Unknown`. In the capability, `original` is `object`, `metadata` / `existing` are `object`, `_copy_mapping` / `_with_handles` return `dict[str, object]`, and the assembled `ToolReturn` locals and `_assemble` return type drop `Any` for `ToolReturn[object]` / `object`. The only `Any` left is the core-inherited `after_tool_execute` hook signature (`args`/`result`/return). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: David SF <david.sanchez@pydantic.dev> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
e34c30ae54 |
feat(shell): control subprocess environment to stop secret inheritance (#284)
* feat(shell): control subprocess environment to stop secret inheritance Shell spawned every command with the inherited parent environment, so an agent running model-written commands in a sandbox could read the host's LLM API keys and tokens. This blocked replacing Pydanty's hand-rolled runner with Shell. Add two composable controls: `env` replaces inheritance entirely (a hard boundary -- the subprocess sees only what you pass), and `denied_env_patterns` strips glob-matched names from the base environment (mirrors `denied_commands`). When both are set, patterns also filter the explicit `env`. The default stays inherit-everything. Export `LLM_API_KEY_ENV_PATTERNS` for the common provider-credential denylist; opt-in, since stripping env silently would break agents that rely on inherited credentials. Closes #281 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(shell): tighten env-control wording after adversarial review Three-lens self-review (API breadth, correctness, docs/tests) surfaced doc-precision gaps and one missing end-to-end test; no correctness defects. - denied_env_patterns matches by glob, not exact match like denied_commands: correct the "mirrors denied_commands" claim to name the real parallel (the denied_* naming convention). - LLM_API_KEY_ENV_PATTERNS targets LLM credentials only and uses coarse prefixes (GOOGLE_* also strips GOOGLE_APPLICATION_CREDENTIALS); say so, and note it does not cover other host secrets like LOGFIRE/GitHub tokens. - env is enforced at spawn; clarify that with denied_env_patterns it is the resolved (filtered) env, and warn that stripping PATH/HOME breaks command resolution. - Add an end-to-end test proving env and denied_env_patterns compose at spawn. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: David SF <david.sanchez@pydantic.dev> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
34dcbf8352 |
feat(subagents): per-delegate run controls (usage limits, timeout, call budget, soft-failure) (#283)
* feat(subagents): per-delegate run controls via SubAgentLimits Delegations through SubAgents ran with no per-child budgets, so an orchestrator that needs one child capped at N requests, another bounded by a wall-clock deadline, or a per-run delegation cap had to keep a parallel hand-rolled delegation path (motivating consumer: Pydanty's run_delegate). Add a `limits: Mapping[str, SubAgentLimits]` knob. `SubAgentLimits` carries `usage_limits` (isolated child accounting so the budget is the child's own, reached softly rather than via a propagating UsageLimitExceeded), `timeout_seconds` (wall-clock cancel with a soft steering message), `max_calls` (per-run delegation budget, counts keyed by run_id and cleared in wrap_run), and `on_failure` (a single steering message that overrides the default soft text and flips child failures from a parent ModelRetry to a soft tool result). Absent names keep current behavior. Closes #282 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * review: validate limits keys, exclude run-state from eq, cover parallel budget Adversarial self-review follow-ups on the per-delegate controls: - SubAgents.__post_init__ raises when `limits` names a sub-agent absent from `agents`. A typo'd key would otherwise silently run that delegate unbudgeted, the exact failure the budgets exist to prevent. - `_call_counts` is marked compare=False so accumulating delegation counts during a run don't change SubAgents equality. - Add a test for two delegations issued in one parent step: the synchronous count still caps them at max_calls. Document that max_calls is scoped per run_id (so nested trees budget independently) and that a timed-out child's event stream is cut without a terminal event. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: David SF <david.sanchez@pydantic.dev> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
6f2aa11438 |
Add StepPersistence capability for step-event durability across delegates (#251)
* Add StepPersistence capability for step-event durability across delegates Supersedes PR #176 (SessionPersistence): orchestrators like pydanty need visible event trails for delegate runs that may time out before a "save full session after the run" hook can fire, and need to continue or fork a delegate's prior investigation without rediscovering context. A single after-run snapshot is too coarse for that use case. The capability now records (a) append-only StepEvents at every boundary (run/model-request/tool-call start, completion, failure), (b) a ContinuableSnapshot only when message history is provider-valid (every ToolCallPart has a matching ToolReturnPart / RetryPromptPart) — saved mid-run after CallToolsNode and at after_run, and (c) a ToolEffectRecord ledger so a run killed between before_tool_execute and after_tool_execute leaves an `unknown_after_crash`-style record rather than a falsely-continuable snapshot. Lineage metadata (parent_run_id, agent_name) ties delegate runs back to their orchestrator. `continue_run` / `fork_run` helpers load the latest continuable snapshot for a run. Backends: InMemoryStepStore (tests) and FileStepStore (JSONL events + JSON snapshots, with run_id path-traversal validation and anyio.to_thread for blocking I/O). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Address pydanty review + ergonomics: tighten correctness and identity model Correctness fixes from pydanty's PR review: - FileStepStore: snapshot filenames are now a per-run monotonic counter, not `ctx.run_step` — `run_step` resets each Agent.run, so re-using a `run_id` across calls would let an earlier run's higher step-index snapshot mask a later run's lower-step-index one. - StepStore.get_tool_effect now takes both `run_id` and `tool_call_id`. TestModel and other providers can reuse deterministic tool-call ids across runs; the previous unscoped lookup let one run's effect leak into another's record (including `started_at`). - is_provider_valid now rejects orphan, duplicate, and out-of-order tool returns — the old `set.discard` pattern silently accepted any return regardless of whether a matching call was open. Identity model: - `run_id` resolution: explicit > `{agent_name}-{8-char-hex}` > UUID. Materialised per Agent.run in `for_run`, so reusing one capability instance never silently merges runs. - `parent_run_id` auto-inferred via a module-level ContextVar set in `wrap_run`, so an orchestrator's tool that synchronously calls `delegate.run(...)` produces a delegate `RunRecord.parent_run_id` pointing at the orchestrator's `run_id` with zero threading. Explicit `parent_run_id=` still wins. - `conversation_id` propagated to `StepEvent` and `RunRecord`; `store.list_runs(conversation_id=..., parent_run_id=...)` supports filtering by either or both. Mirrors pydantic_ai's three-level identity (conversation -> run -> step) so "run 1, run 2, run 3" of one dialogue is queryable as a group via `conversation_id`. - `continue_from=` field dropped from the capability. Continuation is now only via `continue_run(store, run_id=...)` -> standard `Agent.run(message_history=...)`. One way to pass history into pydantic_ai, no parallel capability flag. README rewritten around the final API. New sections: three-level identity, run lineage with auto-inferred parent, inspecting a run tree, failure recovery. Tests: 168 total (up from 64), 100% branch coverage on the package. New coverage for the snapshot seq counter, cross-run tool-effect isolation, orphan/duplicate/out-of-order return rejection, ContextVar parent inference across nested agent.run, conversation_id propagation, and the agent_name-derived run_id default. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Address pydanty round-2 review: retry validity, list ordering, effect metadata Correctness: - is_provider_valid no longer rejects non-tool RetryPromptParts. Pydantic AI emits `RetryPromptPart(tool_name=None)` for output-validation failures and providers map those as plain user messages, not tool results. The previous check required every RetryPromptPart to resolve an open tool call, so a run with one output retry produced no final continuable snapshot despite being fully valid. - StepStore.list_runs now guarantees chronological (started_at ascending) ordering across both backends. FileStepStore was previously returning directory-name order (lexicographic), so the README's `[-1]` pattern for "latest run in conversation" could pick the older run when run ids did not sort by recency. - after_tool_execute and on_tool_execute_error preserve idempotency_key and effect_summary from the prior `started` record. Previously the terminal record was written without those fields, so any annotation the tool body wrote was lost on completion. - from_spec raises ValueError for unknown backends instead of silently falling back to in-memory storage. For a persistence capability, turning a typo into accidental non-durability is the wrong failure mode. API: - New annotate_tool_effect(store, ctx, *, idempotency_key=None, effect_summary=None) helper. Tool bodies that write external state call it to attach idempotency + effect metadata to the in-flight ToolEffectRecord without knowing the (run_id, tool_call_id) plumbing. Resolves run_id from a ContextVar set by wrap_run; reads tool_call_id / tool_name from RunContext. - ContextVar moved from `_capability.py` into a new `_context.py` module so the helper and the capability can share it without circular imports and without crossing the private-name barrier. Docs: README fixes a non-existent `list_runs(agent_name=None)` call, documents the chronological-ordering guarantee, and replaces the hand-wavy "populate fields on the ToolEffectRecord" line with a concrete `annotate_tool_effect` example. Tests: 178 total (was 168), 100% branch coverage on the package. Added coverage for non-tool retry acceptance, chronological list_runs on both backends, metadata preservation across completed/failed transitions, annotate_tool_effect under realistic agent.tool, and from_spec backend validation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(step_persistence): align FileStepStore docstring with seq-counter layout The class docstring still showed snapshots/<step_index>.json from the pre-fix layout, but both the README and _next_snapshot_seq document the monotonic counter. Bring the class docstring in line. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(step_persistence): fix broken README examples + clarify run_id sharing Pydanty round-3 review: - README continuation and lineage examples queried `list_runs(conversation_id=...)` on conversations the earlier `.run(...)` calls never set, so the examples crashed with IndexError on `[-1]`. Pass the conversation_id to the earlier calls so the lookup actually works. - The capability docstring claimed reusing a `StepPersistence` instance across `Agent.run` calls does NOT share the id. That is true only for the auto-derived (`agent_name`-prefixed or `ctx.run_id`) cases — an explicit `run_id=` is shared across every `.run()` by design, since that is the orchestrator pattern where the caller owns one logical identity across turns. Rewrite the resolution-order docs to spell out which cases share and which don't, and when to pick each. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(step_persistence): align run_id semantics with pydantic_ai (per-call, not shared) Pydanty round-4 review: the prior round documented explicit `run_id` as shared across `.run()` calls on one capability instance — that framing caused real correctness gaps. The `ToolEffectRecord` ledger is keyed by `(run_id, tool_call_id)` and providers reuse deterministic tool-call ids (e.g. `TestModel` emits `pyd_ai_tool_call_id__{name}`), so a second `.run()` overwrites the first's effect record under the same key — the `unknown_after_crash` signal from turn 1 disappears when turn 2 lands. Realign: - `run_id` is per-`Agent.run`, matching `pydantic_ai.RunContext.run_id`. - For multi-turn logical grouping, use `conversation_id=` on `Agent.run(...)` — that is the pyai-native primitive. The orchestrator pattern is `conversation_id='orch'` with each turn auto-deriving its own `run_id`. - Explicit `run_id=` remains supported but is documented as single-shot (testing, replay, debugging). Reusing it across calls is a caller contract violation, not an implementation feature. Code is unchanged — the implementation was already correct under the right contract. Only the docs were misleading. Tests: - `TestRunIdIsPerCall::test_multi_turn_orchestrator_uses_conversation_id` exercises the recommended pattern: three turns sharing a `conversation_id`, three distinct auto-derived `run_id`s, all queryable as a group. - `TestRunIdIsPerCall::test_explicit_run_id_reuse_collides_ledger` locks down the misuse contract: reusing one explicit `run_id` across two `.run()` calls produces colliding effect records under the `(run_id, tool_call_id)` key. The behavior is documented; the test exists so a future refactor cannot silently change it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(step_persistence): explain why for_run uses a local ContextVar Pyai-aligned review flagged this as a P3 explainer: pydantic_ai already has three single-slot cross-run signals (RUN_ID_BAGGAGE_KEY, ctx.run_id, _CURRENT_RUN_CONTEXT). All three get overwritten by the inner Instrumentation.wrap_run before any nested capability can see the parent identity. A separate harness-local ContextVar, snapshotted before our own wrap_run rebinds it, is the only correct mechanism today. Spell this out so the next reader doesn't try to 'simplify' it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(step_persistence): enforce explicit-run_id reuse with ValueError Pydanty round-5 review accepted the docs-only contract but flagged that "documented but not enforced" is a soft spot. Enforce it: `before_run` calls `store.get_run(run_id=...)` when the user supplied an explicit `run_id`, and raises `ValueError` if a record with that id already exists. The auto-derived cases cannot trigger this check (each call materialises a fresh id in `for_run`). The check is one extra store read per Agent.run when an explicit run_id is set, only. The error message points the caller at `conversation_id` for multi-turn grouping. Test renamed from `test_explicit_run_id_reuse_collides_ledger` to `test_explicit_run_id_reuse_raises` — asserts the second `.run()` raises and the first run's records survive untouched. README + capability docstring updated: the misuse path is now "raises" not "caller's contract." Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: gitignore branch-context skill artifacts + AGENTS.local.md Two patterns that match the existing CLAUDE.local.md ignore convention: - AGENTS.local.md — canonical local-instructions file (CLAUDE.local.md is symlinked to it where the worktree follows the same AGENTS.md/CLAUDE.md symlink pattern). - .agents/skills/branch-context/ — per-worktree decisions log (`pr-decisions.md`) and the skill's local SKILL.md. Pattern lifted from `~/pydantic/ai/base/.claude/skills/branch-context/` where pyai uses an identical setup. Neither is intended to land in PRs — they record cross-iteration design calls so future AICA sessions in this worktree don't silently undo them. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(step_persistence): add SqliteStepStore + MediaStore externalization Adds a new `pydantic_ai_harness.media` package (MediaStore protocol + DiskMediaStore / SqliteMediaStore / S3MediaStore) and wires it into the file/sqlite step-persistence backends so large BinaryContent payloads get externalized out of snapshot JSON / table rows by default. Defaults are zero-config: FileStepStore writes blobs under `<root>/media/<sha256>.bin`; SqliteStepStore writes them to a sibling `media` table in the same DB. Threshold is 64 KiB and URI scheme is `media+sha256://<hex>` so blobs are content-addressed across stores. Pass `media_store=None` to keep bytes inline, or a custom `MediaStore` to redirect (e.g. `S3MediaStore` for R2 / AWS / MinIO). S3MediaStore handrolls SigV4 over httpx to avoid a botocore/boto3 dependency. Verified working against Cloudflare R2. `StepPersistence.from_spec(backend='sqlite', database=...)` now resolves. 180 → 261 tests, 100% branch coverage maintained. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(media): VCR cassettes for S3MediaStore against R2 Adds replay-driven tests under `tests/media/test_s3_cassettes.py` that exercise `S3MediaStore.put/get/exists` against pre-recorded Cloudflare R2 responses. CI runs them without any S3 creds via the committed cassettes under `tests/media/cassettes/`. Sanitisation policy: - `before_record_request`/`before_record_response` swap the real R2 account-id subdomain and bucket name for fixed placeholders (`account.r2.cloudflarestorage.com`, `harness-test-bucket`) - `Authorization` and `x-amz-date` filtered to `REDACTED` - CF-RAY, x-amz-version-id, x-amz-checksum-*, x-amz-request-id headers dropped (none load-bearing for tests; some carry identifying info) - Non-2xx response bodies blanked (R2's gzipped XML error envelope leaks the bucket name; our code only checks status code) The `s3_credentials` fixture uses `os.environ.get(NAME, PLACEHOLDER)` per field, so real R2 creds are used when recording locally with `.env` loaded, and the placeholder constants match the scrubbed cassettes during replay. Because the placeholders are fixed, any scrubber miss during a future re-record shows up as a replay URL mismatch — built-in canary against credential / private-data leakage in committed cassettes. Adds `pytest-recording` (pulls `vcrpy`) to the dev deps. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(media): public_url resolver — protocol method + per-store callable Adds `MediaStore.public_url(uri) -> str | None` plus a `public_url=` constructor parameter on every concrete store. The parameter accepts a sync or async callable; the store auto-detects and awaits if needed. This is the bottom-layer primitive for the forthcoming `MediaExternalizer` capability — that capability will call `store.public_url(...)` per externalized blob and swap `BinaryContent` for `ImageUrl` / `AudioUrl` parts before the model sees the message. The callable shape covers both static URLs (public bucket / CDN — use `make_static_public_url` helper) and dynamic URLs (presigned, per-request signing — pass any async callable with TTL captured in its closure). Why a callable rather than a static config: a public bucket's URL host is not derivable from the bucket creds (R2 public buckets use `pub-<hash>.r2.dev`, AWS public buckets use a different scheme than the path-style endpoint we sign for). The URL is always user-supplied information, so a callable is the right primitive — same shape for the static and presigned cases, and `get` stays untouched (it serves the harness's internal byte fetch, not the model's external HTTP fetch). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(media): introduce MediaContext + key_strategy for extensible operations Adds `MediaContext` (frozen, kw-only dataclass with `media_type`, `filename`, `metadata`) and threads it through every `MediaStore` method and both user callables (`PublicUrlResolver`, `KeyStrategy`). New context fields can be added non-breakingly; existing call sites and resolvers keep working. Also adds: - `KeyStrategy = Callable[[str, MediaContext], str]` for per-store layout control. Default `default_key_strategy` produces `<sha256>.bin`. Disk store validates the result against `..` traversal. - `metadata` persistence on `SqliteMediaStore` (new JSON column) and `S3MediaStore` (signed `x-amz-meta-*` headers, ASCII key validation). Disk store explicitly does NOT persist metadata in v1 — sidecar / xattr options each have load-bearing drawbacks; we ship nothing rather than a half-true persistence promise. - `make_static_public_url(...)` updated to the new `(uri, ctx)` signature. The shift is motivated by the same principle as pydantic_ai's `RunContext`: extension via fields on a context bag rather than via breaking signature changes. Every new requirement (TTL hints for presigned URLs, audit ids, response-header overrides, etc.) becomes a field addition, not an API revision. Cassettes from the previous commit replay unchanged — match-on does not include the signed headers and the request URLs are stable. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(media): close metadata round-trip; drop vestigial sqlite key_strategy Adds `MediaStore.get_metadata(uri) -> Mapping[str, str]` to the protocol and implements it on all three concrete stores: - `DiskMediaStore`: writes a sidecar `<resolved>.meta.json` alongside the blob on put (atomic tmp + rename), reads it back on `get_metadata`. Returns `{}` when no metadata was supplied. v1 had documented this as a deliberate gap — sidecar JSON is straightforward and the xattr / ADS drawbacks don't apply. - `SqliteMediaStore`: `SELECT metadata FROM <table> WHERE sha256=?` + `json.loads`. Raises `FileNotFoundError` for unknown URIs. - `S3MediaStore`: HEAD + collects `x-amz-meta-*` response headers, strips the prefix. Reuses the existing 404 / non-2xx error shape. Drops `key_strategy=` from `SqliteMediaStore`. The digest is the primary key by content-addressing construction — a user-chosen key would either break dedup or be a no-op. Kept on Disk + S3 where bucket / directory layout is a real concern. README + branch-context entries updated to reflect: all three stores round-trip metadata; key_strategy is Disk + S3 only. Coverage stays at 100% branch. * fix(media): reject absolute paths from DiskMediaStore key_strategy `Path(root) / absolute_path` returns `absolute_path` — the root is silently discarded — so a custom `key_strategy` returning `/etc/passwd` (or similar) escapes the store directory even though the previous check only blocked `..`. Tighten the validator to reject both shapes. Caught by pydanty during its #251 integration review. * fix(step_persistence): dedupe terminal snapshot; document sqlite thread-affinity The terminal CallToolsNode already saves the final provider-valid snapshot with the correct step_index. after_run was re-saving the same tail stamped with step_index=0 (ctx.run_step is reset by then), so latest_snapshot reported a misleading step and every run wrote a duplicate. Track whether a node snapshot was taken via a task-local ContextVar and make after_run a fallback that only fires when the run reached no provider-valid boundary. Also document that a caller-owned sqlite connection= must set check_same_thread=False (store SQL runs on anyio worker threads), on both SqliteStepStore and SqliteMediaStore, and correct the WAL-on-every-connection claim. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(media): SigV4 wire path matches signed path; walker preserves unknown fields S3MediaStore signed `_canonical_uri(path)` (each segment percent-encoded) but sent the raw path, letting httpx apply looser encoding. A custom key_prefix / key_strategy emitting reserved chars (`@`, `(`, `=`, ...) diverged from the signed path -> SignatureDoesNotMatch. Send the canonical bytes via httpx `raw_path` so signer and sender agree. Default `<hex>.bin` keys are unaffected. The externalize/restore walker hardcoded the BinaryContent key set, silently dropping any field pydantic_ai adds upstream. Copy the node and swap only `data` <-> marker keys so unknown fields round-trip. Adds tests for reserved-char path agreement, unknown-field preservation, and restore over a pruned blob. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: gate step_persistence + media behind experimental namespace Move both packages under `pydantic_ai_harness.experimental`, matching the convention introduced for compaction/planning/subagents (#191): the old `pydantic_ai_harness.step_persistence` / `.media` paths and the top-level re-exports are gone, so the only import path is now from pydantic_ai_harness.experimental.step_persistence import StepPersistence from pydantic_ai_harness.experimental.media import S3MediaStore Both package __init__s call `warn_experimental(...)`, so importing either emits a `HarnessExperimentalWarning` (silenced category-wide by one filter). This keeps us from committing to a public surface before the capability has real usage. README gains the standard experimental banner; warning tests cover both new packages. 100% branch coverage retained. * docs(media): fix metadata/key_strategy drift, convert em-dashes Subagent review of the experimental move surfaced doc drift: - README "Persistence by store" implied `get_metadata` returns `media_type`; it returns only the user `metadata` mapping. Reworded. - `KeyStrategy` docstring still listed `SqliteMediaStore` / "DB primary key" as a user; sqlite has no `key_strategy`. Dropped it. - README understated `DiskMediaStore` traversal protection (it rejects absolute paths as well as `..`). - README had a stale paragraph that read as `key_strategy` but described `public_url`, and omitted S3. Rewritten to name `public_url` and all three stores. - README `MediaContext` method list now includes `get_metadata`. - Converted em-dashes to `--` across the step_persistence + media trees per the writing-style rule (#270). Other experimental packages' pre-existing em-dashes are left for a separate sweep. No code behavior change; 100% branch coverage retained. * docs(experimental): add warning banner to planning/subagents, finish em-dash sweep Uniformity pass across the experimental packages: - Add the standard `[!WARNING]` experimental banner to planning/README.md and subagents/README.md (compaction and step_persistence already had it; these two lacked it). - Convert remaining em-dashes to `--` in the compaction package and the shared `_warn.py`, per the writing-style rule (#270). The whole experimental source tree is now em-dash-free. Docs only; 100% branch coverage retained. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: David SF <david.sanchez@pydantic.dev> |
||
|
David SFandGitHub
|
d85b6b2031 | Guide agents building harness capabilities (#247) | ||
|
David SFandGitHub
|
518ae87f90 | Allow fresh PydanticAI package installs (#238) | ||
|
DavidandGitHub
|
31ae664b3e | Public launch infrastructure: docs, GitHub infra, template (#165) | ||
|
David Sanchez
|
27849b336d | first readme | ||
|
David Sanchez
|
72a1052efb | initial commit |