ci: add pydantic-ai v2 beta early-warning job and make code_mode v1/v2 compatible (#291)

* ci: add non-blocking pydantic-ai v2 beta test job

The harness targets the pydantic-ai v1 line, but the `>=1.105.0` floor also
admits v2 prereleases once prerelease resolution is enabled. Nothing in CI
exercised that path, so v2-breaking changes were invisible until release.

This adds a `test-v2-beta` job that resolves the latest v2 beta and runs the
suite against it. It surfaces real breakage today: v2 dropped the `calls`
argument from `ToolManager.get_parallel_execution_mode`, which CodeMode still
calls with the v1 signature, so `code_mode` raises `TypeError` under v2.

The job is intentionally kept out of the `check` gate so an expected red
result on the unsupported v2 line never blocks a merge.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci: cap the v2-beta pytest step so a hang fails fast

The v2 beta job is expected to surface breakage, but some v2 breaks hang
instead of failing cleanly. When code mode raises an unhandled exception
inside a DBOS workflow, DBOS's background recovery thread stays alive and the
Python process never exits, so the step rode to the 20-minute job timeout and
burned a full runner slot on every push.

Wrap the pytest invocation in `timeout -k 30 300` so any such hang fails fast
(exit 124). A per-test timeout plugin would not help: the hang happens during
interpreter shutdown, after the test body completes, so the cap has to be on
the process.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(code_mode): call get_parallel_execution_mode compatibly across v1/v2

pydantic-ai v2 (#5339, shipped in 2.0.0b1) dropped the `calls` argument from
`ToolManager.get_parallel_execution_mode`: it now reads the run-scoped mode
from a context var and applies per-tool `sequential` barriers separately. The
harness passed `[]` specifically to isolate the context var from per-tool
flags, which is exactly what the no-arg v2 call returns, so the two are
equivalent.

Inspect the method arity and call the matching shape. Inspecting rather than
catching TypeError avoids swallowing a genuine TypeError raised inside the
method. The `Callable[...]` annotation erases the bound signature so both call
shapes typecheck whichever major's stubs pyright resolves.

Without this, every code_mode run under v2 raises TypeError; inside a DBOS
workflow that unhandled error also wedged the process (a non-daemon recovery
thread blocked interpreter shutdown), which is what hung the v2-beta CI job.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(code_mode): cover both arities of the v1/v2 mode dispatch

The arity-based dispatch added an `else` branch that the v1-pinned coverage
gate never executes (the v2 no-arg call only runs under pydantic-ai v2), so
total branch coverage fell to 99% and failed the `fail_under=100` gate.

Extract the dispatch into `_global_mode_is_sequential` and unit-test both call
shapes directly, so both branches are exercised whichever major is installed.
This is honest coverage rather than a `# pragma: no cover` that would hide a
branch that does run (in the v2-beta job).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci: deselect v1-only OTel assertions from the v2-beta job

Two instrumentation tests in test_managed_prompt.py assert pydantic-ai v1's
OpenTelemetry attribute and span names. v2 deliberately renamed these
(aggregated-usage attributes, GenAI-semconv span names), so the tests are
expected-red on v2 and carry no signal in this job. Deselecting them keeps the
v2-beta job a meaningful early-warning for capability breakage (code mode,
durable execution) instead of going red on documented instrumentation drift.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: keep managed prompt v2 signal

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: David SF <david.sanchez@pydantic.dev>

.github/workflows/main.yml

@@ -148,6 +148,69 @@ jobs:
       - run: uv sync --locked --group dev --all-extras
       - run: uv run --no-sync pytest
 
+  # Early-warning signal for the pydantic-ai v2 beta line. The harness targets
+  # the v1 line today (the `>=1.105.0` floor and the lock both resolve to v1),
+  # but that `>=` constraint also admits the v2 prereleases once prerelease
+  # resolution is enabled. This job pins the latest v2 beta so we see v2
+  # breakage as it lands -- e.g. v2 dropped the `calls` argument from
+  # `ToolManager.get_parallel_execution_mode`, which CodeMode still calls with
+  # the v1 signature -- rather than discovering it the day v2 is released.
+  #
+  # Deliberately absent from the `check` gate below: v2 is unreleased and the
+  # harness does not yet claim to support it, so a red result here is expected
+  # and must not block merges. It is a visible signal, not a required check.
+  test-v2-beta:
+    runs-on: ubuntu-latest
+    name: test on pydantic-ai v2 beta
+    timeout-minutes: 20
+    # The whole point of the job is to resolve v2 prereleases; allow them for
+    # every uv invocation in this job rather than threading flags per step.
+    env:
+      UV_PRERELEASE: allow
+    steps:
+      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          persist-credentials: false
+
+      - uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
+        with:
+          python-version: '3.14'
+          enable-cache: true # zizmor: ignore[cache-poisoning] -- Job does not produce release artifacts and does not have sensitive permissions
+          cache-suffix: v2-beta
+
+      # `env -u UV_LOCKED`: same as `test-latest` -- under the workflow-level
+      # `UV_LOCKED=1`, `uv lock` would assert the lockfile is current instead of
+      # applying the upgrade, so this step would fail the moment a newer v2 beta
+      # exists rather than testing it.
+      - run: |
+          env -u UV_LOCKED uv lock \
+            --prerelease allow \
+            --upgrade-package pydantic-ai-slim \
+            --upgrade-package pydantic-graph
+      - run: uv sync --locked --group dev --all-extras
+      # A v2 break can hang instead of failing cleanly: e.g. an unhandled
+      # exception raised inside a DBOS workflow leaves DBOS's background
+      # (non-daemon) recovery thread alive, so the Python process never exits
+      # and the step would otherwise ride to the 20-minute job timeout. The
+      # `timeout` wrapper caps the step so any such hang fails fast (exit 124)
+      # rather than burning a full runner slot. `-k 30` escalates to SIGKILL if
+      # pytest ignores the initial SIGTERM. A per-test plugin (pytest-timeout)
+      # would not catch this -- the hang happens after the test body finishes,
+      # during interpreter shutdown -- so the cap has to be on the process.
+      #
+      # Two instrumentation assertions depend on pydantic-ai v1's OpenTelemetry
+      # attribute and span names (`gen_ai.usage.*`, span `agent run`). v2 deliberately
+      # renamed these (aggregated-usage attributes, and GenAI-semconv span names
+      # such as `invoke_agent {name}`), so they are expected-red on v2 and carry
+      # no signal here. Deselect them so this job stays a meaningful early-warning
+      # for capability breakage (e.g. code mode) rather than documented
+      # instrumentation drift. If either test is renamed the deselect stops
+      # matching and the job goes red, which is the right prompt to revisit.
+      - run: |
+          timeout -k 30 300 uv run --no-sync pytest \
+            --deselect "tests/logfire_variables/test_managed_prompt.py::test_baggage_propagates_to_run_and_child_spans" \
+            --deselect "tests/logfire_variables/test_managed_prompt.py::test_provider_backed_resolution_tags_v1_instrumentation_spans"
+
   coverage:
     needs: [test]
     runs-on: ubuntu-latest

pydantic_ai_harness/code_mode/_toolset.py

@@ -3,6 +3,7 @@
 from __future__ import annotations
 
 import asyncio
+import inspect
 import keyword
 import re
 import warnings
@@ -22,7 +23,7 @@ from pydantic_ai.messages import (
     ToolReturnPart,
     is_multi_modal_content,
 )
-from pydantic_ai.tool_manager import ToolManager
+from pydantic_ai.tool_manager import ParallelExecutionMode, ToolManager
 from pydantic_ai.tools import AgentDepsT, ToolDenied, ToolSelector, matches_tool_selector
 from pydantic_ai.toolsets.abstract import SchemaValidatorProt, ToolsetTool
 
@@ -196,6 +197,25 @@ def _sanitize_tool_name(name: str) -> str:
     return sanitized or '_'
 
 
+def _global_mode_is_sequential(get_mode: Callable[..., ParallelExecutionMode]) -> bool:
+    """Whether the run-scoped execution mode forces sandbox tool calls to run sequentially.
+
+    pydantic-ai v1's `get_parallel_execution_mode` took the pending calls list
+    and folded per-tool `sequential` flags into the result; v2 dropped the
+    argument and returns only the run-scoped context-var mode. Passing `[]` in
+    v1 isolated that context var from per-tool flags, which is exactly what the
+    no-arg v2 call returns, so the two are equivalent.
+
+    Inspect the arity rather than catch `TypeError` so a genuine `TypeError`
+    raised inside the method is not swallowed. The `Callable[...]` parameter
+    type erases the bound signature so both call shapes typecheck whichever
+    major's stubs pyright resolves.
+    """
+    if inspect.signature(get_mode).parameters:
+        return get_mode([]) != 'parallel'
+    return get_mode() != 'parallel'
+
+
 @dataclass(kw_only=True)
 class _RunCodeTool(ToolsetTool[AgentDepsT]):
     """ToolsetTool subclass that caches data computed during `get_tools`.
@@ -428,7 +448,7 @@ class CodeModeToolset(WrapperToolset[AgentDepsT]):
         #   to isolate the context var from per-tool flags.
         # - sequential_tools: per-tool `sequential` flags on ToolDefinition.
         #   These tools are rendered as `def` (sync) and resolved inline.
-        global_sequential = tool_manager.get_parallel_execution_mode([]) != 'parallel'
+        global_sequential = _global_mode_is_sequential(tool_manager.get_parallel_execution_mode)
         sequential_tools = {name for name, td in callable_defs.items() if td.sequential}
 
         # Collect nested tool calls and returns keyed by tool_call_id so they

tests/code_mode/test_code_mode.py

@@ -20,7 +20,9 @@ from pydantic_ai import (
     ToolDefinition,
 )
 from pydantic_ai.exceptions import ModelRetry
+from pydantic_ai.messages import ToolCallPart
 from pydantic_ai.models.test import TestModel
+from pydantic_ai.tool_manager import ParallelExecutionMode
 from pydantic_ai.toolsets.abstract import ToolsetTool
 from pydantic_ai.toolsets.function import FunctionToolset
 from pydantic_ai.usage import RunUsage
@@ -33,6 +35,7 @@ from pydantic_ai_harness.code_mode import CodeModeToolset
 from pydantic_ai_harness.code_mode._toolset import (  # pyright: ignore[reportPrivateUsage]
     _SEARCH_TOOLS_MODIFIER,
     _TOOL_SEARCH_ADDENDUM,
+    _global_mode_is_sequential,
     _PrintCapture,
     _sanitize_tool_name,
 )
@@ -2494,3 +2497,32 @@ def _search_tool_def(description: str = 'Search for tools.') -> ToolDefinition:
         description=description,
         parameters_json_schema={'type': 'object', 'properties': {'keywords': {'type': 'string'}}},
     )
+
+
+class TestGlobalModeIsSequential:
+    """`_global_mode_is_sequential` dispatches across pydantic-ai v1 and v2.
+
+    v1's `get_parallel_execution_mode` takes the pending calls list; v2 dropped
+    the argument. The helper inspects arity and calls the matching shape, so
+    both code paths are exercised here regardless of which major is installed.
+    """
+
+    def test_v1_signature_with_calls_argument(self) -> None:
+        def parallel(calls: list[ToolCallPart]) -> ParallelExecutionMode:
+            return 'parallel'
+
+        def sequential(calls: list[ToolCallPart]) -> ParallelExecutionMode:
+            return 'sequential'
+
+        assert _global_mode_is_sequential(parallel) is False
+        assert _global_mode_is_sequential(sequential) is True
+
+    def test_v2_signature_without_arguments(self) -> None:
+        def parallel() -> ParallelExecutionMode:
+            return 'parallel'
+
+        def sequential() -> ParallelExecutionMode:
+            return 'sequential'
+
+        assert _global_mode_is_sequential(parallel) is False
+        assert _global_mode_is_sequential(sequential) is True

tests/logfire_variables/test_managed_prompt.py

@@ -428,8 +428,8 @@ async def test_resolved_property_exposes_active_resolution() -> None:
     assert capability.resolved is None
 
 
-async def test_provider_backed_resolution_uses_remote_value_and_label(capfire: CaptureLogfire) -> None:
-    config = VariablesConfig(
+def _remote_prompt_config() -> VariablesConfig:
+    return VariablesConfig(
         variables={
             'prompt__remote_slug': VariableConfig(
                 name='prompt__remote_slug',
@@ -439,11 +439,11 @@ async def test_provider_backed_resolution_uses_remote_value_and_label(capfire: C
             )
         }
     )
-    with _variables_provider_configured(capfire, config):
-        agent = Agent(
-            TestModel(),
-            capabilities=[ManagedPrompt('remote_slug', default='fallback', label='production'), Instrumentation()],
-        )
+
+
+async def test_provider_backed_resolution_uses_remote_value_and_label(capfire: CaptureLogfire) -> None:
+    with _variables_provider_configured(capfire, _remote_prompt_config()):
+        agent = Agent(TestModel(), capabilities=[ManagedPrompt('remote_slug', default='fallback', label='production')])
 
         result = await agent.run('hello')
 
@@ -455,6 +455,18 @@ async def test_provider_backed_resolution_uses_remote_value_and_label(capfire: C
     assert resolution['attributes']['reason'] == 'resolved'
     assert resolution['attributes']['value'] == '"You are the PRODUCTION prompt."'
     assert resolution['attributes']['label'] == 'production'
+
+
+async def test_provider_backed_resolution_tags_v1_instrumentation_spans(capfire: CaptureLogfire) -> None:
+    with _variables_provider_configured(capfire, _remote_prompt_config()):
+        agent = Agent(
+            TestModel(),
+            capabilities=[ManagedPrompt('remote_slug', default='fallback', label='production'), Instrumentation()],
+        )
+
+        await agent.run('hello')
+
+    spans = capfire.exporter.exported_spans_as_dict()
     # Child spans are tagged with the resolved label via baggage.
     tagged = {s['name'] for s in spans if s['attributes'].get('logfire.variables.prompt__remote_slug') == 'production'}
     assert {'agent run', 'chat test'} <= tagged