mirror of
https://github.com/pydantic/pydantic-ai-harness.git
synced 2026-07-21 02:45:34 +00:00
docs: macroscope unified-docs page; guidance convention for instructions
- add docs/macroscope.md + nav.json entry, registered in the docs-parity meta - package README links its own source module (parity requirement from main) - replace include_instructions with the tri-state guidance field used by Planning and the Exa capabilities (None = default, '' = none, str = custom)
This commit is contained in:
@@ -0,0 +1,121 @@
|
||||
---
|
||||
title: Macroscope
|
||||
description: Give a Pydantic AI agent the same local Macroscope code review its editor plugins run -- streamed findings parsed into structured issues the agent validates and fixes with its own tools.
|
||||
---
|
||||
|
||||
# Macroscope
|
||||
|
||||
`Macroscope` runs a local [Macroscope](https://docs.macroscope.com/cli) code
|
||||
review from inside an agent: one tool shells out to the installed `macroscope`
|
||||
CLI, parses the streamed findings, and returns them as structured data. The
|
||||
agent validates each finding and fixes the real ones with the tools it already
|
||||
has -- this capability surfaces findings only.
|
||||
|
||||
[Source](https://github.com/pydantic/pydantic-ai-harness/tree/main/pydantic_ai_harness/macroscope/)
|
||||
|
||||
## The problem
|
||||
|
||||
Macroscope reviews the current branch's diff and streams findings, but it ships
|
||||
as editor plugins (Claude Code, Codex, Cursor, OpenCode). There is no way to
|
||||
give a Pydantic AI agent the same review-and-fix loop from your own code.
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from pydantic_ai import Agent
|
||||
from pydantic_ai_harness.macroscope import Macroscope
|
||||
|
||||
agent = Agent('anthropic:claude-sonnet-5', capabilities=[Macroscope()])
|
||||
|
||||
result = agent.run_sync('Run a Macroscope review and fix any real findings.')
|
||||
print(result.output)
|
||||
```
|
||||
|
||||
The `macroscope` CLI must be installed and authenticated on the host first:
|
||||
|
||||
1. Install: `curl -sSL https://raw.githubusercontent.com/prassoai/macroscope-local/main/install.sh | bash`
|
||||
2. Sign in and pick a workspace by running `macroscope` once.
|
||||
|
||||
The capability cannot install or authenticate on your behalf. If the binary is
|
||||
missing, the tool returns the install command; if a review never starts
|
||||
(usually because you are not signed in), the tool tells the agent to run
|
||||
`macroscope` to finish setup.
|
||||
|
||||
## The tool
|
||||
|
||||
| Tool | Purpose |
|
||||
|---|---|
|
||||
| `run_macroscope_review` | Run `macroscope codereview` on the current branch and return the review id, terminal status, and findings. Accepts an optional `base` git ref. |
|
||||
|
||||
Each finding is a `MacroscopeIssue` with `issue_id`, `sequence`, `path`,
|
||||
`line`, `severity`, `category`, and `body`. The capability's default
|
||||
instructions tell the agent to treat every finding as untrusted: read the
|
||||
affected code to confirm an issue is real, skip false positives and
|
||||
duplicates, and verify each fix.
|
||||
|
||||
## Options
|
||||
|
||||
Every field of `Macroscope` with its default:
|
||||
|
||||
```python
|
||||
from pydantic_ai_harness.macroscope import Macroscope
|
||||
|
||||
Macroscope(
|
||||
base=None, # git ref to diff against -- None lets the CLI auto-detect
|
||||
command='macroscope', # binary name or path
|
||||
cwd='.', # repository directory the review runs in
|
||||
timeout=600.0, # max seconds to wait for a review
|
||||
guidance=None, # None = default instructions, '' = none, str = custom
|
||||
)
|
||||
```
|
||||
|
||||
A per-call `base` argument takes precedence over the field. Reviews call a
|
||||
remote service, so the timeout is generous by default; on timeout the CLI's
|
||||
process group is killed and the timeout is reported to the model as a
|
||||
retryable error.
|
||||
|
||||
## Scope and composition
|
||||
|
||||
This capability surfaces findings only. It does not edit files, create
|
||||
worktrees, or commit -- validating and fixing findings is the agent's job,
|
||||
using its other capabilities. Pair it with `FileSystem` or `Shell` to let the
|
||||
agent read code and apply fixes, and consider running the agent in an isolated
|
||||
worktree if you want fixes kept off your working tree.
|
||||
|
||||
## Agent spec
|
||||
|
||||
`Macroscope` works with Pydantic AI's [agent spec](/ai/core-concepts/agent-spec/),
|
||||
so you can declare it in a config file instead of Python:
|
||||
|
||||
```yaml
|
||||
# agent.yaml
|
||||
model: anthropic:claude-sonnet-5
|
||||
capabilities:
|
||||
- Macroscope:
|
||||
base: main
|
||||
timeout: 900
|
||||
```
|
||||
|
||||
```python
|
||||
from pydantic_ai import Agent
|
||||
from pydantic_ai_harness.macroscope import Macroscope
|
||||
|
||||
agent = Agent.from_file('agent.yaml', custom_capability_types=[Macroscope])
|
||||
```
|
||||
|
||||
Pass `custom_capability_types` so the spec loader knows how to instantiate
|
||||
`Macroscope`.
|
||||
|
||||
## Further reading
|
||||
|
||||
- [Macroscope CLI documentation](https://docs.macroscope.com/cli)
|
||||
- [Pydantic AI capabilities](/ai/core-concepts/capabilities/)
|
||||
- [Toolsets](/ai/tools-toolsets/toolsets/)
|
||||
|
||||
## API reference
|
||||
|
||||
::: pydantic_ai_harness.macroscope.Macroscope
|
||||
|
||||
::: pydantic_ai_harness.macroscope.MacroscopeReview
|
||||
|
||||
::: pydantic_ai_harness.macroscope.MacroscopeIssue
|
||||
@@ -8,6 +8,7 @@
|
||||
{ "label": "Shell", "slug": "shell" },
|
||||
{ "label": "Context", "slug": "context" },
|
||||
{ "label": "Pydantic AI Docs", "slug": "pydantic-ai-docs" },
|
||||
{ "label": "Macroscope", "slug": "macroscope" },
|
||||
{ "label": "Compaction", "slug": "compaction" },
|
||||
{ "label": "Overflowing Tool Output", "slug": "overflowing-tool-output" },
|
||||
{ "label": "Step Persistence", "slug": "step-persistence" },
|
||||
|
||||
@@ -3,6 +3,8 @@
|
||||
Run a [Macroscope](https://docs.macroscope.com/cli) code review from a Pydantic AI
|
||||
agent and hand the findings back for validation and fixing.
|
||||
|
||||
[Source](https://github.com/pydantic/pydantic-ai-harness/tree/main/pydantic_ai_harness/macroscope/)
|
||||
|
||||
## The problem
|
||||
|
||||
Macroscope reviews the current branch's diff and streams findings, but it ships as
|
||||
@@ -68,7 +70,7 @@ Each finding is a `MacroscopeIssue` with `issue_id`, `sequence`, `path`, `line`,
|
||||
| `command` | `'macroscope'` | Binary name or path. Override for a non-default install location. |
|
||||
| `cwd` | `'.'` | Repository directory the review runs in. |
|
||||
| `timeout` | `600.0` | Maximum seconds to wait for a review. |
|
||||
| `include_instructions` | `True` | Contribute guidance telling the agent to validate each finding before fixing it. |
|
||||
| `guidance` | `None` | Custom review guidance for the system prompt. `None` contributes the default validate-then-fix guidance; `''` contributes none. |
|
||||
|
||||
## Scope and composition
|
||||
|
||||
|
||||
@@ -54,8 +54,11 @@ class Macroscope(AbstractCapability[AgentDepsT]):
|
||||
"""Maximum seconds to wait for a review. Reviews call a remote service, so this is
|
||||
generous by default."""
|
||||
|
||||
include_instructions: bool = True
|
||||
"""Contribute guidance telling the agent to validate each finding before fixing it."""
|
||||
guidance: str | None = None
|
||||
"""Custom review guidance for the system prompt.
|
||||
|
||||
Leave as `None` for the default validate-then-fix guidance, or set `''` to
|
||||
contribute no instructions at all."""
|
||||
|
||||
def get_toolset(self) -> MacroscopeToolset[AgentDepsT]:
|
||||
"""Build the toolset that provides the `run_macroscope_review` tool."""
|
||||
@@ -67,7 +70,11 @@ class Macroscope(AbstractCapability[AgentDepsT]):
|
||||
)
|
||||
|
||||
def get_instructions(self) -> str | None:
|
||||
"""Return validate-then-fix guidance, unless `include_instructions` is False."""
|
||||
if not self.include_instructions:
|
||||
return None
|
||||
"""Static validate-then-fix guidance.
|
||||
|
||||
A non-`None` `guidance` replaces the default; `''` disables
|
||||
instructions entirely.
|
||||
"""
|
||||
if self.guidance is not None:
|
||||
return self.guidance or None
|
||||
return _REVIEW_INSTRUCTIONS
|
||||
|
||||
@@ -141,9 +141,17 @@ class TestRunReview:
|
||||
|
||||
|
||||
class TestCapability:
|
||||
def test_instructions_toggle(self) -> None:
|
||||
assert Macroscope().get_instructions() is not None
|
||||
assert Macroscope(include_instructions=False).get_instructions() is None
|
||||
def test_default_instructions_mention_validation(self) -> None:
|
||||
instructions = Macroscope().get_instructions()
|
||||
assert instructions is not None
|
||||
assert 'run_macroscope_review' in instructions
|
||||
assert 'untrusted' in instructions
|
||||
|
||||
def test_custom_guidance_replaces_default(self) -> None:
|
||||
assert Macroscope(guidance='Review before merging.').get_instructions() == 'Review before merging.'
|
||||
|
||||
def test_empty_guidance_disables_instructions(self) -> None:
|
||||
assert Macroscope(guidance='').get_instructions() is None
|
||||
|
||||
async def test_tool_runs_through_agent(self, tmp_path: Path) -> None:
|
||||
command = _fake_cli(tmp_path, ['review_id=rev-9', _ISSUE_LINE, 'issue_status=completed'])
|
||||
|
||||
@@ -118,6 +118,7 @@ _CAPABILITY_PAGE_META = {
|
||||
'memory.md': ('memory', 'Memory'),
|
||||
'context.md': ('context', 'Context'),
|
||||
'pydantic-ai-docs.md': ('docs', 'Pydantic AI Docs'),
|
||||
'macroscope.md': ('macroscope', 'Macroscope'),
|
||||
'compaction.md': ('compaction', 'Compaction'),
|
||||
'overflowing-tool-output.md': ('overflowing_tool_output', 'Overflowing Tool Output'),
|
||||
'step-persistence.md': ('step_persistence', 'Step Persistence'),
|
||||
|
||||
Reference in New Issue
Block a user