mirror of
https://github.com/furyhawk/deepresearch.git
synced 2026-07-20 09:47:59 +00:00
Add detailed research report on agentic AI breakthroughs and session management updates
This commit is contained in:
@@ -0,0 +1,125 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
uv sync
|
||||
|
||||
# Install with PDF/HTML export support
|
||||
uv sync --extra export
|
||||
|
||||
# Run the server (opens on http://localhost:8080)
|
||||
uv run deepresearch
|
||||
|
||||
# Start Excalidraw canvas (separate terminal)
|
||||
docker compose up -d excalidraw-canvas
|
||||
|
||||
# Run without Excalidraw
|
||||
EXCALIDRAW_ENABLED=0 uv run deepresearch
|
||||
```
|
||||
|
||||
No tests, linters, or formatters are configured in this project.
|
||||
|
||||
## Environment
|
||||
|
||||
Copy `.env.example` to `.env` and configure:
|
||||
- `OPENAI_API_KEY` or `ANTHROPIC_API_KEY` — LLM provider
|
||||
- At least one search API key: `TAVILY_API_KEY`, `BRAVE_API_KEY`, or `JINA_API_KEY`
|
||||
- Default model is `openai-responses:o4-mini` (override via `MODEL_NAME`)
|
||||
- When `LLM_BASE_URL` is set, uses a local OpenAI-compatible endpoint instead
|
||||
|
||||
## Architecture
|
||||
|
||||
**Stack:** Python 3.12+ / FastAPI / WebSocket / pydantic-ai + pydantic-deep
|
||||
|
||||
### Source Layout
|
||||
|
||||
```
|
||||
src/deepresearch/
|
||||
app.py # FastAPI server, WebSocket /ws/chat, REST endpoints
|
||||
agent.py # Agent factory — assembles the research agent with all toolsets
|
||||
config.py # MCP server creation, model config, path constants
|
||||
prompts.py # System prompt for the research agent (~300 lines of instructions)
|
||||
middleware.py # Three capabilities: AuditCapability, PermissionCapability, RateLimitRetryCapability
|
||||
todo_toolset.py # ForgiveWriteTodosCapability — handles malformed tool calls from local LLMs
|
||||
types.py # Pydantic models: Source, Finding, ReportSection, ResearchReport
|
||||
static/
|
||||
index.html # Single-page frontend
|
||||
app.js # WebSocket client, tool rendering, file preview, Excalidraw iframe
|
||||
styles.css # Dark theme UI
|
||||
skills/
|
||||
research-methodology/SKILL.md # Loaded agent skill: source evaluation, search strategy
|
||||
report-writing/SKILL.md # Loaded agent skill: report structure, citation format
|
||||
diagram-design/SKILL.md # Loaded agent skill: Excalidraw diagram best practices
|
||||
workspace/
|
||||
DEEP.md # Context file injected into every session (file organization conventions)
|
||||
MEMORY.md # Persistent agent memory (written via `remember()` tool)
|
||||
workspaces/ # Per-session data: history.json, meta.json, events.jsonl, canvas.json
|
||||
```
|
||||
|
||||
### Runtime Flow
|
||||
|
||||
1. **Lifespan startup** (`app.py:lifespan`): Creates MCP servers (Tavily, Brave, Jina, Excalidraw, Playwright, Firecrawl per API keys in env) and the research agent. Retry loop drops failing MCP servers and restarts.
|
||||
|
||||
2. **WebSocket session** (`/ws/chat`): Each browser tab connects with a session UUID. A per-user Docker sandbox container is created (python-datascience runtime). Message history persists to `workspaces/<uuid>/history.json`.
|
||||
|
||||
3. **Agent execution** (`run_agent_with_streaming`): Uses `agent.iter()` for streaming — processes nodes (ModelRequest → stream text deltas, CallTools → stream tool calls/results) and sends WebSocket JSON events to the frontend.
|
||||
|
||||
4. **Approval flow**: Tools like `execute` trigger `DeferredToolRequests` — the frontend shows an approval dialog, and the agent resumes with `DeferredToolResults`.
|
||||
|
||||
5. **Checkpointing**: Auto-saves after every turn to `InMemoryCheckpointStore`. Supports `rewind` and `fork` via REST endpoints.
|
||||
|
||||
### Agent Architecture
|
||||
|
||||
The agent (`create_research_agent` in `agent.py`) uses **pydantic-deep**'s `create_deep_agent` with:
|
||||
|
||||
- **Toolsets**: MCP servers (search/browser/diagrams) + filesystem + execute + subagents + teams + TODO + skills + checkpoints + agent-factory + remember
|
||||
- **Subagents**: `general-purpose` (research executor), `planner` (plan mode), `code-reviewer`, plus dynamic agents via factory
|
||||
- **Hooks**: `audit_logger` (POST_TOOL_USE, background) + `safety_gate` (PRE_TOOL_USE on `execute`, blocks rm -rf /, mkfs, dd, etc.)
|
||||
- **Middleware stack**: `ForgiveWriteTodosCapability` → `AuditCapability` → `PermissionCapability` → optionally `RateLimitRetryCapability`
|
||||
- **Context management**: Sliding window (50 msg trigger, keep 30), token eviction (20k limit), context files from `/workspace/DEEP.md` and `/workspace/MEMORY.md`
|
||||
|
||||
### MCP Servers (config.py)
|
||||
|
||||
All optional, enabled by API key presence:
|
||||
- **Tavily** — AI search (`npx tavily-mcp`)
|
||||
- **Brave Search** — web search (`npx @anthropic-ai/brave-search-mcp`)
|
||||
- **Jina** — URL→markdown reader (HTTP MCP at `https://mcp.jina.ai/v1`)
|
||||
- **Excalidraw** — live diagrams (Podman container, requires `podman` binary)
|
||||
- **Playwright** — browser automation (requires `PLAYWRIGHT_MCP=1`)
|
||||
- **Firecrawl** — web scraping/crawling (`npx firecrawl-mcp`)
|
||||
|
||||
### Research Workflow (prompts.py)
|
||||
|
||||
The system prompt encodes a detailed 7-step research workflow:
|
||||
1. **Plan** — dispatch planner subagent for complex topics
|
||||
2. **Todo** — create progress tracking with `write_todos`
|
||||
3. **Parallel subagents** — dispatch all research tasks async
|
||||
4. **Wait** — `wait_tasks` for results
|
||||
5. **Handle failures** — never stop, use own knowledge as fallback
|
||||
6. **Write iteratively** — chapter by chapter via `edit_file`
|
||||
7. **Present** — summarize and offer deeper analysis
|
||||
|
||||
### REST Endpoints
|
||||
|
||||
| Endpoint | Purpose |
|
||||
|---|---|
|
||||
| `GET /` | Serves `static/index.html` |
|
||||
| `GET /health` | Health check with agent/session status |
|
||||
| `GET /config` | Full feature configuration |
|
||||
| `POST /upload` | File upload to sandbox |
|
||||
| `GET /files?session_id=` | List files in sandbox |
|
||||
| `GET /files/content/...` | Read file content |
|
||||
| `GET /files/binary/...` | Read binary files (images) |
|
||||
| `GET /todos?session_id=` | Get current TODOs |
|
||||
| `GET /checkpoints?session_id=` | List checkpoints |
|
||||
| `POST /checkpoints/:id/rewind` | Rewind to checkpoint |
|
||||
| `POST /checkpoints/:id/fork` | Fork new session from checkpoint |
|
||||
| `GET /history?session_id=` | Get conversation history |
|
||||
| `GET /sessions` | List all sessions |
|
||||
| `DELETE /sessions/:id` | Delete session |
|
||||
| `GET /export/:fmt` | Export report (md/html/pdf) |
|
||||
| `GET /preview/:session_id/...` | Serve raw files from container |
|
||||
@@ -3773,3 +3773,7 @@
|
||||
{"type": "checkpoint_saved", "checkpoint_id": "f31d3b56-1b3b-4e5d-a806-234f167e932e", "label": "turn-9", "turn": 9, "message_count": 33, "metadata": {}, "_ts": "2026-06-17T08:41:59.812102+00:00"}
|
||||
{"type": "response", "content": "I have completed the research on agentic AI breakthroughs for the past three months (March\u2013June 2026). The high-level overview highlights a significant shift toward **Coordination Engineering** and **Full-Horizon Planning**.\n\n### Key Breakthroughs Identified:\n* **Planning Horizons:** A transition from \"Single-Step\" to **Full-Horizon (FH) Planning**, which allows agents to generate complete multi-step plans upfront, significantly increasing token efficiency for data-centric tasks.\n* **Agentic Browsing:** The emergence of **agentic browsers** (e.g., Perplexity's Comet, OpenAI Operator) that interact with websites semantically and visually rather than relying on brittle scripts.\n* **Memory as Ontology:** A paradigm shift where agent memory is treated as a persistent digital identity governed by a **Constitutional Memory Architecture (CMA)**, featuring stability-based storage tiers.\n* **Swarm Intelligence:** The rise of **Portable Swarm Skills**, allowing complex multi-agent coordination protocols to be shared across different frameworks like AutoGen and LangGraph without custom adapters.\n* **Self-Evolution:** New methods like **Trajectory Distillation** allow agent teams to automatically improve their own coordination patterns by analyzing successful execution logs.\n\nThe full report, including specific model details (GPT-5.2-high, Kimi K2 Thinking) and academic citations, has been saved to:\n` /workspace/report.md`", "_ts": "2026-06-17T08:41:59.925824+00:00"}
|
||||
{"type": "done", "_ts": "2026-06-17T08:41:59.925901+00:00"}
|
||||
{"type": "session_created", "session_id": "fa1132cd-34db-4576-aa0c-9ea7c6e8b923", "_ts": "2026-06-17T08:45:42.269188+00:00"}
|
||||
{"type": "canvas_ready", "session_id": "fa1132cd-34db-4576-aa0c-9ea7c6e8b923", "_ts": "2026-06-17T08:45:42.471606+00:00"}
|
||||
{"type": "session_created", "session_id": "fa1132cd-34db-4576-aa0c-9ea7c6e8b923", "_ts": "2026-06-17T08:45:42.934413+00:00"}
|
||||
{"type": "canvas_ready", "session_id": "fa1132cd-34db-4576-aa0c-9ea7c6e8b923", "_ts": "2026-06-17T08:45:42.934787+00:00"}
|
||||
|
||||
Reference in New Issue
Block a user