* feat(detectors): add static blocking IO inventory
* refactor(detectors): drop superseded runtime probe; clarify static report path
- Remove the #2924 custom runtime blocking IO probe entirely:
backend/tests/support/detectors/blocking_io.py,
backend/tests/test_blocking_io_detector.py,
backend/tests/test_blocking_io_probe_integration.py, and the
pytest_addoption / pytest_runtest_call / pytest_runtest_teardown /
pytest_sessionfinish / pytest_terminal_summary hooks plus the
blocking_io_detector fixture from backend/tests/conftest.py.
Its narrow DEFAULT_BLOCKING_CALL_SPECS (time.sleep, requests, httpx,
os.walk, Path.resolve, Path.read_text, Path.write_text) cannot serve
as a CI gate; a Blockbuster-backed runtime detector will land in a
separate follow-up PR. Leaving the half-coverage probe alongside
the static inventory in this PR added a redundant detect path with
no production value.
- Address Copilot review comments on backend/README.md and
backend/CLAUDE.md by stating explicitly that the JSON report writes
to .deer-flow/blocking-io-findings.json at the repository root,
whether the target is invoked from the repo root or from backend/.
Verified: pytest tests/test_detect_blocking_io_static.py (18 passed),
ruff check + format on touched files (passed), make detect-blocking-io
from both repo root and backend/ produce the same 105-finding report
at <repo-root>/.deer-flow/blocking-io-findings.json.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(tests): add Blockbuster runtime gate for event-loop blocking IO
Adds a strict runtime gate that fails CI when sync blocking IO calls run
on the asyncio event loop thread through DeerFlow business code.
Components:
- backend/tests/support/detectors/blocking_io_runtime.py — Blockbuster
context scoped to `app.*` and `deerflow.*` so test infrastructure,
pytest internals, and third-party libraries stay silent.
- backend/tests/blocking_io/conftest.py — pytest_runtest_protocol
hookwrapper that wraps every item (setup + call + teardown) with the
strict context. Respects `@pytest.mark.allow_blocking_io` opt-out.
- backend/tests/blocking_io/test_skills_load.py — regression anchor for
the #1917 fix (asyncio.to_thread offload around
LocalSkillStorage.load_skills).
- backend/tests/blocking_io/test_sqlite_lifespan.py — regression anchor
for the #1912 fix (asyncio.to_thread offload around
ensure_sqlite_parent_dir).
- backend/tests/blocking_io/test_gate_smoke.py — meta-test asserting the
gate actually catches unoffloaded blocking IO and that the
`@pytest.mark.allow_blocking_io` opt-out works.
- backend/Makefile — `make test-blocking-io` target.
- .github/workflows/backend-blocking-io-tests.yml — hard-fail PR gate on
ubuntu-latest. Windows matrix deferred to follow-up.
Dependencies:
- blockbuster>=1.5.26,<1.6 added to dev group.
Coverage boundary (called out in PR body): the gate only catches blocking
IO on code paths the test suite actually exercises. Static AST inventory
(separate, informational) is the complementary coverage tool. Three blind
spot categories — untested paths, mocked-away paths, env-mismatched paths
— are documented in the PR description.
Findings surfaced while authoring this PR:
- resolve_sqlite_conn_str in runtime/store/_sqlite_utils.py:19 does sync
Path.resolve() -> os.path.abspath on the lifespan loop thread, ahead of
the #1912 fix. Not addressed here; tracked as follow-up.
Tests: 4 passed locally (`make test-blocking-io`).
Lint/format: clean (`ruff check` and `ruff format --check`).
* fix(tests): scope Blockbuster gate to blocking-io suite
* fix(tests): harden Blockbuster runtime gate
* test(blocking-io): add project rule extension point
* test(blocking-io): address review cleanup
langgraph-api emits → and ⚠️ characters in version-check log lines.
On Windows with cp1252 as the default stream encoding, each such line
throws a UnicodeEncodeError inside the logging handler, littering
startup output with tracebacks (though the server still boots).
#1550 already fixed this for scripts/check.py via stream.reconfigure().
Apply the same treatment to the backend Makefile dev/gateway/test
targets by setting PYTHONIOENCODING=utf-8 and PYTHONUTF8=1 before each
uv run invocation. Both variables are no-ops on Linux/macOS where UTF-8
is already the default.
Closes#2337
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
- Updated documentation and comments to reflect the transition from LangGraph Server to Gateway.
- Changed default URLs in ChannelManager and tests to point to Gateway.
- Removed references to LangGraph Server in deployment scripts and configurations.
- Updated Nginx configuration to route API traffic to Gateway.
- Adjusted frontend configurations to utilize Gateway's API.
- Removed LangGraph service from Docker Compose files, consolidating services under Gateway.
- Added regression tests to ensure Gateway integration works as expected.
Co-authored-by: Copilot <copilot@github.com>
#1623 added this flag to both Docker Compose files but missed the
backend Makefile used by `make dev`. Without it `langgraph dev`
defaults to n_jobs_per_worker=1, so all conversation runs are
serialised and concurrent requests block.
This mirrors the Docker configuration.
* refactor: extract shared utils to break harness→app cross-layer imports
Move _validate_skill_frontmatter to src/skills/validation.py and
CONVERTIBLE_EXTENSIONS + convert_file_to_markdown to src/utils/file_conversion.py.
This eliminates the two reverse dependencies from client.py (harness layer)
into gateway/routers/ (app layer), preparing for the harness/app package split.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: split backend/src into harness (deerflow.*) and app (app.*)
Physically split the monolithic backend/src/ package into two layers:
- **Harness** (`packages/harness/deerflow/`): publishable agent framework
package with import prefix `deerflow.*`. Contains agents, sandbox, tools,
models, MCP, skills, config, and all core infrastructure.
- **App** (`app/`): unpublished application code with import prefix `app.*`.
Contains gateway (FastAPI REST API) and channels (IM integrations).
Key changes:
- Move 13 harness modules to packages/harness/deerflow/ via git mv
- Move gateway + channels to app/ via git mv
- Rename all imports: src.* → deerflow.* (harness) / app.* (app layer)
- Set up uv workspace with deerflow-harness as workspace member
- Update langgraph.json, config.example.yaml, all scripts, Docker files
- Add build-system (hatchling) to harness pyproject.toml
- Add PYTHONPATH=. to gateway startup commands for app.* resolution
- Update ruff.toml with known-first-party for import sorting
- Update all documentation to reflect new directory structure
Boundary rule enforced: harness code never imports from app.
All 429 tests pass. Lint clean.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: add harness→app boundary check test and update docs
Add test_harness_boundary.py that scans all Python files in
packages/harness/deerflow/ and fails if any `from app.*` or
`import app.*` statement is found. This enforces the architectural
rule that the harness layer never depends on the app layer.
Update CLAUDE.md to document the harness/app split architecture,
import conventions, and the boundary enforcement test.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add config versioning with auto-upgrade on startup
When config.example.yaml schema changes, developers' local config.yaml
files can silently become outdated. This adds a config_version field and
auto-upgrade mechanism so breaking changes (like src.* → deerflow.*
renames) are applied automatically before services start.
- Add config_version: 1 to config.example.yaml
- Add startup version check warning in AppConfig.from_file()
- Add scripts/config-upgrade.sh with migration registry for value replacements
- Add `make config-upgrade` target
- Auto-run config-upgrade in serve.sh and start-daemon.sh before starting services
- Add config error hints in service failure messages
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix comments
* fix: update src.* import in test_sandbox_tools_security to deerflow.*
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: handle empty config and search parent dirs for config.example.yaml
Address Copilot review comments on PR #1131:
- Guard against yaml.safe_load() returning None for empty config files
- Search parent directories for config.example.yaml instead of only
looking next to config.yaml, fixing detection in common setups
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: correct skills root path depth and config_version type coercion
- loader.py: fix get_skills_root_path() to use 5 parent levels (was 3)
after harness split, file lives at packages/harness/deerflow/skills/
so parent×3 resolved to backend/packages/harness/ instead of backend/
- app_config.py: coerce config_version to int() before comparison in
_check_config_version() to prevent TypeError when YAML stores value
as string (e.g. config_version: "1")
- tests: add regression tests for both fixes
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: update test imports from src.* to deerflow.*/app.* after harness refactor
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat(subagents): make subagent timeout configurable via config.yaml
- Add SubagentsAppConfig supporting global and per-agent timeout_seconds
- Load subagents config section in AppConfig.from_file()
- Registry now applies config.yaml overrides without mutating builtin defaults
- Polling safety-net in task_tool is now dynamic (execution timeout + 60s buffer)
- Document subagents section in config.example.yaml
- Add make test command and enforce TDD policy in CLAUDE.md
- Add 38 unit tests covering config validation, timeout resolution, registry
override behavior, and polling timeout formula
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(subagents): add logging for subagent timeout config and execution
- Log loaded timeout config (global default + per-agent overrides) on startup
- Log debug message in registry when config.yaml overrides a builtin timeout
- Include timeout in executor's async execution start log
- Log effective timeout and polling limit when a task is dispatched
- Fix UnboundLocalError: move max_poll_count assignment before logger.info
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* ci(backend): add lint step and run all unit tests via Makefile
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix lint
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Add a root-level Makefile to manage frontend, backend, and nginx services:
- `make check` validates required dependencies (Node.js 22+, pnpm, uv, nginx)
- `make install` installs all project dependencies
- `make dev` starts all services with unified port 2026
- `make stop` and `make clean` for cleanup
Update nginx configuration:
- Change port from 8000 to 2026
- Add frontend upstream and routing (port 3000)
- Add /api/langgraph/* routing with path rewriting to LangGraph server
- Keep other /api/* routes to Gateway API
- Route non-API requests to frontend
Update frontend configuration:
- Use relative URLs through nginx proxy by default
- Support environment variables for direct backend access
- Construct full URL for LangGraph SDK compatibility
Clean up backend Makefile by removing nginx and serve targets.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* chore: add .claude/ to .gitignore
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat: add gateway module with FastAPI server
- Add new gateway module with FastAPI app for API routing
- Add gateway and serve commands to Makefile
- Add fastapi, httpx, uvicorn, sse-starlette dependencies
- Fix model config retrieval in lead_agent (support both model_name and model)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
- Add AioSandboxProvider for Docker-based sandbox execution with
configurable container lifecycle, volume mounts, and port management
- Add TitleMiddleware to auto-generate thread titles after first
user-assistant exchange using LLM
- Add Claude Code documentation (CLAUDE.md, AGENTS.md)
- Extend SandboxConfig with Docker-specific options (image, port, mounts)
- Fix hardcoded mount path to use expanduser
- Add agent-sandbox and dotenv dependencies
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
AochenShen99