Compare commits

..

1179 Commits

Author SHA1 Message Date
greatmengqi 27b66d6753 feat(auth): authentication module with multi-tenant isolation (RFC-001)
Introduce an always-on auth layer with auto-created admin on first boot,
multi-tenant isolation for threads/stores, and a full setup/login flow.

Backend
- JWT access tokens with `ver` field for stale-token rejection; bump on
  password/email change
- Password hashing, HttpOnly+Secure cookies (Secure derived from request
  scheme at runtime)
- CSRF middleware covering both REST and LangGraph routes
- IP-based login rate limiting (5 attempts / 5-min lockout) with bounded
  dict growth and X-Forwarded-For bypass fix
- Multi-worker-safe admin auto-creation (single DB write, WAL once)
- needs_setup + token_version on User model; SQLite schema migration
- Thread/store isolation by owner; orphan thread migration on first admin
  registration
- thread_id validated as UUID to prevent log injection
- CLI tool to reset admin password
- Decorator-based authz module extracted from auth core

Frontend
- Login and setup pages with SSR guard for needs_setup flow
- Account settings page (change password / email)
- AuthProvider + route guards; skips redirect when no users registered
- i18n (en-US / zh-CN) for auth surfaces
- Typed auth API client; parseAuthError unwraps FastAPI detail envelope

Infra & tooling
- Unified `serve.sh` with gateway mode + auto dep install
- Public PyPI uv.toml pin for CI compatibility
- Regenerated uv.lock with public index

Tests
- HTTP vs HTTPS cookie security tests
- Auth middleware, rate limiter, CSRF, setup flow coverage
2026-04-08 00:31:43 +08:00
yangzheli 636053fb6d fix(frontend): add missing rel="noopener noreferrer" to target="_blank" links (#1741)
* fix(frontend): add missing rel="noopener noreferrer" to target="_blank" links

Prevent tabnabbing attacks and referrer leakage by ensuring all
external links with target="_blank" include both noopener and
noreferrer in the rel attribute.

Made-with: Cursor

* style: fix code formatting
2026-04-02 17:32:52 +08:00
moose-lab f56d0b4869 fix(sandbox): exclude URL paths from absolute path validation (#1385) (#1419)
* fix(sandbox): URL路径被误判为不安全绝对路径 (#1385)

在本地沙箱模式下,bash工具对命令做绝对路径安全校验时,会把curl命令中的
HTTPS URL(如 https://example.com/api/v1/check)误识别为本地绝对路径并拦截。

根因:_ABSOLUTE_PATH_PATTERN 正则的负向后行断言 (?<![:\w]) 只排除了冒号和
单词字符,但 :// 中第二个斜杠前面是第一个斜杠(/),不在排除列表中,导致
//example.com/api/... 被匹配为绝对路径 /example.com/api/...。

修复:在负向后行断言中增加斜杠字符,改为 (?<![:\w/]),使得 :// 中的连续
斜杠不会触发绝对路径匹配。同时补充了URL相关的单元测试用例。

Signed-off-by: moose-lab <moose-lab@users.noreply.github.com>

* fix(sandbox): refine absolute path regex to preserve file:// defense-in-depth

Change lookbehind from (?<![:\w/]) to (?<![:\w])(?<!:/) so only the
second slash in :// sequences is excluded. This keeps URL paths from
false-positiving while still letting the regex detect /etc/passwd in
file:///etc/passwd. Also add explicit file:// URL blocking and tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Signed-off-by: moose-lab <moose-lab@users.noreply.github.com>
Co-authored-by: moose-lab <moose-lab@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-04-02 16:09:14 +08:00
Varian_米泽 a2cb38f62b fix: prevent concurrent subagent file write conflicts in sandbox tools (#1714)
* fix: prevent concurrent subagent file write conflicts

Serialize same-path str_replace operations in sandbox tools

Guard AioSandbox write_file/update_file with the existing sandbox lock

Add regression tests for concurrent str_replace and append races

Verify with backend full tests and ruff lint checks

* fix(sandbox): Fix the concurrency issue of file operations on the same path in isolated sandboxes.

Ensure that different sandbox instances use independent locks for file operations on the same virtual path to avoid concurrency conflicts. Change the lock key from a single path to a composite key of (sandbox.id, path), and add tests to verify the concurrent safety of isolated sandboxes.

* feat(sandbox): Extract file operation lock logic to standalone module and fix concurrency issues

Extract file operation lock related logic from tools.py into a separate file_operation_lock.py module.
Fix data race issues during concurrent str_replace and write_file operations.
2026-04-02 15:39:41 +08:00
dependabot[bot] 3aab2445a6 build(deps): bump aiohttp from 3.13.3 to 3.13.4 in /backend (#1750)
---
updated-dependencies:
- dependency-name: aiohttp
  dependency-version: 3.13.4
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-02 15:26:51 +08:00
knukn f8fb8d6fb1 feat/per agent skill filter (#1650)
* feat(agent): 为AgentConfig添加skills字段并更新lead_agent系统提示

在AgentConfig中添加skills字段以支持配置agent可用技能
更新lead_agent的系统提示模板以包含可用技能信息

* fix: resolve agent skill configuration edge cases and add tests

* Update backend/packages/harness/deerflow/agents/lead_agent/prompt.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* refactor(agent): address PR review comments for skills configuration

- Add detailed docstring to `skills` field in `AgentConfig` to clarify the semantics of `None` vs `[]`.
- Add unit tests in `test_custom_agent.py` to verify `load_agent_config()` correctly parses omitted skills and explicit empty lists.
- Fix `test_make_lead_agent_empty_skills_passed_correctly` to include `agent_name` in the runtime config, ensuring it exercises the real code path.

* docs: 添加关于按代理过滤技能的配置说明

在配置示例文件和文档中添加说明,解释如何通过代理的config.yaml文件限制加载的技能

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-02 15:02:09 +08:00
totoyang 2d1f90d5dc feat(tracing): add optional Langfuse support (#1717)
* feat(tracing): add optional Langfuse support

* Fix tracing fail-fast behavior for explicitly enabled providers

* fix(lint)
2026-04-02 13:06:10 +08:00
3a672b39c7 Fix/1681 llm call retry handling (#1683)
* fix(runtime): handle llm call errors gracefully

* fix(runtime): preserve graph control flow in llm retry middleware

---------

Co-authored-by: luoxiao6645 <luoxiao6645@gmail.com>
2026-04-02 10:12:17 +08:00
SHIYAO ZHANG df5339b5d0 feat(sandbox): truncate oversized bash and read_file tool outputs (#1677)
* feat(sandbox): truncate oversized bash and read_file tool outputs

Long tool outputs (large directory listings, multi-MB source files) can
overflow the model's context window. Two new configurable limits:

- bash_output_max_chars (default 20000): middle-truncates bash output,
  preserving both head and tail so stderr at the end is not lost
- read_file_output_max_chars (default 50000): head-truncates file output
  with a hint to use start_line/end_line for targeted reads

Both limits are enforced at the tool layer (sandbox/tools.py) rather
than middleware, so truncation is guaranteed regardless of call path.
Setting either limit to 0 disables truncation entirely.

Measured: read_file on a 250KB source file drops from 63,698 tokens to
19,927 tokens (69% reduction) with the default limit.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(tests): remove unused pytest import and fix import sort order

* style: apply ruff format to sandbox/tools.py

* refactor(sandbox): address Copilot review feedback on truncation feature

- strict hard cap: while-loop ensures result (including marker) ≤ max_chars
- max_chars=0 now returns "" instead of original output
- get_app_config() wrapped in try/except with fallback to defaults
- sandbox_config.py: add ge=0 validation on truncation limit fields
- config.example.yaml: bump config_version 4→5
- tests: add len(result) <= max_chars assertions, edge-case (max=0, small
  max, various sizes) tests; fix skipped-count test for strict hard cap

* refactor(sandbox): replace while-loop truncation with fixed marker budget

Use a pre-allocated constant (_MARKER_MAX_LEN) instead of a convergence
loop to ensure result <= max_chars. Simpler, safer, and skipped-char
count in the marker is now an exact predictable value.

* refactor(sandbox): compute marker budget dynamically instead of hardcoding

* fix(sandbox): make max_chars=0 disable truncation instead of returning empty string

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: JeffJiang <for-eleven@hotmail.com>
2026-04-02 09:22:41 +08:00
Admire 0eb6550cf4 fix(frontend): persist model selection per thread (#1553)
* fix(frontend): persist model selection per thread

* fix(frontend): apply thread model override on fallback

* refactor(frontend): split thread settings hook

* fix frontend local storage guards
2026-04-01 23:27:03 +08:00
LYU Yichen 0a379602b8 fix: avoid treating Feishu file paths as commands (#1654)
Feishu channel classified any slash-prefixed text (including absolute
paths such as /mnt/user-data/...) as a COMMAND, causing them to be
misrouted through the command pipeline instead of the chat pipeline.

Fix by introducing a shared KNOWN_CHANNEL_COMMANDS frozenset in
app/channels/commands.py — the single authoritative source for the set
of supported slash commands.  Both the Feishu inbound parser and the
ChannelManager's unknown-command reply now derive from it, so adding
or removing a command requires only one edit.

Changes:
- app/channels/commands.py (new): defines KNOWN_CHANNEL_COMMANDS
- app/channels/feishu.py: replace local KNOWN_FEISHU_COMMANDS with the
  shared constant; _is_feishu_command() now gates on it
- app/channels/manager.py: import KNOWN_CHANNEL_COMMANDS and use it in
  the unknown-command fallback reply so the displayed list stays in sync
- tests/test_feishu_parser.py: parametrize over every entry in
  KNOWN_CHANNEL_COMMANDS (each must yield msg_type=command) and add
  parametrized chat cases for /unknown, absolute paths, etc.

Made with Cursor

Made-with: Cursor

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-04-01 23:23:00 +08:00
Jason 1fb5acee39 fix(gateway): prevent 400 error when client sends context with configurable (#1660)
* fix(gateway): prevent 400 error when client sends context with configurable

Fixes #1290

LangGraph >= 0.6.0 rejects requests that include both 'configurable' and
'context' in the run config. If the client (e.g. useStream hook) sends
a 'context' key, we now honour it and skip creating our own
'configurable' dict to avoid the conflict.

When no 'context' is provided, we fall back to the existing
'configurable' behaviour with thread_id.

* fix(gateway): address review feedback — warn on dual keys, fix runtime injection, add tests

- Log a warning when client sends both 'context' and 'configurable' so
  it's no longer silently dropped (reviewer feedback)
- Ensure thread_id is available in config['context'] when present so
  middlewares can find it there too
- Add test coverage for the context path, the both-keys-present case,
  passthrough of other keys, and the no-config fallback

* style: ruff format services.py

---------

Co-authored-by: JasonOA888 <JasonOA888@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-04-01 23:21:32 +08:00
Admire 82c3dbbc6b Fix Windows startup and dependency checks (#1709)
* windows check and dev fixes

* fix windows startup scripts

* fix windows startup scripts

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-04-01 23:13:00 +08:00
Alian e97c8c9943 fix(skills): support parsing multiline YAML strings in SKILL.md frontmatter (#1703)
* fix(skills): support parsing multiline YAML strings in SKILL.md frontmatter

* test(skills): add tests for multiline YAML descriptions
2026-04-01 23:08:30 +08:00
DanielWalnut 68d44f6755 fix: share .deer-flow in docker-compose-dev for uploads (#1718)
* fix: share dev thread data between gateway and langgraph

* refactor: drop redundant dev .deer-flow bind mounts
2026-04-01 21:04:13 +08:00
rayhpeng c2ff59a5b1 fix(gateway): merge context field into configurable for langgraph-compat runs (#1699) (#1707)
The langgraph-compat layer dropped the DeerFlow-specific `context` field
from run requests, causing agent config (subagent_enabled, is_plan_mode,
thinking_enabled, etc.) to fall back to defaults. Add `context` to
RunCreateRequest and merge allowlisted keys into config.configurable in
start_run, with existing configurable values taking precedence.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 17:17:09 +08:00
Shengyuan Wang 2f3744f807 refactor: replace sync requests with async httpx in Jina AI client (#1603)
* refactor: replace sync requests with async httpx in Jina AI client

Replace synchronous `requests.post()` with `httpx.AsyncClient` in
JinaClient.crawl() and make web_fetch_tool async. This is part of the
planned async concurrency optimization for the agent hot path
(see docs/TODO.md).

* fix: address Copilot review feedback on async Jina client

- Short-circuit error strings in web_fetch_tool before passing to
  ReadabilityExtractor, preventing misleading extraction results
- Log missing JINA_API_KEY warning only once per process to reduce
  noise under concurrent async fetching
- Use logger.exception instead of logger.error in crawl exception
  handler to preserve stack traces for debugging
- Add async web_fetch_tool tests and warn-once coverage

* fix: mock get_app_config in web_fetch_tool tests for CI

The web_fetch_tool tests failed in CI because get_app_config requires
a config.yaml file that isn't present in the test environment. Mock
the config loader to remove the filesystem dependency.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-04-01 17:02:39 +08:00
Llugaes 52c8c06cf2 fix: add --n-jobs-per-worker 10 to local dev Makefile (#1694)
#1623 added this flag to both Docker Compose files but missed the
backend Makefile used by `make dev`.  Without it `langgraph dev`
defaults to n_jobs_per_worker=1, so all conversation runs are
serialised and concurrent requests block.

This mirrors the Docker configuration.
2026-04-01 16:45:51 +08:00
AochenShen99 0cdecf7b30 feat(memory): structured reflection + correction detection in MemoryMiddleware (#1620) (#1668)
* feat(memory): add structured reflection and correction detection

* fix(memory): align sourceError schema and prompt guidance

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-04-01 16:45:29 +08:00
LYU Yichen 3e461d9d08 fix: use safe docker bind mount syntax for sandbox mounts (#1655)
Docker's -v host:container syntax is ambiguous for Windows drive-letter
paths (e.g. D:/...) because ':' is both the drive separator and the
volume separator, causing mount failures on Windows hosts.

Introduce _format_container_mount() which uses '--mount type=bind,...'
for Docker (unambiguous on all platforms) and keeps '-v' for Apple
Container runtime which does not support the --mount flag yet.

Adds unit tests covering Windows paths, read-only mounts, and Apple
Container pass-through.

Made-with: Cursor
2026-04-01 11:42:12 +08:00
JeffJiang cf43584d24 fix(artifact): enhance artifact content loading to include URL for non-write files (#1678) 2026-04-01 11:38:55 +08:00
d 🔹 6ff60f2af1 fix(gateway): forward assistant_id as agent_name in build_run_config (#1667)
* fix(gateway): forward assistant_id as agent_name in build_run_config

Fixes #1644

When the LangGraph Platform-compatible /runs endpoint receives a custom
assistant_id (e.g. 'finalis'), the Gateway's build_run_config() silently
ignored it — configurable['agent_name'] was never set, so make_lead_agent
fell through to the default lead agent and SOUL.md was never loaded.

Root cause (introduced in #1403):
  resolve_agent_factory() correctly falls back to make_lead_agent for all
  assistant_id values, but build_run_config() had no assistant_id parameter
  and never injected configurable['agent_name'].  The full call chain:

    POST /runs (assistant_id='finalis')
      → resolve_agent_factory('finalis')   # returns make_lead_agent ✓
      → build_run_config(thread_id, ...)   # no agent_name injected ✗
        → make_lead_agent(config)
          → cfg.get('agent_name') → None
            → load_agent_soul(None) → base SOUL.md (doesn't exist) → None

Fix:
- Add keyword-only  parameter to build_run_config().
- When assistant_id is set and differs from 'lead_agent', inject it as
  configurable['agent_name'] (matching the channel manager's existing
  _resolve_run_params() logic for IM channels).
- Honour an explicit configurable['agent_name'] in the request body;
  assistant_id mapping only fills the gap when it is absent.
- Remove stale log-only branch from resolve_agent_factory(); update
  docstring to explain the factory/configurable split.

Tests added (test_gateway_services.py):
- Custom assistant_id injects configurable['agent_name']
- 'lead_agent' assistant_id does NOT inject agent_name
- None assistant_id does NOT inject agent_name
- Explicit configurable['agent_name'] in request is not overwritten
- resolve_agent_factory returns make_lead_agent for all inputs

* style: format with ruff

* fix: validate and normalize assistant_id to prevent path traversal

Addresses Copilot review: strip/lowercase/replace underscores and
reject names that don't match [a-z0-9-]+, consistent with
ChannelManager._normalize_custom_agent_name().

---------

Co-authored-by: voidborne-d <voidborne-d@users.noreply.github.com>
2026-04-01 11:15:56 +08:00
Matt Van Horn a3bfea631c fix(sandbox): serialize concurrent exec_command calls in AioSandbox (#1435)
* fix(sandbox): serialize concurrent exec_command calls in AioSandbox

The AIO sandbox container maintains a single persistent shell session
that corrupts when multiple exec_command requests arrive concurrently
(e.g. when ToolNode issues parallel tool_calls). The corrupted session
returns 'ErrorObservation' strings as output, cascading into subsequent
commands.

Add a threading.Lock to AioSandbox to serialize shell commands. As a
secondary defense, detect ErrorObservation in output and retry with a
fresh session ID.

Fixes #1433

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(sandbox): address Copilot review findings

- Fix shell injection in list_dir: use shlex.quote(path) to escape
  user-provided paths in the find command
- Narrow ErrorObservation retry condition from broad substring match
  to the specific corruption signature to prevent false retries
- Improve test_lock_prevents_concurrent_execution: use threading.Barrier
  to ensure all workers contend for the lock simultaneously
- Improve test_list_dir_uses_lock: assert lock.locked() is True during
  exec_command to verify lock acquisition

* style: auto-format with ruff

---------

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 22:33:35 +08:00
Admire aae59a8ba8 fix: surface configured sandbox mounts to agents (#1638)
* fix: surface configured sandbox mounts to agents

* fix: address PR review feedback

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-31 22:22:30 +08:00
Admire 3ff15423d6 fix Windows Docker sandbox path mounting (#1634)
* fix windows docker sandbox paths

* fix windows sandbox mount validation

* fix backend checks for windows sandbox path PR
2026-03-31 22:19:27 +08:00
SHIYAO ZHANG c2f7be37b3 fix(tools): move sandbox.tools import in view_image_tool to break circular import (#1674)
view_image_tool.py had a top-level import of deerflow.sandbox.tools, which
created a circular dependency chain:

  sandbox.tools
    -> deerflow.agents.thread_state (triggers agents/__init__.py)
      -> agents/factory.py
        -> tools/builtins/__init__.py
          -> view_image_tool.py
            -> deerflow.sandbox.tools  <-- circular!

This caused ImportError when any test directly imported sandbox.tools,
making test_sandbox_tools_security.py fail to collect since #1522.

Fix: move the sandbox.tools import inside the view_image_tool function body.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 22:05:23 +08:00
tryag 09a9209724 fix: improve Windows compatibility in dependency check (#1550)
* fix: improve Windows compatibility in dependency check

* fix: tolerate stdio reconfigure failures in check script

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-31 21:54:41 +08:00
Rosemary1812 b356a13da5 fix(frontend): improve network error message for agent name check (#1605)
* fix(frontend): distinguish CORS errors   from generic name check failures

* fix(frontend): improve network error message for agent name check

* Fix network error message in zh-CN locale

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-31 21:14:05 +08:00
1316151417 ac9a6ee6a2 fix(langgraph): correct config.yaml mount path in docker-compose (#1679)
Co-authored-by: zhoujie172 <zhoujie172@ke.com>
2026-03-31 19:40:49 +08:00
ZHANG Ning 64e0f5329a fix: remove LANGSMITH_TRACING override that ignores .env value (#1640)
The `environment` section in docker-compose.yaml set
`LANGSMITH_TRACING=${LANGSMITH_TRACING:-false}`, which always resolves
to `false` because Docker Compose evaluates `${}` substitutions from
the host shell environment, not from `env_file`.

Since `environment` entries take precedence over `env_file`, setting
`LANGSMITH_TRACING=true` in `.env` had no effect — tracing stayed
disabled despite following the documented instructions.

Remove the explicit `LANGSMITH_TRACING` from `environment` so the
value from `.env` (loaded via `env_file`) is used as intended.
2026-03-31 09:42:33 +08:00
Admire 9e3d484858 fix(frontend): route agent checks to gateway (#1572)
* fix(frontend): route agent checks to gateway

* fix(frontend): proxy langgraph requests locally

* fix(frontend): keep zh-CN text readable

* fix(frontend): add exact local api rewrites

* fix(frontend): support docker-safe internal rewrites

* Update frontend/src/core/agents/api.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-30 21:04:59 +08:00
Admire 4bb3c101a8 chore(uv): speed up Docker builds with mirrors (#1600)
* docker mirror defaults

* fix: make docker mirror defaults overridable

* fix docker compose default pypi index

* fix: restore upstream pypi defaults

* docs: remove misleading env example mirrors

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-30 20:16:44 +08:00
ZHANG Ning b21792d9be fix: run uv sync before dev services to keep venv up-to-date (#1626)
The dev Docker Compose uses named volumes (langgraph-venv, gateway-venv)
to persist .venv across container restarts. Docker only populates named
volumes from the image on first creation — subsequent rebuilds do NOT
refresh existing volume contents.

When new dependencies are added to packages/harness/pyproject.toml
(e.g. langchain-anthropic), the stale named volume still contains
the old .venv missing the new packages, causing ModuleNotFoundError
at runtime.

Add `uv sync` before launching both gateway and langgraph services.
When deps are already satisfied this is a no-op (~1s), but when the
volume is stale it installs missing packages before the service starts.

Fixes #1624

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-30 20:02:30 +08:00
ZHANG Ning 0f1b023a2a fix: add --n-jobs-per-worker 10 to langgraph dev command in Docker (#1623)
`langgraph dev` defaults `n_jobs_per_worker` to 1 when the flag is not
explicitly passed (see langgraph_api/cli.py), even though the
`N_JOBS_PER_WORKER` env-var default is 10.

This causes the LangGraph server to run with a single background worker,
meaning all conversation runs are processed serially. When one run is
busy (e.g. summarization or long tool-calling chains), all other threads
are blocked until it finishes.

Add `--n-jobs-per-worker 10` to both production and dev Docker Compose
files to match the intended default concurrency.
2026-03-30 19:50:02 +08:00
Admire 9a557751d6 feat: support memory import and export (#1521)
* feat: support memory import and export

* fix(memory): address review feedback

* style: format memory settings page

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-30 17:25:47 +08:00
JeffJiang 2330c38209 fix(config): update SSR fallback in getBaseOrigin function (#1617) 2026-03-30 16:13:32 +08:00
rayhpeng 34e835bc33 feat(gateway): implement LangGraph Platform API in Gateway, replace langgraph-cli (#1403)
* feat(gateway): implement LangGraph Platform API in Gateway, replace langgraph-cli

Implement all core LangGraph Platform API endpoints in the Gateway,
allowing it to fully replace the langgraph-cli dev server for local
development. This eliminates a heavyweight dependency and simplifies
the development stack.

Changes:
- Add runs lifecycle endpoints (create, stream, wait, cancel, join)
- Add threads CRUD and search endpoints
- Add assistants compatibility endpoints (search, get, graph, schemas)
- Add StreamBridge (in-memory pub/sub for SSE) and async provider
- Add RunManager with atomic create_or_reject (eliminates TOCTOU race)
- Add worker with interrupt/rollback cancel actions and runtime context injection
- Route /api/langgraph/* to Gateway in nginx config
- Skip langgraph-cli startup by default (SKIP_LANGGRAPH_SERVER=0 to restore)
- Add unit tests for RunManager, SSE format, and StreamBridge

* fix: drain bridge queue on client disconnect to prevent backpressure

When on_disconnect=continue, keep consuming events from the bridge
without yielding, so the worker is not blocked by a full queue.
Only on_disconnect=cancel breaks out immediately.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: remove pytest import

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: Fix default stream_mode to ["values", "messages-tuple"]

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: Remove unused if_exists field from ThreadCreateRequest

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: address review comments on gateway LangGraph API

- Mount runs.py router in app.py (missing include_router)
- Normalize interrupt_before/after "*" to node list before run_agent()
- Use entry.id for SSE event ID instead of counter
- Drain bridge queue on disconnect when on_disconnect=continue
- Reuse serialization helper in wait_run() for consistent wire format
- Reject unsupported multitask_strategy with 400
- Remove SKIP_LANGGRAPH_SERVER fallback, always use Gateway

* feat: extract app.state access into deps.py

Encapsulate read/write operations for singleton objects (RunManager,
StreamBridge, checkpointer) held in app.state into a shared utility,
reducing repeated access patterns across router modules.

* feat: extract deerflow.runtime.serialization module with tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: replace duplicated serialization with deerflow.runtime.serialization

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: extract app/gateway/services.py with run lifecycle logic

Create a service layer that centralizes SSE formatting, input/config
normalization, and run lifecycle management. Router modules will delegate
to these functions instead of using private cross-imported helpers.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: wire routers to use services layer, remove cross-module private imports

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: apply ruff formatting to refactored files

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(runtime): support LangGraph dev server and add compat route

- Enable official LangGraph dev server for local development workflow
- Decouple runtime components from agents package for better separation
- Provide gateway-backed fallback route when dev server is skipped
- Simplify lifecycle management using context manager in gateway

* feat(runtime): add Store providers with auto-backend selection

- Add async_provider.py and provider.py under deerflow/runtime/store/
- Support memory, sqlite, postgres backends matching checkpointer config
- Integrate into FastAPI lifespan via AsyncExitStack in deps.py
- Replace hardcoded InMemoryStore with config-driven factory

* refactor(gateway): migrate thread management from checkpointer to Store and resolve multiple endpoint failures

- Add Store-backed CRUD helpers (_store_get, _store_put, _store_upsert)
- Replace checkpoint-scanning search with two-phase strategy:
  phase 1 reads Store (O(threads)), phase 2 backfills from checkpointer
  for legacy/LangGraph Server threads with lazy migration
- Extend Store record schema with values field for title persistence
- Sync thread title from checkpoint to Store after run completion
- Fix /threads/{id}/runs/{run_id}/stream 405 by accepting both
  GET and POST methods; POST handles interrupt/rollback actions
- Fix /threads/{id}/state 500 by separating read_config and
  write_config, adding checkpoint_ns to configurable, and
  shallow-copying checkpoint/metadata before mutation
- Sync title to Store on state update for immediate search reflection
- Move _upsert_thread_in_store into services.py, remove duplicate logic
- Add _sync_thread_title_after_run: await run task, read final
  checkpoint title, write back to Store record
- Spawn title sync as background task from start_run when Store exists

* refactor(runtime): deduplicate store and checkpointer provider logic

Extract _ensure_sqlite_parent_dir() helper into checkpointer/provider.py
and use it in all three places that previously inlined the same mkdir logic.
Consolidate duplicate error constants in store/async_provider.py by importing
from store/provider.py instead of redefining them.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(runtime): move SQLite helpers to runtime/store, checkpointer imports from store

_resolve_sqlite_conn_str and _ensure_sqlite_parent_dir now live in
runtime/store/provider.py. agents/checkpointer/provider and
agents/checkpointer/async_provider import from there, reversing the
previous dependency direction (store → checkpointer becomes
checkpointer → store).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(runtime): extract SQLite helpers into runtime/store/_sqlite_utils.py

Move resolve_sqlite_conn_str and ensure_sqlite_parent_dir out of
checkpointer/provider.py into a dedicated _sqlite_utils module.
Functions are now public (no underscore prefix), making cross-module
imports semantically correct. All four provider files import from
the single shared location.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(gateway): use adelete_thread to fully remove thread checkpoints on delete

AsyncSqliteSaver has no adelete method — the previous hasattr check
always evaluated to False, silently leaving all checkpoint rows in the
database. Switch to adelete_thread(thread_id) which deletes every
checkpoint and pending-write row for the thread across all namespaces
(including sub-graph checkpoints).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(gateway): remove dead bridge_cm/ckpt_cm code and fix StrEnum lint

app.py had unreachable code after the async-with lifespan refactor:
bridge_cm and ckpt_cm were referenced but never defined (F821), and
the channel service startup/shutdown was outside the langgraph_runtime
block so it never ran. Move channel service lifecycle inside the
async-with block where it belongs.

Replace str+Enum inheritance in RunStatus and DisconnectMode with
StrEnum as suggested by UP042.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* style: format with ruff

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: JeffJiang <for-eleven@hotmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-30 16:02:23 +08:00
张凯强 7db95926b0 feat(feishu): add configurable domain for Lark international support (#1535)
The lark-oapi SDK defaults to open.feishu.cn (China), but apps on the
international Lark platform (open.larksuite.com) fail to connect with
error 1000040351 'Incorrect domain name'.

Changes:
- Add 'domain' config option to feishu channel (default: open.feishu.cn)
- Pass domain to both API client and WebSocket client
- Update config.example.yaml and all README files
2026-03-30 11:42:07 +08:00
d 🔹 9bcdba6038 fix: promote deferred tools after tool_search returns schema (#1570)
* fix: promote matched tools from deferred registry after tool_search returns schema

After tool_search returns a tool's full schema, the tool is promoted
(removed from the deferred registry) so DeferredToolFilterMiddleware
stops filtering it from bind_tools on subsequent LLM calls.

Without this, deferred tools are permanently filtered — the LLM gets
the schema from tool_search but can never invoke the tool because
the middleware keeps stripping it.

Fixes #1554

* test: add promote() and tool_search promotion tests

Tests cover:
- promote removes tools from registry
- promote nonexistent/empty is no-op
- search returns nothing after promote
- middleware passes promoted tools through
- tool_search auto-promotes matched tools (select + keyword)

* fix: address review — lint blank line + empty registry guard

- Add missing blank line between FakeRequest methods (E301)
- Use 'if not registry' to handle empty registries consistently

---------

Co-authored-by: d 🔹 <258577966+voidborne-d@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-30 11:23:15 +08:00
finallylly ef58bb8d3c fix(config): correct MiniMax M2.7 highspeed model name and add thinking support (#1596)
* fix(config): correct MiniMax M2.7 highspeed model name and add thinking support

- Rename minimax-m2.5-highspeed to minimax-m2.7-highspeed for CN region
- Add supports_thinking: true for both M2.7 and M2.7-highspeed models

* Add supports_thinking option to config examples

Added support_thinking configuration option in examples.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-30 11:13:47 +08:00
Jason c5034c03c7 fix(dev): exclude sandbox dirs from gateway hot-reload watcher (#1519)
* fix(dev): exclude sandbox dirs from gateway hot-reload watcher

The dev-mode gateway uses --reload which watches for file changes.
Sandbox containers mount the repo and write .pyc/__pycache__ during
execution, triggering spurious gateway restarts mid-request.

Add --reload-exclude for .pyc, __pycache__, and sandbox/ paths so
only actual source changes trigger a reload.

Fixes #1513

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: JasonOA888 <JasonOA888@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-30 09:51:07 +08:00
SHIYAO ZHANG 9aa3ff7c48 feat(sandbox): add SandboxAuditMiddleware for bash command security auditing (#1532)
* feat(sandbox): add SandboxAuditMiddleware for bash command security auditing

Addresses the LocalSandbox escape vector reported in #1224 where bash tool
calls can execute destructive commands against the host filesystem.

- Add SandboxAuditMiddleware with three-tier command classification:
  - High-risk (block): rm -rf /, curl|bash, dd if=, mkfs, /etc/shadow access
  - Medium-risk (warn): pip install, apt install, chmod 777
  - Safe (pass): normal workspace operations
- Register middleware after GuardrailMiddleware in _build_runtime_middlewares,
  applied to both lead agent and subagents
- Structured audit log via standard logger (visible in langgraph.log)
- Medium-risk commands execute but append a warning to the tool result,
  allowing the LLM to self-correct without blocking legitimate workflows
- High-risk commands return an error ToolMessage without calling the handler,
  so the agent loop continues gracefully

* fix(lint): sort imports in test_sandbox_audit_middleware

* refactor(sandbox-audit): address Copilot review feedback (3/5/6)

- Fix class docstring to match implementation: medium-risk commands are
  executed with a warning appended (not rejected), and cwd anchoring note
  removed (handled in a separate PR)
- Remove capsys.disabled() from benchmark test to avoid CI log noise;
  keep assertions for recall/precision targets
- Remove misleading 'cwd fix' from test module docstring

* test(sandbox-audit): add async tests for awrap_tool_call

* fix(sandbox-audit): address Copilot review feedback (1/2)

- Narrow rm high-risk regex to only block truly destructive targets
  (/, /*, ~, ~/*, /home, /root); legitimate workspace paths like
  /mnt/user-data/ are no longer false-positived
- Handle list-typed ToolMessage content in _append_warn_to_result;
  append a text block instead of str()-ing the list to avoid breaking
  structured content normalization

* style: apply ruff format to sandbox_audit_middleware files

* fix(sandbox-audit): update benchmark comment to match assert-based implementation

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-30 07:48:31 +08:00
Markus Corazzione 5ceb19f6f6 fix(oauth): Harden Claude OAuth cache-control handling (#1583) 2026-03-30 07:41:18 +08:00
Admire fc7de7fffe feat: support manual add and edit for memory facts (#1538)
* feat: support manual add and edit for memory facts

* fix: restore memory updater save helper

* fix: address memory fact review feedback

* fix: remove duplicate memory fact edit action

* docs: simplify memory fact review setup

* docs: relax memory review startup instructions

* fix: clear rebase marker in memory settings page

* fix: address memory fact review and format issues

* fix: address memory fact review feedback

* refactor: make memory fact updates explicit patch semantics

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-29 23:53:23 +08:00
SHIYAO ZHANG cdb2a3a017 fix(sandbox): anchor relative paths to thread workspace in local mode (#1522)
* fix(task_tool): fallback to configurable thread_id when context is missing

task_tool only read thread_id from runtime.context, but when invoked
via LangGraph Server, thread_id lives in config.configurable instead.
Add the same fallback that ThreadDataMiddleware uses (PR #1237).

Fixes subagent execution failure: 'Thread ID is required in runtime
context or config.configurable'

* remove debug logging from task_tool

* fix(sandbox): anchor relative paths to thread workspace in local mode

In local sandbox mode, bash commands using relative paths were resolved
against the langgraph server process cwd (backend/) instead of the
per-thread workspace directory. This allowed relative-path writes to
escape the thread isolation boundary.

Root cause: validate_local_bash_command_paths and
replace_virtual_paths_in_command only process absolute paths (scanning
for '/' prefix). Relative paths pass through untouched and inherit the
process cwd at subprocess.run time.

Fix: after virtual path translation, prepend `cd {workspace} &&` to
anchor the shell's cwd to the thread-isolated workspace directory before
execution. shlex.quote() ensures paths with spaces or special characters
are handled safely.

This mirrors the approach used by OpenHands (fixed cwd at execution
layer) and is the correct fix for local mode where each subprocess.run
is an independent process with no persistent shell session.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(sandbox): extract _apply_cwd_prefix and add unit tests

Extract the workspace cd-prefix logic from bash_tool into a dedicated
_apply_cwd_prefix() helper so it can be unit-tested in isolation.
Add four tests covering: normal prefix, no thread_data, missing
workspace_path, and paths with spaces (shlex.quote).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* revert: remove unrelated configurable thread_id fallback from sandbox/tools.py

This change belongs in a separate PR.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* style: remove trailing whitespace in test_sandbox_tools_security

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-29 23:21:06 +08:00
Sleepy Ranx 🌙 866cf4ef73 fix(frontend): prevent submit during IME composition (#1562) 2026-03-29 22:36:38 +08:00
Echo-Nie d475de7997 docs: fix some broken links (#1567)
* Fix path for TitleMiddleware implementation

* Fix link to Provisioner Setup Guide in CONFIGURATION.md

* Update file path for TitleMiddleware implementation

* Update image paths in Leica photography article
2026-03-29 21:52:28 +08:00
passer 75c4757f48 fix(nginx): re-resolve upstream DNS in Docker (#1517)
Enable runtime DNS re-resolution for docker-compose upstreams (gateway/langgraph/frontend) to avoid stale container IPs causing persistent 502s.
2026-03-29 21:47:26 +08:00
tryag 580920ef63 fix: use Git Bash for Windows local startup (#1551)
* fix: use Git Bash for Windows local startup

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-29 21:38:29 +08:00
Admire 68c9e09a7a fix: add Windows shell fallback for local sandbox (#1505)
* fix: add Windows shell fallback for local sandbox

* fix: handle PowerShell execution on Windows

* fix: handle Windows local shell execution

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-29 21:31:29 +08:00
13ernkastel 92c7a20cb7 [Security] Address critical host-shell escape in LocalSandboxProvider (#1547)
* fix(security): disable host bash by default in local sandbox

* fix(security): address review feedback for local bash hardening

* fix(ci): sort live test imports for lint

* style: apply backend formatter

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-29 21:03:58 +08:00
mlbo 8b6c333afc fix(docs): Correct security usage recommendations in README_zh.md (#1548)
* fix(docs): Correct security usage recommendations in README_zh.md

Fix formatting and punctuation in security usage recommendations.

* fix(docs): Correct security usage recommendations in README_zh.md
2026-03-29 19:43:03 +08:00
knukn 6091ba83c4 docs(config): add timeout and max_retries examples for model providers (#1549)
Added explicit timeout and retry configurations to `config.example.yaml` to help users properly configure their model connections.

Since different LangChain provider classes use different parameter names, this update maps the correct arguments for each:
- ChatOpenAI (OpenAI, MiniMax, Novita, OpenRouter): added `request_timeout` and `max_retries`
- ChatAnthropic (Claude): added `default_request_timeout` and `max_retries`
- ChatGoogleGenerativeAI (Gemini): added `timeout` and `max_retries`
- PatchedChatDeepSeek (Doubao, DeepSeek, Kimi): added `timeout` and `max_retries`

Default example values are set to 600.0 seconds for timeouts and 2 for max retries.
2026-03-29 19:29:55 +08:00
greatmengqi 70e9f2dd2c docs: add format step to contributing workflow (#1552)
Co-authored-by: greatmengqi <chenmengqi.0376@bytedance.com>
2026-03-29 18:39:10 +08:00
SHIYAO ZHANG 118485a7cb fix(sandbox): fall back to config.configurable for thread_id in lazy sandbox init (#1529)
* fix(sandbox): fall back to config.configurable for thread_id in lazy sandbox init

LangGraph Server injects thread_id via config["configurable"]["thread_id"],
not always via context["thread_id"]. Without the fallback, lazy sandbox
acquisition fails when context is empty.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sandbox): align configurable fallback style with task_tool.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sandbox): guard runtime.config None check for thread_id fallback

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 17:21:04 +08:00
DAN 9e5ba74ecd fix(sandbox): allow MCP filesystem server paths in local bash commands (#1527)
* feat/bug-fix: copy the allowed path configurations in MCP filesystem tools to bash tool. With updated unit test

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-29 17:10:27 +08:00
greatmengqi 25df82cbfd style: format unformatted files and add .omc/ to prettierignore (#1539)
Co-authored-by: greatmengqi <chenmengqi.0376@bytedance.com>
2026-03-29 16:45:31 +08:00
greatmengqi 084dc7e748 ci: enforce code formatting checks for backend and frontend (#1536) 2026-03-29 15:34:38 +08:00
greatmengqi 06a623f9c8 feat: add create_deerflow_agent SDK entry point (Phase 1) (#1203) 2026-03-29 15:31:18 +08:00
Admire 7eb3a150b5 feat: add memory management actions and local filters in memory settings (#1467)
* Add MVP memory management actions

* Fix memory settings locale coverage

* Polish memory management interactions

* Add memory search and type filters

* Refine memory settings review feedback

* docs: simplify memory settings review setup

* fix: restore memory updater compatibility helpers

* fix: address memory settings review feedback

* docs: soften memory sample review wording

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: JeffJiang <for-eleven@hotmail.com>
2026-03-29 13:14:45 +08:00
knukn 481494b9c0 feat(client): support custom middleware injection (#1520)
* feat(client): support custom middleware injection

Add support for custom middleware, allowing custom middleware list to be passed when initializing DeerFlowClient. These middleware will be injected after the default middleware when creating the agent, extending the agent's functionality.

* feat: inject custom middlewares before ClarificationMiddleware to preserve ordering

- Add `custom_middlewares` param to `_build_middlewares`
- Inject custom middlewares right before `ClarificationMiddleware` to keep it as the last in the chain
- Remove unsafe `.extend()` in `client.py`
- Update tests in `test_client.py` and `test_lead_agent_model_resolution.py` to assert correct injection ordering
2026-03-29 11:24:46 +08:00
Nan Gao 89183ae76a fix(channel): reject concurrent same-thread runs (#1465) (#1475)
* fix(channel): reject concurrent same-thread runs (#1465)

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix(lint): sort imports in manager.py and test_channels.py

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(channel): widen _is_thread_busy_error to BaseException and downgrade busy log to warning

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 09:55:47 +08:00
DanielWalnut 18e3487888 Support custom channel assistant IDs via lead_agent (#1500)
* Support custom channel assistant IDs via lead agent

* Normalize custom channel agent names
2026-03-28 19:07:38 +08:00
Nan Gao 520c0352b5 fix(middleware): fall back to configurable thread_id in MemoryMiddleware (#1425) (#1426)
* fix(middleware): fall back to configurable thread_id in MemoryMiddleware (#1425)

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-28 17:00:11 +08:00
SHIYAO ZHANG 690d80f46f fix(task_tool): fallback to configurable thread_id when context is mi… (#1343)
* fix(task_tool): fallback to configurable thread_id when context is missing

task_tool only read thread_id from runtime.context, but when invoked
via LangGraph Server, thread_id lives in config.configurable instead.
Add the same fallback that ThreadDataMiddleware uses (PR #1237).

Fixes subagent execution failure: 'Thread ID is required in runtime
context or config.configurable'

* remove debug logging from task_tool
2026-03-28 16:44:15 +08:00
DanielWalnut c2dd8937ed Fix IM channel backend URLs in Docker (#1497)
* Fix IM channel backend URLs in Docker

* Address Copilot review comments
2026-03-28 16:37:41 +08:00
zihao 9caea0266e fix(frontend): separate mock and default LangGraph clients (#1504)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-28 16:33:22 +08:00
7. Sun 49f2e38fbf fix: prevent SpeechRecognition instance leaks on render (#1369)
* fix: remove unstable dependencies from speech recognition effect

* fix: use refs to prevent stale closures in speech recognition

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-28 16:20:38 +08:00
JeffJiang d22cab8614 fix: refactor to use getBaseOrigin for URL construction in backend and LangGraph base URL functions (#1494) 2026-03-28 12:18:03 +08:00
taka6745 43ef3691a5 fix(oauth): inject billing header for Claude oAuth Models (#1442)
* fix(oauth): inject billing header for non-Haiku model access

The Anthropic Messages API requires a billing identification block
in the system prompt when using Claude Code OAuth tokens (sk-ant-oat*)
to access non-Haiku models (Opus, Sonnet). Without it, the API returns
a generic 400 "Error" with no actionable detail.

This was discovered by intercepting Claude Code CLI requests — the CLI
injects an `x-anthropic-billing-header` text block as the first system
prompt entry on every request. Third-party consumers of the same OAuth
tokens must do the same.

Changes:
- Add `_apply_oauth_billing()` to `ClaudeChatModel` that prepends the
  billing header block to the system prompt when `_is_oauth` is True
- Add `metadata.user_id` with device/session identifiers (required by
  the API alongside the billing header)
- Called from `_get_request_payload()` before prompt caching runs

Verified with Claude Max OAuth tokens against all three model tiers:
- claude-opus-4-6: 200 OK
- claude-sonnet-4-6: 200 OK
- claude-haiku-4-5-20251001: 200 OK (was already working)

Closes #1245

* fix(oauth): address review feedback on billing header injection

- Make OAUTH_BILLING_HEADER configurable via ANTHROPIC_BILLING_HEADER env var
- Normalize billing block to always be first in system list (strip + reinsert)
- Guard metadata with isinstance check for non-dict values
- Replace os.uname() with socket.gethostname() for Windows compat
- Fix docstrings to say "all OAuth requests" instead of "non-Haiku"
- Move inline imports to module level (fixes ruff I001)
- Add 9 unit tests for _apply_oauth_billing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 08:49:34 +08:00
luo jiyin ca20b48601 chore(ci): align workflow action versions (#1484) 2026-03-27 23:25:55 +08:00
moose-lab 03b144f9c9 fix: replace print() with logging across harness package (#1282)
Replace all bare print() calls with proper logging using Python's
standard logging module across the deerflow harness package.

Changes across 8 files (16 print statements replaced):

- agents/middlewares/clarification_middleware.py: use logger.info/debug
- agents/middlewares/memory_middleware.py: use logger.debug
- agents/middlewares/thread_data_middleware.py: use logger.debug
- agents/middlewares/view_image_middleware.py: use logger.debug
- agents/memory/queue.py: use logger.info/debug/warning/error
- agents/lead_agent/prompt.py: use logger.error
- skills/loader.py: use logger.warning
- skills/parser.py: use logger.error

Each file follows the established codebase convention:
  import logging
  logger = logging.getLogger(__name__)

Log levels chosen based on message semantics:
- debug: routine operational details (directory creation, timer resets)
- info: significant state changes (memory queued, updates processed)
- warning: recoverable issues (config load failures, skipped updates)
- error: unexpected failures (parsing errors, memory update errors)

Note: client.py is intentionally excluded as it uses print() for
CLI output, which is the correct behavior for a command-line client.

Co-authored-by: moose-lab <moose-lab@users.noreply.github.com>
2026-03-27 23:15:35 +08:00
luo jiyin 9a4e8f438a docs(frontend): update better-auth README notes (#1487) 2026-03-27 22:58:55 +08:00
Zeng Qingwen 6bf23ba0a3 docs(README): add missing cross-language README links (#1479)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-27 22:49:59 +08:00
luo jiyin 50db51d0fb chore(frontend): align format scripts with Makefile and README (#1481)
* chore(frontend): add format scripts

* docs(frontend): document format commands
2026-03-27 22:49:41 +08:00
Zeng Qingwen 18b0794125 docs(SETUP): correct setup documentation links (#1478) 2026-03-27 22:44:01 +08:00
Andrew Barnes 50f50d7654 test: add unit tests for skill frontmatter validation (#1309)
* test: add unit tests for skill frontmatter validation

Cover _validate_skill_frontmatter logic:
- Valid minimal and full-field skills
- Missing SKILL.md, missing frontmatter, invalid YAML
- Required field validation (name, description)
- Unexpected key rejection
- Name format: hyphen-case, no leading/trailing/consecutive hyphens
- Name and description length limits
- Angle bracket rejection in description

* test: fix unused variables flagged by ruff F841

Replace unused tuple elements with _ and add assertions on
msg/name return values in success-path tests.

* test: address review feedback on unused variables

* test: consolidate validation tests into single module

Move the UTF-8/windows-locale test from test_skills_router.py into
test_skills_validation.py and remove test_skills_router.py to eliminate
duplicated assertions and future maintenance drift.

* fix: match assertion strings to actual validation messages

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-27 20:20:31 +08:00
DanielWalnut 8590249db4 feat(acp): add env field to ACPAgentConfig for subprocess env injection (#1447)
Allow per-agent environment variables to be declared in config.yaml under
acp_agents.<name>.env. Values prefixed with $ are resolved from the host
environment at invocation time, consistent with other config fields.
Passes None to spawn_agent_process when env is empty so the subprocess
inherits the parent environment unchanged.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 20:03:30 +08:00
Admire 40a4acbbed fix(sandbox): Relax upload permissions for aio sandbox sync (#1409)
* Relax upload permissions for aio sandbox sync

* Harden upload permission sync checks

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-27 17:37:44 +08:00
Jason 4708700723 fix(middleware): return proper content format when no images viewed (#1454)
- Fix OpenAI BadRequestError: 'No images have been viewed.' was returned as
  a plain string array instead of a properly formatted content block
- The OpenAI API expects message content to be either a string or an array
  of objects with 'type' field, not an array of plain strings
- Changed return from ['No images have been viewed.'] to
  [{'type': 'text', 'text': 'No images have been viewed.'}]

Fixes #1441

Co-authored-by: JasonOA888 <noreply@github.com>
2026-03-27 17:33:17 +08:00
luo jiyin 43a19f9627 fix(task): avoid blocking in task tool polling (#1320)
* fix: avoid blocking in task tool polling

* test: adapt task tool polling tests for async tool

* fix: clean up cancelled task tool polling

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-27 17:12:40 +08:00
yangzheli a4e4bb21e3 docs: add LangSmith tracing configuration and documentation (#1414)
Add LangSmith tracing setup instructions across the project:
- .env.example: add LANGSMITH_* env vars (commented out)
- README.md + translations (zh/ja/fr/ru): add LangSmith Tracing section
  under Advanced with setup steps and env var reference
- backend/README.md: add detailed LangSmith Tracing section with setup,
  env var table, how-it-works explanation, and Docker notes
- docker-compose.yaml: update LANGCHAIN_TRACING_V2 to LANGSMITH_TRACING
  for naming consistency with the rest of the project

Made-with: Cursor

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-27 14:17:45 +08:00
Matt Van Horn 99965057c1 fix(config): add Docker service name guidance for channel URLs (#1437)
The channels config section uses localhost URLs by default, which don't
work inside Docker containers. Add inline comments showing the Docker
service names (langgraph, gateway) that match the docker-compose service
definitions.

Fixes #1421

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 14:15:48 +08:00
Kaushik Rajan 8ae023574e fix: add build-arg support for proxies and mirrors in Docker builds (#1346)
* fix: add build-arg support for proxies and mirrors in Docker builds (#1260)

Pin Debian images to bookworm, make UV source image configurable,
and pass APT_MIRROR/NPM_REGISTRY/UV_IMAGE through docker-compose.

* fix: ensure build args use consistent defaults across compose and Dockerfiles

UV_IMAGE: ${UV_IMAGE:-} resolved to empty when unset, overriding the
Dockerfile ARG default and breaking `FROM ${UV_IMAGE}`. Also configure
COREPACK_NPM_REGISTRY before pnpm download and propagate NPM_REGISTRY
into the prod stage.

* fix: dearmor NodeSource GPG key to resolve signing error

Pipe the downloaded key through gpg --dearmor so apt can verify
the repository signature (fixes NO_PUBKEY 2F59B5F99B1BE0B4).

---------

Co-authored-by: JeffJiang <for-eleven@hotmail.com>
2026-03-27 10:35:40 +08:00
SCPZ24 6b13f5c9fb feat: Support gitHub PAT configuration for higher github API accessing rate. (#1374)
* feat: Add github PAT configs, allowing larger github API rates.

* Update comment to English for better clarity

* fix: Remove unused config lines in config.example.yaml and unreferenced declarations in app_config. Fix lint issues and update documentation.

* fix: Remove unused imports, and passed the ruff check.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-27 09:54:14 +08:00
Ben Piper c13793386f Implement DuckDuckGo search (#1432)
* Implement DuckDuckGo search

* docs: add DuckDuckGo web search to config example
2026-03-27 09:20:22 +08:00
knukn 1c542ab7f1 feat(memory): Introduce configurable memory storage abstraction (#1353)
* feat(内存存储): 添加可配置的内存存储提供者支持

实现内存存储的抽象基类 MemoryStorage 和文件存储实现 FileMemoryStorage
重构内存数据加载和保存逻辑到存储提供者中
添加 storage_class 配置项以支持自定义存储提供者

* refactor(memory): 重构内存存储模块并更新相关测试

将内存存储逻辑从updater模块移动到独立的storage模块
使用存储接口模式替代直接文件操作
更新所有相关测试以使用新的存储接口

* Update backend/packages/harness/deerflow/agents/memory/storage.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update backend/packages/harness/deerflow/agents/memory/storage.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix(内存存储): 添加线程安全锁并增加测试用例

添加线程锁确保内存存储单例初始化的线程安全
增加对无效代理名称的验证测试
补充单例线程安全性和异常处理的测试用例

* Update backend/tests/test_memory_storage.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix(agents): 使用统一模式验证代理名称

修改代理名称验证逻辑以使用仓库中定义的AGENT_NAME_PATTERN模式,确保代码库一致性并防止路径遍历等安全问题。同时更新测试用例以覆盖更多无效名称情况。

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-27 07:41:06 +08:00
DanielWalnut e1853df06a docs: add install.md agent setup guide (#1402)
* docs: add install.md agent setup guide

* docs: tighten install.md setup flow

* docs: address copilot review comments
2026-03-26 21:39:34 +08:00
Henry Li f80d1743ab Add security alerts to documents (#1413) 2026-03-26 21:24:52 +08:00
7. Sun d7bdb1a4b9 fix: remove unused radix Icon import from suggestion (#1368)
* fix: use create_chat_model for summarization alias

* fix: remove unused radix Icon import from suggestion

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-26 21:14:56 +08:00
Henry Li 227967df3d feat: hide model ID for safety reason, only show the display_name (#1410)
Co-authored-by: Henry Li <lixin.henry@bytedance.com>
2026-03-26 21:13:32 +08:00
13ernkastel 0d3cefaa5a fix(gateway): enforce safe download for active artifact MIME types to mitigate stored XSS (#1389)
* docs: refocus security review on high-confidence artifact XSS

* fix(gateway): block inline active-content artifacts to mitigate XSS

* chore: remove security review markdown from PR

* Delete SECURITY_REVIEW.md

* fix(gateway): harden artifact attachment handling
2026-03-26 17:44:25 +08:00
Admire b9583f7204 Fix Windows backend test compatibility (#1384)
* Fix Windows backend test compatibility

* Preserve ACP path style on Windows

* Fix installer import ordering

* Address review comments for Windows fixes

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-26 17:39:16 +08:00
dependabot[bot] b3d3287b80 build(deps): bump requests from 2.32.5 to 2.33.0 in /backend (#1395)
Bumps [requests](https://github.com/psf/requests) from 2.32.5 to 2.33.0.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.32.5...v2.33.0)

---
updated-dependencies:
- dependency-name: requests
  dependency-version: 2.33.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-26 16:18:20 +08:00
RockeyDon c0a6b81852 Add packages section to pnpm-workspace.yaml (#1382)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-26 16:09:35 +08:00
JeffJiang 4d1a69a938 fix(config): return full URLs for backend and LangGraph base URLs (#1392) 2026-03-26 15:43:37 +08:00
Willem Jiang a087fe7bcc fix(LLM): fixing Gemini thinking + tool calls via OpenAI gateway (#1180) (#1205)
* fix(LLM): fixing Gemini thinking + tool calls via OpenAI gateway (#1180)

When using Gemini with thinking enabled through an OpenAI-compatible gateway,
the API requires that  fields on thinking content blocks are
preserved and echoed back verbatim in subsequent requests. Standard
 silently drops these signatures when serializing
messages, causing HTTP 400 errors:

Changes:
- Add PatchedChatOpenAI adapter that re-injects signed thinking blocks into
  request payloads, preserving the signature chain across multi-turn
  conversations with tool calls.
- Support two LangChain storage patterns: additional_kwargs.thinking_blocks
  and content list.
- Add 11 unit tests covering signed/unsigned blocks, storage patterns, edge
  cases, and precedence rules.
- Update config.example.yaml with Gemini + thinking gateway example.
- Update CONFIGURATION.md with detailed guidance and error explanation.

Fixes: #1180

* Updated the patched_openai.py with thought_signature of function call

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* docs: fix inaccurate thought_signature description in CONFIGURATION.md (#1220)

* Initial plan

* docs: fix CONFIGURATION.md wording for thought_signature - tool-call objects, not thinking blocks

Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>
Agent-Logs-Url: https://github.com/bytedance/deer-flow/sessions/360f5226-4631-48a7-a050-189094af8ffe

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
2026-03-26 15:07:05 +08:00
Admire 080a03f3bc fix(config): fix summarization model alias resolution (#1378)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-26 14:48:45 +08:00
xiangxiang-all-in-AI ae6a791c71 Update config.example.yaml (#1376)
使用deerpseek接口后会报错,因为max_token设置不对。所以从example里改

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-26 14:34:57 +08:00
DanielWalnut d119214fee feat(harness): integration ACP agent tool (#1344)
* refactor: extract shared utils to break harness→app cross-layer imports

Move _validate_skill_frontmatter to src/skills/validation.py and
CONVERTIBLE_EXTENSIONS + convert_file_to_markdown to src/utils/file_conversion.py.
This eliminates the two reverse dependencies from client.py (harness layer)
into gateway/routers/ (app layer), preparing for the harness/app package split.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: split backend/src into harness (deerflow.*) and app (app.*)

Physically split the monolithic backend/src/ package into two layers:

- **Harness** (`packages/harness/deerflow/`): publishable agent framework
  package with import prefix `deerflow.*`. Contains agents, sandbox, tools,
  models, MCP, skills, config, and all core infrastructure.

- **App** (`app/`): unpublished application code with import prefix `app.*`.
  Contains gateway (FastAPI REST API) and channels (IM integrations).

Key changes:
- Move 13 harness modules to packages/harness/deerflow/ via git mv
- Move gateway + channels to app/ via git mv
- Rename all imports: src.* → deerflow.* (harness) / app.* (app layer)
- Set up uv workspace with deerflow-harness as workspace member
- Update langgraph.json, config.example.yaml, all scripts, Docker files
- Add build-system (hatchling) to harness pyproject.toml
- Add PYTHONPATH=. to gateway startup commands for app.* resolution
- Update ruff.toml with known-first-party for import sorting
- Update all documentation to reflect new directory structure

Boundary rule enforced: harness code never imports from app.
All 429 tests pass. Lint clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: add harness→app boundary check test and update docs

Add test_harness_boundary.py that scans all Python files in
packages/harness/deerflow/ and fails if any `from app.*` or
`import app.*` statement is found. This enforces the architectural
rule that the harness layer never depends on the app layer.

Update CLAUDE.md to document the harness/app split architecture,
import conventions, and the boundary enforcement test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add config versioning with auto-upgrade on startup

When config.example.yaml schema changes, developers' local config.yaml
files can silently become outdated. This adds a config_version field and
auto-upgrade mechanism so breaking changes (like src.* → deerflow.*
renames) are applied automatically before services start.

- Add config_version: 1 to config.example.yaml
- Add startup version check warning in AppConfig.from_file()
- Add scripts/config-upgrade.sh with migration registry for value replacements
- Add `make config-upgrade` target
- Auto-run config-upgrade in serve.sh and start-daemon.sh before starting services
- Add config error hints in service failure messages

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix comments

* fix: update src.* import in test_sandbox_tools_security to deerflow.*

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: handle empty config and search parent dirs for config.example.yaml

Address Copilot review comments on PR #1131:
- Guard against yaml.safe_load() returning None for empty config files
- Search parent directories for config.example.yaml instead of only
  looking next to config.yaml, fixing detection in common setups

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: correct skills root path depth and config_version type coercion

- loader.py: fix get_skills_root_path() to use 5 parent levels (was 3)
  after harness split, file lives at packages/harness/deerflow/skills/
  so parent×3 resolved to backend/packages/harness/ instead of backend/
- app_config.py: coerce config_version to int() before comparison in
  _check_config_version() to prevent TypeError when YAML stores value
  as string (e.g. config_version: "1")
- tests: add regression tests for both fixes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: update test imports from src.* to deerflow.*/app.* after harness refactor

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(harness): add tool-first ACP agent invocation (#37)

* feat(harness): add tool-first ACP agent invocation

* build(harness): make ACP dependency required

* fix(harness): address ACP review feedback

* feat(harness): decouple ACP agent workspace from thread data

ACP agents (codex, claude-code) previously used per-thread workspace
directories, causing path resolution complexity and coupling task
execution to DeerFlow's internal thread data layout. This change:

- Replace _resolve_cwd() with a fixed _get_work_dir() that always uses
  {base_dir}/acp-workspace/, eliminating virtual path translation and
  thread_id lookups
- Introduce /mnt/acp-workspace virtual path for lead agent read-only
  access to ACP agent output files (same pattern as /mnt/skills)
- Add security guards: read-only validation, path traversal prevention,
  command path allowlisting, and output masking for acp-workspace
- Update system prompt and tool description to guide LLM: send
  self-contained tasks to ACP agents, copy results via /mnt/acp-workspace
- Add 11 new security tests for ACP workspace path handling

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(prompt): inject ACP section only when ACP agents are configured

The ACP agent guidance in the system prompt is now conditionally built
by _build_acp_section(), which checks get_acp_agents() and returns an
empty string when no ACP agents are configured. This avoids polluting
the prompt with irrelevant instructions for users who don't use ACP.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix lint

* fix(harness): address Copilot review comments on sandbox path handling and ACP tool

- local_sandbox: fix path-segment boundary bug in _resolve_path (== or startswith +"/")
  and add lookahead in _resolve_paths_in_command regex to prevent /mnt/skills matching
  inside /mnt/skills-extra
- local_sandbox_provider: replace print() with logger.warning(..., exc_info=True)
- invoke_acp_agent_tool: guard getattr(option, "optionId") with None default + continue;
  move full prompt from INFO to DEBUG level (truncated to 200 chars)
- sandbox/tools: fix _get_acp_workspace_host_path docstring to match implementation;
  remove misleading "read-only" language from validate_local_bash_command_paths

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(acp): thread-isolated workspaces, permission guardrail, and ContextVar registry

P1.1 – ACP workspace thread isolation
- Add `Paths.acp_workspace_dir(thread_id)` for per-thread paths
- `_get_work_dir(thread_id)` in invoke_acp_agent_tool now uses
  `{base_dir}/threads/{thread_id}/acp-workspace/`; falls back to
  global workspace when thread_id is absent or invalid
- `_invoke` extracts thread_id from `RunnableConfig` via
  `Annotated[RunnableConfig, InjectedToolArg]`
- `sandbox/tools.py`: `_get_acp_workspace_host_path(thread_id)`,
  `_resolve_acp_workspace_path(path, thread_id)`, and all callers
  (`replace_virtual_paths_in_command`, `mask_local_paths_in_output`,
  `ls_tool`, `read_file_tool`) now resolve ACP paths per-thread

P1.2 – ACP permission guardrail
- New `auto_approve_permissions: bool = False` field in `ACPAgentConfig`
- `_build_permission_response(options, *, auto_approve: bool)` now
  defaults to deny; only approves when `auto_approve=True`
- Document field in `config.example.yaml`

P2 – Deferred tool registry race condition
- Replace module-level `_registry` global with `contextvars.ContextVar`
- Each asyncio request context gets its own registry; worker threads
  inherit the context automatically via `loop.run_in_executor`
- Expose `get_deferred_registry` / `set_deferred_registry` /
  `reset_deferred_registry` helpers

Tests: 831 pass (57 for affected modules, 3 new tests)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sandbox): mount /mnt/acp-workspace in docker sandbox container

The AioSandboxProvider was not mounting the ACP workspace into the
sandbox container, so /mnt/acp-workspace was inaccessible when the lead
agent tried to read ACP results in docker mode.

Changes:
- `ensure_thread_dirs`: also create `acp-workspace/` (chmod 0o777) so
  the directory exists before the sandbox container starts — required
  for Docker volume mounts
- `_get_thread_mounts`: add read-only `/mnt/acp-workspace` mount using
  the per-thread host path (`host_paths.acp_workspace_dir(thread_id)`)
- Update stale CLAUDE.md description (was "fixed global workspace")

Tests: `test_aio_sandbox_provider.py` (4 new tests)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(lint): remove unused imports in test_aio_sandbox_provider

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix config

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 14:20:18 +08:00
Hiren Thakore 792c49e6af fix: align config.example.yaml to use GEMINI_API_KEY (#1367)
The commented google_api_key example referenced $GOOGLE_API_KEY but the
codebase (.env.example, generate.py scripts) uses GEMINI_API_KEY.
Closes #1364
2026-03-26 08:34:25 +08:00
Andrew Barnes ac97dc6d42 test: add unit tests for TodoMiddleware (#1307)
* test: add unit tests for TodoMiddleware

Cover context-loss detection logic:
- _todos_in_messages and _reminder_in_messages helpers
- _format_todos formatting
- Reminder injection when write_todos truncated
- No-op when todos visible or reminder already present
- abefore_model async delegation

* test: fix event loop error in todo middleware async test

Use asyncio.run() instead of get_event_loop().run_until_complete()
to avoid RuntimeError on Python 3.12 where no default event loop
exists in the main thread.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-26 00:20:50 +08:00
Andrew Barnes 1f0ae64e02 test: add unit tests for DanglingToolCallMiddleware (#1305)
* test: add unit tests for DanglingToolCallMiddleware

Cover message patching logic for dangling tool calls:
- No-op when all tool calls have responses
- Synthetic ToolMessage insertion at correct positions
- Mixed responded/dangling scenarios
- wrap_model_call and awrap_model_call integration

* test: fix async tests and strengthen override assertions

- Use @pytest.mark.anyio + async def instead of deprecated
  asyncio.get_event_loop().run_until_complete() (fixes Py3.12 CI failure)
- Assert that override() receives the correct patched messages kwarg
  in both wrap_model_call and awrap_model_call tests

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-26 00:20:08 +08:00
offliner afe325d34e Fix command syntax for container image pull (#1349)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-26 00:14:08 +08:00
吴旭云 d7e510763d fix: add null checks for runtime.context and tighten langgraph constraint (#1326)
- Add null checks for runtime.context in uploads_middleware.py and
  sandbox/middleware.py to prevent NPE when langgraph runtime context is None
- Tighten langgraph version constraint from >=1.0.6 to >=1.0.6,<1.0.10
  to avoid context=None incompatibility with langgraph-api 0.7.x

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-25 21:01:10 +08:00
Simon Su adc51e541c fix(frontend): add stable ids for chat resizable panels (#1341)
Signed-off-by: sysusugan <sugan@foxmail.com>
2026-03-25 20:58:15 +08:00
zhoutianwang fdfe08d4aa Add user configuration template for China region (#1337)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-25 18:56:06 +08:00
Henry Li 12875664f1 docs: add domestic link of coding plan (#1340) 2026-03-25 18:53:31 +08:00
greatmengqi b8bc80d89b refactor: extract shared skill installer and upload manager to harness (#1202)
* refactor: extract shared skill installer and upload manager to harness

Move duplicated business logic from Gateway routers and Client into
shared harness modules, eliminating code duplication.

New shared modules:
- deerflow.skills.installer: 6 functions (zip security, extraction, install)
- deerflow.uploads.manager: 7 functions (normalize, deduplicate, validate,
  list, delete, get_uploads_dir, ensure_uploads_dir)

Key improvements:
- SkillAlreadyExistsError replaces stringly-typed 409 status routing
- normalize_filename rejects backslash-containing filenames
- Read paths (list/delete) no longer mkdir via get_uploads_dir
- Write paths use ensure_uploads_dir for explicit directory creation
- list_files_in_dir does stat inside scandir context (no re-stat)
- install_skill_from_archive uses single is_file() check (one syscall)
- Fix agent config key not reset on update_mcp_config/update_skill

Tests: 42 new (22 installer + 20 upload manager) + client hardening

* refactor: centralize upload URL construction and clean up installer

- Extract upload_virtual_path(), upload_artifact_url(), enrich_file_listing()
  into shared manager.py, eliminating 6 duplicated URL constructions across
  Gateway router and Client
- Derive all upload URLs from VIRTUAL_PATH_PREFIX constant instead of
  hardcoded "mnt/user-data/uploads" strings
- Eliminate TOCTOU pre-checks and double file read in installer — single
  ZipFile() open with exception handling replaces is_file() + is_zipfile()
  + ZipFile() sequence
- Add missing re-exports: ensure_uploads_dir in uploads/__init__.py,
  SkillAlreadyExistsError in skills/__init__.py
- Remove redundant .lower() on already-lowercase CONVERTIBLE_EXTENSIONS
- Hoist sandbox_uploads_dir(thread_id) before loop in uploads router

* fix: add input validation for thread_id and filename length

- Reject thread_id containing unsafe filesystem characters (only allow
  alphanumeric, hyphens, underscores, dots) — prevents 500 on inputs
  like <script> or shell metacharacters
- Reject filenames longer than 255 bytes (OS limit) in normalize_filename
- Gateway upload router maps ValueError to 400 for invalid thread_id

* fix: address PR review — symlink safety, input validation coverage, error ordering

- list_files_in_dir: use follow_symlinks=False to prevent symlink metadata
  leakage; check is_dir() instead of exists() for non-directory paths
- install_skill_from_archive: restore is_file() pre-check before extension
  validation so error messages match the documented exception contract
- validate_thread_id: move from ensure_uploads_dir to get_uploads_dir so
  all entry points (upload/list/delete) are protected
- delete_uploaded_file: catch ValueError from thread_id validation (was 500)
- requires_llm marker: also skip when OPENAI_API_KEY is unset
- e2e fixture: update TitleMiddleware exclusion comment (kept filtering —
  middleware triggers extra LLM calls that add non-determinism to tests)

* chore: revert uv.lock to main — no dependency changes in this PR

* fix: use monkeypatch for global config in e2e fixture to prevent test pollution

The e2e_env fixture was calling set_title_config() and
set_summarization_config() directly, which mutated global singletons
without automatic cleanup. When pytest ran test_client_e2e.py before
test_title_middleware_core_logic.py, the leaked enabled=False caused
5 title tests to fail in CI.

Switched to monkeypatch.setattr on the module-level private variables
so pytest restores the originals after each test.

* fix: address code review — URL encoding, API consistency, test isolation

- upload_artifact_url: percent-encode filename to handle spaces/#/?
- deduplicate_filename: mutate seen set in place (caller no longer
  needs manual .add() — less error-prone API)
- list_files_in_dir: document that size is int, enrich stringifies
- e2e fixture: monkeypatch _app_config instead of set_app_config()
  to prevent global singleton pollution (same pattern as title/summarization fix)
- _make_e2e_config: read LLM connection details from env vars so
  external contributors can override defaults
- Update tests to match new deduplicate_filename contract

* docs: rewrite RFC in English and add alternatives/breaking changes sections

* fix: address code review feedback on PR #1202

- Rename deduplicate_filename to claim_unique_filename to make
  the in-place set mutation explicit in the function name
- Replace PermissionError with PathTraversalError(ValueError) for
  path traversal detection — malformed input is 400, not 403

* fix: set _app_config_is_custom in e2e test fixture to prevent config.yaml lookup in CI

---------

Co-authored-by: greatmengqi <chenmengqi.0376@bytedance.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: DanielWalnut <45447813+hetaoBackend@users.noreply.github.com>
2026-03-25 16:28:33 +08:00
Andrew Barnes ec46ae075d test: add unit tests for SubagentLimitMiddleware (#1306)
* test: add unit tests for SubagentLimitMiddleware

Cover subagent limit enforcement:
- _clamp_subagent_limit boundary clamping
- Task call truncation when exceeding limit
- Non-task tool calls preserved during truncation
- after_model/aafter_model delegation

* Update test_subagent_limit_middleware.py

* Fix import statement for MAX_CONCURRENT_SUBAGENTS

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-25 10:20:16 +08:00
Andrew Barnes afb0f66c73 test: add unit tests for skills parser (#1308)
Cover parse_skill_file logic:
- Valid SKILL.md parsing with all fields
- Missing required fields (name, description) return None
- Missing/wrong filename returns None
- Optional license field handling
- Custom and default relative_path behavior
- Colons in description values
- Empty front matter handling

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-25 10:17:40 +08:00
luo jiyin 97ad67db6b docs: fix typo and grammar issues in docs (#1315)
* docs: fix security policy wording

* docs: fix backend agents typo
2026-03-25 10:01:36 +08:00
Matthew 2eca58bd86 fix: add null checks for runtime.context in middlewares and tools (#1269)
Add defensive null checks before accessing runtime.context.get() to
prevent AttributeError when runtime.context is None. This affects:
- UploadsMiddleware
- MemoryMiddleware
- LoopDetectionMiddleware
- SandboxMiddleware
- sandbox tools
- setup_agent_tool
- present_file_tool
- task_tool

Also adds .env loading in serve.sh for environment variable support.

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-25 08:46:42 +08:00
Anna Terek f499f37e94 docs: add Russian README translation (#1311) 2026-03-25 08:39:38 +08:00
Emile Jouannet 21febe1cc9 docs: add French translation of README (#1303) 2026-03-25 08:24:02 +08:00
greatmengqi 16ed797e0e feat: add configurable log level and token usage tracking (#1301)
* feat: add configurable log level and token usage tracking

- Add `log_level` config to control deerflow module log level, synced
  to LangGraph Server via serve.sh `--server-log-level`
- Add `token_usage.enabled` config with TokenUsageMiddleware that logs
  input/output/total tokens per LLM call from usage_metadata
- Add .omc/ to .gitignore

* fix: use info level for token usage logs since feature has its own toggle

* fix: sort imports to pass lint check

---------

Co-authored-by: greatmengqi <chenmengqi.0376@bytedance.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-25 08:13:26 +08:00
d 🔹 77b8ef79ca fix(middleware): use HumanMessage in LoopDetectionMiddleware for Anthropic compat (#1300)
LoopDetectionMiddleware injected SystemMessage mid-conversation to warn
about repetitive tool calls. This crashes Anthropic models because
langchain_anthropic's _format_messages() requires system messages to
appear only at the start of the conversation — interleaved system
messages raise 'Received multiple non-consecutive system messages'.

Switch the warning injection from SystemMessage to HumanMessage, which
works with all providers (Anthropic, OpenAI, Google, etc.).

Fixes #1299

Co-authored-by: voidborne-d <voidborne-d@users.noreply.github.com>
2026-03-25 08:00:01 +08:00
Jason 067b19af00 fix: add Windows compatibility for make dev/start commands (#1297)
* fix: add Windows compatibility for make dev/start commands

On Windows with MinGW/Git Bash, the Makefile's direct shell script
execution fails with 'CreateProcess(NULL, env bash ...)' error.

This fix:
- Detects Windows via $(OS) == Windows_NT
- Uses explicit bash invocation on Windows
- Falls back to direct execution on Unix

Users need Git Bash installed (comes with Git for Windows).

Fixes #1288
Related #1278

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-24 23:01:45 +08:00
knukn a9940c391c fix(mcp): implement sync invocation wrapper for async MCP tools (#1287)
* fix(mcp): implement sync invocation wrapper for async MCP tools

Since DeerFlowClient streams synchronously, invoking async-only MCP tools
(loaded via langchain-mcp-adapters) resulted in a NotImplementedError.
This commit bridges the sync/async gap by dynamically injecting a `func`
wrapper into `StructuredTool` instances that only have a `coroutine`.

Key changes:
- Added `sync_wrapper` in `get_mcp_tools` to execute async tool calls.
- Handled nested event loops by delegating to a global `ThreadPoolExecutor`
  when an event loop is already running, avoiding `RuntimeError`.
- Added detailed error logging within the wrapper for better transparency.
- Added comprehensive test coverage in `test_mcp_sync_wrapper.py` verifying
  tool patching, event loop behavior, and exception propagation.

* refactor(mcp): extract sync wrapper to module level and fix test mocks

Addressed PR review comments:
- Extracted _make_sync_tool_wrapper to module level to avoid nested func definitions.
- Refactored tests to use the actual production helper instead of duplicating logic.
- Fixed AsyncMock patching for awaited dependencies in tests.
- Added atexit hook for graceful thread pool shutdown.
- Fixed PEP8 blank line formatting in tests.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-24 22:38:01 +08:00
kristoffern 6bf526748d fix(skills): follow symlinks when scanning custom skills directory (#1292)
os.walk() does not follow symbolic links by default. This means
custom skills installed as symlinks in skills/custom/ are discovered
as directories but never descended into, so their SKILL.md files
are never found and the skills silently fail to load.

Adding followlinks=True fixes this for users who symlink skill
directories from external projects into the custom skills folder.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-24 22:06:29 +08:00
orbisai0security 14a3fa5290 fix: use subprocess instead of os.system in analyze.py (#1289)
The data analysis skill executes shell commands using os
Resolves V-001

Co-authored-by: orbisai0security <orbisai0security@users.noreply.github.com>
2026-03-24 20:42:03 +08:00
evenboos 4b15f14647 fix: repair frontend check command and docs (#1281)
* fix: repair frontend check command and docs

* docs: 补充 Linux 下 Docker 权限排障说明
2026-03-24 17:02:54 +08:00
dependabot[bot] c5ddc6a171 build(deps): bump h3 from 1.15.5 to 1.15.10 in /frontend (#1280)
Bumps [h3](https://github.com/h3js/h3) from 1.15.5 to 1.15.10.
- [Release notes](https://github.com/h3js/h3/releases)
- [Changelog](https://github.com/h3js/h3/blob/v1.15.10/CHANGELOG.md)
- [Commits](https://github.com/h3js/h3/compare/v1.15.5...v1.15.10)

---
updated-dependencies:
- dependency-name: h3
  dependency-version: 1.15.10
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-24 14:39:57 +08:00
Willem Jiang d0049ad904 chron(ci):setup the lint check in frontend (#1276)
* chron(ci):setup the lint check in frontend

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix(ci): correct lint-check.yml indentation, add Python 3.12 setup, upgrade checkout to v4 (#1277)

* Initial plan

* Fix lint-check.yml: fix steps indentation, add Python 3.12 setup, upgrade checkout to v4

Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>
Agent-Logs-Url: https://github.com/bytedance/deer-flow/sessions/7b4d4fad-f024-453a-9f93-5fc2dd83b471

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
2026-03-24 10:48:18 +08:00
Willem Jiang 48a197555b fix(frontend): fix the build error of i18n (#1274) 2026-03-24 09:55:39 +08:00
Gao Mingfei 0431a67b68 fix(frontend): filter task tool calls when rendering SubtaskCard (#1242)
Only tool calls with name === "task" should be rendered as SubtaskCard.
Previously all tool_calls were mapped to IDs, causing SubtaskCard to
render for non-task tool calls whose IDs were never registered in the
subtask context, resulting in a TypeError on task.status.

Signed-off-by: Gao Mingfei <g199209@gmail.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-24 09:44:36 +08:00
Matt Van Horn b40b05f623 feat(frontend): display token usage per conversation turn (#1229)
Surface the usage_metadata that PR #1218 added to the streaming API.
A compact indicator in the chat header shows cumulative tokens consumed
per thread, with a tooltip breakdown of input/output/total counts.

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-24 08:59:35 +08:00
amdoi7. 8b0f3fe233 fix(threads): clean up local thread data after thread deletion (#1262)
* fix(threads): clean up local thread data after thread deletion

Delete DeerFlow-managed thread directories after the web UI removes a LangGraph thread.
This keeps local thread data in sync with conversation deletion and adds regression coverage for the cleanup flow.

* fix(threads): address thread cleanup review feedback

Encode thread cleanup URLs in the web client, keep cache updates explicit when no thread search data is cached, and return a generic 500 response from the cleanup endpoint while documenting the sanitized error behavior.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-24 00:36:08 +08:00
Jason 79acc3939a fix: add error handling for podcast generation failures (#1257)
* fix: add error handling for podcast generation failures

When TTS processing fails, the system was generating 0-second audio files
without any error indication. This fix adds:

1. Track failed TTS lines and log warning with indices
2. Raise ValueError when all TTS generation fails with helpful message
3. Check for empty audio output in mix_audio and raise error
4. Log success/failure ratio for debugging

Fixes #30

* fix: address Copilot review feedback

- Use `not audio` to catch both None and empty bytes
- Log failed lines with 1-based indices for user-friendly output
- Handle empty script case with clear error message
- Validate env vars before ThreadPoolExecutor for fast-fail on config errors

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-24 00:20:12 +08:00
Willem Jiang 3be1d841aa fix(hotkey):support to open settings with hotkey (#1259) 2026-03-23 18:53:06 +08:00
Matt Van Horn 48031e506b feat(frontend): add Cmd+K command palette and keyboard shortcuts (#1230)
* feat(frontend): add Cmd+K command palette and keyboard shortcuts

Wire up the existing shadcn/ui Command component as a global command
palette. Adds a useGlobalShortcuts hook for Cmd+K (palette), Cmd+Shift+N
(new chat), Cmd+, (settings), and Cmd+/ (shortcuts help overlay).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(frontend): address Copilot review feedback on command palette

- Normalize event.key with toLowerCase() for reliable Shift+key matching
- Replace dead deerflow:open-settings event with router.push navigation
- Use platform-appropriate Shift label (Shift+ on Windows/Linux, glyph on Mac)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-23 18:35:35 +08:00
Uchi Uchibeke a29134d7c9 feat(guardrails): add pre-tool-call authorization middleware with pluggable providers (#1240)
Add GuardrailMiddleware that evaluates every tool call before execution.
Three provider options: built-in AllowlistProvider (zero deps), OAP passport
providers (open standard), or custom providers loaded by class path.

- GuardrailProvider protocol with GuardrailRequest/Decision dataclasses
- GuardrailMiddleware (AgentMiddleware, position 5 in chain)
- AllowlistProvider for simple deny/allow by tool name
- GuardrailsConfig (Pydantic singleton, loaded from config.yaml)
- 25 tests covering allow/deny, fail-closed/open, async, GraphBubbleUp
- Comprehensive docs at backend/docs/GUARDRAILS.md

Closes #1213

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-23 18:07:33 +08:00
knukn fe75cb35ca feat(client): support agent_name injection to enable isolated memory and custom prompts (#1253)
* feat(client): 添加agent_name参数支持自定义代理名称

允许在初始化DeerFlowClient时指定代理名称,该名称将用于中间件构建和系统提示模板

* test: add coverage for agent_name parameter in DeerFlowClient

* fix(client): address PR review comments for agent_name injection

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-23 17:44:21 +08:00
infoquest-byteplus f6c54e0308 infoquest support image-search (#1255) 2026-03-23 17:06:56 +08:00
Ben Ghorbel Mohamed Aziz 38ace61617 feat(web): add conversation export as Markdown and JSON (#1002)
* feat(web): add conversation export as Markdown and JSON (#976)

Add the ability to export conversations in Markdown and JSON formats,
accessible from both the chat header and the sidebar context menu.

- Add export utility (formatThreadAsMarkdown, formatThreadAsJSON) with
  support for user/assistant messages, thinking blocks, and tool calls
- Add ExportTrigger component in chat header (appears when messages exist)
- Add Export submenu to sidebar dropdown (fetches full thread state on demand)
- Add i18n translations for en-US and zh-CN

Closes #976

Made-with: Cursor

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update thread creation timestamp to updated_at

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-23 08:21:54 +08:00
Jason 1c981ead2a fix: add ~/.codex and ~/.claude bind mounts to docker-compose-dev.yaml (#1247)
The dev compose file was missing CLI auth directory mounts that exist in
the production compose file. This caused CodexChatModel to fail with
'Codex CLI credential not found' error in dev mode.

Fixes #1246
2026-03-23 07:44:59 +08:00
Purricane 835ba041f8 feat: add Claude Code OAuth and Codex CLI as LLM providers (#1166)
* feat: add Claude Code OAuth and Codex CLI providers

Port of bytedance/deer-flow#1136 from @solanian's feat/cli-oauth-providers branch.\n\nCarries the feature forward on top of current main without the original CLA-blocked commit metadata, while preserving attribution in the commit message for review.

* fix: harden CLI credential loading

Align Codex auth loading with the current ~/.codex/auth.json shape, make Docker credential mounts directory-based to avoid broken file binds on hosts without exported credential files, and add focused loader tests.

* refactor: tighten codex auth typing

Replace the temporary Any return type in CodexChatModel._load_codex_auth with the concrete CodexCliCredential type after the credential loader was stabilized.

* fix: load Claude Code OAuth from Keychain

Match Claude Code's macOS storage strategy more closely by checking the Keychain-backed credentials store before falling back to ~/.claude/.credentials.json. Keep explicit file overrides and add focused tests for the Keychain path.

* fix: require explicit Claude OAuth handoff

* style: format thread hooks reasoning request

* docs: document CLI-backed auth providers

* fix: address provider review feedback

* fix: harden provider edge cases

* Fix deferred tools, Codex message normalization, and local sandbox paths

* chore: narrow PR scope to OAuth providers

* chore: remove unrelated frontend changes

* chore: reapply OAuth branch frontend scope cleanup

* fix: preserve upload guards with reasoning effort wiring

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-22 22:39:50 +08:00
mxyhi e119dc74ae feat(codex): support explicit OpenAI Responses API config (#1235)
* feat: support explicit OpenAI Responses API config

Co-authored-by: Codex <noreply@openai.com>

* Update backend/packages/harness/deerflow/config/model_config.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-22 20:39:26 +08:00
Gao Mingfei 644501ae07 fix(config): reload AppConfig when config path or mtime changes (#1239)
* fix(config): reload AppConfig when config path or mtime changes

- Track resolved path + mtime; invalidate cache on change
- Preserve set_app_config() injection behavior
- Add regression tests (test_app_config_reload.py)
- Document behavior in README and backend/CLAUDE.md

Signed-off-by: Gao Mingfei <g199209@gmail.com>

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Signed-off-by: Gao Mingfei <g199209@gmail.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-22 20:34:01 +08:00
haoliangxu e6c6770b70 fix(middleware): fallback to configurable thread_id in thread data middleware (#1237)
Co-authored-by: Exploreunive <Exploreunive@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-22 20:14:51 +08:00
Ryanba 894875ab1b fix(gateway): accept output_text suggestion blocks (#1238)
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-22 19:59:54 +08:00
Chetan Sharma 7a90055ede fix(telegram): fix reply ordering race condition (#1231)
* fix(telegram): fix reply ordering race condition

* fix(telegram): address async race condition and add regression test

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-22 19:38:27 +08:00
Willem Jiang 72f01a1638 Update workflow to trigger on push to main
Add push trigger for unit tests on main branch
2026-03-22 17:57:06 +08:00
haoliangxu 3af709097e fix: normalize structured LLM content in serialization and memory updater (#1215)
* fix: normalize ToolMessage structured content in serialization

When models return ToolMessage content as a list of content blocks
(e.g. [{"type": "text", "text": "..."}]), the UI previously displayed
the raw Python repr string instead of the extracted text.

Replace str(msg.content) with the existing _extract_text() helper in
both _serialize_message() and stream() to properly normalize
list-of-blocks content to plain text.

Fixes #1149

Also fixes the same root cause as #1188 (characters displayed one per
line when tool response content is returned as structured blocks).

Added 11 regression tests covering string, list-of-blocks, mixed,
empty, and fallback content types.

* fix(memory): extract text from structured LLM responses in memory updater

When LLMs return response content as list of content blocks
(e.g. [{"type": "text", "text": "..."}]) instead of plain strings,
str() produces Python repr which breaks JSON parsing in the memory
updater. This caused memory updates to silently fail.

Changes:
- Add _extract_text() helper in updater.py for safe content normalization
- Use _extract_text() instead of str(response.content) in update_memory()
- Fix format_conversation_for_update() to handle plain strings in list content
- Fix subagent executor fallback path to extract text from list content
- Replace print() with structured logging (logger.info/warning/error)
- Add 13 regression tests covering _extract_text, format_conversation,
  and update_memory with structured LLM responses

* fix: address Copilot review - defensive text extraction + logger.exception

- client.py _extract_text: use block.get('text') + isinstance check (prevent KeyError/TypeError)
- prompt.py format_conversation_for_update: same defensive check for dict text blocks
- executor.py: type-safe text extraction in both code paths, fallback to placeholder instead of str(raw_content)
- updater.py: use logger.exception() instead of logger.error() for traceback preservation

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: preserve chunked structured content without spurious newlines

* fix: restore backend unit test compatibility

---------

Co-authored-by: Exploreunive <Exploreunive@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-22 17:29:29 +08:00
dependabot[bot] 9fad717977 build(deps): bump h3 from 1.15.5 to 1.15.9 in /frontend (#1234)
Bumps [h3](https://github.com/h3js/h3) from 1.15.5 to 1.15.9.
- [Release notes](https://github.com/h3js/h3/releases)
- [Changelog](https://github.com/h3js/h3/blob/v1.15.9/CHANGELOG.md)
- [Commits](https://github.com/h3js/h3/compare/v1.15.5...v1.15.9)

---
updated-dependencies:
- dependency-name: h3
  dependency-version: 1.15.9
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-22 09:17:31 +08:00
Ikko Eltociear Ashimine 9dbcca579d docs: add Japanese README (#1209)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-21 10:37:32 +08:00
haoliangxu 06cba217c3 feat: track token usage per conversation turn (#1218)
* feat: track token usage per conversation turn

Add token usage tracking to the streaming API so consumers can monitor
cost per turn without additional API calls.

Changes:

1. _serialize_message now includes usage_metadata for AI messages in
   values events, exposing input_tokens/output_tokens/total_tokens
   from LangChain's native metadata.

2. stream() accumulates token usage across all AI messages in a turn
   and emits the cumulative totals in the end event:
   {usage: {input_tokens: N, output_tokens: N, total_tokens: N}}

3. Each messages-tuple AI event with text content now includes a
   per-message usage_metadata field for granular tracking.

This enables the frontend to display token consumption per turn,
support cost-aware UX, and let users monitor API spending.

10 tests added covering serialization passthrough and cumulative
aggregation logic.

Co-Authored-By: OpenClaw <noreply@openclaw.ai>

* fix: address Copilot review - use Mapping access for usage_metadata

- Replace getattr(usage, 'input_tokens', 0) with usage.get('input_tokens', 0)
  since LangChain usage_metadata is a dict, not an object
- Remove unused 'import pytest' (fixes Ruff F401)
- Add proper stream() integration tests for cumulative usage in end event
  and per-message usage_metadata in messages-tuple events

---------

Co-authored-by: Exploreunive <Exploreunive@users.noreply.github.com>
Co-authored-by: OpenClaw <noreply@openclaw.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-21 10:29:52 +08:00
Chetan Sharma e69dc2961f refactor: add channel-based streaming capability check (#1214) 2026-03-20 23:44:09 +08:00
dependabot[bot] 9a99485905 build(deps): bump kysely from 0.28.11 to 0.28.13 in /frontend (#1211)
Bumps [kysely](https://github.com/kysely-org/kysely) from 0.28.11 to 0.28.13.
- [Release notes](https://github.com/kysely-org/kysely/releases)
- [Commits](https://github.com/kysely-org/kysely/compare/v0.28.11...v0.28.13)

---
updated-dependencies:
- dependency-name: kysely
  dependency-version: 0.28.13
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-20 17:29:01 +08:00
Simon Su ceab7fac14 fix: improve MiniMax code plan integration (#1169)
This PR improves MiniMax Code Plan integration in DeerFlow by fixing three issues in the current flow: stream errors were not clearly surfaced in the UI, the frontend could not display the actual provider model ID, and MiniMax reasoning output could leak into final assistant content as inline <think>...</think>. The change adds a MiniMax-specific adapter, exposes real model IDs end-to-end, and adds a frontend fallback for historical messages.
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-20 17:18:59 +08:00
knukn 3b235fd182 fix(feishu): support @bot message in topic groups (#1206)
* fix(feishu): support @bot message in topic groups

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* fix(feishu): preserve rich-text formatting and add parser unit tests

* chore(test): remove unused import to fix ruff lint error

* style: auto-format imports to satisfy ruff

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-20 17:03:39 +08:00
JilongSun c037ed6739 feat(manager): add bootstrap command to initialize soul.md in correct place (#1201)
* feat(manager): add bootstrap command to initialize soul.md in correct place

* feat(channels): add /bootstrap command to IM channels

Add a `/bootstrap` command that routes to the chat handler with
`is_bootstrap: True` in the run context, allowing the agent to invoke
its setup/initialization flow (e.g. `setup_agent`).

- The text after `/bootstrap` is forwarded as the chat message; when
  omitted a default "Initialize workspace" message is used.
- Feishu channels use the streaming path as with normal chat.
- No changes to ChannelStore — bootstrap is stateless and triggered
  purely by the command.
- Update /help output to include /bootstrap.
- Add 5 tests covering: text/no-text variants, Feishu streaming path,
  thread creation, and help text.

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* fix: accept copilot suggestion

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-20 16:54:11 +08:00
greatmengqi accf5b5f8e fix: add sync after_model to TitleMiddleware (#1190) 2026-03-19 15:46:31 +08:00
Ryanba f67c3d2c9e fix(harness): skip duplicate memory facts (#1193)
* fix(harness): skip duplicate memory facts

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* docs: note memory fact deduplication

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* Apply suggestions from code review

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-18 22:41:13 +08:00
BillionToken 423ea59491 fix(scripts): handle docker-init failures gracefully for private registry (#1191)
* fix(scripts): handle docker-init failures gracefully for private registry

The make docker-init command was failing on Linux environments when users
could not access the private Volces container registry. This commonly
occurs in corporate intranet environments with proxies or for users
without registry credentials.

Changes:
- Detect sandbox mode from config.yaml before attempting image pull
- Skip image pull entirely for local sandbox mode (default)
- Gracefully handle pull failures with informative messages
- Update setup-sandbox Makefile target with same error handling

Fixes #1168

* Apply suggestions from code review

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: BillionClaw <billionclaw@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-18 22:06:35 +08:00
Ryanba 4c78188896 fix(gateway): remove generated markdown on upload delete (#1170)
* fix(gateway): remove generated markdown on upload delete

Keep thread upload storage consistent by deleting the generated markdown companion when the original convertible upload is removed.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-18 16:31:26 +08:00
Ryanba f737fbeae8 fix(frontend): block duplicate sends during uploads (#1165)
* fix(frontend): block duplicate sends during uploads

Expose pre-submit upload work as a busy state so the chat input does not allow a second send while the first attachment is still uploading.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* docs(frontend): document upload and stream ownership

Record that thread hooks own upload-before-submit state while the chat page owns composer busy wiring, so future changes do not reintroduce duplicate socket or upload state handling.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* fix(frontend): separate upload busy state from streaming

Keep uploads from reusing the streaming stop state so duplicate submits are blocked without turning the composer into a stop button during file uploads.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

---------

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-18 15:10:27 +08:00
dependabot[bot] beb0eab711 build(deps): bump pyasn1 from 0.6.2 to 0.6.3 in /backend (#1185)
Bumps [pyasn1](https://github.com/pyasn1/pyasn1) from 0.6.2 to 0.6.3.
- [Release notes](https://github.com/pyasn1/pyasn1/releases)
- [Changelog](https://github.com/pyasn1/pyasn1/blob/main/CHANGES.rst)
- [Commits](https://github.com/pyasn1/pyasn1/compare/v0.6.2...v0.6.3)

---
updated-dependencies:
- dependency-name: pyasn1
  dependency-version: 0.6.3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-18 08:05:16 +08:00
dependabot[bot] 4977c43974 build(deps): bump next from 16.1.6 to 16.1.7 in /frontend (#1186)
Bumps [next](https://github.com/vercel/next.js) from 16.1.6 to 16.1.7.
- [Release notes](https://github.com/vercel/next.js/releases)
- [Changelog](https://github.com/vercel/next.js/blob/canary/release.js)
- [Commits](https://github.com/vercel/next.js/compare/v16.1.6...v16.1.7)

---
updated-dependencies:
- dependency-name: next
  dependency-version: 16.1.7
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-18 08:04:24 +08:00
dependabot[bot] 5b37de60b7 build(deps): bump flatted from 3.3.3 to 3.4.2 in /frontend (#1184)
Bumps [flatted](https://github.com/WebReflection/flatted) from 3.3.3 to 3.4.2.
- [Commits](https://github.com/WebReflection/flatted/compare/v3.3.3...v3.4.2)

---
updated-dependencies:
- dependency-name: flatted
  dependency-version: 3.4.2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-18 07:50:32 +08:00
DanielWalnut feac03ecbc fix(harness): allow agent read access to /mnt/skills in local sandbox (#1178)
* fix(harness): allow agent read access to /mnt/skills in local sandbox

Skill files under /mnt/skills/ were blocked by the path validator,
preventing agents from reading skill definitions. This change:

- Refactors `resolve_local_tool_path` into `validate_local_tool_path`,
  a pure security gate that no longer resolves paths (left to the sandbox)
- Permits read-only access to the skills container path (/mnt/skills by
  default, configurable via config.skills.container_path)
- Blocks write access to skills paths (PermissionError)
- Allows /mnt/skills in bash command path validation
- Adds `LocalSandbox.update_path_mappings` and injects per-thread
  user-data mappings into the sandbox so all virtual-path resolution
  is handled uniformly by the sandbox layer
- Covers all new behaviour with tests

Fixes #1177

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(sandbox): unify all virtual path resolution in tools.py

Move skills path resolution from LocalSandbox into tools.py so that all
virtual-to-host path translation (user-data and skills) lives in one
layer.  LocalSandbox becomes a pure execution layer that receives only
real host paths — no more path_mappings, _resolve_path, or reverse
resolve logic.

This addresses architecture feedback that path resolution was split
across two layers (tools.py for user-data, LocalSandbox for skills),
making the flow hard to follow.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(sandbox): address Copilot review — cache-on-success and error path masking

- Replace @lru_cache with manual cache-on-success for _get_skills_container_path
  and _get_skills_host_path so transient failures at startup don't permanently
  disable skills access.
- Add _sanitize_error() helper that masks host filesystem paths in error
  messages via mask_local_paths_in_output before returning them to the agent.
- Apply _sanitize_error() to all catch-all (Exception/OSError) handlers in
  sandbox tool functions to prevent host path leakage in error output.
- Remove unused lru_cache import.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 21:44:36 +08:00
lhd 0091d9f071 feat(tools): add tool_search for deferred MCP tool loading (#1176)
* feat(tools): add tool_search for deferred MCP tool loading

When multiple MCP servers are enabled, total tool count can exceed 30-50,
causing context bloat and degraded tool selection accuracy. This adds a
deferred tool loading mechanism controlled by `tool_search.enabled` config.

- Add ToolSearchConfig with single `enabled` field
- Add DeferredToolRegistry with regex search (select:, +keyword, keyword)
- Add tool_search tool returning OpenAI-compatible function JSON
- Add DeferredToolFilterMiddleware to hide deferred schemas from bind_tools
- Add <available-deferred-tools> section to system prompt
- Enable MCP tool_name_prefix to prevent cross-server name collisions
- Add 34 unit tests covering registry, tool, prompt, and middleware

* fix: reset stale deferred registry and bump config_version

- Reset deferred registry upfront in get_available_tools() to prevent
  stale tool entries when MCP servers are disabled between calls
- Bump config_version to 2 for new tool_search config field

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(tests): mock get_app_config in prompt section tests for CI

CI has no config.yaml, causing TestDeferredToolsPromptSection to fail
with FileNotFoundError. Add autouse fixture to mock get_app_config.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 20:43:55 +08:00
Henry Li f29db80be7 docs: add coding plan from ByteDance Volcengine (#1174)
* docs: add coding plan

* docs: add coding plan
2026-03-17 14:33:47 +08:00
Henry Li cb4cae4064 docs: add README in Chinese (#1172)
Co-authored-by: Henry Li <lixin.henry@bytedance.com>
2026-03-17 13:51:01 +08:00
Karesansui 75c96300cf fix(scripts): add next-server to serve.sh cleanup trap (#1162)
The cleanup() trap kills "next dev" and "next start" but not
"next-server". Since "next start" forks a "next-server" child
process, killing the parent may leave the child running as a
zombie, holding port 3000. The startup teardown block (line 35)
already handles this, but the Ctrl+C / SIGTERM trap did not.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-17 10:07:16 +08:00
lailoo 9809af1f26 feat: add citation/reference support to deep research reports (#1143)
* feat: add citation/reference support to deep research reports (#1141)

- Enhance lead agent system prompt with mandatory citation requirements
  after web_search/web_fetch tool usage
- Add citation examples and best practices to GitHub Deep Research skill
- Add citation hints to report template (Executive Summary, Key Analysis)
- Style regular markdown links in frontend for visual distinction
  (color, underline, hover effect)
- Fix TitleMiddleware being registered when title generation is disabled

* fix: address PR review comments

- Revert TitleMiddleware conditional registration (agent.py) to avoid
  sync/async incompatibility with DeerFlowClient
- Fix markdown link rendering: merge classNames instead of overwriting,
  only set target=_blank for external http(s) URLs
- Remove unrelated package.json/pnpm-lock.yaml changes

* fix: use plain markdown links in Sources section for cleaner rendering

Inline citations in report body use [citation:Title](URL) for pill/badge style.
Sources section uses plain [Title](URL) for simple underlined link style.

* fix(frontend): render plain links as underlined text in artifact markdown

Only links with citation: prefix render as Badge pills.
Regular links in Sources section now render as underlined text links.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-17 09:51:08 +08:00
Ryanba b1913a1902 fix(harness): normalize structured content for titles (#1155)
* fix(harness): normalize structured content for titles

Flatten structured LangChain message content before prompting the title model so list/block payloads don't leak Python reprs into generated thread titles.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* Apply suggestions from code review

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-17 09:10:09 +08:00
Karesansui ab0c10f002 fix(makefile): correct docker-init help description (#1163)
The help text described docker-init as "Build the custom k3s image"
but the actual implementation (scripts/docker.sh init) only pulls
the sandbox image. Updated to match the real behavior.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-16 21:53:58 +08:00
Matt Van Horn 609ff5849f fix(frontend): gracefully handle missing WebGL context (#1147)
Wrap the OGL Renderer instantiation in a try-catch so the app does not
crash when WebGL is unavailable (e.g. hardware acceleration disabled).
The Galaxy background simply does not render instead of taking down the
entire page.

Fixes #1144

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-16 21:22:17 +08:00
Karesansui 3212c7c5a2 fix(scripts): correct Makefile target name in docker.sh restart message (#1161)
docker.sh restart() tells users to run `make docker-dev-logs`, but
this target does not exist. The correct target is `make docker-logs`.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 19:58:59 +08:00
-Astraia- 191b60a326 fix: issue 1138 windows encoding (#1139)
* fix(windows): use utf-8 for text file operations

* fix(windows): normalize sandbox path masking

* fix(windows): preserve utf-8 handling after backend split
2026-03-16 16:53:12 +08:00
DanielWalnut 76803b826f refactor: split backend into harness (deerflow.*) and app (app.*) (#1131)
* refactor: extract shared utils to break harness→app cross-layer imports

Move _validate_skill_frontmatter to src/skills/validation.py and
CONVERTIBLE_EXTENSIONS + convert_file_to_markdown to src/utils/file_conversion.py.
This eliminates the two reverse dependencies from client.py (harness layer)
into gateway/routers/ (app layer), preparing for the harness/app package split.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: split backend/src into harness (deerflow.*) and app (app.*)

Physically split the monolithic backend/src/ package into two layers:

- **Harness** (`packages/harness/deerflow/`): publishable agent framework
  package with import prefix `deerflow.*`. Contains agents, sandbox, tools,
  models, MCP, skills, config, and all core infrastructure.

- **App** (`app/`): unpublished application code with import prefix `app.*`.
  Contains gateway (FastAPI REST API) and channels (IM integrations).

Key changes:
- Move 13 harness modules to packages/harness/deerflow/ via git mv
- Move gateway + channels to app/ via git mv
- Rename all imports: src.* → deerflow.* (harness) / app.* (app layer)
- Set up uv workspace with deerflow-harness as workspace member
- Update langgraph.json, config.example.yaml, all scripts, Docker files
- Add build-system (hatchling) to harness pyproject.toml
- Add PYTHONPATH=. to gateway startup commands for app.* resolution
- Update ruff.toml with known-first-party for import sorting
- Update all documentation to reflect new directory structure

Boundary rule enforced: harness code never imports from app.
All 429 tests pass. Lint clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: add harness→app boundary check test and update docs

Add test_harness_boundary.py that scans all Python files in
packages/harness/deerflow/ and fails if any `from app.*` or
`import app.*` statement is found. This enforces the architectural
rule that the harness layer never depends on the app layer.

Update CLAUDE.md to document the harness/app split architecture,
import conventions, and the boundary enforcement test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add config versioning with auto-upgrade on startup

When config.example.yaml schema changes, developers' local config.yaml
files can silently become outdated. This adds a config_version field and
auto-upgrade mechanism so breaking changes (like src.* → deerflow.*
renames) are applied automatically before services start.

- Add config_version: 1 to config.example.yaml
- Add startup version check warning in AppConfig.from_file()
- Add scripts/config-upgrade.sh with migration registry for value replacements
- Add `make config-upgrade` target
- Auto-run config-upgrade in serve.sh and start-daemon.sh before starting services
- Add config error hints in service failure messages

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix comments

* fix: update src.* import in test_sandbox_tools_security to deerflow.*

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: handle empty config and search parent dirs for config.example.yaml

Address Copilot review comments on PR #1131:
- Guard against yaml.safe_load() returning None for empty config files
- Search parent directories for config.example.yaml instead of only
  looking next to config.yaml, fixing detection in common setups

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: correct skills root path depth and config_version type coercion

- loader.py: fix get_skills_root_path() to use 5 parent levels (was 3)
  after harness split, file lives at packages/harness/deerflow/skills/
  so parent×3 resolved to backend/packages/harness/ instead of backend/
- app_config.py: coerce config_version to int() before comparison in
  _check_config_version() to prevent TypeError when YAML stores value
  as string (e.g. config_version: "1")
- tests: add regression tests for both fixes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: update test imports from src.* to deerflow.*/app.* after harness refactor

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 22:55:52 +08:00
YolenSong 9b49a80dda feat(feishu): stream updates on a single card (#1031)
* feat(feishu): stream updates on a single card

* fix(feishu): ensure final message on stream error and warn on missing card ID

- Wrap streaming loop in try/except/finally so a is_final=True outbound
  message is always published, even when the LangGraph stream breaks
  mid-way. This prevents _running_card_ids memory leaks and ensures the
  Feishu card shows a DONE reaction instead of hanging on "Working on it".
- Log a warning when _ensure_running_card gets no message_id back from
  the Feishu reply API, making silent fallback to new-card behavior
  visible in logs.
- Add test_handle_feishu_stream_error_still_sends_final to cover the
  error path.
- Reformat service.py dict comprehension (ruff format, no logic change).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Avoid blocking inbound on Feishu card creation

---------

Co-authored-by: songyaolun <songyaolun@bytedance.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-14 22:24:35 +08:00
virtaava d18a9ae5aa feat: add LoopDetectionMiddleware to break repetitive tool call loops (#1056)
* feat: add LoopDetectionMiddleware to break repetitive tool call loops

Adds a new AgentMiddleware that detects when the agent is stuck calling
the same tools with the same arguments repeatedly, which currently runs
until the recursion limit kills the run.

Detection: per-thread sliding window of tool call hashes (name + args).
- Warn threshold (default 3): injects a "wrap up" system message
- Hard limit (default 5): strips tool_calls, forcing final text output

Includes 13 unit tests covering hashing, thresholds, window sliding,
reset, and edge cases.

Closes #1055

* fix: address PR #1056 review feedback for LoopDetectionMiddleware

- Remove unused imports (Awaitable, Callable, ModelCallResult,
  ModelRequest, ModelResponse, AIMessage) from loop_detection_middleware
- Remove unused pytest import from test file
- Fix _hash_tool_calls sort key: sort by (name, serialized args) for
  deterministic hashing when multiple calls share the same tool name
- Revert subagent_enabled default to False in agent.py to match
  DeerFlowClient and channel defaults
- Remove unrelated SearxNG tools and Next.js rewrite changes from PR

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address 2nd round review feedback on PR #1056

- Inject loop warning only once per thread (prevents context bloat)
- Add threading.Lock for thread-safe history mutations
- Use runtime.context thread_id instead of workspace_path
- Add LRU eviction for per-thread history (max 100 threads)
- Add 5 new tests covering warn-once, LRU eviction, thread isolation,
  fallback thread_id, and lock presence

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: resolve lint errors in loop detection middleware tests

Sort imports (I001) and remove unused _WARNING_MSG import (F401)
to fix ruff lint failures in CI.

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-14 22:17:54 +08:00
Octopus bbd87df6eb Add MiniMax as an OpenAI-compatible model provider (#1120)
* Add MiniMax as an OpenAI-compatible model provider

MiniMax offers high-performance LLMs (M2.5, M2.5-highspeed) with
204K context windows. This commit adds MiniMax as a selectable
provider in the configuration system.

Changes:
- Add MiniMax to SUPPORTED_MODELS with model definitions
- Add MiniMax provider configuration in conf/config.yaml
- Update documentation with MiniMax setup instructions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Update README to remove MiniMax API details

Removed mention of MiniMax API usage and configuration examples.

---------

Co-authored-by: octo-patch <octo-patch@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-14 22:05:18 +08:00
dependabot[bot] cc192a9846 build(deps): bump pyjwt from 2.10.1 to 2.12.0 in /backend (#1135)
Bumps [pyjwt](https://github.com/jpadilla/pyjwt) from 2.10.1 to 2.12.0.
- [Release notes](https://github.com/jpadilla/pyjwt/releases)
- [Changelog](https://github.com/jpadilla/pyjwt/blob/master/CHANGELOG.rst)
- [Commits](https://github.com/jpadilla/pyjwt/compare/2.10.1...2.12.0)

---
updated-dependencies:
- dependency-name: pyjwt
  dependency-version: 2.12.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-14 10:03:09 +08:00
dependabot[bot] 9983f9d296 build(deps): bump flatted from 3.3.3 to 3.4.1 in /frontend (#1134)
Bumps [flatted](https://github.com/WebReflection/flatted) from 3.3.3 to 3.4.1.
- [Commits](https://github.com/WebReflection/flatted/compare/v3.3.3...v3.4.1)

---
updated-dependencies:
- dependency-name: flatted
  dependency-version: 3.4.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-14 10:00:51 +08:00
Matt(허철진) d197d50146 fix: preserve conversation context in Telegram private chats (#1105)
* fix: preserve conversation context in Telegram private chats

In private (1-on-1) chats, set topic_id=None so all messages map to a
single DeerFlow thread per chat instead of creating a new thread for
every message. Also fix _cmd_generic to use topic_id=None in private
chats so /new correctly targets the default thread.

Group chat behavior is unchanged (reply_to or msg_id as topic_id).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: preserve conversation context in Telegram private chats

Fixes #1101

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: mirror _on_text reply logic in _cmd_generic for group chats

_cmd_generic now prefers reply_to_message.message_id over msg_id in
group/supergroup chats, consistent with _on_text. This ensures commands
like /new and /status target the correct conversation thread when sent
as a reply in group chats.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: JeffJiang <for-eleven@hotmail.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-14 09:47:24 +08:00
dependabot[bot] d6bfadab12 build(deps): bump orjson from 3.11.5 to 3.11.6 in /backend (#1133)
Bumps [orjson](https://github.com/ijl/orjson) from 3.11.5 to 3.11.6.
- [Release notes](https://github.com/ijl/orjson/releases)
- [Changelog](https://github.com/ijl/orjson/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ijl/orjson/compare/3.11.5...3.11.6)

---
updated-dependencies:
- dependency-name: orjson
  dependency-version: 3.11.6
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: DanielWalnut <45447813+hetaoBackend@users.noreply.github.com>
2026-03-13 23:03:23 +08:00
Willem Jiang 253fe4d87f feat(sandbox): harden local file access and mask host paths (#983)
* feat(sandbox): harden local file access and mask host paths

- enforce local sandbox file tools to only accept /mnt/user-data paths
- add path traversal checks against thread workspace/uploads/outputs roots
- preserve requested virtual paths in tool error messages (no host path leaks)
- mask local absolute paths in bash output back to virtual sandbox paths
- update bash tool guidance to prefer thread-local venv + python -m pip
- add regression tests for path mapping, masking, and access restrictions

Fixes #968

* feat(sandbox): restrict risky absolute paths in local bash commands

- validate absolute path usage in local-mode bash commands
- allow only /mnt/user-data virtual paths for user data access
- keep a small allowlist for system executable/device paths
- return clear permission errors for unsafe command paths
- add regression tests for bash path validation rules

* test(sandbox): add success path test for resolve_local_tool_path (#992)

* Initial plan

* test(sandbox): add success path test for resolve_local_tool_path

Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>

* fix(sandbox): reject bare virtual root early with clear error in resolve_local_tool_path (#991)

* Initial plan

* fix(sandbox): reject bare virtual root early with clear error in resolve_local_tool_path

Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
2026-03-13 22:38:32 +08:00
Frank 918ba6b5bf docs: clarify OpenRouter configuration (#1123)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-13 22:12:30 +08:00
Ryanba 5a8481416f fix(frontend): surface upload API error details (#1113)
Preserve backend upload/list/delete error details in the frontend API layer so users see the actual server failure instead of a generic message.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-13 21:55:33 +08:00
Willem Jiang a79d414695 fix: make check/config cross-platform for Windows (#1080) (#1093)
* fix: make check/config cross-platform for Windows (#1080)

- replace shell-based check/config recipes with Python entrypoints
- add a cross-platform dependency checker script
- add a cross-platform config bootstrap script
- route make targets through a Python variable for consistent invocation
- preserve existing config-abort behavior when config file already exists

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-13 21:33:12 +08:00
Ryanba b155923ab0 fix(gateway): ignore archive metadata wrappers (#1108)
* fix(gateway): ignore archive metadata wrappers

Treat top-level __MACOSX and dotfile entries as packaging metadata so valid .skill archives still resolve to their real skill directory.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* Apply suggestions from code review

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-13 21:27:54 +08:00
Ryanba cda9fb7bca fix(gateway): allow standard skill frontmatter metadata (#1103)
* fix(gateway): allow standard skill frontmatter metadata

Accept standard optional frontmatter fields during .skill installs so external skills with version, author, or compatibility metadata do not fail validation.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* docs: sync skill installer metadata behavior

Document the skill install allowlist so user-facing and backend contributor docs match the gateway validation contract.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* Apply suggestions from code review

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-13 21:23:35 +08:00
Ryanba 03cafea715 fix(gateway): normalize suggestion response content (#1098)
* fix(gateway): normalize suggestion response content

Handle list-style model content before JSON parsing so provider wrappers do not silently drop follow-up suggestions.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* docs: sync suggestions endpoint behavior

Document the rich-content normalization path so the README and backend gateway notes stay aligned with the current router contract.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-13 21:20:15 +08:00
Willem Jiang b5fcb1334a fix(memory): inject stored facts into system prompt memory context (#1083)
* fix(memory): inject stored facts into system prompt memory context

- add Facts section rendering in format_memory_for_injection
- rank facts by confidence and coerce confidence values safely
- enforce max token budget while appending fact lines
- add regression tests for fact inclusion, ordering, and budget behavior

Fixes #1059

* Update the document with the latest status

* fix(memory): harden fact injection — NaN/inf confidence, None content, incremental token budget (#1090)

* Initial plan

* fix(memory): address review feedback on confidence coercion, None content, and token budget

Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
2026-03-13 14:37:40 +08:00
Liu Jice 3521cc2668 fix(middleware): degrade tool-call exceptions to error tool messages (#1110)
* fix(middleware): degrade tool-call exceptions to error tool messages

* update script

* fix(middleware): preserve LangGraph control-flow exceptions in tool error handling
2026-03-13 09:41:59 +08:00
JeffJiang 08ea9d3038 feat: enhance Docker support with production setup and deployment script (#1086)
* feat: add `make start` command for local previewing

* Update Makefile

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: update help text for `make dev` and `make start` commands

* feat: enhance Docker support with production setup and deployment script

* feat: add production commands to Makefile

* feat: remove PostgreSQL and Redis services from Docker Compose and update deploy script

* fix: address Copilot review suggestions from Docker production PR #1086 (#10)

* Initial plan

* fix: address all review suggestions from PR #1086

Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>

* Update docker/docker-compose.yaml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* feat: remove deprecated Dockerfile.langgraph to clean up repository

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-12 22:18:18 +08:00
JeffJiang fdacb1c3a5 fix(chat): update navigation method to prevent state loss during thread remount (#1107) 2026-03-12 14:57:17 +08:00
Willem Jiang e5a21b9ba0 fix(makefile):quick fix of the makefile formate error (#1085) 2026-03-11 16:00:40 +08:00
Ryanba 4bae3c724c fix(client): Harden upload validation and conversion flow (#989)
* fix(client): Harden upload validation and conversion flow

* test(client): cover event-loop upload conversion reuse

* test(client): remove unused import in upload regression coverage

* fix(client): load optional shared checkpointer fallback

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* docs(backend): document config preflight and IM channels

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-03-11 15:17:31 +08:00
Orion a0c38a5cf3 feat: add dev-daemon target for background development mode (#1047)
* feat: add dev-daemon target for background development mode

Add a new make dev-daemon target that allows running DeerFlow services
in background mode without keeping the terminal connection.

Following the pattern of PR #1042, the implementation uses a dedicated
shell script (scripts/start-daemon.sh) for better maintainability.

- Create scripts/start-daemon.sh for daemon mode startup
- Add dev-daemon target to Makefile
- Each service writes logs to separate files (langgraph, gateway, frontend, nginx)
- Services can be stopped with make stop
- Use nohup for proper daemon process detachment
- Add cleanup on failure when services fail to start
- Use more specific pkill pattern to avoid killing unrelated nginx processes

* refactor: use wait-for-port.sh instead of hardcoded sleep in daemon script

* refactor: use specific nginx process pattern to avoid killing unrelated processes

* Revert "refactor: use specific nginx process pattern to avoid killing unrelated processes"

This reverts commit 4c369155bf.

* refactor: use consistent nginx kill pattern across all scripts

* chore(daemon): add trap for cleanup on interrupt signals

* fix(daemon): pass repo root as positional argument to nginx command

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-11 15:10:40 +08:00
LofiSu 5d4fd9cf72 fix(frontend): fix new-chat navigation stale state issue (#1077)
- Use router.replace() instead of history.replaceState() so Next.js
  router's internal state is updated on chat start. This ensures
  subsequent "New Chat" clicks are treated as a real cross-route
  navigation (actual-id → "new") rather than a no-op same-path
  navigation, which was causing stale content to persist.

- In ChatLayout, increment the SubtasksProvider key only when
  navigating TO "new" from a non-"new" route. This forces a full
  remount for a fresh new-chat state without remounting when the URL
  transitions from "new" → actual-id (which would interrupt streaming).

Made-with: Cursor

Co-authored-by: DanielWalnut <45447813+hetaoBackend@users.noreply.github.com>
2026-03-11 13:51:51 +08:00
JeffJiang 2e7964d0aa feat: add make start command for local previewing (#1078)
* feat: add `make start` command for local previewing

* Update Makefile

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update Makefile

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update scripts/check.sh

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: update help text for `make dev` and `make start` commands

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-11 13:46:05 +08:00
Willem Jiang 06c4b0c828 chore(issue):create an issue template to provide runtime information (#1069) 2026-03-11 10:28:07 +08:00
Willem Jiang 96dbee00e3 fix(tracing): support LANGCHAIN_* env fallback for LangSmith config (#1065)
* fix(tracing): support LANGCHAIN_* env fallback for LangSmith config

- add backward-compatible env parsing in tracing_config.py
- support fallback keys:
   LANGCHAIN_TRACING_V2 / LANGCHAIN_TRACING
   LANGCHAIN_API_KEY
   LANGCHAIN_PROJECT
   LANGCHAIN_ENDPOINT
- keep LANGSMITH_* as preferred source when both are present
- add regression tests in test_tracing_config.py

* fix(tracing): correct LANGSMITH_* precedence over LANGCHAIN_* for enabled flag (#1067)

* Initial plan

* fix(tracing): use first-present-wins logic for enabled flag, add precedence docs and test

Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
2026-03-11 10:26:56 +08:00
JeffJiang f836d8e17c chore(docker): Refactor sandbox state management and improve Docker integration (#1068)
* Refactor sandbox state management and improve Docker integration

- Removed FileSandboxStateStore and SandboxStateStore classes for a cleaner architecture.
- Enhanced LocalContainerBackend to handle port allocation retries and introduced environment variable support for sandbox host configuration.
- Updated Paths class to include host_base_dir for Docker volume mounts and ensured proper permissions for sandbox directories.
- Modified ExtensionsConfig to improve error handling when loading configuration files and adjusted environment variable resolution.
- Updated sandbox configuration to include a replicas option for managing concurrent sandbox containers.
- Improved logging and context management in SandboxMiddleware for better sandbox lifecycle handling.
- Enhanced network port allocation logic to bind to 0.0.0.0 for compatibility with Docker.
- Updated Docker Compose files to ensure proper volume management and environment variable configuration.
- Created scripts to ensure necessary configuration files are present before starting services.
- Cleaned up unused MCP server configurations in extensions_config.example.json.

* Address Copilot review suggestions from PR #1068 (#9)

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
2026-03-11 10:03:01 +08:00
aworki 6ae7f0c0ee fix: load all thread pages in thread lists (#1044)
* fix(frontend): load all thread pages in thread lists

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-10 23:11:08 +08:00
Xinmin Zeng d5135ab757 fix(frontend): sanitize unsupported langgraph stream modes (#1050)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-10 18:56:19 +08:00
-Astraia- 19604e7f47 fix: improve port detection in WSL (#1061) 2026-03-10 18:52:25 +08:00
JeffJiang f5bd691172 feat(middleware): introduce TodoMiddleware for context-loss detection in todo management (#1041)
* feat(middleware): introduce TodoMiddleware for context-loss detection in todo management

* Address PR #1041 review suggestions: todo reminder dedup, thread switching, artifact deselect, debug log (#8)

* Initial plan

* Handle all suggestions from PR #1041 review

Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>

* fix(chat-box): prevent automatic deselection of artifacts when switching threads
fix(hooks): reset thread state on new thread creation

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>
2026-03-10 11:24:53 +08:00
Willem Jiang cf1c4a68ea docs: fix stream_mode examples for runs stream (#1033) (#1039) 2026-03-10 11:19:31 +08:00
DanielWalnut 33f086b612 feat(channels): upload file attachments via IM channels (Slack, Telegram, Feishu) (#1040) 2026-03-10 09:11:57 +08:00
momorebi 0409f8cefd fix(subagents): cleanup background tasks after completion to prevent memory leak (#1030)
* fix(subagents): cleanup background tasks after completion to prevent memory leak

Added cleanup_background_task() function to remove completed subagent results
from the global _background_tasks dict. Found a small issue: completed tasks
were never removed, causing memory to grow indefinitely with each subagent
execution.

Alternative approaches considered:
- Future + SubagentHandle pattern: Not chosen due to requiring refactoring

Chose the simple cleanup approach for minimal code changes while effectively
resolving the memory leak.

Changes:
- Add cleanup_background_task() in executor.py
- Call cleanup in all task_tool return paths (completed, failed, timed out)

* fix(subagents): prevent race condition in background task cleanup

Address Copilot review feedback on memory leak fix:

- Add terminal state check in cleanup_background_task() to only remove
  tasks that are COMPLETED/FAILED/TIMED_OUT or have completed_at set
- Remove cleanup call from polling safety-timeout branch in task_tool
  since the task may still be running
- Add comprehensive tests for cleanup behavior including:
  - Verification that cleanup is called on terminal states
  - Verification that cleanup is NOT called on polling timeout
  - Tests for terminal state check logic in executor

This prevents KeyError when the background executor tries to update
a task that was prematurely removed from _background_tasks.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-10 07:41:48 +08:00
JeffJiang f6508e0677 feat(dev): refactor service startup to use dedicated start script (#1042) 2026-03-10 07:38:19 +08:00
Willem Jiang 46918f0786 Revert "feat(threads): paginate full history via summaries endpoint (#1022)" (#1037)
This reverts commit 2f47f1ced2.
2026-03-09 16:25:08 +08:00
aworki 2f47f1ced2 feat(threads): paginate full history via summaries endpoint (#1022)
* feat(threads): add paginated summaries API and load full history

* fix(threads): address summaries review feedback

- validate summaries sort params and log gateway failures
- page frontend thread summaries without stale query keys or silent truncation
- export router modules and tighten thread list typing

Refs: 2901804166, 2901804176, 2901804179, 2901804180, 2901804183, 2901804187, 2901804191

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-09 16:08:02 +08:00
lailoo 959b4f2b09 fix(checkpointer): return InMemorySaver instead of None when not configured (#1016) (#1019)
* fix(checkpointer): return InMemorySaver instead of None when not configured (#1016)

* fix(checkpointer): also fix get_checkpointer() to return InMemorySaver

Make all three checkpointer functions consistent:
- make_checkpointer() (async) → InMemorySaver
- checkpointer_context() (sync) → InMemorySaver
- get_checkpointer() (sync singleton) → InMemorySaver

This ensures DeerFlowClient always has a valid checkpointer.

* fix: address CI failure and Copilot review feedback

- Fix import order in test_checkpointer_none_fix.py (I001 ruff error)
- Fix type annotation: _checkpointer should be Checkpointer | None
- Update docstring: change "None if not configured" to "InMemorySaver if not configured"
- Ensure app config is loaded before checking checkpointer config to prevent incorrect InMemorySaver fallback

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-09 15:48:27 +08:00
Ashwek-Werghi 4f0a8da2ee fix(frontend): enable HTML preview for generated artifacts using srcDoc (#1001)
* fix(frontend): enable HTML preview for generated artifacts using srcDoc

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-09 15:34:44 +08:00
aworki ac1e1915ef feat(channels): make mobile session settings configurable by channel and user (#1021) 2026-03-08 22:19:40 +08:00
DanielWalnut 8871fca5cb feat: add claude-to-deerflow skill for DeerFlow API integration (#1024)
* feat: add claude-to-deerflow skill for DeerFlow API integration

Add a new skill that enables Claude Code to interact with the DeerFlow
AI agent platform via its HTTP API, including chat streaming and status
checking capabilities.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: fix telegram channel

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 22:06:24 +08:00
JeffJiang 3721c82ba8 Update Nginx configuration for uploads and improve thread ID handling (#1023)
* fix: update nginx configuration for uploads endpoint and improve thread ID handling in hooks

* Update docker/nginx/nginx.local.conf

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update frontend/src/core/threads/hooks.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-08 21:15:03 +08:00
Willem Jiang 6b5c4fe6dd fix(dev): improve gateway startup diagnostics for config errors (#1020) 2026-03-08 21:06:57 +08:00
JeffJiang cf9af1fe75 Enhance chat UI and compatible anthropic thinking messages (#1018) 2026-03-08 20:19:31 +08:00
JeffJiang 3512279ce3 feat: add thinking settings to compatible anthropic api (#1017) 2026-03-08 20:18:21 +08:00
Jason 511e9eaf5e fix(docker): remove cache_from to prevent missing cache warnings (#1013)
Partially addresses #1011

The cache_from options reference /tmp/docker-cache-* directories
that don't exist by default, causing WARN messages on startup:

WARN local cache import at /tmp/docker-cache-gateway not found
WARN local cache import at /tmp/docker-cache-langgraph not found

Fix: Comment out cache_from with setup instructions.

To re-enable caching, create the directories:
  mkdir -p /tmp/docker-cache-gateway /tmp/docker-cache-langgraph

Note: This PR only fixes the cache warnings. The main NoneType error
in #1011 requires further investigation.
2026-03-08 19:47:46 +08:00
DanielWalnut 75b7302000 feat: add IM channels for Feishu, Slack, and Telegram (#1010)
* feat: add IM channels system for Feishu, Slack, and Telegram integration

Bridge external messaging platforms to DeerFlow via LangGraph Server with
async message bus, thread management, and per-channel configuration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address review comments on IM channels system

Fix topic_id handling in store remove/list_entries and manager commands,
correct Telegram reply threading, remove unused imports/variables, update
docstrings and docs to match implementation, and prevent config mutation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* update skill creator

* fix im reply text

* fix comments

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 15:21:18 +08:00
JeffJiang d664ae5a4b Support langgraph checkpointer (#1005)
* Add checkpointer configuration to config.example.yaml

- Introduced a new section for checkpointer configuration to enable state persistence for the embedded DeerFlowClient.
- Documented supported types: memory, sqlite, and postgres, along with examples for each.
- Clarified that the LangGraph Server manages its own state persistence separately.

* refactor(checkpointer): streamline checkpointer initialization and logging

* fix(uv.lock): update revision and add new wheel URLs for brotlicffi package

* feat: add langchain-anthropic dependency and update related configurations

* Fix checkpointer lifecycle, docstring, and path resolution bugs from PR #1005 review (#4)

* Initial plan

* Address all review suggestions from PR #1005

Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>

* Fix resolve_path to always return real Path; move SQLite special-string handling to callers

Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>
2026-03-07 21:07:21 +08:00
Xinmin Zeng 09325ca28f fix: normalize presented artifact paths (#998)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-06 22:51:27 +08:00
null4536251 9d2144d431 feat: may_ask (#981)
* feat: u may ask

* chore: adjust code according to CR

* chore: adjust code according to CR

* ut: test for suggestions.py

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-06 22:39:58 +08:00
Willem Jiang 2e90101be8 chore(config):comment out the LLM model setting by default (#975)
* chore(config):comment out the LLM model setting by default

* config: update the configure of the LLM models
2026-03-06 17:47:01 +08:00
Willem Jiang cfae751902 chore(ci):add copilot instructions file (#996) 2026-03-06 17:45:02 +08:00
infoquest-byteplus 28e1257e1e support infoquest (#960)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-06 15:32:13 +08:00
FangHao 3e4a24f48b fix(subagent): support async MCP tools in subagent executor (#917)
* fix(subagent): support async MCP tools in subagent executor

SubagentExecutor.execute() was synchronous and could not handle async-only                                                                                                                                  tools like MCP tools. This caused failures when trying to use MCP tools within subagents.

Changes:
- Add _aexecute() async method using agent.astream() for async execution
- Refactor execute() to use asyncio.run() wrapping _aexecute()
- This allows subagents to use async tools (MCP) within ThreadPoolExecutor

* test(subagent): add unit tests for executor async/sync paths

Add comprehensive tests covering:
- Async _aexecute() with success/error cases
- Sync execute() wrapper using asyncio.run()
- Async tool (MCP) support verification
- Thread pool execution safety

* fix(subagent): subagent-test-circular-depend

- Use session-scoped fixture with delayed import to handle circular dependencies
    without affecting other test modules

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-06 14:40:56 +08:00
Willem Jiang 3a5e0b935d fix(backend): upgrade langgraph-api to 0.7 and stabilize memory path tests (#984)
- replace  with explicit runtime deps:
- regenerate  after dependency changes
- make  deterministic by patching
  to avoid leaked global  affecting expected paths
2026-03-06 09:44:40 +08:00
Xinmin Zeng 0c7c96d75e fix(nginx): use cross-platform local paths for pid and logs (#977)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-05 17:50:02 +08:00
JeffJiang 1b3939cb78 fix(chat): handle empty uploaded files case and improve artifact selection logic (#979)
* fix(chat): handle empty uploaded files case and improve artifact selection logic

* Update frontend/src/components/workspace/chats/chat-box.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: address code review suggestions from PR #979 (#3)

* Initial plan

* fix: address PR #979 review suggestions

- utils.ts: scope (empty) check inside <uploaded_files> tag content
- chat-box.tsx: remove stale `artifacts` from useEffect deps
- context.tsx: wrap select/deselect with useCallback for stable refs
- test: add test_empty_new_files_produces_empty_marker

Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>
2026-03-05 17:45:25 +08:00
JeffJiang b17c087174 Implement optimistic UI for file uploads and enhance message handling (#967)
* feat(upload): implement optimistic UI for file uploads and enhance message handling

* feat(middleware): enhance file handling by collecting historical uploads from directory

* feat(thread-title): update page title handling for new threads and improve loading state

* feat(uploads-middleware): enhance file extraction by verifying file existence in uploads directory

* feat(thread-stream): update file path reference to use virtual_path for uploads

* feat(tests): add core behaviour tests for UploadsMiddleware

* feat(tests): remove unused pytest import from test_uploads_middleware_core_logic.py

* feat: enhance file upload handling and localization support

- Update UploadsMiddleware to validate filenames more robustly.
- Modify MessageListItem to parse uploaded files from raw content for backward compatibility.
- Add localization for uploading messages in English and Chinese.
- Introduce parseUploadedFiles utility to extract uploaded files from message content.
2026-03-05 11:16:34 +08:00
DanielWalnut 3ada4f98b1 fix(memory): prevent file upload events from persisting in long-term memory (#971)
* fix(memory): prevent file upload events from persisting in long-term memory

Uploaded files are session-scoped and unavailable in future sessions.
Previously, upload interactions were recorded in memory, causing the
agent to search for non-existent files in subsequent conversations.

Changes:
- memory_middleware: skip human messages containing <uploaded_files>
  and their paired AI responses from the memory queue
- updater: post-process generated memory to strip upload mentions
  before saving to file
- prompt: instruct the memory LLM to ignore file upload events

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(memory): address Copilot review feedback on upload filtering

- memory_middleware: strip <uploaded_files> block from human messages
  instead of dropping the entire turn; only skip the turn (and paired
  AI response) when nothing remains after stripping
- updater: narrow the upload-scrubbing regex to explicit upload events
  (avoids false-positive removal of "User works with CSV files" etc.);
  also filter upload-event facts from the facts array
- prompt: move `import re` to module scope; skip upload-only human
  messages (empty after stripping) rather than appending "User: "

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(memory): allow optional words between 'upload' and 'file' in scrub regex

The previous pattern required 'uploading file' with no intervening words,
so 'uploading a test file' was not matched and leaked into long-term memory.
Allow up to 3 modifier words between the verb and noun (e.g. 'uploading a
test file', 'uploaded the attachment').

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test(memory): add unit tests for upload filtering in memory pipeline

Covers _filter_messages_for_memory and _strip_upload_mentions_from_memory
per Copilot review suggestion.  15 test cases verify:

- Upload-only turns (and paired AI responses) are excluded from memory queue
- User's real question is preserved when combined with an upload block
- Upload file paths are never present in filtered message content
- Intermediate tool messages are always excluded
- Multi-turn conversations: only the upload turn is dropped
- Multimodal (list-content) human messages are handled
- Upload-event sentences are removed from summaries and facts
- Legitimate file-related facts (CSV preferences, PDF exports) are preserved
- "uploading a test file" (words between verb and noun) is caught by regex

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-05 11:14:34 +08:00
Tao 6ac0042cfe fix(readme): correct typo Offiical to Official (#972) 2026-03-05 07:25:30 +08:00
Chris Chen 7149f0c9b5 Add CORS_ORIGINS to .env.example for custom frontend port support (#969)
Fixes issue #47: CORS error when frontend port isn't 3000

Users running the frontend on a port other than 3000 need to set
CORS_ORIGINS to allow cross-origin requests. This addition to
.env.example makes this configuration option visible.

Co-authored-by: GitHub Agent <agent@example.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-04 20:07:37 +08:00
Chetan Sharma cff78206da refactor: reduce repeated configurable dict lookups (#970)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-04 20:06:28 +08:00
Stable Genius 452595255e docs: add make install step before local dev (#955) (#963)
Co-authored-by: Stable Genius <259448942+stablegenius49@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-04 18:29:11 +08:00
Willem Jiang a3c8efb00b fix(make):added make config command in make file (#964) 2026-03-04 09:51:15 +08:00
JeffJiang 14d1e01149 Refactor hooks and improve error handling in chat functionality (#962)
* refactor: update useThreadChat and useThreadStream hooks for improved state management

* fix: improve error handling in agent configuration loading and enhance chat page functionality

* fix: enhance error handling in agent configuration loading

* Update frontend/src/app/workspace/agents/[agent_name]/chats/[thread_id]/page.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-04 09:50:45 +08:00
JeffJiang 7de94394d4 feat(agent):Supports custom agent and chat experience with refactoring (#957)
* feat: add agent management functionality with creation, editing, and deletion

* feat: enhance agent creation and chat experience

- Added AgentWelcome component to display agent description on new thread creation.
- Improved agent name validation with availability check during agent creation.
- Updated NewAgentPage to handle agent creation flow more effectively, including enhanced error handling and user feedback.
- Refactored chat components to streamline message handling and improve user experience.
- Introduced new bootstrap skill for personalized onboarding conversations, including detailed conversation phases and a structured SOUL.md template.
- Updated localization files to reflect new features and error messages.
- General code cleanup and optimizations across various components and hooks.

* Refactor workspace layout and agent management components

- Updated WorkspaceLayout to use useLayoutEffect for sidebar state initialization.
- Removed unused AgentFormDialog and related edit functionality from AgentCard.
- Introduced ArtifactTrigger component to manage artifact visibility.
- Enhanced ChatBox to handle artifact selection and display.
- Improved message list rendering logic to avoid loading states.
- Updated localization files to remove deprecated keys and add new translations.
- Refined hooks for local settings and thread management to improve performance and clarity.
- Added temporal awareness guidelines to deep research skill documentation.

* feat: refactor chat components and introduce thread management hooks

* feat: improve artifact file detail preview logic and clean up console logs

* feat: refactor lead agent creation logic and improve logging details

* feat: validate agent name format and enhance error handling in agent setup

* feat: simplify thread search query by removing unnecessary metadata

* feat: update query key in useDeleteThread and useRenameThread for consistency

* feat: add isMock parameter to thread and artifact handling for improved testing

* fix: reorder import of setup_agent for consistency in builtins module

* feat: append mock parameter to thread links in CaseStudySection for testing purposes

* fix: update load_agent_soul calls to use cfg.name for improved clarity

* fix: update date format in apply_prompt_template for consistency

* feat: integrate isMock parameter into artifact content loading for enhanced testing

* docs: add license section to SKILL.md for clarity and attribution

* feat(agent): enhance model resolution and agent configuration handling

* chore: remove unused import of _resolve_model_name from agents

* feat(agent): remove unused field

* fix(agent): set default value for requested_model_name in _resolve_model_name function

* feat(agent): update get_available_tools call to handle optional agent_config and improve middleware function signature

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-03 21:32:01 +08:00
Xinmin Zeng 8342e88534 fix(models): handle google provider import errors and add dependency (#952)
* fix(models): improve provider import guidance and add google provider dep

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix(reflection): prefer provider install hint on transitive import errors

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-03 14:56:54 +08:00
Xinmin Zeng 7754c49217 feat(skills): support recursive nested skill loading (#950)
* feat(skills): support recursive nested skill loading

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-02 21:02:03 +08:00
Zhiyunyao a138d5388a feat: add reasoning_effort configuration support for Doubao/GPT-5 models (#947)
* feat: Add reasoning effort configuration support

* Add `reasoning_effort` parameter to model config and agent initialization
* Support reasoning effort levels (minimal/low/medium/high) for Doubao/GPT-5 models
* Add UI controls in input box for reasoning effort selection
* Update doubao-seed-1.8 example config with reasoning effort support

Fixes & Cleanup:
* Ensure UTF-8 encoding for file operations
* Remove unused imports

* fix: set reasoning_effort to None for unsupported models

* fix: unit test error

* Update frontend/src/components/workspace/input-box.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-02 20:49:41 +08:00
haibow e399d09e8f Fix line numbering (#954)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-02 20:43:58 +08:00
Zhiyunyao 72df234636 refactor(frontend): optimize network queries and improve code readability (#919)
* refactor(frontend): optimize network queries and improve code readability
- useThreadStream: Add useStream with fetchStateHistory: {limit: 1}
- useThreads: Add select fields and disable refetchOnWindowFocus
- useModels: Disable refetchOnWindowFocus
- ChatPage and titleOfThread: Improve thread title logic
- loadModels: Refactor code for better readability

* fix: address the review comments of Copilot

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-02 20:35:46 +08:00
Willem Jiang a2f91c7594 feat(mcp): add OAuth support for HTTP/SSE MCP servers (#908)
add oauth schema to MCP server config (extensions_config.json)
support client_credentials and refresh_token grants
implement token manager with caching and pre-expiry refresh
inject OAuth Authorization header for MCP tool discovery and tool calls
extend MCP gateway config models to read/write OAuth settings
update docs and examples for OAuth configuration
add unit tests for token fetch/cache and header injection
2026-03-01 22:38:58 +08:00
エイカク 80316c131e fix(backend): Fix readability extraction crash when Node parser fails (#937)
* Fix readability fallback when Node extraction fails

* Narrow readability fallback errors and enrich logs

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-03-01 22:24:02 +08:00
atian8179 d728bb26d5 fix: use shell fallback instead of hardcoded /bin/zsh in LocalSandbox (#939)
* fix: use shell fallback instead of hardcoded /bin/zsh in LocalSandbox

Replace hardcoded /bin/zsh executable with dynamic shell detection
that falls back through /bin/zsh → /bin/bash → /bin/sh. This fixes
skill execution failures in Docker containers (python:3.12-slim)
where zsh is not available.

Closes #935

* Update backend/src/sandbox/local/local_sandbox.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: atian8179 <atian8179@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-01 22:08:07 +08:00
Willem Jiang 8c6dd9e264 fix(uploads): persist thread uploads canonically and fail fast on upload errors (#943)
* fix(uploads): persist thread uploads canonically and fail fast on upload errors

 - write uploads to thread-scoped storage first to guarantee agent visibility
 - sync files to sandbox virtual path only for non-local sandboxes
 - fix markdown conversion flow to operate on canonical saved files and sync converted files when needed
 - prevent silent attachment upload failures in frontend submit flow (show error + abort submit)
 - add regression tests for local vs non-local upload behavior
 - update upload docs with thread-first persistence and troubleshooting notes

* Update frontend/src/core/threads/hooks.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix(uploads): reject "." and ".." filenames in upload sanitization (#944)

* Initial plan

* fix(uploads): reject '.' and '..' filenames in upload sanitization

Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
2026-03-01 15:35:30 +08:00
layla 5a1ac6287e Fix typo: Offiical to Official (#942) 2026-03-01 13:09:57 +08:00
YolenSong 3d3ea84a57 test(backend): add core logic unit tests for task/title/mcp (#936)
* test(backend): add core logic unit tests for task/title/mcp

* test(backend): fix lint issues in client test modules

---------

Co-authored-by: songyaolun <songyaolun@bytedance.com>
2026-03-01 12:36:09 +08:00
Henry Li f2123efdb9 docs: #1 on GitHub Trending (#932) 2026-02-28 16:20:26 +08:00
greatmengqi 30d948711f test: add Gateway conformance tests for DeerFlowClient (#931)
Validate that all dict-returning client methods conform to Gateway
Pydantic response models (ModelsListResponse, ModelResponse,
SkillsListResponse, SkillResponse, SkillInstallResponse,
McpConfigResponse, UploadResponse, MemoryConfigResponse,
MemoryStatusResponse). Pydantic ValidationError in CI catches
schema drift between client and Gateway with zero production coupling.

Also includes prior review fixes: enhanced client methods, expanded
unit tests (67→77), live integration test improvements, and updated
documentation.

Co-authored-by: greatmengqi <chenmengqi.0376@bytedance.com>
2026-02-28 16:08:04 +08:00
greatmengqi 9d48c42a20 feat: add DeerFlowClient for embedded programmatic access (#926)
Add `DeerFlowClient` class that provides direct in-process access to
DeerFlow's agent and Gateway capabilities without requiring LangGraph
Server or Gateway API processes. This enables users to import and use
DeerFlow as a Python library.

Co-authored-by: greatmengqi <chenmengqi.0376@bytedance.com>
2026-02-28 14:38:15 +08:00
Willem Jiang 5ad8a657f4 fix(git):add .gitattributes to avoid 'bash\r' issue (#924) 2026-02-28 10:32:56 +08:00
Alex e62b3d4167 feat: add Novita AI as optional LLM provider (#910)
* feat: add Novita AI as optional LLM provider

Adds Novita AI (https://novita.ai) as an optional, OpenAI-compatible
LLM provider. 

Changes:
- Added Novita model configuration example in config.example.yaml
- Added NOVITA_API_KEY to .env.example

Usage: Set NOVITA_API_KEY in your environment and use novita-gpt-4
as the model name.

* update correct model info

* Update README.md

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-02-27 11:16:31 +08:00
Xinmin Zeng e9adaab7a6 fix(i18n): normalize locale and prevent undefined translations (#914)
* fix(i18n): guard locale input and add safe translation fallback

* refactor(i18n): isolate locale utils and normalize server cookie decode

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-02-27 08:10:38 +08:00
Salman Chishti 902ff3b9f3 Upgrade GitHub Actions to latest versions (#913)
Signed-off-by: Salman Muin Kayser Chishti <13schishti@gmail.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-02-26 22:49:32 +08:00
Salman Chishti 32a22069e9 Upgrade GitHub Actions for Node 24 compatibility (#912)
Signed-off-by: Salman Muin Kayser Chishti <13schishti@gmail.com>
2026-02-26 20:23:57 +08:00
Xinmin Zeng 6a55860a15 fix: recover from stale model context when configured models change (#898)
* fix: recover from stale model context after config model changes

* fix: fail fast on missing model config and expand model resolution tests

* fix: remove duplicate get_app_config imports

* fix: align model resolution tests with runtime imports

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: remove duplicate model resolution test case

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-26 13:54:29 +08:00
dependabot[bot] 3e6e4b0b5f build(deps): bump minimatch from 3.1.2 to 3.1.4 in /frontend (#906)
Bumps [minimatch](https://github.com/isaacs/minimatch) from 3.1.2 to 3.1.4.
- [Changelog](https://github.com/isaacs/minimatch/blob/main/changelog.md)
- [Commits](https://github.com/isaacs/minimatch/compare/v3.1.2...v3.1.4)

---
updated-dependencies:
- dependency-name: minimatch
  dependency-version: 3.1.4
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-02-25 22:34:42 +08:00
DanielWalnut d27a7a5f54 fix(middleware): fix DanglingToolCallMiddleware inserting patches at wrong position (#904)
Previously used before_model which returned {"messages": patches}, causing
LangGraph's add_messages reducer to append patches at the end of the message
list. This resulted in invalid ordering (ToolMessage after a HumanMessage)
that LLMs reject with tool call ID mismatch errors.

Switch to wrap_model_call/awrap_model_call to insert synthetic ToolMessages
immediately after each dangling AIMessage before the request reaches the LLM,
without persisting the patches to state.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-25 22:29:33 +08:00
JeffJiang 33595f0bac fix(skill): enhance data authenticity protocols and clarify reporting guidelines (#905) 2026-02-25 22:25:23 +08:00
JeffJiang 3a7251c95e fix(docker): update nginx configuration and simplify docker script (#903) 2026-02-25 22:16:43 +08:00
JeffJiang d24a66ffd3 Refactor base paths with centralized path management (#901)
* Initial plan

* refactor: centralize path management and improve memory storage configuration

* fix: update memory storage path in config.example.yaml for clarity

* Initial plan

* Address PR #901 review comments: security fixes and documentation improvements

Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>
2026-02-25 21:30:33 +08:00
JeffJiang adfe5c4b44 Enforces config env var checks and improves startup handling (#892)
* Enforces config env var checks and improves startup handling

Ensures critical environment variables are validated during config resolution,
raising clear errors if missing. Improves server startup reliability by
verifying that backend services are listening and by terminating on
misconfiguration at launch. Adds more robust feedback to developers when
API startup fails, reducing silent misconfigurations and speeding up
troubleshooting.

* Initial plan

* Implement suggestions from PR #892: fix env var checks and improve error logging

Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: foreleven <4785594+foreleven@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-02-25 16:12:59 +08:00
dependabot[bot] 6d1878fb1a build(deps): bump cryptography from 46.0.3 to 46.0.5 in /backend (#896)
Bumps [cryptography](https://github.com/pyca/cryptography) from 46.0.3 to 46.0.5.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/46.0.3...46.0.5)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-version: 46.0.5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-02-25 09:42:39 +08:00
DanielWalnut faa422072c feat(subagents): make subagent timeout configurable via config.yaml (#897)
* feat(subagents): make subagent timeout configurable via config.yaml

- Add SubagentsAppConfig supporting global and per-agent timeout_seconds
- Load subagents config section in AppConfig.from_file()
- Registry now applies config.yaml overrides without mutating builtin defaults
- Polling safety-net in task_tool is now dynamic (execution timeout + 60s buffer)
- Document subagents section in config.example.yaml
- Add make test command and enforce TDD policy in CLAUDE.md
- Add 38 unit tests covering config validation, timeout resolution, registry
  override behavior, and polling timeout formula

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(subagents): add logging for subagent timeout config and execution

- Log loaded timeout config (global default + per-agent overrides) on startup
- Log debug message in registry when config.yaml overrides a builtin timeout
- Include timeout in executor's async execution start log
- Log effective timeout and polling limit when a task is dispatched
- Fix UnboundLocalError: move max_poll_count assignment before logger.info

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* ci(backend): add lint step and run all unit tests via Makefile

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix lint

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-25 08:39:29 +08:00
dependabot[bot] 310d54e443 build(deps): bump python-multipart from 0.0.21 to 0.0.22 in /backend (#895)
Bumps [python-multipart](https://github.com/Kludex/python-multipart) from 0.0.21 to 0.0.22.
- [Release notes](https://github.com/Kludex/python-multipart/releases)
- [Changelog](https://github.com/Kludex/python-multipart/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Kludex/python-multipart/compare/0.0.21...0.0.22)

---
updated-dependencies:
- dependency-name: python-multipart
  dependency-version: 0.0.22
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-24 23:11:46 +08:00
dependabot[bot] af374cd40b build(deps): bump pillow from 12.1.0 to 12.1.1 in /backend (#894)
Bumps [pillow](https://github.com/python-pillow/Pillow) from 12.1.0 to 12.1.1.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](https://github.com/python-pillow/Pillow/compare/12.1.0...12.1.1)

---
updated-dependencies:
- dependency-name: pillow
  dependency-version: 12.1.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-02-24 22:58:54 +08:00
dependabot[bot] 50d9174123 build(deps): bump protobuf from 6.33.4 to 6.33.5 in /backend (#893)
Bumps [protobuf](https://github.com/protocolbuffers/protobuf) from 6.33.4 to 6.33.5.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Commits](https://github.com/protocolbuffers/protobuf/commits)

---
updated-dependencies:
- dependency-name: protobuf
  dependency-version: 6.33.5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-24 22:53:45 +08:00
Willem Jiang 03705acf3a fix(sandbox):deer-flow-provisioner container fails to start in local execution mode (#889) 2026-02-24 08:31:52 +08:00
Willem Jiang b5c11baece docs(config):updated the configuration of deepseek-v3 (#885) 2026-02-21 22:06:01 +08:00
CHANGXUBO 85af540076 feat: add LangSmith tracing integration (#878)
* feat: add LangSmith tracing integration

Add optional LangSmith tracing support that can be enabled via environment
variables (LANGSMITH_TRACING, LANGSMITH_API_KEY, LANGSMITH_PROJECT,
LANGSMITH_ENDPOINT). When enabled, a LangChainTracer callback is attached
to chat models and run metadata is injected for trace tagging.

Co-Authored-By: Claude <noreply@anthropic.com>

* Update backend/src/config/tracing_config.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update backend/src/agents/lead_agent/agent.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update backend/src/agents/lead_agent/agent.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update backend/src/models/factory.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Add threading lock to ensure thread-safe access to tracing configuration

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-21 16:41:34 +08:00
Zhiyunyao 75226b2fe6 docs: make README easier to follow and update related docs (#884) 2026-02-21 07:48:20 +08:00
Zhiyunyao 0d7c0826f0 chore: add a Makefile command to create all required local configuration files (#883)
* fix: polish the makefile to provide config command for local config setup

* docs: polish the instructions in README
2026-02-19 09:04:37 +08:00
Willem Jiang ea4e0139af docs: Update Quick Start instructions in README (#881)
Fixes the issue #880
2026-02-18 19:29:25 +08:00
CHANGXUBO 9f74589d09 fix: HTML artifact preview renders blank in preview mode (#876)
The condition guarding ArtifactFilePreview only allowed markdown files
through, which prevented HTML files from reaching the preview component.
Added `language === "html"` to the condition so HTML artifacts render
correctly in preview mode.

Fixes #873

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-18 10:06:21 +08:00
CHANGXUBO 67dbb10c2a fix: use /tmp/nginx.pid to avoid permission denied errors (#877)
Set pid directive to /tmp/nginx.pid in nginx.conf and nginx.local.conf
to prevent permission denied errors when running nginx as a non-root user.

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-18 10:01:51 +08:00
Willem Jiang 79c841db9e chore: Ignore legacy web folder in .gitignore (#866)
Add rule to ignore the legacy 'web' folder
2026-02-15 10:45:59 +08:00
Henry Li 2d3a22aeb0 docs: add videos and official website (#865)
* docs: add videos and official website

* docs: use public video URL
2026-02-14 23:48:29 +08:00
Willem Jiang d796c5a328 docs:Add security policy documentation (#864) 2026-02-14 21:34:41 +08:00
Willem Jiang 8039da2fc4 docs:Update README.md (#863)
Updated the link for the original Deep Research framework to point to the correct branch.
2026-02-14 21:30:37 +08:00
Willem Jiang da1bcf0573 Merge remote-tracking branch 'deer-flow-2/experimental' into main-2.x 2026-02-14 16:29:38 +08:00
Willem Jiang a66d8c94fa Prepare to merge deer-flow-2 2026-02-14 16:28:12 +08:00
Henry Li 88e89921b9 docs: update LICENSE 2026-02-13 11:49:51 +08:00
Henry Li 8f44ca595b docs: update README.md 2026-02-13 09:38:35 +08:00
Henry Li 15df224856 docs: update README.md 2026-02-13 09:25:15 +08:00
JeffJiang 4d5fdcb8db Consolidates market and data analysis skills; adds chart viz (#36)
Unifies market analysis, data analysis, and consulting reporting into a comprehensive consulting-analysis skill, enabling a two-phase workflow from analysis framework design to professional report generation. Introduces a DuckDB-based data analysis utility for Excel/CSV files and a chart-visualization skill with a flexible JS interface and extensive chart type documentation. Removes the legacy market analysis skill to streamline report generation and improve extensibility for consulting and data-driven workflows.
2026-02-12 11:08:09 +08:00
JeffJiang 300e5a519a Adds Kubernetes sandbox provisioner support (#35)
* Adds Kubernetes sandbox provisioner support

* Improves Docker dev setup by standardizing host paths

Replaces hardcoded host paths with a configurable root directory,
making the development environment more portable and easier to use
across different machines. Automatically sets the root path if not
already defined, reducing manual setup steps.
2026-02-12 11:02:09 +08:00
hetao e87fd74e17 docs(ppt-generation): enforce sequential slide image generation
Explicitly prohibit parallel image generation to ensure each slide
can use the previous slide as a reference image for visual consistency.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-11 15:36:51 +08:00
hetao 770d92fe36 feat: make max concurrent subagents configurable via runtime config
Support configuring max_concurrent_subagents (2-4, default 3) through
config.configurable, with automatic clamping and dynamic prompt generation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 11:51:11 +08:00
hetao 4a85c5de7b feat: enable skills support for subagents
Extract get_skills_prompt_section() from apply_prompt_template() so
subagents can also receive the available skills list in their system
prompt. This allows subagents to discover and load skills via read_file,
just like the lead agent.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 11:04:50 +08:00
Henry Li ebf4ec2786 chore: add pnpm-workspace.yaml 2026-02-10 22:07:33 +08:00
Henry Li eb287f095a chore: add .npmrc back 2026-02-10 22:07:25 +08:00
Henry Li 595dba6c35 chore: upgrade langchain and langgraph 2026-02-10 22:07:17 +08:00
Henry Li aeadc00f96 Merge remote-tracking branch 'refs/remotes/origin/experimental' into experimental 2026-02-10 17:30:57 +08:00
LofiSu 192078e50e Merge pull request #32 from LofiSu/experimental
Experimental
2026-02-10 12:42:38 +08:00
LofiSu c8f7bc28e1 docs: 添加技能名称冲突修复的详细文档
- 记录 public 和 custom 技能同名冲突问题的解决方案
- 详细说明所有代码改动(后端配置、API、前端)
- 包含配置格式变更、API 变更说明
- 标注已知问题暂时保留,待后续版本修复
- 提供测试建议和回滚方案

相关改动:
- 使用组合键 {category}:{name} 存储配置
- API 支持可选的 category 查询参数
- 添加类别内重复技能名称检查
- 前端传递 category 参数确保唯一性

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-10 12:41:58 +08:00
LofiSu b3a1f018ab fix: 修复新建技能后输入框无法编辑的问题
问题:点击新建技能按钮后,对话框中预设的文字无法删除或修改
原因:useEffect 依赖项包含 promptInputController.textInput,该对象在每次输入时都会重新创建,导致 useEffect 重复执行并覆盖用户输入
解决:使用 useRef 保存 setInput 方法,并跟踪已设置的初始值,确保 useEffect 只在初始值变化时执行一次

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-10 12:39:44 +08:00
LofiSu 3b2001d89d Merge pull request #31 from LofiSu/experimental
Experimental
2026-02-10 12:31:55 +08:00
LofiSu cc88823a64 fix:memory 为空时i18n字体显示 2026-02-10 12:29:14 +08:00
LofiSu f87d5678f3 feat: 改进设置页面UI和国际化支持 / Improve settings pages UI and i18n support
- 添加 rehype-raw 依赖以支持在 markdown 中渲染 HTML
  Add rehype-raw dependency to support HTML rendering in markdown

- 重构 memory-settings-page,提取 formatMemorySection 函数减少重复代码
  Refactor memory-settings-page by extracting formatMemorySection function to reduce code duplication

- 改进空状态显示,使用 HTML span 标签替代 markdown 斜体,提供更好的样式控制
  Improve empty state display by using HTML span tags instead of markdown italics for better style control

- 为 skill-settings-page 添加完整的国际化支持,替换硬编码的英文文本
  Add complete i18n support for skill-settings-page, replacing hardcoded English text

- 更新国际化文件,添加技能设置页面的空状态文本(中英文)
  Update i18n files with empty state text for skill settings page (both Chinese and English)

- 在 streamdown 插件配置中添加 rehypeRaw 以支持 HTML 渲染
  Add rehypeRaw to streamdown plugins configuration to support HTML rendering

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-10 12:15:37 +08:00
LofiSu 76292125ff Merge pull request #30 from LofiSu/experimental
fix: citations prompt
2026-02-10 11:54:41 +08:00
LofiSu 6109216d54 fix: citations prompt 2026-02-10 00:18:33 +08:00
Henry Li 13b3032d02 fix: eslint 2026-02-09 23:28:36 +08:00
Henry Li df3668ecd5 fix: eslint 2026-02-09 23:28:36 +08:00
LofiSu 11489ae4da Strip citation prefix in citation badges 2026-02-09 22:57:54 +08:00
LofiSu f8e4fe05b2 Strip citation prefix in citation badges 2026-02-09 22:57:54 +08:00
LofiSu d51e6e2f43 Merge branch 'experimental' of github.com:hetaoBackend/deer-flow into feat/citations 2026-02-09 22:05:44 +08:00
LofiSu 1af14bf7e4 Merge branch 'experimental' of github.com:hetaoBackend/deer-flow into feat/citations 2026-02-09 22:05:44 +08:00
JeffJiang 7b7e32f262 Add Kubernetes-based sandbox provider for multi-instance support (#19)
* feat: adds docker-based dev environment

* docs: updates Docker command help

* fix local dev

* feat(sandbox): add Kubernetes-based sandbox provider for multi-instance support

* fix: skills path in k8s

* feat: add example config for k8s sandbox

* fix: docker config

* fix: load skills on docker dev

* feat: support sandbox execution to Kubernetes Deployment model

* chore: rename web service name
2026-02-09 21:59:13 +08:00
JeffJiang b6da3a219e Add Kubernetes-based sandbox provider for multi-instance support (#19)
* feat: adds docker-based dev environment

* docs: updates Docker command help

* fix local dev

* feat(sandbox): add Kubernetes-based sandbox provider for multi-instance support

* fix: skills path in k8s

* feat: add example config for k8s sandbox

* fix: docker config

* fix: load skills on docker dev

* feat: support sandbox execution to Kubernetes Deployment model

* chore: rename web service name
2026-02-09 21:59:13 +08:00
LofiSu c89bd9edc9 Merge upstream/experimental: resolve conflicts (keep feat/citations)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 21:56:02 +08:00
LofiSu 8a2cac7b5a Merge upstream/experimental: resolve conflicts (keep feat/citations)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 21:56:02 +08:00
LofiSu 2f50e5d969 feat(citations): inline citation links with [citation:Title](URL)
- Backend: add citation format to lead_agent and general_purpose prompts
- Add CitationLink component (Badge + HoverCard) for citation cards
- MarkdownContent: detect citation: prefix in link text, render CitationLink
- Message/artifact/subtask: use MarkdownContent or Streamdown with CitationLink
- message-list-item: pass img via components prop (remove isHuman/img)
- message-group, subtask-card: drop unused imports; fix import order (lint)

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 21:40:20 +08:00
Henry Li 69c8b41186 feat: basic implmenetation 2026-02-09 19:02:21 +08:00
Henry Li 554ec7a91e feat: basic implmenetation 2026-02-09 19:02:21 +08:00
LofiSu 715d7436f1 Merge pull request #28 from LofiSu/experimental
chore: 移除所有 Citations 相关逻辑,为后续重构做准备
2026-02-09 16:28:37 +08:00
LofiSu 47bceca87c Merge pull request #28 from LofiSu/experimental
chore: 移除所有 Citations 相关逻辑,为后续重构做准备
2026-02-09 16:28:37 +08:00
LofiSu 6a540d8408 Merge upstream/experimental: resolve conflict in lead_agent/prompt.py
- Keep upstream subagent HARD LIMIT (max 3 task calls, batching) in subagent_reminder
- Keep our removal of Citations: do not add back 'Citations when synthesizing'

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 16:27:36 +08:00
ruitanglin 1860e10451 Merge upstream/experimental: resolve conflict in lead_agent/prompt.py
- Keep upstream subagent HARD LIMIT (max 3 task calls, batching) in subagent_reminder
- Keep our removal of Citations: do not add back 'Citations when synthesizing'

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 16:27:36 +08:00
LofiSu 46048c76ce chore: 移除所有 Citations 相关逻辑,为后续重构做准备
- Backend: 删除 lead_agent / general_purpose 中的 citations_format 与引用相关 reminder;artifacts 下载不再对 markdown 做 citation 清洗,统一走 FileResponse,保留 Response 用于二进制 inline
- Frontend: 删除 core/citations 模块、inline-citation、safe-citation-content;新增 MarkdownContent 仅做 Markdown 渲染;消息/artifact 预览与复制均使用原始 content
- i18n: 移除 citations 命名空间(loadingCitations、loadingCitationsWithCount)
- 技能与 demo: 措辞改为 references,demo 数据去掉 <citations> 块
- 文档: 更新 CLAUDE/AGENTS/README 描述,新增按文件 diff 的代码变更总结

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 16:24:01 +08:00
ruitanglin 8747873b8d chore: 移除所有 Citations 相关逻辑,为后续重构做准备
- Backend: 删除 lead_agent / general_purpose 中的 citations_format 与引用相关 reminder;artifacts 下载不再对 markdown 做 citation 清洗,统一走 FileResponse,保留 Response 用于二进制 inline
- Frontend: 删除 core/citations 模块、inline-citation、safe-citation-content;新增 MarkdownContent 仅做 Markdown 渲染;消息/artifact 预览与复制均使用原始 content
- i18n: 移除 citations 命名空间(loadingCitations、loadingCitationsWithCount)
- 技能与 demo: 措辞改为 references,demo 数据去掉 <citations> 块
- 文档: 更新 CLAUDE/AGENTS/README 描述,新增按文件 diff 的代码变更总结

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 16:24:01 +08:00
LofiSu 0f0346f73e Merge pull request #27 from LofiSu/experimental
Experimental
2026-02-09 15:59:41 +08:00
LofiSu 0093134db0 Merge pull request #27 from LofiSu/experimental
Experimental
2026-02-09 15:59:41 +08:00
LofiSu cef8d389fd refactor(frontend): consolidate citation logic, slim exports and impl
- SafeCitationContent: add loadingOnly and renderBody props.
  - loadingOnly: show only loading indicator or null (e.g. write_file step).
  - renderBody(parsed): custom body renderer (e.g. artifact preview).

- message-group write_file: use SafeCitationContent(content, isLoading,
  rehypePlugins, loadingOnly) instead of local useParsedCitations +
  shouldShowCitationLoading + CitationsLoadingIndicator. Pass rehypePlugins
  into ToolCall.

- artifact-file-detail markdown preview: use SafeCitationContent with
  renderBody((p) => <ArtifactFilePreview ... cleanContent={p.cleanContent}
  citationMap={p.citationMap} />). Remove local shouldShowCitationLoading
  and CitationsLoadingIndicator branch.

- core/citations: inline buildCitationMap into use-parsed-citations, remove
  from utils; stop exporting hasCitationsBlock (internal to shouldShowCitationLoading).

- inline-citation: make InlineCitationCard, InlineCitationCardBody,
  InlineCitationSource file-private (no longer exported).

Co-authored-by: Cursor <cursoragent@cursor.com>

---
refactor(前端): 收拢引用逻辑、精简导出与实现

- SafeCitationContent 新增 loadingOnly、renderBody。
  - loadingOnly:仅显示加载或 null(如 write_file 步骤)。
  - renderBody(parsed):自定义正文渲染(如 artifact 预览)。

- message-group write_file:改用 SafeCitationContent(loadingOnly),去掉
  本地 useParsedCitations + shouldShowCitationLoading + CitationsLoadingIndicator,
  并向 ToolCall 传入 rehypePlugins。

- artifact-file-detail 的 markdown 预览:改用 SafeCitationContent +
  renderBody 渲染 ArtifactFilePreview,去掉本地加载判断与
  CitationsLoadingIndicator 分支。

- core/citations:buildCitationMap 内联到 use-parsed-citations 并从 utils
  删除;hasCitationsBlock 不再导出(仅 shouldShowCitationLoading 内部使用)。

- inline-citation:InlineCitationCard/Body/Source 改为文件内私有,不再导出。
2026-02-09 15:58:59 +08:00
ruitanglin 59c8fec7e7 refactor(frontend): consolidate citation logic, slim exports and impl
- SafeCitationContent: add loadingOnly and renderBody props.
  - loadingOnly: show only loading indicator or null (e.g. write_file step).
  - renderBody(parsed): custom body renderer (e.g. artifact preview).

- message-group write_file: use SafeCitationContent(content, isLoading,
  rehypePlugins, loadingOnly) instead of local useParsedCitations +
  shouldShowCitationLoading + CitationsLoadingIndicator. Pass rehypePlugins
  into ToolCall.

- artifact-file-detail markdown preview: use SafeCitationContent with
  renderBody((p) => <ArtifactFilePreview ... cleanContent={p.cleanContent}
  citationMap={p.citationMap} />). Remove local shouldShowCitationLoading
  and CitationsLoadingIndicator branch.

- core/citations: inline buildCitationMap into use-parsed-citations, remove
  from utils; stop exporting hasCitationsBlock (internal to shouldShowCitationLoading).

- inline-citation: make InlineCitationCard, InlineCitationCardBody,
  InlineCitationSource file-private (no longer exported).

Co-authored-by: Cursor <cursoragent@cursor.com>

---
refactor(前端): 收拢引用逻辑、精简导出与实现

- SafeCitationContent 新增 loadingOnly、renderBody。
  - loadingOnly:仅显示加载或 null(如 write_file 步骤)。
  - renderBody(parsed):自定义正文渲染(如 artifact 预览)。

- message-group write_file:改用 SafeCitationContent(loadingOnly),去掉
  本地 useParsedCitations + shouldShowCitationLoading + CitationsLoadingIndicator,
  并向 ToolCall 传入 rehypePlugins。

- artifact-file-detail 的 markdown 预览:改用 SafeCitationContent +
  renderBody 渲染 ArtifactFilePreview,去掉本地加载判断与
  CitationsLoadingIndicator 分支。

- core/citations:buildCitationMap 内联到 use-parsed-citations 并从 utils
  删除;hasCitationsBlock 不再导出(仅 shouldShowCitationLoading 内部使用)。

- inline-citation:InlineCitationCard/Body/Source 改为文件内私有,不再导出。
2026-02-09 15:58:59 +08:00
LofiSu d9a86c10e8 fix(frontend): no half-finished citations, correct state when SSE ends
Citations:
- In shouldShowCitationLoading, treat any unreplaced [cite-N] in cleanContent
  as show-loading (no body). Fixes Ultra and other modes when refs arrive
  before the <citations> block in the stream.
- Single rule: hasUnreplacedCitationRefs(cleanContent) => true forces loading;
  then isLoading && hasCitationsBlock(rawContent) for streaming indicator.

SSE end state:
- When stream finishes, SDK may set isLoading=false before client state has
  the final message content, so UI stayed wrong until refresh.
- Store onFinish(state) as finalState in chat page; clear when stream starts.
- Pass messagesOverride={finalState.messages} to MessageList when not loading
  so the list uses server-complete messages right after SSE ends (no refresh).

Chore:
- Stop tracking .githooks/pre-commit; add .githooks/ to .gitignore (local only).

Co-authored-by: Cursor <cursoragent@cursor.com>

---
fix(前端): 杜绝半成品引用,SSE 结束时展示正确状态

引用:
- shouldShowCitationLoading 中只要 cleanContent 仍含未替换的 [cite-N] 就
  只显示加载、不渲染正文,解决流式时引用块未到就出现 [cite-1] 的问题。
- 规则:hasUnreplacedCitationRefs(cleanContent) 为真则一律显示加载;
  此外 isLoading && hasCitationsBlock 用于流式时显示「正在整理引用」。

SSE 结束状态:
- 流结束时 SDK 可能先置 isLoading=false,客户端 messages 尚未包含
  最终内容,导致需刷新才显示正确。
- 在对话页保存 onFinish(state) 为 finalState,流开始时清空。
- 非加载时向 MessageList 传入 messagesOverride={finalState.messages},
  列表在 SSE 结束后立即用服务端完整消息渲染,无需刷新。

杂项:
- 取消跟踪 .githooks/pre-commit,.gitignore 增加 .githooks/(仅本地)。
2026-02-09 15:15:20 +08:00
ruitanglin 53509eaeb1 fix(frontend): no half-finished citations, correct state when SSE ends
Citations:
- In shouldShowCitationLoading, treat any unreplaced [cite-N] in cleanContent
  as show-loading (no body). Fixes Ultra and other modes when refs arrive
  before the <citations> block in the stream.
- Single rule: hasUnreplacedCitationRefs(cleanContent) => true forces loading;
  then isLoading && hasCitationsBlock(rawContent) for streaming indicator.

SSE end state:
- When stream finishes, SDK may set isLoading=false before client state has
  the final message content, so UI stayed wrong until refresh.
- Store onFinish(state) as finalState in chat page; clear when stream starts.
- Pass messagesOverride={finalState.messages} to MessageList when not loading
  so the list uses server-complete messages right after SSE ends (no refresh).

Chore:
- Stop tracking .githooks/pre-commit; add .githooks/ to .gitignore (local only).

Co-authored-by: Cursor <cursoragent@cursor.com>

---
fix(前端): 杜绝半成品引用,SSE 结束时展示正确状态

引用:
- shouldShowCitationLoading 中只要 cleanContent 仍含未替换的 [cite-N] 就
  只显示加载、不渲染正文,解决流式时引用块未到就出现 [cite-1] 的问题。
- 规则:hasUnreplacedCitationRefs(cleanContent) 为真则一律显示加载;
  此外 isLoading && hasCitationsBlock 用于流式时显示「正在整理引用」。

SSE 结束状态:
- 流结束时 SDK 可能先置 isLoading=false,客户端 messages 尚未包含
  最终内容,导致需刷新才显示正确。
- 在对话页保存 onFinish(state) 为 finalState,流开始时清空。
- 非加载时向 MessageList 传入 messagesOverride={finalState.messages},
  列表在 SSE 结束后立即用服务端完整消息渲染,无需刷新。

杂项:
- 取消跟踪 .githooks/pre-commit,.gitignore 增加 .githooks/(仅本地)。
2026-02-09 15:15:20 +08:00
LofiSu 4f9d1d524e feat(frontend): unify citation logic and prevent half-finished citations
- Add SafeCitationContent as single component for citation-aware body:
  useParsedCitations + shouldShowCitationLoading; show loading until
  citations complete, then render body with createCitationMarkdownComponents.
  Supports optional remarkPlugins, rehypePlugins, isHuman, img.

- Refactor MessageListItem: assistant message body now uses
  SafeCitationContent only; remove duplicate useParsedCitations,
  shouldShowCitationLoading, createCitationMarkdownComponents and
  CitationsLoadingIndicator logic. Human messages keep plain
  AIElementMessageResponse (no citation parsing).

- Use SafeCitationContent for clarification, present-files (message-list),
  thinking steps and write_file loading (message-group), subtask result
  (subtask-card). Artifact markdown preview keeps same guard
  (shouldShowCitationLoading) with ArtifactFilePreview.

- Unify loading condition: shouldShowCitationLoading(rawContent,
  cleanContent, isLoading) is the single source of truth. Show loading when
  (isLoading && hasCitationsBlock(rawContent)) or when
  (hasCitationsBlock(rawContent) && hasUnreplacedCitationRefs(cleanContent))
  so Pro/Ultra modes also show "loading citations" and half-finished
  [cite-N] never appear.

- message-group write_file: replace hasCitationsBlock + threadIsLoading
  with shouldShowCitationLoading(fileContent, cleanContent,
  threadIsLoading && isLast) for consistency.

- citations/utils: parse incomplete <citations> during streaming;
  remove isCitationsBlockIncomplete; keep hasUnreplacedCitationRefs
  internal; document display rule in file header.

Co-authored-by: Cursor <cursoragent@cursor.com>

---
feat(前端): 统一引用逻辑并杜绝半成品引用

- 新增 SafeCitationContent 作为引用正文的唯一出口:内部使用
  useParsedCitations + shouldShowCitationLoading,在引用未就绪时只显示
  「正在整理引用」,就绪后用 createCitationMarkdownComponents 渲染正文;
  支持可选 remarkPlugins、rehypePlugins、isHuman、img。

- 重构 MessageListItem:助手消息正文仅通过 SafeCitationContent 渲染,
  删除重复的 useParsedCitations、shouldShowCitationLoading、
  createCitationMarkdownComponents、CitationsLoadingIndicator 等逻辑;
  用户消息仍用 AIElementMessageResponse,不做引用解析。

- 澄清、present-files(message-list)、思考步骤与 write_file 加载
  (message-group)、子任务结果(subtask-card)均使用
  SafeCitationContent;Artifact 的 markdown 预览仍用同一 guard
  shouldShowCitationLoading,正文由 ArtifactFilePreview 渲染。

- 统一加载条件:shouldShowCitationLoading(rawContent, cleanContent,
  isLoading) 为唯一判断。在「流式中且已有引用块」或「有引用块且
  cleanContent 中仍有未替换的 [cite-N]」时仅显示加载,从而在 Pro/Ultra
  下也能看到「正在整理引用」,且永不出现半成品 [cite-N]。

- message-group 的 write_file:用 shouldShowCitationLoading(
  fileContent, cleanContent, threadIsLoading && isLast) 替代
  hasCitationsBlock + threadIsLoading,与其他场景一致。

- citations/utils:流式时解析未闭合的 <citations>;移除
  isCitationsBlockIncomplete;hasUnreplacedCitationRefs 保持内部使用;
  在文件头注释中说明展示规则。
2026-02-09 15:01:51 +08:00
ruitanglin a4268cb6d3 feat(frontend): unify citation logic and prevent half-finished citations
- Add SafeCitationContent as single component for citation-aware body:
  useParsedCitations + shouldShowCitationLoading; show loading until
  citations complete, then render body with createCitationMarkdownComponents.
  Supports optional remarkPlugins, rehypePlugins, isHuman, img.

- Refactor MessageListItem: assistant message body now uses
  SafeCitationContent only; remove duplicate useParsedCitations,
  shouldShowCitationLoading, createCitationMarkdownComponents and
  CitationsLoadingIndicator logic. Human messages keep plain
  AIElementMessageResponse (no citation parsing).

- Use SafeCitationContent for clarification, present-files (message-list),
  thinking steps and write_file loading (message-group), subtask result
  (subtask-card). Artifact markdown preview keeps same guard
  (shouldShowCitationLoading) with ArtifactFilePreview.

- Unify loading condition: shouldShowCitationLoading(rawContent,
  cleanContent, isLoading) is the single source of truth. Show loading when
  (isLoading && hasCitationsBlock(rawContent)) or when
  (hasCitationsBlock(rawContent) && hasUnreplacedCitationRefs(cleanContent))
  so Pro/Ultra modes also show "loading citations" and half-finished
  [cite-N] never appear.

- message-group write_file: replace hasCitationsBlock + threadIsLoading
  with shouldShowCitationLoading(fileContent, cleanContent,
  threadIsLoading && isLast) for consistency.

- citations/utils: parse incomplete <citations> during streaming;
  remove isCitationsBlockIncomplete; keep hasUnreplacedCitationRefs
  internal; document display rule in file header.

Co-authored-by: Cursor <cursoragent@cursor.com>

---
feat(前端): 统一引用逻辑并杜绝半成品引用

- 新增 SafeCitationContent 作为引用正文的唯一出口:内部使用
  useParsedCitations + shouldShowCitationLoading,在引用未就绪时只显示
  「正在整理引用」,就绪后用 createCitationMarkdownComponents 渲染正文;
  支持可选 remarkPlugins、rehypePlugins、isHuman、img。

- 重构 MessageListItem:助手消息正文仅通过 SafeCitationContent 渲染,
  删除重复的 useParsedCitations、shouldShowCitationLoading、
  createCitationMarkdownComponents、CitationsLoadingIndicator 等逻辑;
  用户消息仍用 AIElementMessageResponse,不做引用解析。

- 澄清、present-files(message-list)、思考步骤与 write_file 加载
  (message-group)、子任务结果(subtask-card)均使用
  SafeCitationContent;Artifact 的 markdown 预览仍用同一 guard
  shouldShowCitationLoading,正文由 ArtifactFilePreview 渲染。

- 统一加载条件:shouldShowCitationLoading(rawContent, cleanContent,
  isLoading) 为唯一判断。在「流式中且已有引用块」或「有引用块且
  cleanContent 中仍有未替换的 [cite-N]」时仅显示加载,从而在 Pro/Ultra
  下也能看到「正在整理引用」,且永不出现半成品 [cite-N]。

- message-group 的 write_file:用 shouldShowCitationLoading(
  fileContent, cleanContent, threadIsLoading && isLast) 替代
  hasCitationsBlock + threadIsLoading,与其他场景一致。

- citations/utils:流式时解析未闭合的 <citations>;移除
  isCitationsBlockIncomplete;hasUnreplacedCitationRefs 保持内部使用;
  在文件头注释中说明展示规则。
2026-02-09 15:01:51 +08:00
Henry Li cbe0f3b32f feat: update translations 2026-02-09 13:57:46 +08:00
Henry Li 738c509c7e feat: update translations 2026-02-09 13:57:46 +08:00
hetao f68b3c26c3 feat: enforce subagent concurrency limit of 3 per turn with batch execution
Strengthen the SUBAGENT_SECTION prompt to prevent the model from launching
more than 3 subagents in a single response. When >3 sub-tasks are needed,
the model is now explicitly instructed to plan and execute in sequential
batches of ≤3. Reinforced at three prompt injection points: thinking style,
main subagent instructions, and critical reminders.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 13:50:54 +08:00
hetao 3aa45ff035 feat: enforce subagent concurrency limit of 3 per turn with batch execution
Strengthen the SUBAGENT_SECTION prompt to prevent the model from launching
more than 3 subagents in a single response. When >3 sub-tasks are needed,
the model is now explicitly instructed to plan and execute in sequential
batches of ≤3. Reinforced at three prompt injection points: thinking style,
main subagent instructions, and critical reminders.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 13:50:54 +08:00
LofiSu 804d988002 chore: add pre-commit hook to reject *@bytedance.com author/committer email
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 13:46:18 +08:00
ruitanglin 79c85d6410 chore: add pre-commit hook to reject *@bytedance.com author/committer email
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 13:46:18 +08:00
hetao caf12da0f2 feat: add DanglingToolCallMiddleware and SubagentLimitMiddleware
Add two new middlewares to improve robustness of the agent pipeline:
- DanglingToolCallMiddleware injects placeholder ToolMessages for
  interrupted tool calls, preventing LLM errors from malformed history
- SubagentLimitMiddleware truncates excess parallel task tool calls at
  the model response level, replacing the runtime check in task_tool

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 13:22:49 +08:00
hetao 48e3039055 feat: add DanglingToolCallMiddleware and SubagentLimitMiddleware
Add two new middlewares to improve robustness of the agent pipeline:
- DanglingToolCallMiddleware injects placeholder ToolMessages for
  interrupted tool calls, preventing LLM errors from malformed history
- SubagentLimitMiddleware truncates excess parallel task tool calls at
  the model response level, replacing the runtime check in task_tool

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 13:22:49 +08:00
LofiSu 2c3ddbb9e5 Merge pull request #26 from LofiSu/experimental
引用(Citations)优化、Gateway 路径工具抽离、模式悬停说明与中英文国际化
2026-02-09 13:10:37 +08:00
LofiSu dda6b57b46 Merge pull request #26 from LofiSu/experimental
引用(Citations)优化、Gateway 路径工具抽离、模式悬停说明与中英文国际化
2026-02-09 13:10:37 +08:00
LofiSu da9b8e3333 Merge upstream/experimental, resolve conflicts in input-box, message-group, en-US, zh-CN
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 13:10:14 +08:00
ruitanglin 2bac0b904f Merge upstream/experimental, resolve conflicts in input-box, message-group, en-US, zh-CN
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 13:10:14 +08:00
ruitanglin 412923708f Merge upstream/experimental, resolve conflicts in input-box, message-group, en-US, zh-CN
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 13:10:14 +08:00
LofiSu 1c9a969a70 i18n(zh-CN): keep Pro and Ultra as English in mode labels
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 13:02:54 +08:00
ruitanglin 9af66f384b i18n(zh-CN): keep Pro and Ultra as English in mode labels
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 13:02:54 +08:00
LofiSu 0fff2880d3 Merge upstream/experimental and resolve conflicts; citations + path_utils + mode-hover
## 冲突解决 (Resolve conflicts)
- input-box.tsx: 保留 ModeHoverGuide 包裹的模式选择器(PR #26 的 mode-hover-guide)
- message-group.tsx: 保留 getCleanContent / hasCitationsBlock / useParsedCitations
- message-list-item.tsx: 保留 useParsedCitations,移除重复的 MessageLink(使用 CitationAwareLink)
- artifact-file-detail.tsx: 保留 CitationAwareLink、useParsedCitations、contentWithoutCitationsFromParsed
- artifacts.py: 保留 path_utils 与 _extract_citation_urls + remove_citations_block 精简实现
- citations/index.ts: 保留并补充 contentWithoutCitationsFromParsed 导出
- en-US.ts: 保留 Ultra 模式描述 "Reasoning, planning and execution with subagents..."
- zh-CN.ts: 保留「超级」标签,描述保留「思考、计划并执行,可调用子代理分工协作...」

## PR #26 代码改动汇总

### 1. Citations(引用)
- lead_agent prompt: 增加 Web search 与子代理合成时的 citation 提示
- general_purpose: 子代理 system prompt 增加 <citations_format> 说明
- frontend utils: 新增 contentWithoutCitationsFromParsed,removeAllCitations 基于单次解析
- frontend artifact: 使用 contentWithoutCitationsFromParsed(parsed) 避免对同一内容解析两次
- backend artifacts: _extract_citation_urls + remove_citations_block,json 提到顶部

### 2. path_utils(路径解析)
- 新增 backend/src/gateway/path_utils.py:resolve_thread_virtual_path,防 path traversal
- artifacts.py / skills.py:删除内联路径解析,统一使用 path_utils

### 3. Mode hover guide
- input-box: 模式选择器外包 ModeHoverGuide,悬停展示模式说明

### 4. i18n
- en: ultraModeDescription 与 zh: ultraMode / ultraModeDescription 与上游对齐并保留 PR 文案

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 13:01:01 +08:00
ruitanglin 58e10b0bca Merge upstream/experimental and resolve conflicts; citations + path_utils + mode-hover
## 冲突解决 (Resolve conflicts)
- input-box.tsx: 保留 ModeHoverGuide 包裹的模式选择器(PR #26 的 mode-hover-guide)
- message-group.tsx: 保留 getCleanContent / hasCitationsBlock / useParsedCitations
- message-list-item.tsx: 保留 useParsedCitations,移除重复的 MessageLink(使用 CitationAwareLink)
- artifact-file-detail.tsx: 保留 CitationAwareLink、useParsedCitations、contentWithoutCitationsFromParsed
- artifacts.py: 保留 path_utils 与 _extract_citation_urls + remove_citations_block 精简实现
- citations/index.ts: 保留并补充 contentWithoutCitationsFromParsed 导出
- en-US.ts: 保留 Ultra 模式描述 "Reasoning, planning and execution with subagents..."
- zh-CN.ts: 保留「超级」标签,描述保留「思考、计划并执行,可调用子代理分工协作...」

## PR #26 代码改动汇总

### 1. Citations(引用)
- lead_agent prompt: 增加 Web search 与子代理合成时的 citation 提示
- general_purpose: 子代理 system prompt 增加 <citations_format> 说明
- frontend utils: 新增 contentWithoutCitationsFromParsed,removeAllCitations 基于单次解析
- frontend artifact: 使用 contentWithoutCitationsFromParsed(parsed) 避免对同一内容解析两次
- backend artifacts: _extract_citation_urls + remove_citations_block,json 提到顶部

### 2. path_utils(路径解析)
- 新增 backend/src/gateway/path_utils.py:resolve_thread_virtual_path,防 path traversal
- artifacts.py / skills.py:删除内联路径解析,统一使用 path_utils

### 3. Mode hover guide
- input-box: 模式选择器外包 ModeHoverGuide,悬停展示模式说明

### 4. i18n
- en: ultraModeDescription 与 zh: ultraMode / ultraModeDescription 与上游对齐并保留 PR 文案

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 13:01:01 +08:00
LofiSu 2a39947830 feat: citations prompts, path_utils, and citation code cleanup
- Prompt: add citation reminders for web_search and subagent synthesis (lead_agent, general_purpose)
- Gateway: add path_utils for shared thread virtual path resolution; refactor artifacts and skills to use it
- Citations: simplify removeAllCitations (single parse); backend _extract_citation_urls and remove_citations_block cleanup

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 12:55:12 +08:00
ruitanglin eb5782b93b feat: citations prompts, path_utils, and citation code cleanup
- Prompt: add citation reminders for web_search and subagent synthesis (lead_agent, general_purpose)
- Gateway: add path_utils for shared thread virtual path resolution; refactor artifacts and skills to use it
- Citations: simplify removeAllCitations (single parse); backend _extract_citation_urls and remove_citations_block cleanup

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 12:55:12 +08:00
LofiSu 8168ea47b3 chore(frontend): remove unused Citation UI components from inline-citation
- Remove InlineCitation, InlineCitationText, InlineCitationCardTrigger
- Remove InlineCitationCarousel and all Carousel subcomponents (Content, Item, Header, Index, Prev, Next)
- Remove InlineCitationQuote
- Drop Carousel/carousel and ArrowLeft/ArrowRight icon imports; keep only CitationLink, CitationAwareLink, CitationsLoadingIndicator and their dependencies

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 12:49:31 +08:00
ruitanglin 2b10b97bb9 chore(frontend): remove unused Citation UI components from inline-citation
- Remove InlineCitation, InlineCitationText, InlineCitationCardTrigger
- Remove InlineCitationCarousel and all Carousel subcomponents (Content, Item, Header, Index, Prev, Next)
- Remove InlineCitationQuote
- Drop Carousel/carousel and ArrowLeft/ArrowRight icon imports; keep only CitationLink, CitationAwareLink, CitationsLoadingIndicator and their dependencies

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 12:49:31 +08:00
LofiSu d265bdb245 feat(frontend): add mode hover guide and adjust mode i18n
## 中文

### 代码改动
- **新增** `frontend/src/components/workspace/mode-hover-guide.tsx`
  - 新增 ModeHoverGuide 组件:接收 mode (flash/thinking/pro/ultra) 与 children,用 Tooltip 包裹
  - hover 时展示该模式名称与简介,支持 showTitle 控制是否显示模式名
  - 文案通过 useI18n 从 inputBox 的 *Mode / *ModeDescription 读取,中英文已支持
- **修改** `frontend/src/components/workspace/input-box.tsx`
  - 在模式选择器触发按钮外包一层 ModeHoverGuide,悬停当前模式时显示说明
- **修改** `frontend/src/core/i18n/locales/zh-CN.ts`
  - ultraModeDescription:改为完整描述「思考、计划并执行,可调用子代理分工协作,适合复杂多步骤任务,能力最强」(不再仅写「专业模式加子代理」)
  - proMode / ultraMode:中文环境下保留英文原文 "Pro"、"Ultra",不再翻译为「专业」「超级」
- **修改** `frontend/src/core/i18n/locales/en-US.ts`
  - ultraModeDescription:改为 "Reasoning, planning and execution with subagents to divide work; best for complex multi-step tasks"

### 说明
为 Flash / 思考 / Pro / Ultra 四种模式增加 hover 说明,并统一超级模式文案与 Pro/Ultra 在中文下的展示。

Co-authored-by: Cursor <cursoragent@cursor.com>

---

## English

### Code changes
- **Add** `frontend/src/components/workspace/mode-hover-guide.tsx`
  - New ModeHoverGuide component: takes mode (flash/thinking/pro/ultra) and children, wraps in Tooltip
  - On hover shows mode name and short description; showTitle toggles mode name in tooltip
  - Copy from useI18n (inputBox *Mode / *ModeDescription), i18n in zh-CN and en-US
- **Update** `frontend/src/components/workspace/input-box.tsx`
  - Wrap mode selector trigger with ModeHoverGuide so hovering shows current mode description
- **Update** `frontend/src/core/i18n/locales/zh-CN.ts`
  - ultraModeDescription: full description (reasoning, planning, execution, subagents, complex tasks); no longer "Pro + subagents" only
  - proMode / ultraMode: keep English "Pro" and "Ultra" in zh locale instead of "专业" / "超级"
- **Update** `frontend/src/core/i18n/locales/en-US.ts`
  - ultraModeDescription: "Reasoning, planning and execution with subagents to divide work; best for complex multi-step tasks"

### Summary
Hover guide for all four modes (Flash / Reasoning / Pro / Ultra); clearer Ultra copy and Pro/Ultra labels in Chinese.
2026-02-09 12:33:16 +08:00
ruitanglin 5e000f1a99 feat(frontend): add mode hover guide and adjust mode i18n
## 中文

### 代码改动
- **新增** `frontend/src/components/workspace/mode-hover-guide.tsx`
  - 新增 ModeHoverGuide 组件:接收 mode (flash/thinking/pro/ultra) 与 children,用 Tooltip 包裹
  - hover 时展示该模式名称与简介,支持 showTitle 控制是否显示模式名
  - 文案通过 useI18n 从 inputBox 的 *Mode / *ModeDescription 读取,中英文已支持
- **修改** `frontend/src/components/workspace/input-box.tsx`
  - 在模式选择器触发按钮外包一层 ModeHoverGuide,悬停当前模式时显示说明
- **修改** `frontend/src/core/i18n/locales/zh-CN.ts`
  - ultraModeDescription:改为完整描述「思考、计划并执行,可调用子代理分工协作,适合复杂多步骤任务,能力最强」(不再仅写「专业模式加子代理」)
  - proMode / ultraMode:中文环境下保留英文原文 "Pro"、"Ultra",不再翻译为「专业」「超级」
- **修改** `frontend/src/core/i18n/locales/en-US.ts`
  - ultraModeDescription:改为 "Reasoning, planning and execution with subagents to divide work; best for complex multi-step tasks"

### 说明
为 Flash / 思考 / Pro / Ultra 四种模式增加 hover 说明,并统一超级模式文案与 Pro/Ultra 在中文下的展示。

Co-authored-by: Cursor <cursoragent@cursor.com>

---

## English

### Code changes
- **Add** `frontend/src/components/workspace/mode-hover-guide.tsx`
  - New ModeHoverGuide component: takes mode (flash/thinking/pro/ultra) and children, wraps in Tooltip
  - On hover shows mode name and short description; showTitle toggles mode name in tooltip
  - Copy from useI18n (inputBox *Mode / *ModeDescription), i18n in zh-CN and en-US
- **Update** `frontend/src/components/workspace/input-box.tsx`
  - Wrap mode selector trigger with ModeHoverGuide so hovering shows current mode description
- **Update** `frontend/src/core/i18n/locales/zh-CN.ts`
  - ultraModeDescription: full description (reasoning, planning, execution, subagents, complex tasks); no longer "Pro + subagents" only
  - proMode / ultraMode: keep English "Pro" and "Ultra" in zh locale instead of "专业" / "超级"
- **Update** `frontend/src/core/i18n/locales/en-US.ts`
  - ultraModeDescription: "Reasoning, planning and execution with subagents to divide work; best for complex multi-step tasks"

### Summary
Hover guide for all four modes (Flash / Reasoning / Pro / Ultra); clearer Ultra copy and Pro/Ultra labels in Chinese.
2026-02-09 12:33:16 +08:00
LofiSu 30e1760211 refactor(frontend): simplify and deduplicate Citation-related code
- Extract removeCitationsBlocks in utils, reuse in parseCitations and removeAllCitations
- Add hasCitationsBlock; isCitationsBlockIncomplete now uses it
- Add useParsedCitations hook (parseCitations + buildCitationMap) for message/artifact
- Add CitationAwareLink to unify link rendering (message-list-item + artifact-file-detail)
- Add getCleanContent helper; message-group uses it and useParsedCitations
- ArtifactFileDetail: single useParsedCitations, pass cleanContent/citationMap to Preview
- Stop exporting buildCitationMap and removeCitationsBlocks from citations index
- Remove duplicate MessageLink and inline link logic in artifact preview

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 12:13:06 +08:00
ruitanglin 175c1d2e3b refactor(frontend): simplify and deduplicate Citation-related code
- Extract removeCitationsBlocks in utils, reuse in parseCitations and removeAllCitations
- Add hasCitationsBlock; isCitationsBlockIncomplete now uses it
- Add useParsedCitations hook (parseCitations + buildCitationMap) for message/artifact
- Add CitationAwareLink to unify link rendering (message-list-item + artifact-file-detail)
- Add getCleanContent helper; message-group uses it and useParsedCitations
- ArtifactFileDetail: single useParsedCitations, pass cleanContent/citationMap to Preview
- Stop exporting buildCitationMap and removeCitationsBlocks from citations index
- Remove duplicate MessageLink and inline link logic in artifact preview

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 12:13:06 +08:00
LofiSu f0423b88ac Merge branch 'hetaoBackend:experimental' into experimental 2026-02-09 11:58:56 +08:00
LofiSu 302211696e Merge branch 'hetaoBackend:experimental' into experimental 2026-02-09 11:58:56 +08:00
Henry Li 8b053a4415 feat: update workspace header to conditionally render title based on environment variable 2026-02-09 09:20:32 +08:00
Henry Li fd4f6c679a feat: update workspace header to conditionally render title based on environment variable 2026-02-09 09:20:32 +08:00
Henry Li 3ad2cd936f feat: update workspace header to conditionally render title based on environment variable 2026-02-09 09:20:32 +08:00
Henry Li 305e8969ef feat: make it golden 2026-02-09 09:15:39 +08:00
Henry Li 189fcab4c5 feat: make it golden 2026-02-09 09:15:39 +08:00
Henry Li e6261469ef feat: make it golden 2026-02-09 09:15:39 +08:00
Henry Li ddbda4e38f feat: make the title golden in Ultra mode 2026-02-09 08:59:40 +08:00
Henry Li db79ab27f4 feat: make the title golden in Ultra mode 2026-02-09 08:59:40 +08:00
Henry Li 76cdb0e16e feat: make the title golden in Ultra mode 2026-02-09 08:59:40 +08:00
LofiSu 2d70aaa969 fix(frontend): citations display + refactor link/citation utils
- Citations: no underline while streaming (message links); artifact markdown external links as citation cards
- Refactor: add isExternalUrl, syntheticCitationFromLink in core/citations; shared externalLinkClass in lib/utils; simplify message-list-item and artifact-file-detail link rendering

修复引用展示并抽离链接/引用工具
- 引用:流式输出时链接不这下划线;Artifact 内 Markdown 外链以引用卡片展示
- 重构:core/citations 新增 isExternalUrl、syntheticCitationFromLink;lib/utils 共享 externalLinkClass;精简消息与 Artifact 中的链接渲染逻辑

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 04:03:15 +08:00
ruitanglin 509ea874f7 fix(frontend): citations display + refactor link/citation utils
- Citations: no underline while streaming (message links); artifact markdown external links as citation cards
- Refactor: add isExternalUrl, syntheticCitationFromLink in core/citations; shared externalLinkClass in lib/utils; simplify message-list-item and artifact-file-detail link rendering

修复引用展示并抽离链接/引用工具
- 引用:流式输出时链接不这下划线;Artifact 内 Markdown 外链以引用卡片展示
- 重构:core/citations 新增 isExternalUrl、syntheticCitationFromLink;lib/utils 共享 externalLinkClass;精简消息与 Artifact 中的链接渲染逻辑

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 04:03:15 +08:00
LofiSu d72aad8063 fix(frontend): build + remove hover tooltips in step links
- Fix Turbopack build: replace raw-loader .md import with inlined about-content.ts; drop raw-loader from next.config and package.json
- Remove all hover tooltips on step-area links (web_fetch, read_file, ls, bash, write_file, web_search) so hidden steps no longer show popups

修复:构建错误与步骤链接悬停提示
- 修复 Turbopack 构建:用内联 about-content.ts 替代 raw-loader 导入 about.md,并移除 next.config 与 package.json 中的 raw-loader
- 移除步骤区域内所有链接的悬停提示(查看网页、读文件、列目录、bash、写文件、网页搜索),隐藏步骤悬停不再弹出内容

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 03:42:16 +08:00
ruitanglin 8cb14ad4fb fix(frontend): build + remove hover tooltips in step links
- Fix Turbopack build: replace raw-loader .md import with inlined about-content.ts; drop raw-loader from next.config and package.json
- Remove all hover tooltips on step-area links (web_fetch, read_file, ls, bash, write_file, web_search) so hidden steps no longer show popups

修复:构建错误与步骤链接悬停提示
- 修复 Turbopack 构建:用内联 about-content.ts 替代 raw-loader 导入 about.md,并移除 next.config 与 package.json 中的 raw-loader
- 移除步骤区域内所有链接的悬停提示(查看网页、读文件、列目录、bash、写文件、网页搜索),隐藏步骤悬停不再弹出内容

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 03:42:16 +08:00
LofiSu fe06be8258 Revert "fix(frontend): Turbopack about page + remove hover on web search/citations"
This reverts commit 7e9e061f20fcec1f1a9c35be40d9407c05ed82be.
2026-02-09 03:23:51 +08:00
ruitanglin f577ff115b Revert "fix(frontend): Turbopack about page + remove hover on web search/citations"
This reverts commit 7e9e061f20fcec1f1a9c35be40d9407c05ed82be.
2026-02-09 03:23:51 +08:00
LofiSu 842c4ecac0 fix(frontend): Turbopack about page + remove hover on web search/citations
- About: use aboutMarkdown from about-content.ts instead of raw-loader for
  about.md (fixes Turbopack 'Cannot find module raw-loader')
- Web search: remove Tooltip from web_search and web_fetch result links
- Citations: remove HoverCard from CitationLink so no hover popup on badges

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 03:23:02 +08:00
ruitanglin 77859d01b8 fix(frontend): Turbopack about page + remove hover on web search/citations
- About: use aboutMarkdown from about-content.ts instead of raw-loader for
  about.md (fixes Turbopack 'Cannot find module raw-loader')
- Web search: remove Tooltip from web_search and web_fetch result links
- Citations: remove HoverCard from CitationLink so no hover popup on badges

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 03:23:02 +08:00
Henry Li cebf2599c9 feat: add mode in welcome 2026-02-09 00:41:25 +08:00
Henry Li 143f9f1f4d feat: add mode in welcome 2026-02-09 00:41:25 +08:00
Henry Li d197ee8f28 feat: add mode in welcome 2026-02-09 00:41:25 +08:00
Henry Li 25b60e732f feat: set golden color for ultra 2026-02-09 00:30:20 +08:00
Henry Li 9da3a1dcb2 feat: set golden color for ultra 2026-02-09 00:30:20 +08:00
Henry Li d9b60778a9 feat: set golden color for ultra 2026-02-09 00:30:20 +08:00
Henry Li f146e35ee7 feat: rewording 2026-02-08 23:44:36 +08:00
Henry Li a4e1e1a95e feat: rewording 2026-02-08 23:44:36 +08:00
Henry Li eb9af00d1d feat: rewording 2026-02-08 23:44:36 +08:00
hetaoBackend 6eb4cdd3ec feat: disallow present_files tool in subagents and add market-analysis skill
Add "present_files" to disallowed_tools for bash and general-purpose
subagents to prevent them from presenting files directly. Also add the
new market-analysis skill for generating consulting-grade reports.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 23:38:55 +08:00
hetaoBackend f9b769b5c3 feat: disallow present_files tool in subagents and add market-analysis skill
Add "present_files" to disallowed_tools for bash and general-purpose
subagents to prevent them from presenting files directly. Also add the
new market-analysis skill for generating consulting-grade reports.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 23:38:55 +08:00
hetaoBackend 54f2f1bd3a feat: disallow present_files tool in subagents and add market-analysis skill
Add "present_files" to disallowed_tools for bash and general-purpose
subagents to prevent them from presenting files directly. Also add the
new market-analysis skill for generating consulting-grade reports.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 23:38:55 +08:00
Henry Li 8a2351593c feat: add special effect for Ultra mode 2026-02-08 23:22:51 +08:00
Henry Li d36fbcdfc1 feat: add special effect for Ultra mode 2026-02-08 23:22:51 +08:00
Henry Li 0d55230016 feat: add special effect for Ultra mode 2026-02-08 23:22:51 +08:00
hetaoBackend 2703eb0b22 docs: revise backend README and CLAUDE.md to reflect full architecture
Updated documentation to accurately cover all backend subsystems including
subagents, memory, middleware chain, sandbox, MCP, skills, and gateway API.
Fixed broken MCP_SETUP.md link in root README.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 22:50:42 +08:00
hetaoBackend fdd25c1bb8 docs: revise backend README and CLAUDE.md to reflect full architecture
Updated documentation to accurately cover all backend subsystems including
subagents, memory, middleware chain, sandbox, MCP, skills, and gateway API.
Fixed broken MCP_SETUP.md link in root README.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 22:50:42 +08:00
hetaoBackend d891a8a37c docs: revise backend README and CLAUDE.md to reflect full architecture
Updated documentation to accurately cover all backend subsystems including
subagents, memory, middleware chain, sandbox, MCP, skills, and gateway API.
Fixed broken MCP_SETUP.md link in root README.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 22:50:42 +08:00
Henry Li 010aba1e28 feat: add realtime subagent status report 2026-02-08 22:43:51 +08:00
Henry Li 7ed1be32fd feat: add realtime subagent status report 2026-02-08 22:43:51 +08:00
Henry Li 7d4b5eb3ca feat: add realtime subagent status report 2026-02-08 22:43:51 +08:00
hetaoBackend 808e028338 feat: limit concurrent subagents to 3 per turn
Prevent resource exhaustion by capping the number of parallel subagents.
Adds runtime enforcement in task_tool and updates prompts/examples accordingly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 22:12:21 +08:00
hetaoBackend 9e2b3f1f39 feat: limit concurrent subagents to 3 per turn
Prevent resource exhaustion by capping the number of parallel subagents.
Adds runtime enforcement in task_tool and updates prompts/examples accordingly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 22:12:21 +08:00
hetaoBackend faa327b3cd feat: limit concurrent subagents to 3 per turn
Prevent resource exhaustion by capping the number of parallel subagents.
Adds runtime enforcement in task_tool and updates prompts/examples accordingly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 22:12:21 +08:00
hetaoBackend 96bace7ab6 feat: add real-time streaming of subagent AI messages
Enable task tool to capture and stream AI messages as they are generated by subagents. This replaces simple polling status updates with detailed message-level progress updates.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-08 21:29:45 +08:00
hetaoBackend 0a27a7561a feat: add real-time streaming of subagent AI messages
Enable task tool to capture and stream AI messages as they are generated by subagents. This replaces simple polling status updates with detailed message-level progress updates.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-08 21:29:45 +08:00
hetaoBackend 54772947cb feat: add real-time streaming of subagent AI messages
Enable task tool to capture and stream AI messages as they are generated by subagents. This replaces simple polling status updates with detailed message-level progress updates.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-08 21:29:45 +08:00
Henry Li 0355493a16 feat: rewording and add initial animation 2026-02-08 21:24:17 +08:00
Henry Li 2b3dc96e40 feat: rewording and add initial animation 2026-02-08 21:24:17 +08:00
Henry Li ff7437f830 feat: rewording and add initial animation 2026-02-08 21:24:17 +08:00
hetaoBackend 17365e40d5 fix: fix sub agent timeout 2026-02-08 21:09:18 +08:00
hetaoBackend 19a1d03fc8 fix: fix sub agent timeout 2026-02-08 21:09:18 +08:00
hetaoBackend f01c470e64 fix: fix sub agent timeout 2026-02-08 21:09:18 +08:00
Henry Li 5d4cecbb84 refactor: optimize task handling in message list 2026-02-07 18:42:24 +08:00
Henry Li 4f9150229c refactor: optimize task handling in message list 2026-02-07 18:42:24 +08:00
Henry Li 542b04588a refactor: optimize task handling in message list 2026-02-07 18:42:24 +08:00
Henry Li de8ff9d336 feat: add ambilight 2026-02-07 18:42:08 +08:00
Henry Li 01aa035905 feat: add ambilight 2026-02-07 18:42:08 +08:00
Henry Li a4e89cc96b feat: add ambilight 2026-02-07 18:42:08 +08:00
Henry Li d9a52f07e7 feat: add handling for task timeout and enhance Streamdown plugin for word animation 2026-02-07 18:06:22 +08:00
Henry Li 99e8f22d1d feat: add handling for task timeout and enhance Streamdown plugin for word animation 2026-02-07 18:06:22 +08:00
Henry Li 0810917b69 feat: add handling for task timeout and enhance Streamdown plugin for word animation 2026-02-07 18:06:22 +08:00
Henry Li 260953fb81 feat: adjust position 2026-02-07 18:00:24 +08:00
Henry Li dce82c1db4 feat: adjust position 2026-02-07 18:00:24 +08:00
Henry Li 4dc3cdac48 feat: adjust position 2026-02-07 18:00:24 +08:00
Henry Li b135449c07 fix: adjust suggestion positioning and height for improved UI layout 2026-02-07 17:56:06 +08:00
Henry Li 2510991698 fix: adjust suggestion positioning and height for improved UI layout 2026-02-07 17:56:06 +08:00
Henry Li 17b2630b73 fix: adjust suggestion positioning and height for improved UI layout 2026-02-07 17:56:06 +08:00
hetaoBackend f41d9b3be5 refactor: optimize task tool parameter order and improve task tracking
- Reorder task tool parameters to prioritize description first for better usability
- Add tool_call_id injection for better task traceability
- Use tool_call_id as task_id in executor for consistent tracking
- Simplify event messages by removing redundant task_type field
- Update task examples in prompt to reflect new parameter order

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-07 16:56:13 +08:00
hetaoBackend 1425294f9b refactor: optimize task tool parameter order and improve task tracking
- Reorder task tool parameters to prioritize description first for better usability
- Add tool_call_id injection for better task traceability
- Use tool_call_id as task_id in executor for consistent tracking
- Simplify event messages by removing redundant task_type field
- Update task examples in prompt to reflect new parameter order

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-07 16:56:13 +08:00
hetaoBackend a6db74baba refactor: optimize task tool parameter order and improve task tracking
- Reorder task tool parameters to prioritize description first for better usability
- Add tool_call_id injection for better task traceability
- Use tool_call_id as task_id in executor for consistent tracking
- Simplify event messages by removing redundant task_type field
- Update task examples in prompt to reflect new parameter order

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-07 16:56:13 +08:00
Henry Li 3e2883e2a3 feat: support subtasks 2026-02-07 16:14:48 +08:00
Henry Li a016332a37 feat: support subtasks 2026-02-07 16:14:48 +08:00
Henry Li 46798c0931 feat: support subtasks 2026-02-07 16:14:48 +08:00
Henry Li 39a5d8dc30 Merge remote-tracking branch 'refs/remotes/origin/experimental' into experimental 2026-02-07 16:13:48 +08:00
Henry Li 617559a900 Merge remote-tracking branch 'refs/remotes/origin/experimental' into experimental 2026-02-07 16:13:48 +08:00
Henry Li e04e70c7a8 Merge remote-tracking branch 'refs/remotes/origin/experimental' into experimental 2026-02-07 16:13:48 +08:00
LofiSu e4eb4a65cf Merge pull request #25 from LofiSu/feat/citations
feat(citations): add shared citation components and optimize code
2026-02-07 12:13:32 +08:00
LofiSu afb7a36739 Merge pull request #25 from LofiSu/feat/citations
feat(citations): add shared citation components and optimize code
2026-02-07 12:13:32 +08:00
LofiSu 9f8d9e4da2 Merge pull request #25 from LofiSu/feat/citations
feat(citations): add shared citation components and optimize code
2026-02-07 12:13:32 +08:00
Henry Li 91a05acdf8 feat: enhance workspace navigation menu with conditional rendering and mounted state 2026-02-07 11:10:24 +08:00
Henry Li 4ac637a0eb feat: enhance workspace navigation menu with conditional rendering and mounted state 2026-02-07 11:10:24 +08:00
Henry Li e7cd5287f1 feat: enhance workspace navigation menu with conditional rendering and mounted state 2026-02-07 11:10:24 +08:00
Henry Li 60be7ee20d docs: update description for surprise-me skill to enhance clarity 2026-02-07 10:51:43 +08:00
Henry Li 86ad92a1a6 docs: update description for surprise-me skill to enhance clarity 2026-02-07 10:51:43 +08:00
Henry Li 85767c8470 docs: update description for surprise-me skill to enhance clarity 2026-02-07 10:51:43 +08:00
Henry Li c758a28a3e styles: format 2026-02-07 10:50:08 +08:00
Henry Li 70191783ad styles: format 2026-02-07 10:50:08 +08:00
Henry Li 1d3ed9f43b styles: format 2026-02-07 10:50:08 +08:00
Henry Li a122f76e36 feat: add animations 2026-02-07 10:30:35 +08:00
Henry Li a2af464a6f feat: add animations 2026-02-07 10:30:35 +08:00
Henry Li fc543a9b30 feat: add animations 2026-02-07 10:30:35 +08:00
LofiSu f0075e0d64 Merge upstream/experimental into feat/citations
Resolved conflicts:
- backend/src/gateway/routers/artifacts.py: Keep citations block removal for markdown downloads
- frontend/src/components/workspace/messages/message-list-item.tsx: Keep improved citation handling with rehypePlugins, humanMessagePlugins, and CitationsLoadingIndicator

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-07 00:53:16 +08:00
ruitanglin 2331c67446 Merge upstream/experimental into feat/citations
Resolved conflicts:
- backend/src/gateway/routers/artifacts.py: Keep citations block removal for markdown downloads
- frontend/src/components/workspace/messages/message-list-item.tsx: Keep improved citation handling with rehypePlugins, humanMessagePlugins, and CitationsLoadingIndicator

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-07 00:53:16 +08:00
ruitanglin ea543ce1f4 Merge upstream/experimental into feat/citations
Resolved conflicts:
- backend/src/gateway/routers/artifacts.py: Keep citations block removal for markdown downloads
- frontend/src/components/workspace/messages/message-list-item.tsx: Keep improved citation handling with rehypePlugins, humanMessagePlugins, and CitationsLoadingIndicator

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-07 00:53:16 +08:00
Henry Li 5ed15d79c9 fix: fix markdown table 2026-02-06 22:00:55 +08:00
Henry Li 8f1a42a8e0 fix: fix markdown table 2026-02-06 22:00:55 +08:00
Henry Li c3f9089e95 fix: fix markdown table 2026-02-06 22:00:55 +08:00
Henry Li 6b56e68ff2 Merge pull request #24 from LofiSu/fix/upload-files-alignment
fix: 修复用户消息中上传文件的右对齐显示
2026-02-06 21:53:01 +08:00
Henry Li 537687c2c5 Merge pull request #24 from LofiSu/fix/upload-files-alignment
fix: 修复用户消息中上传文件的右对齐显示
2026-02-06 21:53:01 +08:00
Henry Li 5016a5f7d9 Merge pull request #24 from LofiSu/fix/upload-files-alignment
fix: 修复用户消息中上传文件的右对齐显示
2026-02-06 21:53:01 +08:00
hetao 9e4f2512f3 fix: fix subagent prompt 2026-02-06 20:32:15 +08:00
hetao a423dfb9fd fix: fix subagent prompt 2026-02-06 20:32:15 +08:00
hetao d1d275bb81 fix: fix subagent prompt 2026-02-06 20:32:15 +08:00
hetao 9bf3a12c30 feat: send custom event 2026-02-06 17:48:15 +08:00
hetao 172813720a feat: send custom event 2026-02-06 17:48:15 +08:00
hetao 4f15670455 feat: send custom event 2026-02-06 17:48:15 +08:00
hetao 9f367b5563 feat: fix task polling issue 2026-02-06 17:48:15 +08:00
hetao 41d8d2fd5c feat: fix task polling issue 2026-02-06 17:48:15 +08:00
hetao 498c8b3ec0 feat: fix task polling issue 2026-02-06 17:48:15 +08:00
hetao 449ffbad75 feat: add ultra mode 2026-02-06 17:48:14 +08:00
hetao 926c322c36 feat: add ultra mode 2026-02-06 17:48:14 +08:00
hetao 96baab12a2 feat: add ultra mode 2026-02-06 17:48:14 +08:00
LofiSu 5484233548 fix(citations): hide citations block in reasoning/thinking content
The reasoning content in message-group.tsx was not being processed
through parseCitations, causing raw <citations> blocks to be visible.
Now reasoning content is parsed to remove citations blocks.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 16:12:43 +08:00
ruitanglin ca6bcaa31c fix(citations): hide citations block in reasoning/thinking content
The reasoning content in message-group.tsx was not being processed
through parseCitations, causing raw <citations> blocks to be visible.
Now reasoning content is parsed to remove citations blocks.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 16:12:43 +08:00
ruitanglin 50ced32722 fix(citations): hide citations block in reasoning/thinking content
The reasoning content in message-group.tsx was not being processed
through parseCitations, causing raw <citations> blocks to be visible.
Now reasoning content is parsed to remove citations blocks.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 16:12:43 +08:00
LofiSu 582bfaee39 fix(citations): only citation links in citationMap render as badges
Revert streaming logic - only links that are actually in citationMap
should render as badges. This prevents project URLs and other regular
links from being incorrectly rendered as citation badges.

During streaming, links may initially appear as plain links until the
citations block is fully parsed, then they will update to badge style.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 16:10:29 +08:00
ruitanglin 666b747b8a fix(citations): only citation links in citationMap render as badges
Revert streaming logic - only links that are actually in citationMap
should render as badges. This prevents project URLs and other regular
links from being incorrectly rendered as citation badges.

During streaming, links may initially appear as plain links until the
citations block is fully parsed, then they will update to badge style.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 16:10:29 +08:00
ruitanglin e8ee19821d fix(citations): only citation links in citationMap render as badges
Revert streaming logic - only links that are actually in citationMap
should render as badges. This prevents project URLs and other regular
links from being incorrectly rendered as citation badges.

During streaming, links may initially appear as plain links until the
citations block is fully parsed, then they will update to badge style.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 16:10:29 +08:00
LofiSu e7ea0fc551 fix(citations): render external links as badges during streaming
During streaming when citations are still loading (isLoadingCitations=true),
all external links should be rendered as badges since we don't know yet
which links are citations. After streaming completes, only links in
citationMap are rendered as badges.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 16:09:03 +08:00
ruitanglin 697c683dfa fix(citations): render external links as badges during streaming
During streaming when citations are still loading (isLoadingCitations=true),
all external links should be rendered as badges since we don't know yet
which links are citations. After streaming completes, only links in
citationMap are rendered as badges.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 16:09:03 +08:00
ruitanglin e444817c5d fix(citations): render external links as badges during streaming
During streaming when citations are still loading (isLoadingCitations=true),
all external links should be rendered as badges since we don't know yet
which links are citations. After streaming completes, only links in
citationMap are rendered as badges.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 16:09:03 +08:00
LofiSu f1c3f908c9 fix(citations): parse citations in reasoning content
When only reasoning content exists (no main content), the citations
block was not being parsed and removed. Now reasoning content also
goes through parseCitations to hide the raw citations block.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 16:04:49 +08:00
ruitanglin 579dccbdce fix(citations): parse citations in reasoning content
When only reasoning content exists (no main content), the citations
block was not being parsed and removed. Now reasoning content also
goes through parseCitations to hide the raw citations block.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 16:04:49 +08:00
ruitanglin e9648b11cd fix(citations): parse citations in reasoning content
When only reasoning content exists (no main content), the citations
block was not being parsed and removed. Now reasoning content also
goes through parseCitations to hide the raw citations block.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 16:04:49 +08:00
LofiSu 7c21d8f3a6 fix(artifacts): only render citation badges for links in citationMap
Same fix as message-list-item: project URLs and regular links in
artifact file preview should be rendered as plain links, not badges.
Only actual citations (in citationMap) should be rendered as badges.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 15:55:53 +08:00
ruitanglin 365e3f4304 fix(artifacts): only render citation badges for links in citationMap
Same fix as message-list-item: project URLs and regular links in
artifact file preview should be rendered as plain links, not badges.
Only actual citations (in citationMap) should be rendered as badges.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 15:55:53 +08:00
ruitanglin 0cf8ba86d1 fix(artifacts): only render citation badges for links in citationMap
Same fix as message-list-item: project URLs and regular links in
artifact file preview should be rendered as plain links, not badges.
Only actual citations (in citationMap) should be rendered as badges.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 15:55:53 +08:00
Henry Li 70989a949e feat: add 'about' page 2026-02-06 15:18:37 +08:00
Henry Li 44742c6353 feat: add 'about' page 2026-02-06 15:18:37 +08:00
Henry Li f9811671d8 feat: add 'about' page 2026-02-06 15:18:37 +08:00
Henry Li bc7837ed6f docs: rewording 2026-02-06 15:18:19 +08:00
Henry Li dd4a7aae36 docs: rewording 2026-02-06 15:18:19 +08:00
Henry Li ee41324887 docs: rewording 2026-02-06 15:18:19 +08:00
LofiSu 5d8c08d3ba fix(citations): only render citation badges for links in citationMap
Project URLs and regular links should be rendered as plain underlined
links, not as citation badges. Only links that are actual citations
(present in citationMap) should be rendered as badges.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 15:15:45 +08:00
ruitanglin 1ce154fa71 fix(citations): only render citation badges for links in citationMap
Project URLs and regular links should be rendered as plain underlined
links, not as citation badges. Only links that are actual citations
(present in citationMap) should be rendered as badges.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 15:15:45 +08:00
ruitanglin 7a3a5f5196 fix(citations): only render citation badges for links in citationMap
Project URLs and regular links should be rendered as plain underlined
links, not as citation badges. Only links that are actual citations
(present in citationMap) should be rendered as badges.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 15:15:45 +08:00
LofiSu 49f7cf1662 fix(citations): use markdown link text as fallback for display
When citation data is not available, use the markdown link text
(children) as display text instead of just the domain. This ensures
that links like [OpenJudge](github.com/...) show 'OpenJudge' instead
of just 'github.com'.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 15:06:51 +08:00
ruitanglin acbf2fb453 fix(citations): use markdown link text as fallback for display
When citation data is not available, use the markdown link text
(children) as display text instead of just the domain. This ensures
that links like [OpenJudge](github.com/...) show 'OpenJudge' instead
of just 'github.com'.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 15:06:51 +08:00
ruitanglin c87f176fac fix(citations): use markdown link text as fallback for display
When citation data is not available, use the markdown link text
(children) as display text instead of just the domain. This ensures
that links like [OpenJudge](github.com/...) show 'OpenJudge' instead
of just 'github.com'.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 15:06:51 +08:00
Henry Li 23c082f05d docs: add CLAUDE.md 2026-02-06 14:40:52 +08:00
Henry Li a711c5f310 docs: add CLAUDE.md 2026-02-06 14:40:52 +08:00
Henry Li 8bd20ab4e6 docs: add CLAUDE.md 2026-02-06 14:40:52 +08:00
LofiSu a91302ac72 fix(prompt): clarify citation link format must include URL
AI was outputting bare brackets like [arXiv:xxx] without URLs,
which do not render as links. Updated prompt to explicitly show
correct vs wrong formats and require complete markdown links.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 14:38:31 +08:00
ruitanglin f43522bd27 fix(prompt): clarify citation link format must include URL
AI was outputting bare brackets like [arXiv:xxx] without URLs,
which do not render as links. Updated prompt to explicitly show
correct vs wrong formats and require complete markdown links.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 14:38:31 +08:00
ruitanglin b46a19e116 fix(prompt): clarify citation link format must include URL
AI was outputting bare brackets like [arXiv:xxx] without URLs,
which do not render as links. Updated prompt to explicitly show
correct vs wrong formats and require complete markdown links.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 14:38:31 +08:00
Henry Li 78b6164770 docs: add AGENTS.md 2026-02-06 14:32:48 +08:00
Henry Li 5b33a62f05 docs: add AGENTS.md 2026-02-06 14:32:48 +08:00
Henry Li 30cd2387f2 docs: add AGENTS.md 2026-02-06 14:32:48 +08:00
LofiSu 738b71be47 fix(messages): prevent URL autolink bleeding into adjacent text
For human messages, disable remark-gfm autolink feature to prevent
URLs from incorrectly including adjacent text (especially Chinese
characters) as part of the link. This ensures that when users input
"https://example.com 帮我分析", only the URL becomes a link.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 14:30:57 +08:00
ruitanglin c8c4d2fc95 fix(messages): prevent URL autolink bleeding into adjacent text
For human messages, disable remark-gfm autolink feature to prevent
URLs from incorrectly including adjacent text (especially Chinese
characters) as part of the link. This ensures that when users input
"https://example.com 帮我分析", only the URL becomes a link.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 14:30:57 +08:00
ruitanglin 34a199c6f3 fix(messages): prevent URL autolink bleeding into adjacent text
For human messages, disable remark-gfm autolink feature to prevent
URLs from incorrectly including adjacent text (especially Chinese
characters) as part of the link. This ensures that when users input
"https://example.com 帮我分析", only the URL becomes a link.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 14:30:57 +08:00
LofiSu 6f968242d6 fix(citations): only render CitationLink badges for AI messages
Human messages should display links as plain underlined text,
not as citation badges. This preserves the original user input
appearance when users paste URLs in their messages.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 14:28:28 +08:00
ruitanglin 1b0c016093 fix(citations): only render CitationLink badges for AI messages
Human messages should display links as plain underlined text,
not as citation badges. This preserves the original user input
appearance when users paste URLs in their messages.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 14:28:28 +08:00
ruitanglin bcbbf9cf3f fix(citations): only render CitationLink badges for AI messages
Human messages should display links as plain underlined text,
not as citation badges. This preserves the original user input
appearance when users paste URLs in their messages.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 14:28:28 +08:00
Henry Li b74cf65275 feat: update surprise-me functionality with localization support 2026-02-06 14:21:03 +08:00
Henry Li 765e35fc70 feat: update surprise-me functionality with localization support 2026-02-06 14:21:03 +08:00
Henry Li bbb1a731a5 feat: update surprise-me functionality with localization support 2026-02-06 14:21:03 +08:00
Henry Li 22dea3fd43 feat: add surprise-me 2026-02-06 14:04:15 +08:00
Henry Li 26e078df7d feat: add surprise-me 2026-02-06 14:04:15 +08:00
Henry Li 697ea8e845 feat: add surprise-me 2026-02-06 14:04:15 +08:00
Henry Li f391060573 feat: adjust position 2026-02-06 09:39:20 +08:00
Henry Li 254efe7391 feat: adjust position 2026-02-06 09:39:20 +08:00
Henry Li dedfa1bfb5 feat: adjust position 2026-02-06 09:39:20 +08:00
hetao 85128f5f14 feat: add configuration to enable/disable subagents
Add subagents.enabled flag in config.yaml to control subagent feature:
- When disabled, task/task_status tools are not loaded
- When disabled, system prompt excludes subagent documentation
- Default is enabled for backward compatibility

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 20:49:02 +08:00
hetao b7bf027aa5 feat: add configuration to enable/disable subagents
Add subagents.enabled flag in config.yaml to control subagent feature:
- When disabled, task/task_status tools are not loaded
- When disabled, system prompt excludes subagent documentation
- Default is enabled for backward compatibility

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 20:49:02 +08:00
hetao b7ba237c36 feat: add configuration to enable/disable subagents
Add subagents.enabled flag in config.yaml to control subagent feature:
- When disabled, task/task_status tools are not loaded
- When disabled, system prompt excludes subagent documentation
- Default is enabled for backward compatibility

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 20:49:02 +08:00
hetao ef379a3100 feat: support sub agent mechanism 2026-02-05 19:59:25 +08:00
hetao cbd2fe66de feat: support sub agent mechanism 2026-02-05 19:59:25 +08:00
hetao 6e3f43c943 feat: support sub agent mechanism 2026-02-05 19:59:25 +08:00
Henry Li 43ebce3b37 feat: remove demo 2026-02-05 09:46:05 +08:00
Henry Li c31175defd feat: remove demo 2026-02-05 09:46:05 +08:00
Henry Li 118fc00368 feat: remove demo 2026-02-05 09:46:05 +08:00
hetao db0461142e feat: enhance memory system with tiktoken and improved prompt guidelines
Add accurate token counting using tiktoken library and significantly enhance
memory update prompts with detailed section guidelines, multilingual support,
and improved fact extraction. Update deep-research skill to be more proactive
for research queries.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-04 20:44:26 +08:00
hetao 0d245d6e31 feat: enhance memory system with tiktoken and improved prompt guidelines
Add accurate token counting using tiktoken library and significantly enhance
memory update prompts with detailed section guidelines, multilingual support,
and improved fact extraction. Update deep-research skill to be more proactive
for research queries.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-04 20:44:26 +08:00
hetao df1191c90a feat: enhance memory system with tiktoken and improved prompt guidelines
Add accurate token counting using tiktoken library and significantly enhance
memory update prompts with detailed section guidelines, multilingual support,
and improved fact extraction. Update deep-research skill to be more proactive
for research queries.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-04 20:44:26 +08:00
LofiSu 2debcf421c fix(citations): improve citation link rendering and copy behavior
- Use citation.title for display text in CitationLink to ensure correct
  titles show during streaming (instead of generic "Source" text)
- Render all external links as CitationLink badges for consistent styling
  during streaming output
- Add removeAllCitations when copying message content to clipboard
- Simplify citations_format prompt for cleaner AI output

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-04 16:34:12 +08:00
ruitanglin f6e625ec3b fix(citations): improve citation link rendering and copy behavior
- Use citation.title for display text in CitationLink to ensure correct
  titles show during streaming (instead of generic "Source" text)
- Render all external links as CitationLink badges for consistent styling
  during streaming output
- Add removeAllCitations when copying message content to clipboard
- Simplify citations_format prompt for cleaner AI output

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-04 16:34:12 +08:00
ruitanglin 0f9e3d508b fix(citations): improve citation link rendering and copy behavior
- Use citation.title for display text in CitationLink to ensure correct
  titles show during streaming (instead of generic "Source" text)
- Render all external links as CitationLink badges for consistent styling
  during streaming output
- Add removeAllCitations when copying message content to clipboard
- Simplify citations_format prompt for cleaner AI output

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-04 16:34:12 +08:00
LofiSu 644229f968 feat(citations): add shared citation components and optimize code
## New Features
- Add `CitationLink` shared component for rendering citation hover cards
- Add `CitationsLoadingIndicator` component for showing loading state
- Add `removeAllCitations` utility to strip all citations from content
- Add backend support for removing citations when downloading markdown files
- Add i18n support for citation loading messages (en-US, zh-CN)

## Code Optimizations
- Remove duplicate `ExternalLinkBadge` component, reuse `CitationLink` instead
- Consolidate `remarkPlugins` config in `streamdownPlugins` to avoid duplication
- Remove unused imports: `Citation`, `buildCitationMap`, `extractDomainFromUrl`, etc.
- Remove unused `messages` parameter from `ToolCall` component
- Remove unused `isWriteFile` parameter from `ArtifactFilePreview` component
- Remove unused `useI18n` hook from `MessageContent` component

## Bug Fixes
- Fix `remarkGfm` plugin configuration that prevented table rendering
- Fix React Hooks rule violation: move `useMemo` to component top level
- Replace `||` with `??` for nullish coalescing in clipboard data

## Code Cleanup
- Remove debug console.log/info statements from:
  - `threads/hooks.ts`
  - `notification/hooks.ts`
  - `memory-settings-page.tsx`
- Fix import order in `message-group.tsx`

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-04 11:56:10 +08:00
ruitanglin c67f1af889 feat(citations): add shared citation components and optimize code
## New Features
- Add `CitationLink` shared component for rendering citation hover cards
- Add `CitationsLoadingIndicator` component for showing loading state
- Add `removeAllCitations` utility to strip all citations from content
- Add backend support for removing citations when downloading markdown files
- Add i18n support for citation loading messages (en-US, zh-CN)

## Code Optimizations
- Remove duplicate `ExternalLinkBadge` component, reuse `CitationLink` instead
- Consolidate `remarkPlugins` config in `streamdownPlugins` to avoid duplication
- Remove unused imports: `Citation`, `buildCitationMap`, `extractDomainFromUrl`, etc.
- Remove unused `messages` parameter from `ToolCall` component
- Remove unused `isWriteFile` parameter from `ArtifactFilePreview` component
- Remove unused `useI18n` hook from `MessageContent` component

## Bug Fixes
- Fix `remarkGfm` plugin configuration that prevented table rendering
- Fix React Hooks rule violation: move `useMemo` to component top level
- Replace `||` with `??` for nullish coalescing in clipboard data

## Code Cleanup
- Remove debug console.log/info statements from:
  - `threads/hooks.ts`
  - `notification/hooks.ts`
  - `memory-settings-page.tsx`
- Fix import order in `message-group.tsx`

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-04 11:56:10 +08:00
ruitanglin 1e2675beb3 feat(citations): add shared citation components and optimize code
## New Features
- Add `CitationLink` shared component for rendering citation hover cards
- Add `CitationsLoadingIndicator` component for showing loading state
- Add `removeAllCitations` utility to strip all citations from content
- Add backend support for removing citations when downloading markdown files
- Add i18n support for citation loading messages (en-US, zh-CN)

## Code Optimizations
- Remove duplicate `ExternalLinkBadge` component, reuse `CitationLink` instead
- Consolidate `remarkPlugins` config in `streamdownPlugins` to avoid duplication
- Remove unused imports: `Citation`, `buildCitationMap`, `extractDomainFromUrl`, etc.
- Remove unused `messages` parameter from `ToolCall` component
- Remove unused `isWriteFile` parameter from `ArtifactFilePreview` component
- Remove unused `useI18n` hook from `MessageContent` component

## Bug Fixes
- Fix `remarkGfm` plugin configuration that prevented table rendering
- Fix React Hooks rule violation: move `useMemo` to component top level
- Replace `||` with `??` for nullish coalescing in clipboard data

## Code Cleanup
- Remove debug console.log/info statements from:
  - `threads/hooks.ts`
  - `notification/hooks.ts`
  - `memory-settings-page.tsx`
- Fix import order in `message-group.tsx`

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-04 11:56:10 +08:00
hetao 5959ef87b8 feat: add Apple Container support with comprehensive documentation and dev tools
Add native Apple Container support for better performance on macOS while
maintaining full Docker compatibility. Enhance documentation with memory system
details, development guidelines, and sandbox setup instructions. Improve dev
experience with container image pre-pulling and unified cleanup tools.

Key changes:
- Auto-detect and prefer Apple Container on macOS with Docker fallback
- Add APPLE_CONTAINER.md with complete usage and troubleshooting guide
- Document memory system architecture in CLAUDE.md
- Add make setup-sandbox for pre-pulling container images
- Create cleanup-containers.sh for cross-runtime container cleanup
- Update all related documentation (README, SETUP, config examples)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-03 20:41:36 +08:00
hetao 70a27b49c0 feat: add Apple Container support with comprehensive documentation and dev tools
Add native Apple Container support for better performance on macOS while
maintaining full Docker compatibility. Enhance documentation with memory system
details, development guidelines, and sandbox setup instructions. Improve dev
experience with container image pre-pulling and unified cleanup tools.

Key changes:
- Auto-detect and prefer Apple Container on macOS with Docker fallback
- Add APPLE_CONTAINER.md with complete usage and troubleshooting guide
- Document memory system architecture in CLAUDE.md
- Add make setup-sandbox for pre-pulling container images
- Create cleanup-containers.sh for cross-runtime container cleanup
- Update all related documentation (README, SETUP, config examples)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-03 20:41:36 +08:00
hetao ef10f3ba41 feat: add Apple Container support with comprehensive documentation and dev tools
Add native Apple Container support for better performance on macOS while
maintaining full Docker compatibility. Enhance documentation with memory system
details, development guidelines, and sandbox setup instructions. Improve dev
experience with container image pre-pulling and unified cleanup tools.

Key changes:
- Auto-detect and prefer Apple Container on macOS with Docker fallback
- Add APPLE_CONTAINER.md with complete usage and troubleshooting guide
- Document memory system architecture in CLAUDE.md
- Add make setup-sandbox for pre-pulling container images
- Create cleanup-containers.sh for cross-runtime container cleanup
- Update all related documentation (README, SETUP, config examples)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-03 20:41:36 +08:00
hetao b773bae407 fix: fix frontend rendering issue 2026-02-03 19:38:10 +08:00
hetao d670cc0ab1 fix: fix frontend rendering issue 2026-02-03 19:38:10 +08:00
hetao 8f8637c3c4 fix: fix frontend rendering issue 2026-02-03 19:38:10 +08:00
LofiSu 3b411fe499 fix: 修复用户消息中上传文件的右对齐显示
在 UploadedFilesList 组件中添加 justify-end 类,确保上传的文件卡片在用户消息中保持右对齐显示,与消息气泡对齐一致。

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-03 19:13:17 +08:00
ruitanglin 1fac83eafa fix: 修复用户消息中上传文件的右对齐显示
在 UploadedFilesList 组件中添加 justify-end 类,确保上传的文件卡片在用户消息中保持右对齐显示,与消息气泡对齐一致。

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-03 19:13:17 +08:00
ruitanglin 901772136e fix: 修复用户消息中上传文件的右对齐显示
在 UploadedFilesList 组件中添加 justify-end 类,确保上传的文件卡片在用户消息中保持右对齐显示,与消息气泡对齐一致。

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-03 19:13:17 +08:00
Henry Li 6b53456b39 feat: add memory settings page 2026-02-03 18:18:56 +08:00
Henry Li 94acb15c0c feat: add memory settings page 2026-02-03 18:18:56 +08:00
Henry Li 552d1c3a9a feat: add memory settings page 2026-02-03 18:18:56 +08:00
Henry Li 4d650f35f8 chore: add /api/memory 2026-02-03 15:21:15 +08:00
Henry Li b8c325eb3a chore: add /api/memory 2026-02-03 15:21:15 +08:00
Henry Li 1cf081120e chore: add /api/memory 2026-02-03 15:21:15 +08:00
hetaoBackend 2c32e8a461 fix: add file mtime-based cache invalidation for memory data
Implement automatic cache invalidation based on file modification time to ensure memory data consistency across Gateway API and agent prompts. The cache now automatically reloads when the memory file is updated externally.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-03 13:50:01 +08:00
hetaoBackend 9e15e609ec fix: add file mtime-based cache invalidation for memory data
Implement automatic cache invalidation based on file modification time to ensure memory data consistency across Gateway API and agent prompts. The cache now automatically reloads when the memory file is updated externally.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-03 13:50:01 +08:00
hetaoBackend 5682f7b67d fix: add file mtime-based cache invalidation for memory data
Implement automatic cache invalidation based on file modification time to ensure memory data consistency across Gateway API and agent prompts. The cache now automatically reloads when the memory file is updated externally.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-03 13:50:01 +08:00
hetaoBackend 3b30913e10 feat: add memory API and optimize memory middleware
- Add memory API endpoints for retrieving memory data:
  - GET /api/memory - get current memory data
  - POST /api/memory/reload - reload from file
  - GET /api/memory/config - get memory configuration
  - GET /api/memory/status - get config and data together
- Optimize MemoryMiddleware to only use user inputs and final
  assistant responses, filtering out intermediate tool calls
- Add memory configuration example to config.example.yaml

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 13:41:04 +08:00
hetaoBackend 7b7a7abaf2 feat: add memory API and optimize memory middleware
- Add memory API endpoints for retrieving memory data:
  - GET /api/memory - get current memory data
  - POST /api/memory/reload - reload from file
  - GET /api/memory/config - get memory configuration
  - GET /api/memory/status - get config and data together
- Optimize MemoryMiddleware to only use user inputs and final
  assistant responses, filtering out intermediate tool calls
- Add memory configuration example to config.example.yaml

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 13:41:04 +08:00
hetaoBackend 74d47ad87f feat: add memory API and optimize memory middleware
- Add memory API endpoints for retrieving memory data:
  - GET /api/memory - get current memory data
  - POST /api/memory/reload - reload from file
  - GET /api/memory/config - get memory configuration
  - GET /api/memory/status - get config and data together
- Optimize MemoryMiddleware to only use user inputs and final
  assistant responses, filtering out intermediate tool calls
- Add memory configuration example to config.example.yaml

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 13:41:04 +08:00
hetaoBackend 0ea666e0cf feat: add global memory mechanism for personalized conversations
Implement a memory system that stores user context and conversation history
in memory.json, uses LLM to summarize conversations, and injects relevant
context into system prompts for personalized responses.

Key components:
- MemoryConfig for configuration management
- MemoryUpdateQueue with debounce for batch processing
- MemoryUpdater for LLM-based memory extraction
- MemoryMiddleware to queue conversations after agent execution
- Memory injection into lead agent system prompt

Note: Add memory section to config.yaml to enable (see config.example.yaml)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 13:31:05 +08:00
hetaoBackend 18d85ab6e5 feat: add global memory mechanism for personalized conversations
Implement a memory system that stores user context and conversation history
in memory.json, uses LLM to summarize conversations, and injects relevant
context into system prompts for personalized responses.

Key components:
- MemoryConfig for configuration management
- MemoryUpdateQueue with debounce for batch processing
- MemoryUpdater for LLM-based memory extraction
- MemoryMiddleware to queue conversations after agent execution
- Memory injection into lead agent system prompt

Note: Add memory section to config.yaml to enable (see config.example.yaml)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 13:31:05 +08:00
hetaoBackend ffd07bbafe feat: add global memory mechanism for personalized conversations
Implement a memory system that stores user context and conversation history
in memory.json, uses LLM to summarize conversations, and injects relevant
context into system prompts for personalized responses.

Key components:
- MemoryConfig for configuration management
- MemoryUpdateQueue with debounce for batch processing
- MemoryUpdater for LLM-based memory extraction
- MemoryMiddleware to queue conversations after agent execution
- Memory injection into lead agent system prompt

Note: Add memory section to config.yaml to enable (see config.example.yaml)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 13:31:05 +08:00
Henry Li 86255511e1 docs: add README.md 2026-02-02 23:24:58 +08:00
Henry Li 0baa8a733a docs: add README.md 2026-02-02 23:24:58 +08:00
Henry Li 4fd9a2de8e docs: add README.md 2026-02-02 23:24:58 +08:00
Henry Li e84715831f fix: fix position 2026-02-02 16:40:43 +08:00
Henry Li 03f84f2b76 fix: fix position 2026-02-02 16:40:43 +08:00
Henry Li 268b7f911c fix: fix position 2026-02-02 16:40:43 +08:00
Henry Li 018241c203 fix: set default state for todo list collapse to true 2026-02-02 14:46:26 +08:00
Henry Li 35c5b6ba6b fix: set default state for todo list collapse to true 2026-02-02 14:46:26 +08:00
Henry Li 8bc9d1b226 fix: set default state for todo list collapse to true 2026-02-02 14:46:26 +08:00
Henry Li 6f6d799051 fix: set default state for todo list collapse to false 2026-02-02 14:46:10 +08:00
Henry Li a745b824d5 fix: set default state for todo list collapse to false 2026-02-02 14:46:10 +08:00
Henry Li e01127eec9 fix: set default state for todo list collapse to false 2026-02-02 14:46:10 +08:00
Henry Li f4f16bfa5c feat: enhance welcome component and input box with skill mode handling and localization updates 2026-02-02 14:44:23 +08:00
Henry Li 010eadecca feat: enhance welcome component and input box with skill mode handling and localization updates 2026-02-02 14:44:23 +08:00
Henry Li 26acd6f3ad feat: enhance welcome component and input box with skill mode handling and localization updates 2026-02-02 14:44:23 +08:00
Henry Li ccf21238af feat: update button in skill settings to include icon 2026-02-02 14:31:44 +08:00
Henry Li 67451df910 feat: update button in skill settings to include icon 2026-02-02 14:31:44 +08:00
Henry Li 9cc41139cb feat: update button in skill settings to include icon 2026-02-02 14:31:44 +08:00
Henry Li efd56fdf51 feat: use list of links 2026-02-02 13:25:21 +08:00
Henry Li a5a0222963 feat: use list of links 2026-02-02 13:25:21 +08:00
Henry Li 207cb2b98d feat: use list of links 2026-02-02 13:25:21 +08:00
Henry Li b7c9bf557b feat: update button styling for artifacts tooltip 2026-02-02 11:43:02 +08:00
Henry Li 44daeaf37d feat: update button styling for artifacts tooltip 2026-02-02 11:43:02 +08:00
Henry Li b5e9eeea99 feat: update button styling for artifacts tooltip 2026-02-02 11:43:02 +08:00
Henry Li 3067f8dd03 feat: add suggestions 2026-02-02 11:21:30 +08:00
Henry Li 154fbb0ba3 feat: add suggestions 2026-02-02 11:21:30 +08:00
Henry Li e673405c00 feat: add suggestions 2026-02-02 11:21:30 +08:00
Henry Li 6c0e5fffd0 feat: integrate PromptInputProvider into ChatLayout and utilize prompt input controller in ChatPage 2026-02-02 10:18:02 +08:00
Henry Li f287022ac0 feat: integrate PromptInputProvider into ChatLayout and utilize prompt input controller in ChatPage 2026-02-02 10:18:02 +08:00
Henry Li b1227bb911 feat: integrate PromptInputProvider into ChatLayout and utilize prompt input controller in ChatPage 2026-02-02 10:18:02 +08:00
Henry Li 867749d7a3 feat: add file icons 2026-02-02 10:02:31 +08:00
Henry Li c587460dbc feat: add file icons 2026-02-02 10:02:31 +08:00
Henry Li f1db301d77 feat: add file icons 2026-02-02 10:02:31 +08:00
Henry Li 37dcee41c0 feat: add file icon 2026-02-02 09:49:44 +08:00
Henry Li 8bb4c35416 feat: add file icon 2026-02-02 09:49:44 +08:00
Henry Li 02400e0e8c feat: add file icon 2026-02-02 09:49:44 +08:00
Henry Li 51b4ed3124 feat: adjust tooltips 2026-02-02 09:32:18 +08:00
Henry Li 7274f9a6ae feat: adjust tooltips 2026-02-02 09:32:18 +08:00
Henry Li 0091da1aee feat: adjust tooltips 2026-02-02 09:32:18 +08:00
Henry Li 6d31c1c5cf feat: wrap path and command in Tooltip for enhanced user experience 2026-02-02 09:23:36 +08:00
Henry Li cb494fe4df feat: wrap path and command in Tooltip for enhanced user experience 2026-02-02 09:23:36 +08:00
Henry Li 076c1f0985 feat: wrap path and command in Tooltip for enhanced user experience 2026-02-02 09:23:36 +08:00
Henry Li a66f76f43d fix: update TooltipContent component to handle sideOffset correctly and add shadow styling 2026-02-02 09:23:24 +08:00
Henry Li ccab24983e fix: update TooltipContent component to handle sideOffset correctly and add shadow styling 2026-02-02 09:23:24 +08:00
Henry Li 33e82a7abe fix: update TooltipContent component to handle sideOffset correctly and add shadow styling 2026-02-02 09:23:24 +08:00
Henry Li 90104291ae docs: add comments 2026-02-02 09:11:05 +08:00
Henry Li 68df848b82 docs: add comments 2026-02-02 09:11:05 +08:00
Henry Li ac16a73a47 docs: add comments 2026-02-02 09:11:05 +08:00
Henry Li 54277b9d9e feat: add skeleton 2026-02-02 09:05:33 +08:00
Henry Li b797ef8168 feat: add skeleton 2026-02-02 09:05:33 +08:00
Henry Li 7da0a03dd0 feat: add skeleton 2026-02-02 09:05:33 +08:00
Henry Li a0a3a3fc02 feat: dynamic title 2026-02-02 09:05:24 +08:00
Henry Li be65130a06 feat: dynamic title 2026-02-02 09:05:24 +08:00
Henry Li 1eb4da6c75 feat: dynamic title 2026-02-02 09:05:24 +08:00
Henry Li b540ad4505 feat: use create skill as title 2026-02-02 08:43:37 +08:00
Henry Li dc1190b228 feat: use create skill as title 2026-02-02 08:43:37 +08:00
Henry Li b50fbf83d0 feat: use create skill as title 2026-02-02 08:43:37 +08:00
hetaoBackend f082ef3d87 feat: add find-skills skill for discovering agent skills
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 23:54:08 +08:00
hetaoBackend e4939216fd feat: add find-skills skill for discovering agent skills
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 23:54:08 +08:00
hetaoBackend 7fd5ba258d feat: add find-skills skill for discovering agent skills
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 23:54:08 +08:00
hetaoBackend 9043c964ca docs: add comprehensive backend documentation
- Add README.md with project overview, quick start, and API reference
- Add CONTRIBUTING.md with development setup and contribution guidelines
- Add docs/ARCHITECTURE.md with detailed system architecture diagrams
- Add docs/API.md with complete API reference for LangGraph and Gateway
- Add docs/README.md as documentation index
- Update CLAUDE.md with improved structure and new features
- Update docs/TODO.md to reflect current status
- Update pyproject.toml description

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 22:18:25 +08:00
hetaoBackend 68c3e3341a docs: add comprehensive backend documentation
- Add README.md with project overview, quick start, and API reference
- Add CONTRIBUTING.md with development setup and contribution guidelines
- Add docs/ARCHITECTURE.md with detailed system architecture diagrams
- Add docs/API.md with complete API reference for LangGraph and Gateway
- Add docs/README.md as documentation index
- Update CLAUDE.md with improved structure and new features
- Update docs/TODO.md to reflect current status
- Update pyproject.toml description

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 22:18:25 +08:00
hetaoBackend 4f4b7cde2e docs: add comprehensive backend documentation
- Add README.md with project overview, quick start, and API reference
- Add CONTRIBUTING.md with development setup and contribution guidelines
- Add docs/ARCHITECTURE.md with detailed system architecture diagrams
- Add docs/API.md with complete API reference for LangGraph and Gateway
- Add docs/README.md as documentation index
- Update CLAUDE.md with improved structure and new features
- Update docs/TODO.md to reflect current status
- Update pyproject.toml description

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 22:18:25 +08:00
Henry Li 9b77070406 feat: update skills 2026-02-01 11:12:08 +08:00
Henry Li 7e11f28d55 feat: update skills 2026-02-01 11:12:08 +08:00
Henry Li 890a8379ce feat: update skills 2026-02-01 11:12:08 +08:00
Henry Li ec444e1f8b docs: update artifacts 2026-02-01 11:05:00 +08:00
Henry Li 37e9810191 docs: update artifacts 2026-02-01 11:05:00 +08:00
Henry Li e28d5d2cf9 docs: update artifacts 2026-02-01 11:05:00 +08:00
Henry Li 22ef5fb5ba feat: add new demo 2026-02-01 10:58:27 +08:00
Henry Li d131a497d7 feat: add new demo 2026-02-01 10:58:27 +08:00
Henry Li 88e1c7c0b3 feat: add new demo 2026-02-01 10:58:27 +08:00
Henry Li f206a574c5 feat: update github-deep-research skill 2026-02-01 10:55:21 +08:00
Henry Li 8c37c9c755 feat: update github-deep-research skill 2026-02-01 10:55:21 +08:00
Henry Li f656fd0768 feat: update github-deep-research skill 2026-02-01 10:55:21 +08:00
Henry Li e1ecf62afa feat: add tooltip for installation 2026-02-01 10:55:08 +08:00
Henry Li 4721f1a890 feat: add tooltip for installation 2026-02-01 10:55:08 +08:00
Henry Li a1267875fa feat: add tooltip for installation 2026-02-01 10:55:08 +08:00
Henry Li 46feff6c16 feat: add github-deep-research skill 2026-02-01 10:54:19 +08:00
Henry Li 16122dd92d feat: add github-deep-research skill 2026-02-01 10:54:19 +08:00
Henry Li 469e044935 feat: add github-deep-research skill 2026-02-01 10:54:19 +08:00
Henry Li f5b1412ac0 fix: add translations 2026-01-31 22:49:59 +08:00
Henry Li 8a2fb353c6 fix: add translations 2026-01-31 22:49:59 +08:00
Henry Li 45fab66a7d fix: add translations 2026-01-31 22:49:59 +08:00
Henry Li ca83ed00f8 docs: rephrasing 2026-01-31 22:42:17 +08:00
Henry Li f3d7fea9ce docs: rephrasing 2026-01-31 22:42:17 +08:00
Henry Li 7d3e7eb1c9 docs: rephrasing 2026-01-31 22:42:17 +08:00
Henry Li bdd2e25e14 feat: implement create skill 2026-01-31 22:31:25 +08:00
Henry Li 8639dde3ad feat: implement create skill 2026-01-31 22:31:25 +08:00
Henry Li 67ec1162cb feat: implement create skill 2026-01-31 22:31:25 +08:00
hetaoBackend 06511f38e1 feat: add .skill file preview support
Enable previewing .skill files (ZIP archives) by extracting and displaying
their SKILL.md content. Add caching to avoid repeated ZIP extraction.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-31 22:27:06 +08:00
hetaoBackend f31258dd10 feat: add .skill file preview support
Enable previewing .skill files (ZIP archives) by extracting and displaying
their SKILL.md content. Add caching to avoid repeated ZIP extraction.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-31 22:27:06 +08:00
hetaoBackend 41f8b931c9 feat: add .skill file preview support
Enable previewing .skill files (ZIP archives) by extracting and displaying
their SKILL.md content. Add caching to avoid repeated ZIP extraction.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-31 22:27:06 +08:00
hetaoBackend a9e11f6341 feat: add skill installation API endpoint
Add POST /api/skills/install endpoint to install .skill files from
thread's user-data directory. The endpoint extracts the ZIP archive,
validates SKILL.md frontmatter, and installs to skills/custom/.

Frontend Install buttons now call the API instead of downloading.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-31 22:10:05 +08:00
hetaoBackend 624f758163 feat: add skill installation API endpoint
Add POST /api/skills/install endpoint to install .skill files from
thread's user-data directory. The endpoint extracts the ZIP archive,
validates SKILL.md frontmatter, and installs to skills/custom/.

Frontend Install buttons now call the API instead of downloading.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-31 22:10:05 +08:00
hetaoBackend 5834b15af7 feat: add skill installation API endpoint
Add POST /api/skills/install endpoint to install .skill files from
thread's user-data directory. The endpoint extracts the ZIP archive,
validates SKILL.md frontmatter, and installs to skills/custom/.

Frontend Install buttons now call the API instead of downloading.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-31 22:10:05 +08:00
hetaoBackend 1899333b95 refactor: update skills XML format in prompt template
Change skills rendering from attribute-based format to nested element format
with <available_skills>, <skill>, <name>, <description>, and <location> tags
for better readability and structure.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-31 21:54:29 +08:00
hetaoBackend 3c3bf901e7 refactor: update skills XML format in prompt template
Change skills rendering from attribute-based format to nested element format
with <available_skills>, <skill>, <name>, <description>, and <location> tags
for better readability and structure.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-31 21:54:29 +08:00
hetaoBackend c76481d8f7 refactor: update skills XML format in prompt template
Change skills rendering from attribute-based format to nested element format
with <available_skills>, <skill>, <name>, <description>, and <location> tags
for better readability and structure.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-31 21:54:29 +08:00
Henry Li d3ff5f9d3c fix: fix eslint errors and warnings 2026-01-31 21:46:31 +08:00
Henry Li 718bb947d0 fix: fix eslint errors and warnings 2026-01-31 21:46:31 +08:00
Henry Li 8ecb6b3d1d fix: fix eslint errors and warnings 2026-01-31 21:46:31 +08:00
Henry Li cf961328a9 feat: preview the message if possible 2026-01-31 20:22:15 +08:00
Henry Li 20a023ee90 feat: preview the message if possible 2026-01-31 20:22:15 +08:00
Henry Li 9c3b928f1d feat: preview the message if possible 2026-01-31 20:22:15 +08:00
Henry Li e858ef0250 fix: fix eslint errors 2026-01-31 11:11:13 +08:00
Henry Li b8281be892 fix: fix eslint errors 2026-01-31 11:11:13 +08:00
Henry Li 2ba687b239 fix: fix eslint errors 2026-01-31 11:11:13 +08:00
Henry Li 5295f5b5b9 feat: add notification 2026-01-31 11:08:27 +08:00
Henry Li 47fe2f8195 feat: add notification 2026-01-31 11:08:27 +08:00
Henry Li c62caf95c4 feat: add notification 2026-01-31 11:08:27 +08:00
Henry Li 835fd4d0c7 feat: change email 2026-01-30 22:32:47 +08:00
Henry Li cb660c2643 feat: change email 2026-01-30 22:32:47 +08:00
Henry Li 4e0571f3b3 feat: change email 2026-01-30 22:32:47 +08:00
hetaoBackend 43ee8a2968 fix: fix aio sandbox shutdown bug 2026-01-30 22:02:07 +08:00
hetaoBackend 733c020c58 fix: fix aio sandbox shutdown bug 2026-01-30 22:02:07 +08:00
hetaoBackend 8182ed3737 fix: fix aio sandbox shutdown bug 2026-01-30 22:02:07 +08:00
Henry Li c07c0228f6 fix: fix condition of displaying artifacts 2026-01-30 21:51:18 +08:00
Henry Li 697f094ba9 fix: fix condition of displaying artifacts 2026-01-30 21:51:18 +08:00
Henry Li 21e12d91eb fix: fix condition of displaying artifacts 2026-01-30 21:51:18 +08:00
Henry Li c1182c680c feat: support Github Flavored Markdown 2026-01-30 16:41:18 +08:00
Henry Li 618b3e1e8f feat: support Github Flavored Markdown 2026-01-30 16:41:18 +08:00
Henry Li 1bb91bb267 feat: support Github Flavored Markdown 2026-01-30 16:41:18 +08:00
Henry Li 7d024326dc chore: remove 2026-01-30 11:01:09 +08:00
Henry Li 3339e70c25 chore: remove 2026-01-30 11:01:09 +08:00
Henry Li 05794e29d1 chore: remove 2026-01-30 11:01:09 +08:00
Henry Li 4dffad89ca feat: re-arrange icons 2026-01-29 16:17:41 +08:00
Henry Li cbcbbbe0a8 feat: re-arrange icons 2026-01-29 16:17:41 +08:00
Henry Li 939745d027 feat: re-arrange icons 2026-01-29 16:17:41 +08:00
Henry Li a4f749f939 fix: add max width 2026-01-29 16:12:30 +08:00
Henry Li c265f5410d fix: add max width 2026-01-29 16:12:30 +08:00
Henry Li 66deedf3b2 fix: add max width 2026-01-29 16:12:30 +08:00
Henry Li a135ddfa48 feat: display mode 2026-01-29 15:57:08 +08:00
Henry Li 86ed750a38 feat: display mode 2026-01-29 15:57:08 +08:00
Henry Li 79955d2e6c feat: display mode 2026-01-29 15:57:08 +08:00
Henry Li 62ac3b6b03 feat: use "mode" instead of "thinking_enabled" and "is_plan_mode" 2026-01-29 15:48:50 +08:00
Henry Li 7bf15cb777 feat: use "mode" instead of "thinking_enabled" and "is_plan_mode" 2026-01-29 15:48:50 +08:00
Henry Li 98e08a85c9 feat: use "mode" instead of "thinking_enabled" and "is_plan_mode" 2026-01-29 15:48:50 +08:00
Henry Li 4411af68f5 fix: fix renaming 2026-01-29 15:31:56 +08:00
Henry Li caf469d2ab fix: fix renaming 2026-01-29 15:31:56 +08:00
Henry Li 0ba82a9fd7 fix: fix renaming 2026-01-29 15:31:56 +08:00
Henry Li 9d889434c4 feat: add placeholder for image 2026-01-29 15:01:18 +08:00
Henry Li 4fc54a7408 feat: add placeholder for image 2026-01-29 15:01:18 +08:00
Henry Li 16a9626d54 feat: add placeholder for image 2026-01-29 15:01:18 +08:00
hetao 2c7a56dd33 feat: optimize vision tools and image handling
- Add model-aware vision tool loading based on supports_vision flag
- Move view_image_tool from config to builtin tools for dynamic inclusion
- Add timeout to image search to prevent hanging requests
- Optimize image search results format using thumbnails
- Add image validation for reference images in generation
- Improve error handling with detailed messages

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-29 14:57:26 +08:00
hetao 314ea41781 feat: optimize vision tools and image handling
- Add model-aware vision tool loading based on supports_vision flag
- Move view_image_tool from config to builtin tools for dynamic inclusion
- Add timeout to image search to prevent hanging requests
- Optimize image search results format using thumbnails
- Add image validation for reference images in generation
- Improve error handling with detailed messages

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-29 14:57:26 +08:00
hetao 7aa10b980f feat: optimize vision tools and image handling
- Add model-aware vision tool loading based on supports_vision flag
- Move view_image_tool from config to builtin tools for dynamic inclusion
- Add timeout to image search to prevent hanging requests
- Optimize image search results format using thumbnails
- Add image validation for reference images in generation
- Improve error handling with detailed messages

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-29 14:57:26 +08:00
hetao 75801d9817 fix: fix frontend bug 2026-01-29 13:55:30 +08:00
hetao 2c6dbbe065 fix: fix frontend bug 2026-01-29 13:55:30 +08:00
hetao 3cbf54b2eb fix: fix frontend bug 2026-01-29 13:55:30 +08:00
hetao 09d9c18a28 feat: add view_image tool and optimize web fetch tools
Add image viewing capability for vision-enabled models with ViewImageMiddleware and view_image_tool. Limit web_fetch tool output to 4096 characters to prevent excessive content. Update model config to support vision capability flag.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-29 13:44:39 +08:00
hetao 7414947cc6 feat: add view_image tool and optimize web fetch tools
Add image viewing capability for vision-enabled models with ViewImageMiddleware and view_image_tool. Limit web_fetch tool output to 4096 characters to prevent excessive content. Update model config to support vision capability flag.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-29 13:44:39 +08:00
hetao 9dc2405555 feat: add view_image tool and optimize web fetch tools
Add image viewing capability for vision-enabled models with ViewImageMiddleware and view_image_tool. Limit web_fetch tool output to 4096 characters to prevent excessive content. Update model config to support vision capability flag.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-29 13:44:39 +08:00
Henry Li 73a5a7972e Merge pull request #22 from LofiSu/experimental
feat: add inline citations and thread management features
2026-01-29 13:06:00 +08:00
Henry Li d77e7c32b2 Merge pull request #22 from LofiSu/experimental
feat: add inline citations and thread management features
2026-01-29 13:06:00 +08:00
Henry Li 294a1614b8 Merge pull request #22 from LofiSu/experimental
feat: add inline citations and thread management features
2026-01-29 13:06:00 +08:00
LofiSu 588673d043 merge: upstream/experimental with citations feature
- Merge upstream changes including image search, tooltips, and UI improvements
- Keep citations feature with inline hover cards
- Resolve conflict in message-list-item.tsx: use upstream img max-width (90%) while preserving citations logic
- Maintain file upload improvements with citations support

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-01-29 12:55:43 +08:00
ruitanglin ac283b92aa merge: upstream/experimental with citations feature
- Merge upstream changes including image search, tooltips, and UI improvements
- Keep citations feature with inline hover cards
- Resolve conflict in message-list-item.tsx: use upstream img max-width (90%) while preserving citations logic
- Maintain file upload improvements with citations support

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-01-29 12:55:43 +08:00
ruitanglin 5120022d6d merge: upstream/experimental with citations feature
- Merge upstream changes including image search, tooltips, and UI improvements
- Keep citations feature with inline hover cards
- Resolve conflict in message-list-item.tsx: use upstream img max-width (90%) while preserving citations logic
- Maintain file upload improvements with citations support

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-01-29 12:55:43 +08:00
LofiSu 849cc4d771 feat: improve file upload message handling and UI
Backend:
- Handle both string and list format for message content in uploads middleware
- Extract text content from structured message blocks
- Add logging for debugging file upload flow

Frontend:
- Separate file display from message bubble for human messages
- Show uploaded files outside the message bubble for cleaner layout
- Improve file card border styling with subtle border color
- Add debug logging for message submission with files

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-01-29 12:51:21 +08:00
ruitanglin ce9731c10a feat: improve file upload message handling and UI
Backend:
- Handle both string and list format for message content in uploads middleware
- Extract text content from structured message blocks
- Add logging for debugging file upload flow

Frontend:
- Separate file display from message bubble for human messages
- Show uploaded files outside the message bubble for cleaner layout
- Improve file card border styling with subtle border color
- Add debug logging for message submission with files

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-01-29 12:51:21 +08:00
ruitanglin 341397562a feat: improve file upload message handling and UI
Backend:
- Handle both string and list format for message content in uploads middleware
- Extract text content from structured message blocks
- Add logging for debugging file upload flow

Frontend:
- Separate file display from message bubble for human messages
- Show uploaded files outside the message bubble for cleaner layout
- Improve file card border styling with subtle border color
- Add debug logging for message submission with files

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-01-29 12:51:21 +08:00
Henry Li eff241f9f2 feat: enable images in content 2026-01-29 12:51:04 +08:00
Henry Li f809b67c47 feat: enable images in content 2026-01-29 12:51:04 +08:00
LofiSu e2e0fbf114 fix: hide incomplete citations block during streaming
Improve UX by hiding citations block while it's being streamed:
- Remove complete citations blocks (existing logic)
- Also remove incomplete citations blocks during streaming
- Prevents flickering of raw citations XML in the UI

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-01-29 12:50:09 +08:00
ruitanglin 6ae4868780 fix: hide incomplete citations block during streaming
Improve UX by hiding citations block while it's being streamed:
- Remove complete citations blocks (existing logic)
- Also remove incomplete citations blocks during streaming
- Prevents flickering of raw citations XML in the UI

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-01-29 12:50:09 +08:00
ruitanglin 2ec506d590 fix: hide incomplete citations block during streaming
Improve UX by hiding citations block while it's being streamed:
- Remove complete citations blocks (existing logic)
- Also remove incomplete citations blocks during streaming
- Prevents flickering of raw citations XML in the UI

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-01-29 12:50:09 +08:00
LofiSu c14378a312 feat: refine citations format and improve content presentation
Backend:
- Simplify citations prompt format and rules
- Add clear distinction between chat responses and file content
- Enforce full URL usage in markdown links, prohibit [cite-1] format
- Require content-first approach: write full content, then add citations at end

Frontend:
- Hide <citations> block in both chat messages and markdown preview
- Remove top-level Citations/Sources list for cleaner UI
- Auto-remove <citations> block in code editor view for markdown files
- Keep inline citation hover cards for reference details

This ensures citations are presented like Claude: clean content with inline reference badges.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-01-29 12:29:13 +08:00
ruitanglin 4b63e70b7e feat: refine citations format and improve content presentation
Backend:
- Simplify citations prompt format and rules
- Add clear distinction between chat responses and file content
- Enforce full URL usage in markdown links, prohibit [cite-1] format
- Require content-first approach: write full content, then add citations at end

Frontend:
- Hide <citations> block in both chat messages and markdown preview
- Remove top-level Citations/Sources list for cleaner UI
- Auto-remove <citations> block in code editor view for markdown files
- Keep inline citation hover cards for reference details

This ensures citations are presented like Claude: clean content with inline reference badges.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-01-29 12:29:13 +08:00
ruitanglin e8a8b5e56b feat: refine citations format and improve content presentation
Backend:
- Simplify citations prompt format and rules
- Add clear distinction between chat responses and file content
- Enforce full URL usage in markdown links, prohibit [cite-1] format
- Require content-first approach: write full content, then add citations at end

Frontend:
- Hide <citations> block in both chat messages and markdown preview
- Remove top-level Citations/Sources list for cleaner UI
- Auto-remove <citations> block in code editor view for markdown files
- Keep inline citation hover cards for reference details

This ensures citations are presented like Claude: clean content with inline reference badges.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-01-29 12:29:13 +08:00
Henry Li 6b030d7589 feat: add tooltips 2026-01-29 09:23:06 +08:00
Henry Li e4d373541f feat: add tooltips 2026-01-29 09:23:06 +08:00
Henry Li c700bd6841 feat: enhance search_image 2026-01-29 09:19:43 +08:00
Henry Li f7ec116c26 feat: enhance search_image 2026-01-29 09:19:43 +08:00
Henry Li 8359d842b5 feat: support image_search 2026-01-29 08:59:55 +08:00
Henry Li d787b1ca54 feat: support image_search 2026-01-29 08:59:55 +08:00
Henry Li 7decdbcc83 fix: improve hasPresentFiles function to check for multiple tool calls 2026-01-29 08:59:45 +08:00
Henry Li 946031b79f fix: improve hasPresentFiles function to check for multiple tool calls 2026-01-29 08:59:45 +08:00
hetaoBackend 1926c58cf2 feat: add image search builtin tool 2026-01-29 08:23:50 +08:00
hetaoBackend 5e62471312 feat: add image search builtin tool 2026-01-29 08:23:50 +08:00
hetaoBackend 248ffe61bc feat: modernize PPT styles and add deep-research skill
Update presentation generation with contemporary design styles
(glassmorphism, dark-premium, neo-brutalist, etc.) and add a new
deep-research skill to guide thorough web research before content
generation tasks.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 01:54:57 +08:00
hetaoBackend af18df480b feat: modernize PPT styles and add deep-research skill
Update presentation generation with contemporary design styles
(glassmorphism, dark-premium, neo-brutalist, etc.) and add a new
deep-research skill to guide thorough web research before content
generation tasks.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 01:54:57 +08:00
hetaoBackend d4bfed271b feat: display ask_clarification tool messages directly in frontend
Simplify clarification message handling by having the frontend detect and
display ask_clarification tool messages directly, instead of relying on
backend to add an extra AIMessage.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 01:25:31 +08:00
hetaoBackend 73a1d32a5b feat: display ask_clarification tool messages directly in frontend
Simplify clarification message handling by having the frontend detect and
display ask_clarification tool messages directly, instead of relying on
backend to add an extra AIMessage.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 01:25:31 +08:00
LofiSu ad85b72064 feat: add inline citations and thread management features
Citations:
- Add citations parsing utilities for extracting source references from AI responses
- Render inline citations as hover card badges in message content
- Display citation cards with title, URL, and description on hover
- Add citation badge rendering in artifact markdown preview
- Update prompt to guide AI to output citations in correct format

Thread Management:
- Add rename functionality for chat threads with dialog UI
- Add share functionality to copy thread link to clipboard
- Share links use Vercel URL for production accessibility
- Add useRenameThread hook for thread title updates

i18n:
- Add translations for rename, share, cancel, save, and linkCopied

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-01-28 19:15:11 +08:00
ruitanglin 33c47e0c56 feat: add inline citations and thread management features
Citations:
- Add citations parsing utilities for extracting source references from AI responses
- Render inline citations as hover card badges in message content
- Display citation cards with title, URL, and description on hover
- Add citation badge rendering in artifact markdown preview
- Update prompt to guide AI to output citations in correct format

Thread Management:
- Add rename functionality for chat threads with dialog UI
- Add share functionality to copy thread link to clipboard
- Share links use Vercel URL for production accessibility
- Add useRenameThread hook for thread title updates

i18n:
- Add translations for rename, share, cancel, save, and linkCopied

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-01-28 19:15:11 +08:00
ruitanglin f8d2d88727 feat: add inline citations and thread management features
Citations:
- Add citations parsing utilities for extracting source references from AI responses
- Render inline citations as hover card badges in message content
- Display citation cards with title, URL, and description on hover
- Add citation badge rendering in artifact markdown preview
- Update prompt to guide AI to output citations in correct format

Thread Management:
- Add rename functionality for chat threads with dialog UI
- Add share functionality to copy thread link to clipboard
- Share links use Vercel URL for production accessibility
- Add useRenameThread hook for thread title updates

i18n:
- Add translations for rename, share, cancel, save, and linkCopied

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-01-28 19:15:11 +08:00
Henry Li a010953880 feat: update notes 2026-01-28 14:42:13 +08:00
Henry Li 453efa1a1d feat: update notes 2026-01-28 14:42:13 +08:00
Henry Li efb0db53bc Merge remote-tracking branch 'refs/remotes/origin/experimental' into experimental 2026-01-28 14:20:24 +08:00
Henry Li 3293dda5db Merge remote-tracking branch 'refs/remotes/origin/experimental' into experimental 2026-01-28 14:20:24 +08:00
Henry Li dd2c2011f1 feat: update a new demo 2026-01-28 14:19:00 +08:00
Henry Li c0980bfa82 feat: update a new demo 2026-01-28 14:19:00 +08:00
hetaoBackend 49f6c001c3 feat: modify the config example yaml 2026-01-28 14:06:38 +08:00
hetaoBackend 055ab1fb04 feat: modify the config example yaml 2026-01-28 14:06:38 +08:00
hetaoBackend fa9fba3f8e fix: preserve reasoning_content in multi-turn conversations
When using thinking-enabled models (like Kimi K2.5, DeepSeek), the API
expects reasoning_content on all assistant messages. The original
ChatDeepSeek stores reasoning_content in additional_kwargs but doesn't
include it when making subsequent API calls, causing "reasoning_content
is missing" errors.

This adds PatchedChatDeepSeek which overrides _get_request_payload to
restore reasoning_content from additional_kwargs into the payload.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 14:04:35 +08:00
hetaoBackend 9d0a0ea022 fix: preserve reasoning_content in multi-turn conversations
When using thinking-enabled models (like Kimi K2.5, DeepSeek), the API
expects reasoning_content on all assistant messages. The original
ChatDeepSeek stores reasoning_content in additional_kwargs but doesn't
include it when making subsequent API calls, causing "reasoning_content
is missing" errors.

This adds PatchedChatDeepSeek which overrides _get_request_payload to
restore reasoning_content from additional_kwargs into the payload.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 14:04:35 +08:00
Henry Li d84a34b7cd feat: add Leica demo 2026-01-28 13:52:41 +08:00
Henry Li 486a06d772 feat: add Leica demo 2026-01-28 13:52:41 +08:00
Henry Li d075e7a234 feat: fallback to error reporting 2026-01-28 12:23:29 +08:00
Henry Li 5980bbde02 feat: fallback to error reporting 2026-01-28 12:23:29 +08:00
Henry Li a249b7178a feat: add another Kimi K2.5 demo 2026-01-28 12:17:21 +08:00
Henry Li 5d5aec43d3 feat: add another Kimi K2.5 demo 2026-01-28 12:17:21 +08:00
Henry Li e2bcc70a84 feat: add kimi-k2.5 demo with vercel deployment 2026-01-28 10:13:32 +08:00
Henry Li ade5426d9e feat: add kimi-k2.5 demo with vercel deployment 2026-01-28 10:13:32 +08:00
Henry Li dab4093103 feat: fallback to textarea when loading 2026-01-28 10:13:17 +08:00
Henry Li 86c8f1a25e feat: fallback to textarea when loading 2026-01-28 10:13:17 +08:00
Henry Li 90782f29a2 chore: remove 2026-01-27 13:37:09 +08:00
Henry Li 6b3e101b66 chore: remove 2026-01-27 13:37:09 +08:00
Henry Li 28361ca03c feat: add scroll indicator 2026-01-27 13:15:49 +08:00
Henry Li 7c42fa5162 feat: add scroll indicator 2026-01-27 13:15:49 +08:00
Henry Li ed31dc6aab fix: hide chats when sidebar is not open 2026-01-27 10:41:08 +08:00
Henry Li ec31e61f95 fix: hide chats when sidebar is not open 2026-01-27 10:41:08 +08:00
Henry Li cc1fe4e50e fix: eslint 2026-01-27 10:39:44 +08:00
Henry Li 7928a6f2e1 fix: eslint 2026-01-27 10:39:44 +08:00
Henry Li eca2b139cc fix: bugfix 2026-01-27 09:50:42 +08:00
Henry Li 0bcbaebb7e fix: bugfix 2026-01-27 09:50:42 +08:00
hetao b8c33e342b feat: add firecrawl community package with web_search and web_fetch tools
Add web_search_tool and web_fetch_tool implementations using the official
firecrawl-py SDK as an alternative to Tavily/Jina AI integrations.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 19:58:08 +08:00
hetao ce7f7258ba feat: add firecrawl community package with web_search and web_fetch tools
Add web_search_tool and web_fetch_tool implementations using the official
firecrawl-py SDK as an alternative to Tavily/Jina AI integrations.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 19:58:08 +08:00
hetao 9215c9cce7 feat: add ppt-generation skill
Creates presentations by generating AI images for each slide and composing
them into PPTX files. Features include:
- Multiple presentation styles (business, academic, minimal, keynote, creative)
- Visual consistency through reference image chaining (each slide uses the
  previous slide as reference)
- Speaker notes from presentation plan

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 19:35:29 +08:00
hetao 0cc7cc08e9 feat: add ppt-generation skill
Creates presentations by generating AI images for each slide and composing
them into PPTX files. Features include:
- Multiple presentation styles (business, academic, minimal, keynote, creative)
- Visual consistency through reference image chaining (each slide uses the
  previous slide as reference)
- Speaker notes from presentation plan

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 19:35:29 +08:00
Henry Li 3ce4968e95 feat: auto select the first model as default model 2026-01-26 17:13:34 +08:00
Henry Li 574dfd2b05 feat: auto select the first model as default model 2026-01-26 17:13:34 +08:00
hetao 22004406a7 perf: parallelize TTS generation in podcast skill
Use ThreadPoolExecutor to generate audio for multiple script lines
concurrently, significantly speeding up podcast generation.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 14:50:56 +08:00
hetao ff7065b085 perf: parallelize TTS generation in podcast skill
Use ThreadPoolExecutor to generate audio for multiple script lines
concurrently, significantly speeding up podcast generation.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 14:50:56 +08:00
hetao dddd745b5b refactor: simplify podcast-generation to use direct JSON script input
- Remove LLM script generation from Python script, model now generates
  JSON script directly (similar to image-generation skill)
- Add --transcript-file option to generate markdown transcript
- Add optional "title" field in JSON for transcript heading
- Remove dependency on OPENAI_API_KEY for podcast generation
- Update SKILL.md with new workflow and JSON format documentation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 14:01:48 +08:00
hetao 68a3be1491 refactor: simplify podcast-generation to use direct JSON script input
- Remove LLM script generation from Python script, model now generates
  JSON script directly (similar to image-generation skill)
- Add --transcript-file option to generate markdown transcript
- Add optional "title" field in JSON for transcript heading
- Remove dependency on OPENAI_API_KEY for podcast generation
- Update SKILL.md with new workflow and JSON format documentation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 14:01:48 +08:00
hetaoBackend 9f5658fa0e feat: add podcast generation skill
- Add podcast-generation skill for creating tech explainer podcasts
- Include generate.py script with TTS synthesis capabilities
- Add tech-explainer template for structured podcast content
- Increase sandbox command timeout from 30s to 600s to support
  longer-running skill scripts
2026-01-26 13:16:35 +08:00
hetaoBackend 3fa16467a2 feat: add podcast generation skill
- Add podcast-generation skill for creating tech explainer podcasts
- Include generate.py script with TTS synthesis capabilities
- Add tech-explainer template for structured podcast content
- Increase sandbox command timeout from 30s to 600s to support
  longer-running skill scripts
2026-01-26 13:16:35 +08:00
hetaoBackend 139063283f fix: ensure MCP and skills config changes are immediately reflected
- Use ExtensionsConfig.from_file() instead of cached config to always
  read latest configuration from disk in LangGraph Server
- Add mtime-based cache invalidation for MCP tools to detect config
  file changes made through Gateway API
- Call reload_extensions_config() in Gateway API after updates to
  refresh the global cache
- Remove unnecessary MCP initialization from Gateway startup since
  MCP tools are only used by LangGraph Server

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 22:40:09 +08:00
hetaoBackend 038f5d44f4 fix: ensure MCP and skills config changes are immediately reflected
- Use ExtensionsConfig.from_file() instead of cached config to always
  read latest configuration from disk in LangGraph Server
- Add mtime-based cache invalidation for MCP tools to detect config
  file changes made through Gateway API
- Call reload_extensions_config() in Gateway API after updates to
  refresh the global cache
- Remove unnecessary MCP initialization from Gateway startup since
  MCP tools are only used by LangGraph Server

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 22:40:09 +08:00
Henry Li f629e134d4 feat: adjust button 2026-01-25 22:10:50 +08:00
Henry Li 044e38aec6 feat: adjust button 2026-01-25 22:10:50 +08:00
Henry Li 598fed797f fix: many minor fixes 2026-01-25 21:57:57 +08:00
Henry Li 756b396a64 fix: many minor fixes 2026-01-25 21:57:57 +08:00
Henry Li ae0e7de3b7 feat: add image and video generation skills 2026-01-25 21:57:44 +08:00
Henry Li b53a2ea5e1 feat: add image and video generation skills 2026-01-25 21:57:44 +08:00
Henry Li af4fc800ee feat: update demo 2026-01-25 21:57:22 +08:00
Henry Li 90c30c8485 feat: update demo 2026-01-25 21:57:22 +08:00
Henry Li 87200d1ad1 feat: update translations 2026-01-25 11:54:49 +08:00
Henry Li e6cac2cae4 feat: update translations 2026-01-25 11:54:49 +08:00
Henry Li c82f705541 fix: fix artifacts in demo mode 2026-01-25 11:42:25 +08:00
Henry Li fecc5faacf fix: fix artifacts in demo mode 2026-01-25 11:42:25 +08:00
Henry Li 74dd09b364 feat: update demos 2026-01-25 11:41:48 +08:00
Henry Li e84fb705ce feat: update demos 2026-01-25 11:41:48 +08:00
Henry Li 78bba47769 feat: add Titanic ADA demo 2026-01-25 00:35:42 +08:00
Henry Li c6dbd9fbf4 feat: add Titanic ADA demo 2026-01-25 00:35:42 +08:00
Henry Li 3ac6e58d4f fix: remove tooltip 2026-01-25 00:06:49 +08:00
Henry Li 9501ec5eed fix: remove tooltip 2026-01-25 00:06:49 +08:00
Henry Li 03b380cb8b fix: fix auto select first artifact 2026-01-24 23:59:41 +08:00
Henry Li 1e2855b533 fix: fix auto select first artifact 2026-01-24 23:59:41 +08:00
Henry Li 35f2aea510 feat: add new demo 2026-01-24 23:51:38 +08:00
Henry Li 03311d43da feat: add new demo 2026-01-24 23:51:38 +08:00
Henry Li a83e5e238d feat: auto expand in demo mode 2026-01-24 23:51:11 +08:00
Henry Li 099fb727cc feat: auto expand in demo mode 2026-01-24 23:51:11 +08:00
Henry Li 2698c26768 chore: update translation 2026-01-24 22:41:40 +08:00
Henry Li 2eb9bb2167 chore: update translation 2026-01-24 22:41:40 +08:00
hetao 6e147a772e feat: add environment variable injection for Docker sandbox
- Add environment field to sandbox config for injecting env vars into container
- Support $VAR syntax to resolve values from host environment variables
- Refactor frontend API modules to use centralized getBackendBaseURL()
- Improve Doraemon skill with explicit input/output path arguments
- Add .env.example file

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-24 22:36:05 +08:00
hetao 5671642dbe feat: add environment variable injection for Docker sandbox
- Add environment field to sandbox config for injecting env vars into container
- Support $VAR syntax to resolve values from host environment variables
- Refactor frontend API modules to use centralized getBackendBaseURL()
- Improve Doraemon skill with explicit input/output path arguments
- Add .env.example file

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-24 22:36:05 +08:00
Henry Li 869af570c9 feat: add i18n 2026-01-24 22:19:37 +08:00
Henry Li 48b5428000 feat: add i18n 2026-01-24 22:19:37 +08:00
Henry Li 2bdda01657 Merge remote-tracking branch 'refs/remotes/origin/experimental' into experimental 2026-01-24 22:05:15 +08:00
Henry Li 481396210b Merge remote-tracking branch 'refs/remotes/origin/experimental' into experimental 2026-01-24 22:05:15 +08:00
Henry Li 5a27a3beeb feat: expand by default in demo mode 2026-01-24 22:03:38 +08:00
Henry Li 56b21e00bf feat: expand by default in demo mode 2026-01-24 22:03:38 +08:00
JeffJiang 38081306fe feat: adds docker-based dev environment (#18)
* feat: adds docker-based dev environment

* docs: updates Docker command help

* fix local dev
2026-01-24 22:01:00 +08:00
JeffJiang 400349c3e0 feat: adds docker-based dev environment (#18)
* feat: adds docker-based dev environment

* docs: updates Docker command help

* fix local dev
2026-01-24 22:01:00 +08:00
Henry Li c468381064 feat: add Doraemon Skill 2026-01-24 21:54:01 +08:00
Henry Li ee9950d6aa feat: add Doraemon Skill 2026-01-24 21:54:01 +08:00
Henry Li cae7e67a1f feat: remove over-scroll 2026-01-24 21:14:33 +08:00
Henry Li 291b899486 feat: remove over-scroll 2026-01-24 21:14:33 +08:00
Henry Li 72e0f3d081 feat: add new demo 2026-01-24 20:59:06 +08:00
Henry Li 2c2a177186 feat: add new demo 2026-01-24 20:59:06 +08:00
Henry Li 08f1af00b6 feat: support absolute path as image src 2026-01-24 20:58:56 +08:00
Henry Li 4aef821344 feat: support absolute path as image src 2026-01-24 20:58:56 +08:00
Henry Li cdcadc3fe3 style: update tooltip background class for consistency 2026-01-24 20:58:36 +08:00
Henry Li c47455e1eb style: update tooltip background class for consistency 2026-01-24 20:58:36 +08:00
Henry Li 6485ed2a50 chore: add new demo 2026-01-24 19:44:06 +08:00
Henry Li cced422e9d chore: add new demo 2026-01-24 19:44:06 +08:00
Henry Li e88b34c0cb chore: delete 2026-01-24 19:43:44 +08:00
Henry Li 2487d7684f chore: delete 2026-01-24 19:43:44 +08:00
Henry Li 2c8a41dc6c style: reformat 2026-01-24 19:40:44 +08:00
Henry Li 931421f57b style: reformat 2026-01-24 19:40:44 +08:00
Henry Li 72e3ba9b79 feat: add new demo 2026-01-24 19:38:17 +08:00
Henry Li 373fe0cd3c feat: add new demo 2026-01-24 19:38:17 +08:00
Henry Li 27df1b5f73 feat: add uploads 2026-01-24 19:38:08 +08:00
Henry Li 1f4591a4d1 feat: add uploads 2026-01-24 19:38:08 +08:00
Henry Li a3eb03b105 chore: add new demo 2026-01-24 18:53:13 +08:00
Henry Li db27ca4ae0 chore: add new demo 2026-01-24 18:53:13 +08:00
Henry Li 930e6bd46f feat: remove background 2026-01-24 18:48:35 +08:00
Henry Li 0f1bfc3403 feat: remove background 2026-01-24 18:48:35 +08:00
Henry Li 6f24a71e1e feat: update save-demo 2026-01-24 18:33:30 +08:00
Henry Li 3ea1dcac11 feat: update save-demo 2026-01-24 18:33:30 +08:00
Henry Li 584c88f0dd feat: add more links 2026-01-24 18:21:50 +08:00
Henry Li 3c40446ade feat: add more links 2026-01-24 18:21:50 +08:00
Henry Li cd63f41b4c feat: support static website 2026-01-24 18:01:27 +08:00
Henry Li ebda30c7cf feat: support static website 2026-01-24 18:01:27 +08:00
Henry Li 3ffce7667c Merge pull request #16 from amszuidas/experimental
fix: refactor env var resolution to support complex structures and fix in-place mutation bug
2026-01-24 10:14:20 +08:00
Henry Li c66995bcc0 Merge pull request #16 from amszuidas/experimental
fix: refactor env var resolution to support complex structures and fix in-place mutation bug
2026-01-24 10:14:20 +08:00
Henry Li b1e7028ea0 Merge pull request #17 from amszuidas/fix/tavily-api-key-config
fix: support loading tavily ak from config.yaml
2026-01-24 10:14:04 +08:00
Henry Li 9498e783f1 Merge pull request #17 from amszuidas/fix/tavily-api-key-config
fix: support loading tavily ak from config.yaml
2026-01-24 10:14:04 +08:00
amszuidas d6176e86d6 fix: support loading tavily ak from config.yaml 2026-01-23 23:50:40 +08:00
amszuidas c1c8942491 fix: support loading tavily ak from config.yaml 2026-01-23 23:50:40 +08:00
amszuidas 3972485fe0 fix: use return value of resolve_env_variables in config loading 2026-01-23 21:51:48 +08:00
amszuidas 761cb6a7f5 fix: use return value of resolve_env_variables in config loading 2026-01-23 21:51:48 +08:00
hetao f6a20a69e3 feat: implement file upload feature 2026-01-23 18:47:39 +08:00
hetao 1fe37fdb6c feat: implement file upload feature 2026-01-23 18:47:39 +08:00
amszuidas eb802361e1 fix: correct spelling 2026-01-23 18:29:20 +08:00
amszuidas 2ef320f107 fix: correct spelling 2026-01-23 18:29:20 +08:00
amszuidas 82a6ae81bd fix: robust environment variable resolution in config 2026-01-23 17:01:38 +08:00
amszuidas 303e0252ce fix: robust environment variable resolution in config 2026-01-23 17:01:38 +08:00
Henry Li 3f4bcd9433 feat: implement the first version of landing page 2026-01-23 13:24:03 +08:00
Henry Li 0908127bd7 feat: implement the first version of landing page 2026-01-23 13:24:03 +08:00
Henry Li 307972f93e feat: implement the first section of landing page 2026-01-23 00:15:21 +08:00
Henry Li b69c13a3e5 feat: implement the first section of landing page 2026-01-23 00:15:21 +08:00
Henry Li 459d9d0287 fix: fix menu item in side bar collapsed mode 2026-01-22 15:18:42 +08:00
Henry Li 6e1f63e47f fix: fix menu item in side bar collapsed mode 2026-01-22 15:18:42 +08:00
Henry Li e9ab427326 feat: adjust styles 2026-01-22 14:28:10 +08:00
Henry Li dc9d28018c feat: adjust styles 2026-01-22 14:28:10 +08:00
Henry Li c48a3f499d docs: rewording 2026-01-22 14:21:03 +08:00
Henry Li 9df56299c1 docs: rewording 2026-01-22 14:21:03 +08:00
Henry Li e0f491dcdb feat: add main menu 2026-01-22 14:19:54 +08:00
Henry Li e1378123f5 feat: add main menu 2026-01-22 14:19:54 +08:00
Henry Li 80b07bcac0 feat: update opacities 2026-01-22 13:50:09 +08:00
Henry Li cb996f0858 feat: update opacities 2026-01-22 13:50:09 +08:00
Henry Li 8c994293a8 feat: make reasoning mode as default 2026-01-22 13:46:43 +08:00
Henry Li 99eb2474b3 feat: make reasoning mode as default 2026-01-22 13:46:43 +08:00
Henry Li ec4b3a0ead docs: update description 2026-01-22 13:46:31 +08:00
Henry Li b938f40e4c docs: update description 2026-01-22 13:46:31 +08:00
Henry Li 7d4d706738 feat: put all options into '+' 2026-01-22 13:43:45 +08:00
Henry Li 8ef89b3004 feat: put all options into '+' 2026-01-22 13:43:45 +08:00
hetao 31bf49917c feat: add unified development environment with nginx proxy
Add a root-level Makefile to manage frontend, backend, and nginx services:
- `make check` validates required dependencies (Node.js 22+, pnpm, uv, nginx)
- `make install` installs all project dependencies
- `make dev` starts all services with unified port 2026
- `make stop` and `make clean` for cleanup

Update nginx configuration:
- Change port from 8000 to 2026
- Add frontend upstream and routing (port 3000)
- Add /api/langgraph/* routing with path rewriting to LangGraph server
- Keep other /api/* routes to Gateway API
- Route non-API requests to frontend

Update frontend configuration:
- Use relative URLs through nginx proxy by default
- Support environment variables for direct backend access
- Construct full URL for LangGraph SDK compatibility

Clean up backend Makefile by removing nginx and serve targets.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-22 12:00:46 +08:00
hetao 2fac72601e feat: add unified development environment with nginx proxy
Add a root-level Makefile to manage frontend, backend, and nginx services:
- `make check` validates required dependencies (Node.js 22+, pnpm, uv, nginx)
- `make install` installs all project dependencies
- `make dev` starts all services with unified port 2026
- `make stop` and `make clean` for cleanup

Update nginx configuration:
- Change port from 8000 to 2026
- Add frontend upstream and routing (port 3000)
- Add /api/langgraph/* routing with path rewriting to LangGraph server
- Keep other /api/* routes to Gateway API
- Route non-API requests to frontend

Update frontend configuration:
- Use relative URLs through nginx proxy by default
- Support environment variables for direct backend access
- Construct full URL for LangGraph SDK compatibility

Clean up backend Makefile by removing nginx and serve targets.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-22 12:00:46 +08:00
hetao c00f780501 fix: fix nginx conf 2026-01-22 12:00:46 +08:00
hetao 50c25f5c4d fix: fix nginx conf 2026-01-22 12:00:46 +08:00
Henry Li 16a499190b feat: show in-progress 2026-01-22 11:56:45 +08:00
Henry Li 93f70893fc feat: show in-progress 2026-01-22 11:56:45 +08:00
Henry Li aa7436db2f feat: adjust input background in light mode 2026-01-22 11:51:56 +08:00
Henry Li 4f712861a3 feat: adjust input background in light mode 2026-01-22 11:51:56 +08:00
Henry Li 93842e81a4 feat: adjust styles 2026-01-22 11:42:25 +08:00
Henry Li aed2f7ce67 feat: adjust styles 2026-01-22 11:42:25 +08:00
Henry Li 54710960cb docs: remove '/' 2026-01-22 11:31:23 +08:00
Henry Li 3774d0453c docs: remove '/' 2026-01-22 11:31:23 +08:00
Henry Li 11918b5270 fix: update summarization configuration values 2026-01-22 10:36:19 +08:00
Henry Li bd33f72017 fix: update summarization configuration values 2026-01-22 10:36:19 +08:00
Henry Li e8e522c2fe feat: add animations 2026-01-22 09:41:01 +08:00
Henry Li 9e72dc4f63 feat: add animations 2026-01-22 09:41:01 +08:00
Henry Li 37e2c3d3c9 feat: update skill settings 2026-01-22 00:38:20 +08:00
Henry Li b630e1846a feat: update skill settings 2026-01-22 00:38:20 +08:00
Henry Li 1e4e51a80c feat: add Todos 2026-01-22 00:26:11 +08:00
Henry Li 44850d9a61 feat: add Todos 2026-01-22 00:26:11 +08:00
hetao 08101aa432 refactor: refine skills 2026-01-21 21:22:56 +08:00
hetao 085dff8d29 refactor: refine skills 2026-01-21 21:22:56 +08:00
hetao 5a45b9c131 feat: add SSE and HTTP transport support for MCP servers
- Add type, url, and headers fields to MCP server config
- Update MCP client to handle stdio, sse, and http transports
- Add todos field to ThreadState
- Add Deerflow branding requirement to frontend-design skill
- Update extensions_config.example.json with SSE/HTTP examples

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-21 16:14:00 +08:00
hetao 87752cafac feat: add SSE and HTTP transport support for MCP servers
- Add type, url, and headers fields to MCP server config
- Update MCP client to handle stdio, sse, and http transports
- Add todos field to ThreadState
- Add Deerflow branding requirement to frontend-design skill
- Update extensions_config.example.json with SSE/HTTP examples

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-21 16:14:00 +08:00
Henry Li fbe4d27ddd feat: use resolvedTheme instead of systemTheme 2026-01-21 10:48:29 +08:00
Henry Li 68b8083826 feat: use resolvedTheme instead of systemTheme 2026-01-21 10:48:29 +08:00
Henry Li 11c562eb98 refactor: move 2026-01-21 10:46:43 +08:00
Henry Li c7a40d6a23 refactor: move 2026-01-21 10:46:43 +08:00
Henry Li 54d29e254f docs: rewording 2026-01-21 10:46:18 +08:00
Henry Li f907f8ac16 docs: rewording 2026-01-21 10:46:18 +08:00
Henry Li e3d5b4960f feat: adjust colors 2026-01-21 10:35:50 +08:00
Henry Li ce4aa1e154 feat: adjust colors 2026-01-21 10:35:50 +08:00
Henry Li 26587ee970 feat: bring back the deer 2026-01-21 10:31:54 +08:00
Henry Li 1372dbefb2 feat: bring back the deer 2026-01-21 10:31:54 +08:00
Henry Li 220fc1c489 feat: auto open artifact 2026-01-21 09:45:55 +08:00
Henry Li 4467b1860f feat: auto open artifact 2026-01-21 09:45:55 +08:00
Henry Li 48742d1b59 feat: add code editor 2026-01-21 09:33:33 +08:00
Henry Li 6e024d6c8f feat: add code editor 2026-01-21 09:33:33 +08:00
Henry Li 7c6eb4cc8b feat: enlarge shadow 2026-01-21 08:52:30 +08:00
Henry Li 4b7ee2bee2 feat: enlarge shadow 2026-01-21 08:52:30 +08:00
Henry Li d77b9922a6 feat: make artifact "floating" 2026-01-21 08:50:15 +08:00
Henry Li 28d724d55a feat: make artifact "floating" 2026-01-21 08:50:15 +08:00
Henry Li a2ca682b0c feat: change color themes 2026-01-21 08:37:30 +08:00
Henry Li adfce3c79c feat: change color themes 2026-01-21 08:37:30 +08:00
Henry Li 10d253f461 feat: support settings 2026-01-20 23:43:21 +08:00
Henry Li 1b70e00642 feat: support settings 2026-01-20 23:43:21 +08:00
hetaoBackend 3191a3845f feat: integrate todo middleware 2026-01-20 22:38:04 +08:00
hetaoBackend 7ead7c93f8 feat: integrate todo middleware 2026-01-20 22:38:04 +08:00
hetaoBackend adbb03fc26 fix: fix sandbox cp issue 2026-01-20 22:08:36 +08:00
hetaoBackend c5a2771636 fix: fix sandbox cp issue 2026-01-20 22:08:36 +08:00
hetaoBackend 5888a5ba16 fix: fix skill md path 2026-01-20 21:10:05 +08:00
hetaoBackend e58e5f1904 fix: fix skill md path 2026-01-20 21:10:05 +08:00
hetaoBackend abc6c21b11 feat: enable public skills by default 2026-01-20 20:37:51 +08:00
hetaoBackend 2d931105d5 feat: enable public skills by default 2026-01-20 20:37:51 +08:00
Henry Li faba2784e1 feat: save locale in cookies 2026-01-20 16:00:39 +08:00
Henry Li dc8c1f4ed6 feat: save locale in cookies 2026-01-20 16:00:39 +08:00
Henry Li 32a45eb043 feat: implement i18n 2026-01-20 14:06:47 +08:00
Henry Li ac9ef30780 feat: implement i18n 2026-01-20 14:06:47 +08:00
hetaoBackend 6ec023de8b fix: fix config 2026-01-20 13:58:28 +08:00
hetaoBackend 33e6197f65 fix: fix config 2026-01-20 13:58:28 +08:00
hetaoBackend 50810c8212 feat: add skills api 2026-01-20 13:57:36 +08:00
hetaoBackend 66df9b5927 feat: add skills api 2026-01-20 13:57:36 +08:00
hetaoBackend 8434cf4c60 feat: add MCP API endpoint and enhance API documentation
Add new MCP configuration management endpoint and enhance API documentation
with detailed descriptions, examples, and OpenAPI support for better
developer experience.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-20 13:20:50 +08:00
hetaoBackend 411d9d57c3 feat: add MCP API endpoint and enhance API documentation
Add new MCP configuration management endpoint and enhance API documentation
with detailed descriptions, examples, and OpenAPI support for better
developer experience.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-20 13:20:50 +08:00
hetaoBackend d11763dcc8 fix: fix backend 2026-01-20 09:58:27 +08:00
hetaoBackend 5c1bb675ba fix: fix backend 2026-01-20 09:58:27 +08:00
Henry Li a18f37779e docs: rewording 2026-01-20 09:26:29 +08:00
Henry Li b791b28afa docs: rewording 2026-01-20 09:26:29 +08:00
DanielWalnut 513332b746 feat: add nginx reversed proxy (#15)
* docs: add nginx reverse proxy documentation

Add comprehensive nginx configuration documentation to README including:
- Production deployment instructions with step-by-step setup
- Architecture diagram showing traffic routing between services
- Nginx features: unified entry point, CORS handling, SSE support
- Updated project structure with nginx.conf and service ports

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* feat: implement nginx

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-19 23:23:38 +08:00
DanielWalnut 7978e05dc1 feat: add nginx reversed proxy (#15)
* docs: add nginx reverse proxy documentation

Add comprehensive nginx configuration documentation to README including:
- Production deployment instructions with step-by-step setup
- Architecture diagram showing traffic routing between services
- Nginx features: unified entry point, CORS handling, SSE support
- Updated project structure with nginx.conf and service ports

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* feat: implement nginx

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-19 23:23:38 +08:00
Henry Li b8f9678d07 feat: use code block to display bash commands 2026-01-19 21:59:23 +08:00
Henry Li 5d6162d006 feat: use code block to display bash commands 2026-01-19 21:59:23 +08:00
Henry Li fb265f2b1f feat: support NEXT_PUBLIC_LANGGRAPH_BASE_URL 2026-01-19 21:51:40 +08:00
Henry Li 58b5c2fcd5 feat: support NEXT_PUBLIC_LANGGRAPH_BASE_URL 2026-01-19 21:51:40 +08:00
hetaoBackend d6c1e5868d fix: fix proxy 2026-01-19 21:36:35 +08:00
hetaoBackend a6fcdbf50a fix: fix proxy 2026-01-19 21:36:35 +08:00
Henry Li d7dfffad90 feat: add ToggleGroup 2026-01-19 19:41:46 +08:00
Henry Li 24ca87d650 feat: add ToggleGroup 2026-01-19 19:41:46 +08:00
hetaoBackend 1171598b2f feat: add MCP (Model Context Protocol) support
Add comprehensive MCP integration using langchain-mcp-adapters to enable
pluggable external tools from MCP servers.

Features:
- MCP server configuration via mcp_config.json
- Automatic lazy initialization for seamless use in both FastAPI and LangGraph Studio
- Support for multiple MCP servers (filesystem, postgres, github, brave-search, etc.)
- Environment variable resolution in configuration
- Tool caching mechanism for optimal performance
- Complete documentation and setup guide

Implementation:
- Add src/mcp module with client, tools, and cache components
- Integrate MCP config loading in AppConfig
- Update tool system to include MCP tools automatically
- Add eager initialization in FastAPI lifespan handler
- Add lazy initialization fallback for LangGraph Studio

Dependencies:
- Add langchain-mcp-adapters>=0.1.0

Documentation:
- Add MCP_SETUP.md with comprehensive setup guide
- Update CLAUDE.md with MCP system architecture
- Update config.example.yaml with MCP configuration notes
- Update README.md with MCP setup instructions

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-19 18:58:13 +08:00
hetaoBackend 74d4a16492 feat: add MCP (Model Context Protocol) support
Add comprehensive MCP integration using langchain-mcp-adapters to enable
pluggable external tools from MCP servers.

Features:
- MCP server configuration via mcp_config.json
- Automatic lazy initialization for seamless use in both FastAPI and LangGraph Studio
- Support for multiple MCP servers (filesystem, postgres, github, brave-search, etc.)
- Environment variable resolution in configuration
- Tool caching mechanism for optimal performance
- Complete documentation and setup guide

Implementation:
- Add src/mcp module with client, tools, and cache components
- Integrate MCP config loading in AppConfig
- Update tool system to include MCP tools automatically
- Add eager initialization in FastAPI lifespan handler
- Add lazy initialization fallback for LangGraph Studio

Dependencies:
- Add langchain-mcp-adapters>=0.1.0

Documentation:
- Add MCP_SETUP.md with comprehensive setup guide
- Update CLAUDE.md with MCP system architecture
- Update config.example.yaml with MCP configuration notes
- Update README.md with MCP setup instructions

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-19 18:58:13 +08:00
Henry Li 541586dc66 feat: support dynamic loading models 2026-01-19 18:54:04 +08:00
Henry Li 21f35b1d3c feat: support dynamic loading models 2026-01-19 18:54:04 +08:00
hetaoBackend 1a7c853811 fix: use shared httpx client to prevent premature closure in SSE streaming
The proxy was creating a temporary httpx.AsyncClient within an async context manager.
When returning StreamingResponse for SSE endpoints, the client was being closed before
the streaming generator could use it, causing "client has been closed" errors.

This change introduces a shared httpx.AsyncClient that persists for the application
lifecycle, properly cleaned up during shutdown. This also improves performance by
reusing TCP connections across requests.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-19 16:52:30 +08:00
hetaoBackend ffb9ed3198 fix: use shared httpx client to prevent premature closure in SSE streaming
The proxy was creating a temporary httpx.AsyncClient within an async context manager.
When returning StreamingResponse for SSE endpoints, the client was being closed before
the streaming generator could use it, causing "client has been closed" errors.

This change introduces a shared httpx.AsyncClient that persists for the application
lifecycle, properly cleaned up during shutdown. This also improves performance by
reusing TCP connections across requests.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-19 16:52:30 +08:00
hetaoBackend 8ea530e221 fix: stop tracking .claude/settings.local.json
Remove .claude/settings.local.json from git tracking and add it to .gitignore to prevent future accidental commits of local settings.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-19 16:28:03 +08:00
hetaoBackend 3a4149c437 fix: stop tracking .claude/settings.local.json
Remove .claude/settings.local.json from git tracking and add it to .gitignore to prevent future accidental commits of local settings.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-19 16:28:03 +08:00
DanielWalnut 9a3eaea54e feat: implement summarization (#14) 2026-01-19 16:17:31 +08:00
DanielWalnut f0a2381bd5 feat: implement summarization (#14) 2026-01-19 16:17:31 +08:00
Henry Li 1ef04c94ee fix: fix getBackendBaseURL() 2026-01-19 15:42:19 +08:00
Henry Li 1352b0e0ba fix: fix getBackendBaseURL() 2026-01-19 15:42:19 +08:00
Henry Li f3f66ee924 feat: add NEXT_PUBLIC_BACKEND_BASE_URL 2026-01-19 11:23:40 +08:00
Henry Li 9d18e4e12d feat: add NEXT_PUBLIC_BACKEND_BASE_URL 2026-01-19 11:23:40 +08:00
Henry Li d8391ca3ea feat: make new chat always on top 2026-01-19 00:00:35 +08:00
Henry Li b431567666 feat: make new chat always on top 2026-01-19 00:00:35 +08:00
Henry Li 63fa500716 fix: decode URL 2026-01-18 20:26:01 +08:00
Henry Li c321c9293a fix: decode URL 2026-01-18 20:26:01 +08:00
Henry Li dc04042b53 feat: support clarification tool 2026-01-18 20:17:32 +08:00
Henry Li 5624b0cd38 feat: support clarification tool 2026-01-18 20:17:32 +08:00
Henry Li 69b225082b feat: re-implement message group 2026-01-18 19:56:07 +08:00
Henry Li aa44566fef feat: re-implement message group 2026-01-18 19:56:07 +08:00
DanielWalnut 645923c3bc feat: add clarification feature (#13) 2026-01-18 19:55:36 +08:00
DanielWalnut e1a8d544b6 feat: add clarification feature (#13) 2026-01-18 19:55:36 +08:00
Henry Li dd80348b76 feat: support SSE write_file(0 2026-01-18 17:13:15 +08:00
Henry Li ec1964c829 feat: support SSE write_file(0 2026-01-18 17:13:15 +08:00
DanielWalnut c50540e3fc fix: Long thinking but with empty content (#12) 2026-01-18 14:21:40 +08:00
DanielWalnut 6f97dde5d1 fix: Long thinking but with empty content (#12) 2026-01-18 14:21:40 +08:00
DanielWalnut 1397f30f24 feat: implement lazy sandbox and thread data initialization (#11)
Defer sandbox acquisition and thread directory creation until first use to improve performance and reduce resource usage.

Changes:
- Add lazy_init parameter to SandboxMiddleware (default: true)
- Add ensure_sandbox_initialized() helper for lazy sandbox acquisition
- Update all sandbox tools to use lazy initialization
- Add lazy_init parameter to ThreadDataMiddleware (default: true)
- Create thread directories on-demand in AioSandboxProvider
- LocalSandbox already creates directories on write (no changes needed)

Benefits:
- Saves 1-2s Docker container startup for conversations without tools
- Reduces unnecessary directory creation and file system operations
- Backward compatible with lazy_init=false option

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-18 13:38:34 +08:00
DanielWalnut 5f4c58aa82 feat: implement lazy sandbox and thread data initialization (#11)
Defer sandbox acquisition and thread directory creation until first use to improve performance and reduce resource usage.

Changes:
- Add lazy_init parameter to SandboxMiddleware (default: true)
- Add ensure_sandbox_initialized() helper for lazy sandbox acquisition
- Update all sandbox tools to use lazy initialization
- Add lazy_init parameter to ThreadDataMiddleware (default: true)
- Create thread directories on-demand in AioSandboxProvider
- LocalSandbox already creates directories on write (no changes needed)

Benefits:
- Saves 1-2s Docker container startup for conversations without tools
- Reduces unnecessary directory creation and file system operations
- Backward compatible with lazy_init=false option

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-18 13:38:34 +08:00
Henry Li 8f0bd828d5 feat: add recursion_limit 2026-01-18 13:16:27 +08:00
Henry Li 41a22fde91 feat: add recursion_limit 2026-01-18 13:16:27 +08:00
Henry Li 6bf187c1c2 fix: fix message grouping issues 2026-01-18 13:07:56 +08:00
Henry Li 71eadc942f fix: fix message grouping issues 2026-01-18 13:07:56 +08:00
DanielWalnut bfe8a24350 fix: fix backend python execution (#10) 2026-01-18 12:41:48 +08:00
DanielWalnut 5a0912d0fd fix: fix backend python execution (#10) 2026-01-18 12:41:48 +08:00
Henry Li 23dc64fab1 feat: enhance message display 2026-01-18 11:25:46 +08:00
Henry Li 9605cec6d3 feat: enhance message display 2026-01-18 11:25:46 +08:00
Henry Li 59683fc12e feat: dim the placeholder 2026-01-18 09:57:30 +08:00
Henry Li f9242727c7 feat: dim the placeholder 2026-01-18 09:57:30 +08:00
Henry Li 54f58fd7eb Merge remote-tracking branch 'refs/remotes/origin/experimental' into experimental 2026-01-18 09:57:23 +08:00
Henry Li bffe802130 Merge remote-tracking branch 'refs/remotes/origin/experimental' into experimental 2026-01-18 09:57:23 +08:00
Henry Li 92fc19a3aa feat: remove model icon 2026-01-18 09:55:17 +08:00
Henry Li 449f04fc44 feat: remove model icon 2026-01-18 09:55:17 +08:00
DanielWalnut aa030410fc feat: fix todos (#9) 2026-01-17 23:23:12 +08:00
DanielWalnut 3261273ee3 feat: fix todos (#9) 2026-01-17 23:23:12 +08:00
Henry Li 3f1f6af30c feat: change back to 60px height 2026-01-17 22:01:17 +08:00
Henry Li fa07e9e903 feat: change back to 60px height 2026-01-17 22:01:17 +08:00
Henry Li 7ea7a7864e feat: use default sidebar width 2026-01-17 22:01:08 +08:00
Henry Li d0988b3cf0 feat: use default sidebar width 2026-01-17 22:01:08 +08:00
Henry Li caf761be59 fix: fix z index 2026-01-17 21:34:32 +08:00
Henry Li 88eb341115 fix: fix z index 2026-01-17 21:34:32 +08:00
Henry Li 5cda2b90fc feat: refine theme 2026-01-17 21:24:49 +08:00
Henry Li 36b7ac0ce4 feat: refine theme 2026-01-17 21:24:49 +08:00
Henry Li 70cd664d3f feat: adjust dark theme 2026-01-17 21:08:05 +08:00
Henry Li 00fc70536e feat: adjust dark theme 2026-01-17 21:08:05 +08:00
Henry Li 32a77cce84 feat: the DeerFlow theme is back 2026-01-17 20:59:42 +08:00
Henry Li 6c9b0f275b feat: the DeerFlow theme is back 2026-01-17 20:59:42 +08:00
Henry Li 094553ea42 feat: change light theme 2026-01-17 20:32:27 +08:00
Henry Li 79d87de523 feat: change light theme 2026-01-17 20:32:27 +08:00
Henry Li df65010e5f fix: remove unused imports 2026-01-17 19:47:51 +08:00
Henry Li e418eb6110 fix: remove unused imports 2026-01-17 19:47:51 +08:00
Henry Li 2bc5f30c4d feat: welcome, again 2026-01-17 19:46:02 +08:00
Henry Li a6e5ebe898 feat: welcome, again 2026-01-17 19:46:02 +08:00
Henry Li 06068dd07b feat: add reasoning check to message list item rendering 2026-01-17 18:02:19 +08:00
Henry Li 0ea448a220 feat: add reasoning check to message list item rendering 2026-01-17 18:02:19 +08:00
Henry Li b705a44f3c feat: pull up the input box when creating new thread 2026-01-17 18:02:01 +08:00
Henry Li cb54b5dffa feat: pull up the input box when creating new thread 2026-01-17 18:02:01 +08:00
Henry Li 85d9baf2b1 feat:enhance focus status 2026-01-17 17:52:15 +08:00
Henry Li 9bfa49ae07 feat:enhance focus status 2026-01-17 17:52:15 +08:00
Henry Li a64b0d226b feat: redesign step counter 2026-01-17 17:45:13 +08:00
Henry Li 62921ec96a feat: redesign step counter 2026-01-17 17:45:13 +08:00
Henry Li 97dbcc4bd6 fix: remove unused imports 2026-01-17 17:37:36 +08:00
Henry Li 63f3c9e2bb fix: remove unused imports 2026-01-17 17:37:36 +08:00
Henry Li d8f0f91238 feat: extract ThreadTitle component 2026-01-17 17:37:12 +08:00
Henry Li 7b33214a05 feat: extract ThreadTitle component 2026-01-17 17:37:12 +08:00
Henry Li f1c6991194 feat: integrated with artifacts in states 2026-01-17 17:21:37 +08:00
Henry Li 9a3f72869c feat: integrated with artifacts in states 2026-01-17 17:21:37 +08:00
Henry Li 384353d613 feat: remove ring 2026-01-17 17:21:05 +08:00
Henry Li ab65ab3af2 feat: remove ring 2026-01-17 17:21:05 +08:00
Henry Li a66d515214 chore: add TODO for checking duplicate files in state.artifacts 2026-01-17 16:25:51 +08:00
Henry Li d603771291 chore: add TODO for checking duplicate files in state.artifacts 2026-01-17 16:25:51 +08:00
Henry Li 1c74e9996f dos: update backend TODOs 2026-01-17 16:17:59 +08:00
Henry Li 1d3b3e8eb8 dos: update backend TODOs 2026-01-17 16:17:59 +08:00
Henry Li a663bcc37b feat: merge the last thinking with the previous group 2026-01-17 16:10:58 +08:00
Henry Li 1a3b70ac43 feat: merge the last thinking with the previous group 2026-01-17 16:10:58 +08:00
Henry Li 584eed0166 fix: do not display 'Untitled' 2026-01-17 15:48:43 +08:00
Henry Li be1e016ed4 fix: do not display 'Untitled' 2026-01-17 15:48:43 +08:00
Henry Li 56da1c990a feat: implement '/chats' 2026-01-17 15:44:49 +08:00
Henry Li e2d0246827 feat: implement '/chats' 2026-01-17 15:44:49 +08:00
Henry Li 228ec49f70 feat: add date time util 2026-01-17 15:44:38 +08:00
Henry Li c38dfdf0e0 feat: add date time util 2026-01-17 15:44:38 +08:00
Henry Li 0e8fdf6234 feat: shrink card size 2026-01-17 15:22:00 +08:00
Henry Li 31510879f2 feat: shrink card size 2026-01-17 15:22:00 +08:00
Henry Li 5dc40a9ade feat: add open in new window 2026-01-17 15:19:53 +08:00
Henry Li aa2677e9fd feat: add open in new window 2026-01-17 15:19:53 +08:00
Henry Li 962d8f04ec feat: support artifact preview 2026-01-17 15:09:44 +08:00
Henry Li 0c6f8353bf feat: support artifact preview 2026-01-17 15:09:44 +08:00
Henry Li ec5bbf6b51 feat: set artifacts layout 2026-01-17 11:02:33 +08:00
Henry Li 80c928fcf5 feat: set artifacts layout 2026-01-17 11:02:33 +08:00
Henry Li 9d1cf89532 chore: remove unused components 2026-01-17 10:09:43 +08:00
Henry Li 5d23dd0763 chore: remove unused components 2026-01-17 10:09:43 +08:00
Henry Li a973c82a1f chore: downgrade shiki since breaking changes 2026-01-17 10:05:55 +08:00
Henry Li 58ff7f8876 chore: downgrade shiki since breaking changes 2026-01-17 10:05:55 +08:00
Henry Li 664ccb922f style: format 2026-01-17 00:13:24 +08:00
Henry Li dfe8f58c16 style: format 2026-01-17 00:13:24 +08:00
Henry Li 4e7256a9d8 feat: make BETTER_AUTH_* optional 2026-01-17 00:13:17 +08:00
Henry Li 9a4cb616c9 feat: make BETTER_AUTH_* optional 2026-01-17 00:13:17 +08:00
Henry Li bb92dec8d5 feat: ignore components from 3rd parties 2026-01-17 00:12:57 +08:00
Henry Li 1a99ae9c36 feat: ignore components from 3rd parties 2026-01-17 00:12:57 +08:00
Henry Li 4613d6e16e refactor: rename 2026-01-17 00:05:19 +08:00
Henry Li c216093360 refactor: rename 2026-01-17 00:05:19 +08:00
Henry Li 9d64c7e076 feat: integrated with artifacts 2026-01-17 00:02:03 +08:00
Henry Li e5050c6c1e feat: integrated with artifacts 2026-01-17 00:02:03 +08:00
Henry Li 34ca58ed1b fix: fix broken when SSE 2026-01-16 23:15:53 +08:00
Henry Li 16a5ed9a73 fix: fix broken when SSE 2026-01-16 23:15:53 +08:00
DanielWalnut facde645d7 feat: add artifacts logic (#8) 2026-01-16 23:04:38 +08:00
DanielWalnut d5b3052cda feat: add artifacts logic (#8) 2026-01-16 23:04:38 +08:00
Henry Li 6464a67230 feat: remember sidebar state 2026-01-16 23:03:39 +08:00
Henry Li 0d11b21c84 feat: remember sidebar state 2026-01-16 23:03:39 +08:00
Henry Li f9853f037c feat: support basic file presenting 2026-01-16 22:35:20 +08:00
Henry Li 83f367b98a feat: support basic file presenting 2026-01-16 22:35:20 +08:00
DanielWalnut 4b69aed47b feat: add thread-safety and graceful shutdown to AioSandboxProvider (#7)
Add thread-safe port allocation and proper cleanup on process exit to
prevent port conflicts in concurrent environments and ensure containers
are stopped when the application terminates.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 22:28:19 +08:00
DanielWalnut 50a1e407cf feat: add thread-safety and graceful shutdown to AioSandboxProvider (#7)
Add thread-safe port allocation and proper cleanup on process exit to
prevent port conflicts in concurrent environments and ensure containers
are stopped when the application terminates.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 22:28:19 +08:00
Henry Li c0e63c5308 docs: rewording 2026-01-16 22:10:08 +08:00
Henry Li d2845f658f docs: rewording 2026-01-16 22:10:08 +08:00
Henry Li 93a231cfb1 feat: integrated with artifact resizable 2026-01-16 21:55:31 +08:00
Henry Li ca70e2dcf7 feat: integrated with artifact resizable 2026-01-16 21:55:31 +08:00
Henry Li 68fbf53fb2 chore: add resizable 2026-01-16 21:54:54 +08:00
Henry Li e1ddb1ee42 chore: add resizable 2026-01-16 21:54:54 +08:00
Henry Li 1517e8675d feat: add present_file tool 2026-01-16 21:48:00 +08:00
Henry Li 56b26c060e feat: add present_file tool 2026-01-16 21:48:00 +08:00
Henry Li 91eff99f01 feat: add flip display effect 2026-01-16 20:40:09 +08:00
Henry Li e37be40773 feat: add flip display effect 2026-01-16 20:40:09 +08:00
Henry Li c265734c6e feat: adjust layout 2026-01-16 20:06:30 +08:00
Henry Li 6e5dab76cc feat: adjust layout 2026-01-16 20:06:30 +08:00
Henry Li 03f0e3f0c7 refactor: move biz logic to core 2026-01-16 19:51:39 +08:00
Henry Li ce70b67459 refactor: move biz logic to core 2026-01-16 19:51:39 +08:00
Henry Li 7066a3b691 feat: adjust layout and style of tooltip 2026-01-16 19:51:14 +08:00
Henry Li f6c20dbcfe feat: adjust layout and style of tooltip 2026-01-16 19:51:14 +08:00
Henry Li b72eb61302 refactor: simplify parameter 2026-01-16 19:50:41 +08:00
Henry Li a8ba2de579 refactor: simplify parameter 2026-01-16 19:50:41 +08:00
Henry Li df396fc246 feat: add copy button 2026-01-16 19:50:23 +08:00
Henry Li 574b7e59ce feat: add copy button 2026-01-16 19:50:23 +08:00
Henry Li 6bd49ab411 chore: remove 2026-01-16 19:50:17 +08:00
Henry Li b5dcb2b1d3 chore: remove 2026-01-16 19:50:17 +08:00
DanielWalnut 9f755ecc30 feat: add skills system for specialized agent workflows (#6)
Implement a skills framework that enables specialized workflows for
specific tasks (e.g., PDF processing, web page generation). Skills are
discovered from the skills/ directory and automatically mounted in
sandboxes with path mapping support.

- Add SkillsConfig for configuring skills path and container mount point
- Implement dynamic skill loading from SKILL.md files with YAML frontmatter
- Add path mapping in LocalSandbox to translate container paths to local paths
- Mount skills directory in AIO Docker sandbox containers
- Update lead agent prompt to dynamically inject available skills
- Add setup documentation and expand config.example.yaml

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 14:44:51 +08:00
DanielWalnut cfa97f7a96 feat: add skills system for specialized agent workflows (#6)
Implement a skills framework that enables specialized workflows for
specific tasks (e.g., PDF processing, web page generation). Skills are
discovered from the skills/ directory and automatically mounted in
sandboxes with path mapping support.

- Add SkillsConfig for configuring skills path and container mount point
- Implement dynamic skill loading from SKILL.md files with YAML frontmatter
- Add path mapping in LocalSandbox to translate container paths to local paths
- Mount skills directory in AIO Docker sandbox containers
- Update lead agent prompt to dynamically inject available skills
- Add setup documentation and expand config.example.yaml

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 14:44:51 +08:00
Henry Li f19e3ae8ac fix: lastStep could be empty 2026-01-16 14:38:49 +08:00
Henry Li 5ef3cb57ee fix: lastStep could be empty 2026-01-16 14:38:49 +08:00
Henry Li 52b9d0cffc feat: remove scroll button 2026-01-16 14:38:33 +08:00
Henry Li a589fb3dae feat: remove scroll button 2026-01-16 14:38:33 +08:00
Henry Li 2105170d39 Merge remote-tracking branch 'refs/remotes/origin/experimental' into experimental 2026-01-16 14:04:21 +08:00
Henry Li d8b7d06e99 Merge remote-tracking branch 'refs/remotes/origin/experimental' into experimental 2026-01-16 14:04:21 +08:00
Henry Li faf80bb429 feat: rename 'model' to 'model_name' 2026-01-16 14:03:34 +08:00
Henry Li ac075477a0 feat: rename 'model' to 'model_name' 2026-01-16 14:03:34 +08:00
DanielWalnut 7284eb15f1 feat: add gateway module with FastAPI server (#5)
* chore: add .claude/ to .gitignore

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add gateway module with FastAPI server

- Add new gateway module with FastAPI app for API routing
- Add gateway and serve commands to Makefile
- Add fastapi, httpx, uvicorn, sse-starlette dependencies
- Fix model config retrieval in lead_agent (support both model_name and model)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 13:22:26 +08:00
DanielWalnut fb92a472e2 feat: add gateway module with FastAPI server (#5)
* chore: add .claude/ to .gitignore

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add gateway module with FastAPI server

- Add new gateway module with FastAPI app for API routing
- Add gateway and serve commands to Makefile
- Add fastapi, httpx, uvicorn, sse-starlette dependencies
- Fix model config retrieval in lead_agent (support both model_name and model)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 13:22:26 +08:00
Henry Li 1f03fb3749 fix: navigate to home only in open-mode 2026-01-16 09:58:16 +08:00
Henry Li 5c94e6d222 fix: navigate to home only in open-mode 2026-01-16 09:58:16 +08:00
Henry Li 7c6189668c feat: link to home page 2026-01-16 09:56:30 +08:00
Henry Li 5fa98bf6cd feat: link to home page 2026-01-16 09:56:30 +08:00
Henry Li 028f402ff5 feat: store the local settings 2026-01-16 09:55:02 +08:00
Henry Li 3a62deb3fd feat: store the local settings 2026-01-16 09:55:02 +08:00
Henry Li 3f2bfded41 feat: enable edit context options 2026-01-16 09:37:04 +08:00
Henry Li cad12068ef feat: enable edit context options 2026-01-16 09:37:04 +08:00
Henry Li 956f8f0cfb refactor: rename 2026-01-16 09:15:04 +08:00
Henry Li c5b4c90b83 refactor: rename 2026-01-16 09:15:04 +08:00
Henry Li 08e0a1da16 chore: remove 2026-01-16 09:14:25 +08:00
Henry Li 608b703d26 chore: remove 2026-01-16 09:14:25 +08:00
Henry Li e9846c1dda refactor: refine folder structure and rename 2026-01-16 09:13:02 +08:00
Henry Li fe7504daed refactor: refine folder structure and rename 2026-01-16 09:13:02 +08:00
Henry Li 61499624a0 feat: adjust message group layout 2026-01-15 23:56:42 +08:00
Henry Li 7680a5adba feat: adjust message group layout 2026-01-15 23:56:42 +08:00
Henry Li 00ad4206c4 feat: enhance label 2026-01-15 23:47:36 +08:00
Henry Li f353831ac9 feat: enhance label 2026-01-15 23:47:36 +08:00
Henry Li c3cb4c348d feat: remove max-w- 2026-01-15 23:47:28 +08:00
Henry Li d45f48adde feat: remove max-w- 2026-01-15 23:47:28 +08:00
Henry Li 9f2b94ed52 feat: implement basic web app 2026-01-15 23:40:21 +08:00
Henry Li cecc684de1 feat: implement basic web app 2026-01-15 23:40:21 +08:00
DanielWalnut b44144dd2c feat: support function factory (#4) 2026-01-15 22:05:54 +08:00
DanielWalnut c7d68c6d3f feat: support function factory (#4) 2026-01-15 22:05:54 +08:00
DanielWalnut a39f799a7e fix: fix local path for local sandbox (#3) 2026-01-15 14:37:00 +08:00
DanielWalnut 3b879e277e fix: fix local path for local sandbox (#3) 2026-01-15 14:37:00 +08:00
DanielWalnut c92eedc572 feat: add thread data middleware (#2) 2026-01-15 13:22:30 +08:00
DanielWalnut 41442ccc2f feat: add thread data middleware (#2) 2026-01-15 13:22:30 +08:00
DanielWalnut ab427731dc feat: add AIO sandbox provider and auto title generation (#1)
- Add AioSandboxProvider for Docker-based sandbox execution with
  configurable container lifecycle, volume mounts, and port management
- Add TitleMiddleware to auto-generate thread titles after first
  user-assistant exchange using LLM
- Add Claude Code documentation (CLAUDE.md, AGENTS.md)
- Extend SandboxConfig with Docker-specific options (image, port, mounts)
- Fix hardcoded mount path to use expanduser
- Add agent-sandbox and dotenv dependencies

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-14 23:29:18 +08:00
DanielWalnut b2abfecf67 feat: add AIO sandbox provider and auto title generation (#1)
- Add AioSandboxProvider for Docker-based sandbox execution with
  configurable container lifecycle, volume mounts, and port management
- Add TitleMiddleware to auto-generate thread titles after first
  user-assistant exchange using LLM
- Add Claude Code documentation (CLAUDE.md, AGENTS.md)
- Extend SandboxConfig with Docker-specific options (image, port, mounts)
- Fix hardcoded mount path to use expanduser
- Add agent-sandbox and dotenv dependencies

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-14 23:29:18 +08:00
Henry Li de2d18561a feat: integrated with sandbox 2026-01-14 12:32:34 +08:00
Henry Li c1e9340062 chore: use .env 2026-01-14 12:32:15 +08:00
Henry Li 57dfc89ca1 chore: specify project name 2026-01-14 09:58:53 +08:00
Henry Li ce0b6f7754 chore: specify project title 2026-01-14 09:57:52 +08:00
Henry Li 5d6a7442d6 chore: remove tests 2026-01-14 09:52:34 +08:00
Henry Li c628c7f8db chore: create frontend project from boilerplate 2026-01-14 09:50:26 +08:00
Henry Li 3ff7613dd9 chore: mark backend folder as a Python project 2026-01-14 09:24:33 +08:00
Henry Li 2aeaf7c965 style: format 2026-01-14 09:21:19 +08:00
Henry Li 421488a991 chore: add lint and format 2026-01-14 09:20:05 +08:00
Henry Li 2e3a50d847 chore: update workspace structure 2026-01-14 09:19:54 +08:00
Henry Li e5c69cb7ee docs: update tool docs 2026-01-14 09:12:03 +08:00
Henry Li cb611f9270 chore: use ruff to lint and auto-format 2026-01-14 09:08:20 +08:00
Henry Li 7dc063ba25 feat: add agents 2026-01-14 07:20:00 +08:00
Henry Li cbbbac0c2b feat: add tools 2026-01-14 07:19:43 +08:00
Henry Li 57a02acb59 feat: add sandbox and local impl 2026-01-14 07:19:34 +08:00
Henry Li 4b5f529903 feat: integrated with Tavily and Jina AI 2026-01-14 07:17:22 +08:00
Henry Li 83bd7e4309 feat: add model modules 2026-01-14 07:16:45 +08:00
Henry Li 721b26a32f chore: add an empty __init__.py 2026-01-14 07:16:27 +08:00
Henry Li 86524a65f6 feat: add reflection modules 2026-01-14 07:16:07 +08:00
Henry Li 88ed3841c7 feat: add config modules 2026-01-14 07:15:58 +08:00
Henry Li c2a62a2266 chore: add Python and LangGraph stuff 2026-01-14 07:15:02 +08:00
Henry Li 81bd4dafa8 chore: add .gitignore for Python project 2026-01-14 07:14:00 +08:00
Henry Li dd545cfb97 chore: init 2026-01-14 07:09:20 +08:00
1241 changed files with 159439 additions and 140351 deletions
+18
View File
@@ -3,6 +3,7 @@ Dockerfile
.dockerignore
.git
.gitignore
docker/
# Python
__pycache__/
@@ -51,3 +52,20 @@ examples/
assets/
tests/
*.log
# Exclude directories not needed in Docker context
# Frontend build only needs frontend/
# Backend build only needs backend/
scripts/
logs/
docker/
skills/
frontend/.next
frontend/node_modules
backend/.venv
backend/htmlcov
backend/.coverage
*.md
!README.md
!frontend/README.md
!backend/README.md
+33 -120
View File
@@ -1,128 +1,41 @@
# Application Settings
# Set to True to enable debug-level logging (shows detailed LLM prompts and responses)
# Recommended for development and troubleshooting
DEBUG=True
APP_ENV=development
# TAVILY API Key
TAVILY_API_KEY=your-tavily-api-key
# Frontend API URL (used as Docker build arg for Next.js)
# This is a BUILD-TIME variable: it gets embedded into the frontend JS bundle during build.
# Default works for local development (localhost). For remote/LAN deployment, change to your host IP or domain:
# NEXT_PUBLIC_API_URL=http://192.168.1.100:8000/api
# NEXT_PUBLIC_API_URL=https://your-domain.com/api
# Note: When using docker-compose, only this root .env is used (not web/.env).
# If you change this value after building, you must rebuild: docker compose build
NEXT_PUBLIC_API_URL="http://localhost:8000/api"
# Jina API Key
JINA_API_KEY=your-jina-api-key
AGENT_RECURSION_LIMIT=30
# InfoQuest API Key
INFOQUEST_API_KEY=your-infoquest-api-key
# Authentication — JWT secret for session signing
# If not set, an ephemeral secret is auto-generated (sessions lost on restart)
# Generate with: python -c "import secrets; print(secrets.token_urlsafe(32))"
# AUTH_JWT_SECRET=your-secure-jwt-secret-here
# CORS settings
# Comma-separated list of allowed origins for CORS requests
# Example: ALLOWED_ORIGINS=http://localhost:3000,http://example.com
ALLOWED_ORIGINS=http://localhost:3000
# CORS Origins (comma-separated) - e.g., http://localhost:3000,http://localhost:3001
# CORS_ORIGINS=http://localhost:3000
# Enable or disable MCP server configuration, the default is false.
# Please enable this feature before securing your front-end and back-end in a managed environment.
# Otherwise, you system could be compromised.
ENABLE_MCP_SERVER_CONFIGURATION=false
# Optional:
# FIRECRAWL_API_KEY=your-firecrawl-api-key
# VOLCENGINE_API_KEY=your-volcengine-api-key
# OPENAI_API_KEY=your-openai-api-key
# GEMINI_API_KEY=your-gemini-api-key
# DEEPSEEK_API_KEY=your-deepseek-api-key
# NOVITA_API_KEY=your-novita-api-key # OpenAI-compatible, see https://novita.ai
# MINIMAX_API_KEY=your-minimax-api-key # OpenAI-compatible, see https://platform.minimax.io
# FEISHU_APP_ID=your-feishu-app-id
# FEISHU_APP_SECRET=your-feishu-app-secret
# Enable or disable PYTHON_REPL configuration, the default is false.
# Please enable this feature before securing your in a managed environment.
# Otherwise, you system could be compromised.
ENABLE_PYTHON_REPL=false
# SLACK_BOT_TOKEN=your-slack-bot-token
# SLACK_APP_TOKEN=your-slack-app-token
# TELEGRAM_BOT_TOKEN=your-telegram-bot-token
# Search Engine, Supported values: tavily, infoquest (recommended), duckduckgo, brave_search, arxiv, searx, serper
SEARCH_API=tavily
TAVILY_API_KEY=tvly-xxx
INFOQUEST_API_KEY="infoquest-xxx"
# SERPER_API_KEY=xxx # Required only if SEARCH_API is serper
# SEARX_HOST=xxx # Required only if SEARCH_API is searx.(compatible with both Searx and SearxNG)
# BRAVE_SEARCH_API_KEY=xxx # Required only if SEARCH_API is brave_search
# JINA_API_KEY=jina_xxx # Optional, default is None
# Optional, RAG provider
# RAG_PROVIDER=vikingdb_knowledge_base
# VIKINGDB_KNOWLEDGE_BASE_API_URL="api-knowledgebase.mlp.cn-beijing.volces.com"
# VIKINGDB_KNOWLEDGE_BASE_API_AK="AKxxx"
# VIKINGDB_KNOWLEDGE_BASE_API_SK=""
# VIKINGDB_KNOWLEDGE_BASE_RETRIEVAL_SIZE=15
# RAG_PROVIDER=ragflow
# RAGFLOW_API_URL="http://localhost:9388"
# RAGFLOW_API_KEY="ragflow-xxx"
# RAGFLOW_RETRIEVAL_SIZE=10
# RAGFLOW_CROSS_LANGUAGES=English,Chinese,Spanish,French,German,Japanese,Korean # Optional. To use RAGFlow's cross-language search, please separate each language with a single comma
# RAG_PROVIDER=dify
# DIFY_API_URL="https://api.dify.ai/v1"
# DIFY_API_KEY="dataset-xxx"
# MOI is a hybrid database that mainly serves enterprise users (https://www.matrixorigin.io/matrixone-intelligence)
# RAG_PROVIDER=moi
# MOI_API_URL="https://cluster.matrixonecloud.cn"
# MOI_API_KEY="xxx-xxx-xxx-xxx"
# MOI_RETRIEVAL_SIZE=10
# MOI_LIST_LIMIT=10
# RAG_PROVIDER: milvus (using free milvus instance on zilliz cloud: https://docs.zilliz.com/docs/quick-start )
# RAG_PROVIDER=milvus
# MILVUS_URI=<endpoint_of_self_hosted_milvus_or_zilliz_cloud>
# MILVUS_USER=<username_of_self_hosted_milvus_or_zilliz_cloud>
# MILVUS_PASSWORD=<password_of_self_hosted_milvus_or_zilliz_cloud>
# MILVUS_COLLECTION=documents
# MILVUS_EMBEDDING_PROVIDER=openai # support openai,dashscope
# MILVUS_EMBEDDING_BASE_URL=
# MILVUS_EMBEDDING_MODEL=
# MILVUS_EMBEDDING_API_KEY=
# MILVUS_AUTO_LOAD_EXAMPLES=true
# RAG_PROVIDER: milvus (using milvus lite on Mac or Linux)
# RAG_PROVIDER=milvus
# MILVUS_URI=./milvus_demo.db
# MILVUS_COLLECTION=documents
# MILVUS_EMBEDDING_PROVIDER=openai # support openai,dashscope
# MILVUS_EMBEDDING_BASE_URL=
# MILVUS_EMBEDDING_MODEL=
# MILVUS_EMBEDDING_API_KEY=
# MILVUS_AUTO_LOAD_EXAMPLES=true
# RAG_PROVIDER: qdrant (using qdrant cloud or self-hosted: https://qdrant.tech/documentation/quick-start/)
# RAG_PROVIDER=qdrant
# QDRANT_LOCATION=https://xyz-example.eu-central.aws.cloud.qdrant.io:6333
# QDRANT_API_KEY=<your_qdrant_api_key> # Optional, only for cloud/authenticated instances
# QDRANT_COLLECTION=documents
# QDRANT_EMBEDDING_PROVIDER=openai # support openai,dashscope
# QDRANT_EMBEDDING_BASE_URL=
# QDRANT_EMBEDDING_MODEL=text-embedding-ada-002
# QDRANT_EMBEDDING_API_KEY=
# QDRANT_AUTO_LOAD_EXAMPLES=true
# Optional, volcengine TTS for generating podcast
VOLCENGINE_TTS_APPID=xxx
VOLCENGINE_TTS_ACCESS_TOKEN=xxx
# VOLCENGINE_TTS_CLUSTER=volcano_tts # Optional, default is volcano_tts
# VOLCENGINE_TTS_VOICE_TYPE=BV700_V2_streaming # Optional, default is BV700_V2_streaming
# Optional, for langsmith tracing and monitoring
# Highly recommended for production debugging and performance monitoring
# Get your API key from https://smith.langchain.com/
# Enable LangSmith to monitor and debug your LLM calls, agent runs, and tool executions.
# LANGSMITH_TRACING=true
# LANGSMITH_ENDPOINT="https://api.smith.langchain.com"
# LANGSMITH_API_KEY="xxx"
# LANGSMITH_PROJECT="xxx"
# LANGSMITH_ENDPOINT=https://api.smith.langchain.com
# LANGSMITH_API_KEY=your-langsmith-api-key
# LANGSMITH_PROJECT=your-langsmith-project
# Optional, LangChain verbose logging
# Enable these to see detailed LLM interactions in console/logs
# Useful for debugging but can be very verbose
# LANGCHAIN_VERBOSE=true
# LANGCHAIN_DEBUG=true
# [!NOTE]
# For model settings and other configurations, please refer to `docs/configuration_guide.md`
# Option, for langgraph mongodb checkpointer
# Enable LangGraph checkpoint saver, supports MongoDB, Postgres
#LANGGRAPH_CHECKPOINT_SAVER=true
# Set the database URL for saving checkpoints
#LANGGRAPH_CHECKPOINT_DB_URL=mongodb://localhost:27017/
#LANGGRAPH_CHECKPOINT_DB_URL=postgresql://localhost:5432/postgres
# GitHub API Token
# GITHUB_TOKEN=your-github-token
# WECOM_BOT_ID=your-wecom-bot-id
# WECOM_BOT_SECRET=your-wecom-bot-secret
+43
View File
@@ -0,0 +1,43 @@
# Normalize line endings to LF for all text files
* text=auto eol=lf
# Shell scripts and makefiles must always use LF
*.sh text eol=lf
Makefile text eol=lf
**/Makefile text eol=lf
# Common config/source files
*.yml text eol=lf
*.yaml text eol=lf
*.toml text eol=lf
*.json text eol=lf
*.md text eol=lf
*.py text eol=lf
*.ts text eol=lf
*.tsx text eol=lf
*.js text eol=lf
*.jsx text eol=lf
*.css text eol=lf
*.scss text eol=lf
*.html text eol=lf
*.env text eol=lf
# Windows scripts
*.bat text eol=crlf
*.cmd text eol=crlf
# Binary assets
*.png binary
*.jpg binary
*.jpeg binary
*.gif binary
*.webp binary
*.ico binary
*.pdf binary
*.zip binary
*.tar binary
*.gz binary
*.mp4 binary
*.mov binary
*.woff binary
*.woff2 binary
@@ -0,0 +1,128 @@
name: Runtime Information
description: Report runtime/environment details to help reproduce an issue.
title: "[runtime] "
labels:
- needs-triage
body:
- type: markdown
attributes:
value: |
Thanks for sharing runtime details.
Complete this form so maintainers can quickly reproduce and diagnose the problem.
- type: input
id: summary
attributes:
label: Problem summary
description: Short summary of the issue.
placeholder: e.g. make dev fails to start gateway service
validations:
required: true
- type: textarea
id: expected
attributes:
label: Expected behavior
placeholder: What did you expect to happen?
validations:
required: true
- type: textarea
id: actual
attributes:
label: Actual behavior
placeholder: What happened instead? Include key error lines.
validations:
required: true
- type: dropdown
id: os
attributes:
label: Operating system
options:
- macOS
- Linux
- Windows
- Other
validations:
required: true
- type: input
id: platform_details
attributes:
label: Platform details
description: Add architecture and shell if relevant.
placeholder: e.g. arm64, zsh
- type: input
id: python_version
attributes:
label: Python version
placeholder: e.g. Python 3.12.9
- type: input
id: node_version
attributes:
label: Node.js version
placeholder: e.g. v23.11.0
- type: input
id: pnpm_version
attributes:
label: pnpm version
placeholder: e.g. 10.26.2
- type: input
id: uv_version
attributes:
label: uv version
placeholder: e.g. 0.7.20
- type: dropdown
id: run_mode
attributes:
label: How are you running DeerFlow?
options:
- Local (make dev)
- Docker (make docker-dev)
- CI
- Other
validations:
required: true
- type: textarea
id: reproduce
attributes:
label: Reproduction steps
description: Provide exact commands and sequence.
placeholder: |
1. make check
2. make install
3. make dev
4. ...
validations:
required: true
- type: textarea
id: logs
attributes:
label: Relevant logs
description: Paste key lines from logs (for example logs/gateway.log, logs/frontend.log).
render: shell
validations:
required: true
- type: textarea
id: git_info
attributes:
label: Git state
description: Share output of git branch and latest commit SHA.
placeholder: |
branch: feature/my-branch
commit: abcdef1
- type: textarea
id: additional
attributes:
label: Additional context
description: Add anything else that might help triage.
+179 -269
View File
@@ -1,303 +1,213 @@
# GitHub Copilot Instructions for DeerFlow
# Copilot Onboarding Instructions for DeerFlow
This file provides guidance to GitHub Copilot when working with the DeerFlow repository.
Use this file as the default operating guide for this repository. Follow it first, and only search the codebase when this file is incomplete or incorrect.
## Project Overview
## 1) Repository Summary
**DeerFlow** (Deep Exploration and Efficient Research Flow) is a community-driven Deep Research framework built on LangGraph. It orchestrates AI agents to conduct deep research, generate reports, and create content like podcasts and presentations.
DeerFlow is a full-stack "super agent harness".
### Technology Stack
- Backend: Python 3.12, LangGraph + FastAPI gateway, sandbox/tool system, memory, MCP integration.
- Frontend: Next.js 16 + React 19 + TypeScript + pnpm.
- Local dev entrypoint: root `Makefile` starts backend + frontend + nginx on `http://localhost:2026`.
- Docker dev entrypoint: `make docker-*` (mode-aware provisioner startup from `config.yaml`).
- **Backend**: Python 3.12+, FastAPI, LangGraph, LangChain
- **Frontend**: Next.js (React), TypeScript, pnpm
- **Package Management**: uv (Python), pnpm (Node.js)
- **Testing**: pytest (Python), Jest (JavaScript)
- **Linting/Formatting**: Ruff (Python), ESLint/Prettier (JavaScript)
Current repo footprint is medium-large (backend service, frontend app, docker stack, skills library, docs).
## Architecture Overview
## 2) Runtime and Toolchain Requirements
### Core Components
Validated in this repo on macOS:
1. **Multi-Agent System**: Built on LangGraph with state-based workflows
- **Coordinator**: Entry point managing workflow lifecycle
- **Planner**: Decomposes research objectives into structured plans
- **Research Team**: Specialized agents (Researcher, Coder) executing plans
- **Reporter**: Aggregates findings and generates final reports
- **Human-in-the-loop**: Interactive plan modification and approval
- Node.js `>=22` (validated with Node `23.11.0`)
- pnpm (repo expects lockfile generated by pnpm 10; validated with pnpm `10.26.2` and `10.15.0`)
- Python `>=3.12` (CI uses `3.12`)
- `uv` (validated with `0.7.20`)
- `nginx` (required for `make dev` unified local endpoint)
2. **State Management**
- Uses LangGraph StateGraph for agent communication
- MemorySaver for conversation persistence
- Checkpointing with MongoDB/PostgreSQL support
Always run from repo root unless a command explicitly says otherwise.
3. **External Integrations**
- Search engines: Tavily, Brave Search, DuckDuckGo
- Web crawling: Jina for content extraction
- TTS: Volcengine TTS API
- RAG: RAGFlow and VikingDB support
- MCP: Model Context Protocol integration
## 3) Build/Test/Lint/Run - Verified Command Sequences
### Directory Structure
These were executed and validated in this repository.
```
src/
├── agents/ # Agent definitions and behaviors
├── config/ # Configuration management (YAML, env vars)
├── crawler/ # Web crawling and content extraction
├── graph/ # LangGraph workflow definitions
├── llms/ # LLM provider integrations (OpenAI, DeepSeek, etc.)
├── prompts/ # Agent prompt templates
├── server/ # FastAPI web server and endpoints
├── tools/ # External tools (search, TTS, Python REPL)
└── rag/ # RAG integration for private knowledgebases
### A. Bootstrap and install
web/ # Next.js web UI (React, TypeScript)
├── src/app/ # Next.js pages and API routes
├── src/components/ # UI components and design system
└── src/core/ # Frontend utilities and state management
1. Check prerequisites:
tests/ # Test suite
├── unit/ # Unit tests
└── integration/ # Integration tests
```
## Development Workflow
### Environment Setup
1. **Python Environment**:
```bash
# Use uv for dependency management
uv sync
# For development dependencies
uv pip install -e ".[dev]"
uv pip install -e ".[test]"
```
2. **Configuration Files**:
```bash
# Copy and configure environment files
cp .env.example .env
cp conf.yaml.example conf.yaml
```
3. **Frontend Setup**:
```bash
cd web/
pnpm install
```
### Running the Application
- **Backend Development Server**: `uv run server.py --reload`
- **Console UI**: `uv run main.py`
- **Frontend Development**: `cd web && pnpm dev`
- **Full Stack**: `./bootstrap.sh -d` (macOS/Linux) or `bootstrap.bat -d` (Windows)
- **LangGraph Studio**: `make langgraph-dev`
### Testing
- **Python Tests**: `make test` or `pytest tests/`
- **Python Coverage**: `make coverage`
- **Frontend Tests**: `cd web && pnpm test:run`
- **Frontend Lint**: `make lint-frontend`
### Code Quality
- **Python Formatting**: `make format` (uses Ruff)
- **Python Linting**: `make lint` (uses Ruff)
- **Frontend Linting**: `cd web && pnpm lint`
- **Frontend Type Check**: `cd web && pnpm typecheck`
## Coding Standards
### Python Code
1. **Style Guidelines**:
- Follow PEP 8 guidelines
- Use type hints wherever possible
- Line length: 88 characters (Ruff default)
- Python version requirement: >= 3.12
2. **Code Organization**:
- Write clear, documented code with descriptive docstrings
- Keep functions and methods focused and single-purpose
- Comment complex logic
- Use meaningful variable and function names
3. **Testing Requirements**:
- Add tests for new features in `tests/` directory
- Maintain test coverage (minimum 25%)
- Use pytest fixtures for test setup
- Test both unit and integration scenarios
4. **LangGraph Patterns**:
- Agents communicate via LangGraph state
- Each agent has specific tool permissions
- Use persistent checkpoints for conversation history
- Follow the node → edge → state pattern
### TypeScript/JavaScript Code
1. **Style Guidelines**:
- Use TypeScript for type safety
- Follow ESLint configuration
- Use Prettier for consistent formatting
- Prefer functional components with hooks
2. **Component Structure**:
- Place UI components in `web/src/components/`
- Use the established design system
- Keep components focused and reusable
- Export types alongside components
3. **API Integration**:
- API utilities in `web/src/core/api/`
- Handle errors gracefully
- Use proper TypeScript types for API responses
## Configuration Management
### Environment Variables (.env)
Key environment variables to configure:
- `TAVILY_API_KEY`: Web search integration
- `BRAVE_SEARCH_API_KEY`: Alternative search engine
- `LANGSMITH_API_KEY`: LangSmith tracing (optional)
- `LANGGRAPH_CHECKPOINT_DB_URL`: MongoDB/PostgreSQL for persistence
- `RAGFLOW_API_URL`: RAG integration
### Application Configuration (conf.yaml)
- LLM model configurations
- Provider-specific settings
- Search engine preferences
- MCP server configurations
## Common Development Tasks
### Adding New Features
1. **New Agent**:
- Add agent definition in `src/agents/`
- Update graph in `src/graph/builder.py`
- Register agent tools in prompts
2. **New Tool**:
- Implement tool in `src/tools/`
- Register in agent prompts
- Add tests for tool functionality
3. **New Workflow**:
- Create graph builder in `src/[feature]/graph/builder.py`
- Define state management
- Add nodes and edges
4. **Frontend Component**:
- Add component to `web/src/components/`
- Update API in `web/src/core/api/`
- Add corresponding types
### Debugging
- **LangGraph Studio**: `make langgraph-dev` for visual workflow debugging
- **LangSmith**: Configure `LANGSMITH_API_KEY` for tracing
- **Server Logs**: Check FastAPI server output for backend issues
- **Browser DevTools**: Use for frontend debugging
## Important Patterns
### Agent Communication
- Agents communicate through LangGraph state
- State is preserved across checkpoints
- Use proper type annotations for state
### Content Generation Pipeline
1. Planning: Planner creates research plan
2. Research: Researcher gathers information
3. Processing: Coder analyzes data/code
4. Reporting: Reporter synthesizes findings
5. Post-processing: Optional podcast/PPT generation
### Error Handling
- Use try-except blocks with specific exception types
- Log errors with appropriate context
- Provide meaningful error messages to users
- Handle API failures gracefully
### Async Operations
- Use async/await for I/O operations
- Properly handle concurrent operations
- Use appropriate timeout values
- Clean up resources in finally blocks
## Pre-commit Hooks
The repository uses pre-commit hooks for code quality:
```bash
chmod +x pre-commit
ln -s ../../pre-commit .git/hooks/pre-commit
make check
```
## Dependencies
Observed: passes when required tools are installed.
### Adding New Dependencies
2. Install dependencies (recommended order: backend then frontend, as implemented by `make install`):
- **Python**: Add to `pyproject.toml` dependencies, then run `uv sync`
- **JavaScript**: Use `pnpm add <package>` in the `web/` directory
```bash
make install
```
### Dependency Updates
### B. Backend CI-equivalent validation
- Keep dependencies up to date
- Test thoroughly after updates
- Check compatibility with Python 3.12+ and Node.js 22+
Run from `backend/`:
## Documentation
```bash
make lint
make test
```
### When to Update Documentation
Validated results:
- New features: Update relevant docs in `docs/` directory
- API changes: Update `docs/API.md`
- Configuration changes: Update `docs/configuration_guide.md`
- Breaking changes: Clearly document in README and CONTRIBUTING
- `make lint`: pass (`ruff check .`)
- `make test`: pass (`277 passed, 15 warnings in ~76.6s`)
### Documentation Style
CI parity:
- Use clear, concise language
- Include code examples where applicable
- Keep documentation in sync with code
- Use markdown formatting consistently
- `.github/workflows/backend-unit-tests.yml` runs on pull requests.
- CI executes `uv sync --group dev`, then `make lint`, then `make test` in `backend/`.
## Security Considerations
### C. Frontend validation
- Never commit API keys or secrets to the repository
- Use `.env` files for sensitive configuration
- Validate and sanitize user inputs
- Follow security best practices for web applications
- Be cautious with code execution features
Run from `frontend/`.
## Community Guidelines
Recommended reliable sequence:
- Be respectful and inclusive
- Follow the MIT License terms
- Give constructive feedback in code reviews
- Help others learn and grow
- Stay focused on improving the project
```bash
pnpm lint
pnpm typecheck
BETTER_AUTH_SECRET=local-dev-secret pnpm build
```
## Getting Help
Observed failure modes and workarounds:
- Check existing documentation in `docs/`
- Review `Agent.md` for architecture details
- See `CONTRIBUTING` for contribution guidelines
- Check GitHub issues for known problems
- Join community discussions for support
- `pnpm build` fails without `BETTER_AUTH_SECRET` in production-mode env validation.
- Workaround: set `BETTER_AUTH_SECRET` (best) or set `SKIP_ENV_VALIDATION=1`.
- Even with `SKIP_ENV_VALIDATION=1`, Better Auth can still warn/error in logs about default secret; prefer setting a real non-default secret.
- `pnpm check` currently fails (`next lint` invocation is incompatible here and resolves to an invalid directory). Do not rely on `pnpm check`; run `pnpm lint` and `pnpm typecheck` explicitly.
## Additional Resources
### D. Run locally (all services)
- Main README: Comprehensive project overview
- Agent.md: Detailed architecture and agent guidance
- CONTRIBUTING: Full contribution guidelines
- docs/configuration_guide.md: Configuration details
- docs/API.md: API documentation
- docs/mcp_integrations.md: MCP integration guide
From root:
```bash
make dev
```
Behavior:
- Stops existing local services first.
- Starts LangGraph (`2024`), Gateway (`8001`), Frontend (`3000`), nginx (`2026`).
- Unified app endpoint: `http://localhost:2026`.
- Logs: `logs/langgraph.log`, `logs/gateway.log`, `logs/frontend.log`, `logs/nginx.log`.
Stop services:
```bash
make stop
```
If tool sessions/timeouts interrupt `make dev`, run `make stop` again to ensure cleanup.
### E. Config bootstrap
From root:
```bash
make config
```
Important behavior:
- This intentionally aborts if `config.yaml` (or `config.yml`/`configure.yml`) already exists.
- Use `make config` only for first-time setup in a clean clone.
## 4) Command Order That Minimizes Failures
Use this exact order for local code changes:
1. `make check`
2. `make install` (if frontend fails with proxy errors, rerun frontend install with proxy vars unset)
3. Backend checks: `cd backend && make lint && make test`
4. Frontend checks: `cd frontend && pnpm lint && pnpm typecheck`
5. Frontend build (if UI changes or release-sensitive changes): `BETTER_AUTH_SECRET=... pnpm build`
Always run backend lint/tests before opening PRs because that is what CI enforces.
## 5) Project Layout and Architecture (High-Value Paths)
Root-level orchestration and config:
- `Makefile` - main local/dev/docker command entrypoints
- `config.example.yaml` - primary app config template
- `config.yaml` - local active config (gitignored)
- `docker/docker-compose-dev.yaml` - Docker dev topology
- `.github/workflows/backend-unit-tests.yml` - PR validation workflow
Backend core:
- `backend/packages/harness/deerflow/agents/` - lead agent, middleware chain, memory
- `backend/app/gateway/` - FastAPI gateway API
- `backend/packages/harness/deerflow/sandbox/` - sandbox provider + tool wrappers
- `backend/packages/harness/deerflow/subagents/` - subagent registry/execution
- `backend/packages/harness/deerflow/mcp/` - MCP integration
- `backend/langgraph.json` - graph entrypoint (`deerflow.agents:make_lead_agent`)
- `backend/pyproject.toml` - Python deps and `requires-python`
- `backend/ruff.toml` - lint/format policy
- `backend/tests/` - backend unit and integration-like tests
Frontend core:
- `frontend/src/app/` - Next.js routes/pages
- `frontend/src/components/` - UI components
- `frontend/src/core/` - app logic (threads, tools, API, models)
- `frontend/src/env.js` - env schema/validation (critical for build behavior)
- `frontend/package.json` - scripts/deps
- `frontend/eslint.config.js` - lint rules
- `frontend/tsconfig.json` - TS config
Skills and assets:
- `skills/public/` - built-in skill packs loaded by agent runtime
## 6) Pre-Checkin / Validation Expectations
Before submitting changes, run at minimum:
- Backend: `cd backend && make lint && make test`
- Frontend (if touched): `cd frontend && pnpm lint && pnpm typecheck`
- Frontend build when changing env/auth/routing/build-sensitive files: `BETTER_AUTH_SECRET=... pnpm build`
If touching orchestration/config (`Makefile`, `docker/*`, `config*.yaml`), also run `make dev` and verify the four services start.
## 7) Non-Obvious Dependencies and Gotchas
- Proxy env vars can silently break frontend network operations (`pnpm install`/registry access).
- `BETTER_AUTH_SECRET` is effectively required for reliable frontend production build validation.
- Next.js may warn about multiple lockfiles and workspace root inference; this is currently a warning, not a build blocker.
- `make config` is non-idempotent by design when config already exists.
- `make dev` includes process cleanup and can emit shutdown logs/noise if interrupted; this is expected.
## 8) Root Inventory (quick reference)
Important root entries:
- `.github/`
- `backend/`
- `frontend/`
- `docker/`
- `skills/`
- `scripts/`
- `docs/`
- `README.md`
- `CONTRIBUTING.md`
- `Makefile`
- `config.example.yaml`
- `extensions_config.example.json`
## 9) Instruction Priority
Trust this onboarding guide first.
Only do broad repo searches (`grep/find/code search`) when:
- you need file-level implementation details not listed here,
- a command here fails and you need updated replacement behavior,
- or CI/workflow definitions have changed since this file was written.
+40
View File
@@ -0,0 +1,40 @@
name: Unit Tests
on:
push:
branches: [ 'main' ]
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
concurrency:
group: unit-tests-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
backend-unit-tests:
if: github.event.pull_request.draft == false
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@v6
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.12'
- name: Install uv
uses: astral-sh/setup-uv@v7
- name: Install backend dependencies
working-directory: backend
run: uv sync --group dev
- name: Run unit tests of backend
working-directory: backend
run: make test
-95
View File
@@ -1,95 +0,0 @@
name: Publish Containers
on:
push:
branches:
- main-1.x
release:
types: [published]
workflow_dispatch:
jobs:
backend-container:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
attestations: write
id-token: write
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Log in to the Container registry
uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 #v3.4.0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@902fa8ec7d6ecbf8d84d538b9b233a880e428804 #v5.7.0
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
- name: Build and push Docker image
id: push
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 #v6.18.0
with:
context: .
file: Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
- name: Generate artifact attestation
uses: actions/attest-build-provenance@v2
with:
subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME}}
subject-digest: ${{ steps.push.outputs.digest }}
push-to-registry: true
frontend-container:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
attestations: write
id-token: write
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}-web
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Log in to the Container registry
uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 #v3.4.0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@902fa8ec7d6ecbf8d84d538b9b233a880e428804 #v5.7.0
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
- name: Build and push Docker image
id: push
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 #v6.18.0
with:
context: web
file: web/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
- name: Generate artifact attestation
uses: actions/attest-build-provenance@v2
with:
subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME}}
subject-digest: ${{ steps.push.outputs.digest }}
push-to-registry: true
+74
View File
@@ -0,0 +1,74 @@
name: Lint Check
on:
push:
branches: [ 'main' ]
pull_request:
branches: [ '*' ]
permissions:
contents: read
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.12'
- name: Install uv
uses: astral-sh/setup-uv@v7
- name: Install dependencies
working-directory: backend
run: |
uv sync --group dev
- name: Lint backend
working-directory: backend
run: make lint
lint-frontend:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '22'
- name: Enable Corepack
run: corepack enable
- name: Use pinned pnpm version
run: corepack prepare pnpm@10.26.2 --activate
- name: Install frontend dependencies
run: |
cd frontend
pnpm install --frozen-lockfile
- name: Check frontend formatting
run: |
cd frontend
pnpm format
- name: Run frontend linting
run: |
cd frontend
pnpm lint
- name: Check TypeScript types
run: |
cd frontend
pnpm typecheck
- name: Build frontend
run: |
cd frontend
BETTER_AUTH_SECRET=local-dev-secret pnpm build
-69
View File
@@ -1,69 +0,0 @@
name: Lint Check
on:
push:
branches: [ 'main' ]
pull_request:
branches: [ '*' ]
permissions:
contents: read
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install the latest version of uv
uses: astral-sh/setup-uv@d9e0f98d3fc6adb07d1e3d37f3043649ddad06a1 #v6.5.0
with:
version: "latest"
- name: Install dependencies
run: |
uv venv --python 3.12
uv pip install -e ".[dev]"
- name: Run linters
run: |
source .venv/bin/activate
make lint
lint-frontend:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '22'
- name: Install pnpm
run: npm install -g pnpm
- name: Install frontend dependencies
run: |
cd web
pnpm install --frozen-lockfile
- name: Run frontend linting
run: |
cd web
pnpm lint
- name: Check TypeScript types
run: |
cd web
pnpm typecheck
- name: Running the frontend tests
run: |
cd web
pnpm test:run
- name: Build frontend
run: |
cd web
pnpm build
-73
View File
@@ -1,73 +0,0 @@
name: Test Cases Check
on:
push:
branches: [ 'main' ]
pull_request:
branches: [ '*' ]
permissions:
contents: read
jobs:
test:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:15
env:
POSTGRES_DB: checkpointing_db
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
ports: ["5432:5432"]
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
mongodb:
image: mongo:6
env:
MONGO_INITDB_ROOT_USERNAME: admin
MONGO_INITDB_ROOT_PASSWORD: admin
MONGO_INITDB_DATABASE: checkpointing_db
ports: ["27017:27017"]
options: >-
--health-cmd "mongosh --eval 'db.runCommand(\"ping\").ok'"
--health-interval 10s
--health-timeout 5s
--health-retries 3
steps:
- uses: actions/checkout@v3
- name: Install the latest version of uv
uses: astral-sh/setup-uv@d9e0f98d3fc6adb07d1e3d37f3043649ddad06a1 #v6.5.0
with:
version: "latest"
- name: Install dependencies
run: |
uv venv --python 3.12
uv pip install -e ".[dev]"
uv pip install -e ".[test]"
- name: Run test cases with coverage
run: |
source .venv/bin/activate
TAVILY_API_KEY=mock-key DB_TESTS_ENABLED=true make coverage
- name: Generate HTML Coverage Report
run: |
source .venv/bin/activate
python -m coverage html -d coverage_html
- name: Upload Coverage Report
uses: actions/upload-artifact@v4
with:
name: coverage-report
path: coverage_html/
- name: Display Coverage Summary
run: |
source .venv/bin/activate
python -m coverage report
+42 -17
View File
@@ -1,14 +1,20 @@
# Python-generated files
# DeerFlow docker image cache
docker/.cache/
# oh-my-claudecode state
.omc/
# OS generated files
.DS_Store
*.local
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db
# Python cache
__pycache__/
*.py[oc]
build/
dist/
wheels/
*.egg-info
.coverage
.coverage.*
agent_history.gif
static/browser_history/*.gif
*.pyc
*.pyo
# Virtual environments
.venv
@@ -17,16 +23,35 @@ venv/
# Environment variables
.env
# user conf
conf.yaml
# Configuration files
config.yaml
mcp_config.json
extensions_config.json
# IDE
.idea/
.langgraph_api/
.DS_Store
.vscode/
# coverage report
# Coverage report
coverage.xml
coverage/
.deer-flow/
.claude/
skills/custom/*
logs/
log/
# Temporary PPT content files
ppt_content_*.md
# Local git hooks (keep only on this machine, do not push)
.githooks/
# pnpm
.pnpm-store
sandbox_image_cache.tar
# ignore the legacy `web` folder
web/
# Deployment artifacts
backend/Dockerfile.langgraph
config.yaml.bak
.gstack/
-121
View File
@@ -1,121 +0,0 @@
{
"version": "0.2.0",
"configurations": [
{
"name": "Debug Tests",
"type": "debugpy",
"request": "launch",
"module": "pytest",
"args": [
"${workspaceFolder}/tests",
"-v",
"-s"
],
"console": "integratedTerminal",
"justMyCode": false,
"env": {
"PYTHONPATH": "${workspaceFolder}"
}
},
{
"name": "Debug Current Test File",
"type": "debugpy",
"request": "launch",
"module": "pytest",
"args": [
"${file}",
"-v",
"-s"
],
"console": "integratedTerminal",
"justMyCode": false
},
{
"name": "Python: 当前文件",
"type": "debugpy",
"request": "launch",
"program": "${file}",
"console": "integratedTerminal",
"justMyCode": true
},
{
"name": "Python: main.py",
"type": "debugpy",
"request": "launch",
"program": "${workspaceFolder}/main.py",
"console": "integratedTerminal",
"justMyCode": false,
"env": {
"PYTHONPATH": "${workspaceFolder}"
},
"args": [
"--debug", "--max_plan_iterations", "1", "--max_step_num", "1"
]
},
{
"name": "Python: llm.py",
"type": "debugpy",
"request": "launch",
"program": "${workspaceFolder}/src/llms/llm.py",
"console": "integratedTerminal",
"justMyCode": true,
"env": {
"PYTHONPATH": "${workspaceFolder}"
}
},
{
"name": "Python: server.py",
"type": "debugpy",
"request": "launch",
"program": "${workspaceFolder}/server.py",
"console": "integratedTerminal",
"justMyCode": false,
"env": {
"PYTHONPATH": "${workspaceFolder}"
}
},
{
"name": "Python: graph.py",
"type": "debugpy",
"request": "launch",
"program": "${workspaceFolder}/src/ppt/graph/builder.py",
"console": "integratedTerminal",
"justMyCode": false,
"env": {
"PYTHONPATH": "${workspaceFolder}"
}
},
{
"name": "Debug: python server",
"type": "debugpy",
"request": "launch",
"program": "${workspaceFolder}/server.py",
"console": "integratedTerminal",
"justMyCode": false,
"env": {
"PYTHONPATH": "${workspaceFolder}"
},
"args": [
"--reload"
]
},
{
"name": "Debug: nodejs web",
"type": "node",
"request": "launch",
"runtimeExecutable": "pnpm",
"runtimeArgs": [
"dev"
],
"cwd": "${workspaceFolder}/web",
"console": "integratedTerminal"
},
],
"compounds": [
{
"name": "Launch Deerflow",
"configurations": ["Debug: python server", "Debug: nodejs web"]
}
]
}
-7
View File
@@ -1,7 +0,0 @@
{
"python.testing.pytestArgs": [
"tests"
],
"python.testing.unittestEnabled": false,
"python.testing.pytestEnabled": true
}
-186
View File
@@ -1,186 +0,0 @@
# Agent.md
This file provides guidance to AI agents when working with code in this repository.
## Architecture Overview
**DeerFlow** is a multi-agent research framework built on LangGraph that orchestrates AI agents to conduct deep research, generate reports, and create content like podcasts and presentations.
### Core Architecture
The system uses a **modular multi-agent architecture** with these key components:
- **Coordinator**: Entry point managing workflow lifecycle
- **Planner**: Decomposes research objectives into structured plans
- **Research Team**: Specialized agents (Researcher, Coder) executing plans
- **Reporter**: Aggregates findings and generates final reports
- **Human-in-the-loop**: Interactive plan modification and approval
### Graph Structure
Built on **LangGraph** with state-based workflows:
- **StateGraph** manages agent communication
- **MemorySaver** provides conversation persistence
- **Checkpointing** supports MongoDB/PostgreSQL storage
- **Nodes**: coordinator → planner → research_team → reporter
### Key Directories
```
src/
├── agents/ # Agent definitions and behaviors
├── config/ # Configuration management (YAML, env vars)
├── crawler/ # Web crawling and content extraction
├── graph/ # LangGraph workflow definitions
├── llms/ # LLM provider integrations (OpenAI, DeepSeek, etc.)
├── prompts/ # Agent prompt templates
├── server/ # FastAPI web server and endpoints
├── tools/ # External tools (search, TTS, Python REPL)
└── rag/ # RAG integration for private knowledgebases
web/ # Next.js web UI (React, TypeScript)
├── src/app/ # Next.js pages and API routes
├── src/components/ # UI components and design system
└── src/core/ # Frontend utilities and state management
```
## Development Commands
### Backend (Python)
```bash
# Install dependencies
uv sync
# Development server
uv run server.py --reload
# Console UI
uv run main.py
# Run tests
make test # Run all tests
make coverage # Run tests with coverage
pytest tests/unit/test_*.py # Run specific test file
# Code quality
make lint # Ruff linting
make format # Ruff formatting
# LangGraph Studio (debugging)
make langgraph-dev # Start LangGraph development server
```
### Frontend (Web UI)
```bash
cd web/
pnpm install # Install dependencies
pnpm dev # Development server (localhost:3000)
pnpm build # Production build
pnpm typecheck # Type checking
pnpm lint # ESLint
pnpm format:write # Prettier formatting
```
### Full Stack Development
```bash
# Run both backend and frontend
./bootstrap.sh -d # macOS/Linux
bootstrap.bat -d # Windows
```
### Docker
```bash
# Build and run
make build # Build Docker image
docker compose up # Run with Docker Compose
# Production deployment
docker build -t deer-flow-api .
docker run -p 8000:8000 deer-flow-api
```
### Fix GitHub issues
create a branch named `fix/<issue-number>` to address specific GitHub issues.
## Configuration
### Environment Setup
```bash
# Required: Copy example configs
cp .env.example .env
cp conf.yaml.example conf.yaml
# Key environment variables:
# TAVILY_API_KEY # Web search
# BRAVE_SEARCH_API_KEY # Alternative search
# LANGSMITH_API_KEY # LangSmith tracing (optional)
# LANGGRAPH_CHECKPOINT_DB_URL # MongoDB/PostgreSQL for persistence
```
### LangGraph Studio
```bash
# Local debugging with checkpointing
uvx --refresh --from "langgraph-cli[inmem]" --with-editable . --python 3.12 langgraph dev --allow-blocking
```
## Common Development Tasks
### Testing
```bash
# Unit tests
pytest tests/unit/
# Integration tests
pytest tests/integration/
# Specific component
pytest tests/unit/config/test_configuration.py
# With coverage
pytest --cov=src tests/ --cov-report=html
```
### Code Quality
```bash
# Format code
make format
# Check linting
make lint
# Type checking (frontend)
cd web && pnpm typecheck
```
### Adding New Features
1. **New Agent**: Add agent in `src/agents/` + update graph in `src/graph/builder.py`
2. **New Tool**: Add tool in `src/tools/` + register in agent prompts
3. **New Workflow**: Create graph builder in `src/[feature]/graph/builder.py`
4. **Frontend Component**: Add to `web/src/components/` + update API in `web/src/core/api/`
### Configuration Changes
- **LLM Models**: Update `conf.yaml` with new providers
- **Search Engines**: Modify `.env` SEARCH_API variable
- **RAG Integration**: Configure RAGFLOW_API_URL in `.env`
- **MCP Servers**: Add MCP settings in configuration
## Architecture Patterns
### Agent Communication
- **Message Passing**: Agents communicate via LangGraph state
- **Tool Access**: Each agent has specific tool permissions
- **State Management**: Persistent checkpoints for conversation history
### Content Generation Pipeline
1. **Planning**: Planner creates research plan
2. **Research**: Researcher gathers information
3. **Processing**: Coder analyzes data/code
4. **Reporting**: Reporter synthesizes findings
5. **Post-processing**: Optional podcast/PPT generation
### External Integrations
- **Search**: Tavily, Brave Search, DuckDuckGo
- **Crawling**: Jina for web content extraction
- **TTS**: Volcengine TTS API
- **RAG**: RAGFlow and VikingDB support
- **MCP**: Model Context Protocol integration
-144
View File
@@ -1,144 +0,0 @@
# Contributing to DeerFlow
Thank you for your interest in contributing to DeerFlow! We welcome contributions of all kinds from the community.
## Ways to Contribute
There are many ways you can contribute to DeerFlow:
- **Code Contributions**: Add new features, fix bugs, or improve performance
- **Documentation**: Improve README, add code comments, or create examples
- **Bug Reports**: Submit detailed bug reports through issues
- **Feature Requests**: Suggest new features or improvements
- **Code Reviews**: Review pull requests from other contributors
- **Community Support**: Help others in discussions and issues
## Development Setup
1. Fork the repository
2. Clone your fork:
```bash
git clone https://github.com/bytedance/deer-flow.git
cd deer-flow
```
3. Set up your development environment:
```bash
# Install dependencies, uv will take care of the python interpreter and venv creation
uv sync
# For development, install additional dependencies
uv pip install -e ".[dev]"
uv pip install -e ".[test]"
```
4. Configure pre-commit hooks:
```bash
chmod +x pre-commit
ln -s ../../pre-commit .git/hooks/pre-commit
```
## Development Process
1. Create a new branch:
```bash
git checkout -b feature/amazing-feature
```
2. Make your changes following our coding standards:
- Write clear, documented code
- Follow PEP 8 style guidelines
- Add tests for new features
- Update documentation as needed
3. Run tests and checks:
```bash
make test # Run tests
make lint # Run linting
make format # Format code
make coverage # Check test coverage
```
4. Commit your changes:
```bash
git commit -m 'Add some amazing feature'
```
5. Push to your fork:
```bash
git push origin feature/amazing-feature
```
6. Open a Pull Request
## Pull Request Guidelines
- Fill in the pull request template completely
- Include tests for new features
- Update documentation as needed
- Ensure all tests pass and there are no linting errors
- Keep pull requests focused on a single feature or fix
- Reference any related issues
## Code Style
- Follow PEP 8 guidelines
- Use type hints where possible
- Write descriptive docstrings
- Keep functions and methods focused and single-purpose
- Comment complex logic
- Python version requirement: >= 3.12
## Testing
Run the test suite:
```bash
# Run all tests
make test
# Run specific test file
pytest tests/integration/test_workflow.py
# Run with coverage
make coverage
```
## Code Quality
```bash
# Run linting
make lint
# Format code
make format
```
## Community Guidelines
- Be respectful and inclusive
- Follow our code of conduct
- Help others learn and grow
- Give constructive feedback
- Stay focused on improving the project
## Need Help?
If you need help with anything:
- Check existing issues and discussions
- Join our community channels
- Ask questions in discussions
## License
By contributing to DeerFlow, you agree that your contributions will be licensed under the MIT License.
We appreciate your contributions to making DeerFlow better!
+323
View File
@@ -0,0 +1,323 @@
# Contributing to DeerFlow
Thank you for your interest in contributing to DeerFlow! This guide will help you set up your development environment and understand our development workflow.
## Development Environment Setup
We offer two development environments. **Docker is recommended** for the most consistent and hassle-free experience.
### Option 1: Docker Development (Recommended)
Docker provides a consistent, isolated environment with all dependencies pre-configured. No need to install Node.js, Python, or nginx on your local machine.
#### Prerequisites
- Docker Desktop or Docker Engine
- pnpm (for caching optimization)
#### Setup Steps
1. **Configure the application**:
```bash
# Copy example configuration
cp config.example.yaml config.yaml
# Set your API keys
export OPENAI_API_KEY="your-key-here"
# or edit config.yaml directly
```
2. **Initialize Docker environment** (first time only):
```bash
make docker-init
```
This will:
- Build Docker images
- Install frontend dependencies (pnpm)
- Install backend dependencies (uv)
- Share pnpm cache with host for faster builds
3. **Start development services**:
```bash
make docker-start
```
`make docker-start` reads `config.yaml` and starts `provisioner` only for provisioner/Kubernetes sandbox mode.
All services will start with hot-reload enabled:
- Frontend changes are automatically reloaded
- Backend changes trigger automatic restart
- LangGraph server supports hot-reload
4. **Access the application**:
- Web Interface: http://localhost:2026
- API Gateway: http://localhost:2026/api/*
- LangGraph: http://localhost:2026/api/langgraph/*
#### Docker Commands
```bash
# Build the custom k3s image (with pre-cached sandbox image)
make docker-init
# Start Docker services (mode-aware, localhost:2026)
make docker-start
# Stop Docker development services
make docker-stop
# View Docker development logs
make docker-logs
# View Docker frontend logs
make docker-logs-frontend
# View Docker gateway logs
make docker-logs-gateway
```
If Docker builds are slow in your network, you can override the default package registries before running `make docker-init` or `make docker-start`:
```bash
export UV_INDEX_URL=https://pypi.org/simple
export NPM_REGISTRY=https://registry.npmjs.org
```
#### Linux: Docker daemon permission denied
If `make docker-init`, `make docker-start`, or `make docker-stop` fails on Linux with an error like below, your current user likely does not have permission to access the Docker daemon socket:
```text
unable to get image 'deer-flow-dev-langgraph': permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock
```
Recommended fix: add your current user to the `docker` group so Docker commands work without `sudo`.
1. Confirm the `docker` group exists:
```bash
getent group docker
```
2. Add your current user to the `docker` group:
```bash
sudo usermod -aG docker $USER
```
3. Apply the new group membership. The most reliable option is to log out completely and then log back in. If you want to refresh the current shell session instead, run:
```bash
newgrp docker
```
4. Verify Docker access:
```bash
docker ps
```
5. Retry the DeerFlow command:
```bash
make docker-stop
make docker-start
```
If `docker ps` still reports a permission error after `usermod`, fully log out and log back in before retrying.
#### Docker Architecture
```
Host Machine
Docker Compose (deer-flow-dev)
├→ nginx (port 2026) ← Reverse proxy
├→ web (port 3000) ← Frontend with hot-reload
├→ api (port 8001) ← Gateway API with hot-reload
├→ langgraph (port 2024) ← LangGraph server with hot-reload
└→ provisioner (optional, port 8002) ← Started only in provisioner/K8s sandbox mode
```
**Benefits of Docker Development**:
- ✅ Consistent environment across different machines
- ✅ No need to install Node.js, Python, or nginx locally
- ✅ Isolated dependencies and services
- ✅ Easy cleanup and reset
- ✅ Hot-reload for all services
- ✅ Production-like environment
### Option 2: Local Development
If you prefer to run services directly on your machine:
#### Prerequisites
Check that you have all required tools installed:
```bash
make check
```
Required tools:
- Node.js 22+
- pnpm
- uv (Python package manager)
- nginx
#### Setup Steps
1. **Configure the application** (same as Docker setup above)
2. **Install dependencies**:
```bash
make install
```
3. **Run development server** (starts all services with nginx):
```bash
make dev
```
4. **Access the application**:
- Web Interface: http://localhost:2026
- All API requests are automatically proxied through nginx
#### Manual Service Control
If you need to start services individually:
1. **Start backend services**:
```bash
# Terminal 1: Start LangGraph Server (port 2024)
cd backend
make dev
# Terminal 2: Start Gateway API (port 8001)
cd backend
make gateway
# Terminal 3: Start Frontend (port 3000)
cd frontend
pnpm dev
```
2. **Start nginx**:
```bash
make nginx
# or directly: nginx -c $(pwd)/docker/nginx/nginx.local.conf -g 'daemon off;'
```
3. **Access the application**:
- Web Interface: http://localhost:2026
#### Nginx Configuration
The nginx configuration provides:
- Unified entry point on port 2026
- Routes `/api/langgraph/*` to LangGraph Server (2024)
- Routes other `/api/*` endpoints to Gateway API (8001)
- Routes non-API requests to Frontend (3000)
- Centralized CORS handling
- SSE/streaming support for real-time agent responses
- Optimized timeouts for long-running operations
## Project Structure
```
deer-flow/
├── config.example.yaml # Configuration template
├── extensions_config.example.json # MCP and Skills configuration template
├── Makefile # Build and development commands
├── scripts/
│ └── docker.sh # Docker management script
├── docker/
│ ├── docker-compose-dev.yaml # Docker Compose configuration
│ └── nginx/
│ ├── nginx.conf # Nginx config for Docker
│ └── nginx.local.conf # Nginx config for local dev
├── backend/ # Backend application
│ ├── src/
│ │ ├── gateway/ # Gateway API (port 8001)
│ │ ├── agents/ # LangGraph agents (port 2024)
│ │ ├── mcp/ # Model Context Protocol integration
│ │ ├── skills/ # Skills system
│ │ └── sandbox/ # Sandbox execution
│ ├── docs/ # Backend documentation
│ └── Makefile # Backend commands
├── frontend/ # Frontend application
│ └── Makefile # Frontend commands
└── skills/ # Agent skills
├── public/ # Public skills
└── custom/ # Custom skills
```
## Architecture
```
Browser
Nginx (port 2026) ← Unified entry point
├→ Frontend (port 3000) ← / (non-API requests)
├→ Gateway API (port 8001) ← /api/models, /api/mcp, /api/skills, /api/threads/*/artifacts
└→ LangGraph Server (port 2024) ← /api/langgraph/* (agent interactions)
```
## Development Workflow
1. **Create a feature branch**:
```bash
git checkout -b feature/your-feature-name
```
2. **Make your changes** with hot-reload enabled
3. **Format and lint your code** (CI will reject unformatted code):
```bash
# Backend
cd backend
make format # ruff check --fix + ruff format
# Frontend
cd frontend
pnpm format:write # Prettier
```
4. **Test your changes** thoroughly
5. **Commit your changes**:
```bash
git add .
git commit -m "feat: description of your changes"
```
6. **Push and create a Pull Request**:
```bash
git push origin feature/your-feature-name
```
## Testing
```bash
# Backend tests
cd backend
uv run pytest
# Frontend checks
cd frontend
pnpm check
```
### PR Regression Checks
Every pull request runs the backend regression workflow at [.github/workflows/backend-unit-tests.yml](.github/workflows/backend-unit-tests.yml), including:
- `tests/test_provisioner_kubeconfig.py`
- `tests/test_docker_sandbox_mode_detection.py`
## Code Style
- **Backend (Python)**: We use `ruff` for linting and formatting. Run `make format` before committing.
- **Frontend (TypeScript)**: We use ESLint and Prettier. Run `pnpm format:write` before committing.
- CI enforces formatting — PRs with unformatted code will fail the lint check.
## Documentation
- [Configuration Guide](backend/docs/CONFIGURATION.md) - Setup and configuration
- [Architecture Overview](backend/CLAUDE.md) - Technical architecture
- [MCP Setup Guide](backend/docs/MCP_SERVER.md) - Model Context Protocol configuration
## Need Help?
- Check existing [Issues](https://github.com/bytedance/deer-flow/issues)
- Read the [Documentation](backend/docs/)
- Ask questions in [Discussions](https://github.com/bytedance/deer-flow/discussions)
## License
By contributing to DeerFlow, you agree that your contributions will be licensed under the [MIT License](./LICENSE).
-32
View File
@@ -1,32 +0,0 @@
FROM ghcr.io/astral-sh/uv:python3.12-bookworm
# Install uv.
COPY --from=ghcr.io/astral-sh/uv:latest /uv /bin/uv
# Install system dependencies including libpq
RUN apt-get update && apt-get install -y \
libpq-dev \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
# Pre-cache the application dependencies.
RUN --mount=type=cache,target=/root/.cache/uv \
--mount=type=bind,source=uv.lock,target=uv.lock \
--mount=type=bind,source=pyproject.toml,target=pyproject.toml \
uv sync --locked --no-install-project
# Copy the application into the container.
COPY . /app
# Install the application dependencies.
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --locked
EXPOSE 8000
# Run the application.
RUN useradd -m appuser
USER appuser
CMD ["uv", "run", "python", "server.py", "--host", "0.0.0.0", "--port", "8000"]
+87
View File
@@ -0,0 +1,87 @@
# DeerFlow Install
This file is for coding agents. If the DeerFlow repository is not already cloned and open, clone `https://github.com/bytedance/deer-flow.git` first, then continue from the repository root.
## Goal
Bootstrap a DeerFlow local development workspace on the user's machine with the least risky path available.
Default preference:
1. Docker development environment
2. Local development environment
Do not assume API keys or model credentials exist. Set up everything that can be prepared safely, then stop with a concise summary of what the user still needs to provide.
## Operating Rules
- Be idempotent. Re-running this document should not damage an existing setup.
- Prefer existing repo commands over ad hoc shell commands.
- Do not use `sudo` or install system packages without explicit user approval.
- Do not overwrite existing user config values unless the user asks.
- If a step fails, stop, explain the blocker, and provide the smallest next action.
- If multiple setup paths are possible, prefer Docker when Docker is already available.
## Success Criteria
Consider the setup successful when all of the following are true:
- The DeerFlow repository is cloned and the current working directory is the repo root.
- `config.yaml` exists.
- For Docker setup, `make docker-init` completed successfully and Docker prerequisites are prepared, but services are not assumed to be running yet.
- For local setup, `make check` passed or reported no missing prerequisites, and `make install` completed successfully.
- The user receives the exact next command to launch DeerFlow.
- The user also receives any missing model configuration or referenced environment variable names from `config.yaml`, without inspecting secret-bearing files for actual values.
## Steps
- If the current directory is not the DeerFlow repository root, clone `https://github.com/bytedance/deer-flow.git` if needed, then change into the repository root.
- Confirm the current directory is the DeerFlow repository root by checking that `Makefile`, `backend/`, `frontend/`, and `config.example.yaml` exist.
- Detect whether `config.yaml` already exists.
- If `config.yaml` does not exist, run `make config`.
- Detect whether Docker is available and the daemon is reachable with `docker info`.
- If Docker is available:
- Run `make docker-init`.
- Treat this as Docker prerequisite preparation only. Do not claim that app services, compose validation, or image builds have already succeeded.
- Do not start long-running services unless the user explicitly asks or this setup request clearly includes launch verification.
- Tell the user the recommended next command is `make docker-start`.
- If Docker is not available:
- Run `make check`.
- If `make check` reports missing system dependencies such as `node`, `pnpm`, `uv`, or `nginx`, stop and report the missing tools instead of attempting privileged installs.
- If prerequisites are satisfied, run `make install`.
- Tell the user the recommended next command is `make dev`.
- Inspect `config.yaml` only for missing model entries or referenced environment variable placeholders. Do not read `.env`, `frontend/.env`, or other secret-bearing files.
- If no model is configured, tell the user they must add at least one entry under `models` in `config.yaml`.
- If `config.yaml` references variables such as `$OPENAI_API_KEY`, tell the user which variable names still need real values, but do not verify them by opening secret-bearing files.
- If the repository already appears configured, avoid repeating expensive work unless it is necessary to verify the environment.
## Verification
Use the lightest verification that matches the chosen setup path.
For Docker setup:
- Confirm `make docker-init` completed successfully.
- Confirm `config.yaml` exists.
- State explicitly that Docker services were not started and `make docker-start` is still the first real launch step.
- Do not leave background services running unless the user asked for that.
For local setup:
- Confirm `make install` completed successfully.
- Confirm `config.yaml` exists.
- Do not leave background services running unless the user asked for that.
## Final Response Format
Return a short status report with:
1. Setup path used: Docker or local
2. Setup level reached: Docker prerequisites prepared or local dependencies installed
3. Files created or detected: for example `config.yaml`
4. Remaining user action: model config, env var values, auth files, or nothing
5. Exact next command to start DeerFlow
## EXECUTE NOW
Complete the steps above. When finished, stop after the setup boundary and report status instead of continuing into unrelated project work.
+2 -1
View File
@@ -1,6 +1,7 @@
MIT License
Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
Copyright (c) 2025-2026 DeerFlow Authors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
@@ -18,4 +19,4 @@ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
SOFTWARE.
-2
View File
@@ -1,2 +0,0 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
-2
View File
@@ -1,2 +0,0 @@
// Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
// SPDX-License-Identifier: MIT
+221 -31
View File
@@ -1,44 +1,234 @@
.PHONY: help lint format install-dev serve test coverage langgraph-dev lint-frontend add-license-all check-license-all
# DeerFlow - Unified Development Environment
help: ## Show this help message
@echo "Deer Flow - Available Make Targets:"
.PHONY: help config config-upgrade check install dev dev-pro dev-daemon dev-daemon-pro start start-pro start-daemon start-daemon-pro stop up up-pro down clean docker-init docker-start docker-start-pro docker-stop docker-logs docker-logs-frontend docker-logs-gateway
BASH ?= bash
# Detect OS for Windows compatibility
ifeq ($(OS),Windows_NT)
SHELL := cmd.exe
PYTHON ?= python
else
PYTHON ?= python3
endif
help:
@echo "DeerFlow Development Commands:"
@echo " make config - Generate local config files (aborts if config already exists)"
@echo " make config-upgrade - Merge new fields from config.example.yaml into config.yaml"
@echo " make check - Check if all required tools are installed"
@echo " make install - Install all dependencies (frontend + backend)"
@echo " make setup-sandbox - Pre-pull sandbox container image (recommended)"
@echo " make dev - Start all services in development mode (with hot-reloading)"
@echo " make dev-pro - Start in dev + Gateway mode (experimental, no LangGraph server)"
@echo " make dev-daemon - Start dev services in background (daemon mode)"
@echo " make dev-daemon-pro - Start dev daemon + Gateway mode (experimental)"
@echo " make start - Start all services in production mode (optimized, no hot-reloading)"
@echo " make start-pro - Start in prod + Gateway mode (experimental)"
@echo " make start-daemon - Start prod services in background (daemon mode)"
@echo " make start-daemon-pro - Start prod daemon + Gateway mode (experimental)"
@echo " make stop - Stop all running services"
@echo " make clean - Clean up processes and temporary files"
@echo ""
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf " \033[36m%-18s\033[0m %s\n", $$1, $$2}'
@echo "Docker Production Commands:"
@echo " make up - Build and start production Docker services (localhost:2026)"
@echo " make up-pro - Build and start production Docker in Gateway mode (experimental)"
@echo " make down - Stop and remove production Docker containers"
@echo ""
@echo "Usage: make <target>"
@echo "Docker Development Commands:"
@echo " make docker-init - Pull the sandbox image"
@echo " make docker-start - Start Docker services (mode-aware from config.yaml, localhost:2026)"
@echo " make docker-start-pro - Start Docker in Gateway mode (experimental, no LangGraph container)"
@echo " make docker-stop - Stop Docker development services"
@echo " make docker-logs - View Docker development logs"
@echo " make docker-logs-frontend - View Docker frontend logs"
@echo " make docker-logs-gateway - View Docker gateway logs"
install-dev: ## Install development dependencies which could be optional for normal usage
uv pip install -e ".[dev]" && uv pip install -e ".[test]"
config:
@$(PYTHON) ./scripts/configure.py
format: ## Format code using ruff
uv run ruff format --config pyproject.toml .
config-upgrade:
@./scripts/config-upgrade.sh
lint: ## Lint and fix code using ruff
uv run ruff check --fix --select I --config pyproject.toml .
# Check required tools
check:
@$(PYTHON) ./scripts/check.py
lint-frontend: ## Lint frontend code, run tests, and check build
cd web && pnpm install --frozen-lockfile
cd web && pnpm lint
cd web && pnpm typecheck
cd web && pnpm test:run
cd web && pnpm build
# Install all dependencies
install:
@echo "Installing backend dependencies..."
@cd backend && uv sync
@echo "Installing frontend dependencies..."
@cd frontend && pnpm install
@echo "✓ All dependencies installed"
@echo ""
@echo "=========================================="
@echo " Optional: Pre-pull Sandbox Image"
@echo "=========================================="
@echo ""
@echo "If you plan to use Docker/Container-based sandbox, you can pre-pull the image:"
@echo " make setup-sandbox"
@echo ""
serve: ## Start development server with reload
uv run server.py --reload
# Pre-pull sandbox Docker image (optional but recommended)
setup-sandbox:
@echo "=========================================="
@echo " Pre-pulling Sandbox Container Image"
@echo "=========================================="
@echo ""
@IMAGE=$$(grep -A 20 "# sandbox:" config.yaml 2>/dev/null | grep "image:" | awk '{print $$2}' | head -1); \
if [ -z "$$IMAGE" ]; then \
IMAGE="enterprise-public-cn-beijing.cr.volces.com/vefaas-public/all-in-one-sandbox:latest"; \
echo "Using default image: $$IMAGE"; \
else \
echo "Using configured image: $$IMAGE"; \
fi; \
echo ""; \
if command -v container >/dev/null 2>&1 && [ "$$(uname)" = "Darwin" ]; then \
echo "Detected Apple Container on macOS, pulling image..."; \
container pull "$$IMAGE" || echo "⚠ Apple Container pull failed, will try Docker"; \
fi; \
if command -v docker >/dev/null 2>&1; then \
echo "Pulling image using Docker..."; \
if docker pull "$$IMAGE"; then \
echo ""; \
echo "✓ Sandbox image pulled successfully"; \
else \
echo ""; \
echo "⚠ Failed to pull sandbox image (this is OK for local sandbox mode)"; \
fi; \
else \
echo "✗ Neither Docker nor Apple Container is available"; \
echo " Please install Docker: https://docs.docker.com/get-docker/"; \
exit 1; \
fi
test: ## Run tests with pytest, need to run after 'make install-dev' for first time
uv run pytest tests/
# Start all services in development mode (with hot-reloading)
dev:
@$(PYTHON) ./scripts/check.py
ifeq ($(OS),Windows_NT)
@call scripts\run-with-git-bash.cmd ./scripts/serve.sh --dev
else
@./scripts/serve.sh --dev
endif
langgraph-dev: ## Start langgraph development server
uvx --refresh --from "langgraph-cli[inmem]" --with-editable . --python 3.12 langgraph dev --allow-blocking
# Start all services in dev + Gateway mode (experimental: agent runtime embedded in Gateway)
dev-pro:
@$(PYTHON) ./scripts/check.py
ifeq ($(OS),Windows_NT)
@call scripts\run-with-git-bash.cmd ./scripts/serve.sh --dev --gateway
else
@./scripts/serve.sh --dev --gateway
endif
coverage: ## Run tests with coverage report
uv run pytest --cov=src tests/ --cov-report=term-missing --cov-report=xml
# Start all services in production mode (with optimizations)
start:
@$(PYTHON) ./scripts/check.py
ifeq ($(OS),Windows_NT)
@call scripts\run-with-git-bash.cmd ./scripts/serve.sh --prod
else
@./scripts/serve.sh --prod
endif
add-license-all: ## Add license headers to all Python and TypeScript files
@echo "Adding license headers to all source files..."
@uv run python scripts/license_header.py src/ tests/ server.py main.py web/src/ web/tests/ --verbose
# Start all services in prod + Gateway mode (experimental)
start-pro:
@$(PYTHON) ./scripts/check.py
ifeq ($(OS),Windows_NT)
@call scripts\run-with-git-bash.cmd ./scripts/serve.sh --prod --gateway
else
@./scripts/serve.sh --prod --gateway
endif
check-license-all: ## Check if all Python and TypeScript files have license headers
@echo "Checking license headers in all source files..."
@uv run python scripts/license_header.py src/ tests/ server.py main.py web/src/ web/tests/ --check
# Start all services in daemon mode (background)
dev-daemon:
@$(PYTHON) ./scripts/check.py
ifeq ($(OS),Windows_NT)
@call scripts\run-with-git-bash.cmd ./scripts/serve.sh --dev --daemon
else
@./scripts/serve.sh --dev --daemon
endif
# Start daemon + Gateway mode (experimental)
dev-daemon-pro:
@$(PYTHON) ./scripts/check.py
ifeq ($(OS),Windows_NT)
@call scripts\run-with-git-bash.cmd ./scripts/serve.sh --dev --gateway --daemon
else
@./scripts/serve.sh --dev --gateway --daemon
endif
# Start prod services in daemon mode (background)
start-daemon:
@$(PYTHON) ./scripts/check.py
ifeq ($(OS),Windows_NT)
@call scripts\run-with-git-bash.cmd ./scripts/serve.sh --prod --daemon
else
@./scripts/serve.sh --prod --daemon
endif
# Start prod daemon + Gateway mode (experimental)
start-daemon-pro:
@$(PYTHON) ./scripts/check.py
ifeq ($(OS),Windows_NT)
@call scripts\run-with-git-bash.cmd ./scripts/serve.sh --prod --gateway --daemon
else
@./scripts/serve.sh --prod --gateway --daemon
endif
# Stop all services
stop:
@./scripts/serve.sh --stop
# Clean up
clean: stop
@echo "Cleaning up..."
@-rm -rf backend/.deer-flow 2>/dev/null || true
@-rm -rf backend/.langgraph_api 2>/dev/null || true
@-rm -rf logs/*.log 2>/dev/null || true
@echo "✓ Cleanup complete"
# ==========================================
# Docker Development Commands
# ==========================================
# Initialize Docker containers and install dependencies
docker-init:
@./scripts/docker.sh init
# Start Docker development environment
docker-start:
@./scripts/docker.sh start
# Start Docker in Gateway mode (experimental)
docker-start-pro:
@./scripts/docker.sh start --gateway
# Stop Docker development environment
docker-stop:
@./scripts/docker.sh stop
# View Docker development logs
docker-logs:
@./scripts/docker.sh logs
# View Docker development logs
docker-logs-frontend:
@./scripts/docker.sh logs --frontend
docker-logs-gateway:
@./scripts/docker.sh logs --gateway
# ==========================================
# Production Docker Commands
# ==========================================
# Build and start production services
up:
@./scripts/deploy.sh
# Build and start production services in Gateway mode
up-pro:
@./scripts/deploy.sh --gateway
# Stop and remove production containers
down:
@./scripts/deploy.sh down
+585 -577
View File
File diff suppressed because it is too large Load Diff
-610
View File
@@ -1,610 +0,0 @@
# 🦌 DeerFlow
[![Python 3.12+](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![DeepWiki](https://img.shields.io/badge/DeepWiki-bytedance%2Fdeer--flow-blue.svg?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACwAAAAyCAYAAAAnWDnqAAAAAXNSR0IArs4c6QAAA05JREFUaEPtmUtyEzEQhtWTQyQLHNak2AB7ZnyXZMEjXMGeK/AIi+QuHrMnbChYY7MIh8g01fJoopFb0uhhEqqcbWTp06/uv1saEDv4O3n3dV60RfP947Mm9/SQc0ICFQgzfc4CYZoTPAswgSJCCUJUnAAoRHOAUOcATwbmVLWdGoH//PB8mnKqScAhsD0kYP3j/Yt5LPQe2KvcXmGvRHcDnpxfL2zOYJ1mFwrryWTz0advv1Ut4CJgf5uhDuDj5eUcAUoahrdY/56ebRWeraTjMt/00Sh3UDtjgHtQNHwcRGOC98BJEAEymycmYcWwOprTgcB6VZ5JK5TAJ+fXGLBm3FDAmn6oPPjR4rKCAoJCal2eAiQp2x0vxTPB3ALO2CRkwmDy5WohzBDwSEFKRwPbknEggCPB/imwrycgxX2NzoMCHhPkDwqYMr9tRcP5qNrMZHkVnOjRMWwLCcr8ohBVb1OMjxLwGCvjTikrsBOiA6fNyCrm8V1rP93iVPpwaE+gO0SsWmPiXB+jikdf6SizrT5qKasx5j8ABbHpFTx+vFXp9EnYQmLx02h1QTTrl6eDqxLnGjporxl3NL3agEvXdT0WmEost648sQOYAeJS9Q7bfUVoMGnjo4AZdUMQku50McCcMWcBPvr0SzbTAFDfvJqwLzgxwATnCgnp4wDl6Aa+Ax283gghmj+vj7feE2KBBRMW3FzOpLOADl0Isb5587h/U4gGvkt5v60Z1VLG8BhYjbzRwyQZemwAd6cCR5/XFWLYZRIMpX39AR0tjaGGiGzLVyhse5C9RKC6ai42ppWPKiBagOvaYk8lO7DajerabOZP46Lby5wKjw1HCRx7p9sVMOWGzb/vA1hwiWc6jm3MvQDTogQkiqIhJV0nBQBTU+3okKCFDy9WwferkHjtxib7t3xIUQtHxnIwtx4mpg26/HfwVNVDb4oI9RHmx5WGelRVlrtiw43zboCLaxv46AZeB3IlTkwouebTr1y2NjSpHz68WNFjHvupy3q8TFn3Hos2IAk4Ju5dCo8B3wP7VPr/FGaKiG+T+v+TQqIrOqMTL1VdWV1DdmcbO8KXBz6esmYWYKPwDL5b5FA1a0hwapHiom0r/cKaoqr+27/XcrS5UwSMbQAAAABJRU5ErkJggg==)](https://deepwiki.com/bytedance/deer-flow)
<!-- DeepWiki badge generated by https://deepwiki.ryoppippi.com/ -->
[English](./README.md) | [简体中文](./README_zh.md) | [日本語](./README_ja.md) | [Deutsch](./README_de.md) | [Español](./README_es.md) | [Русский](./README_ru.md) | [Portuguese](./README_pt.md)
> Aus Open Source entstanden, an Open Source zurückgeben.
**DeerFlow** (**D**eep **E**xploration and **E**fficient **R**esearch **Flow**) ist ein Community-getriebenes Framework für tiefgehende Recherche, das auf der großartigen Arbeit der Open-Source-Community aufbaut. Unser Ziel ist es, Sprachmodelle mit spezialisierten Werkzeugen für Aufgaben wie Websuche, Crawling und Python-Code-Ausführung zu kombinieren und gleichzeitig der Community, die dies möglich gemacht hat, etwas zurückzugeben.
Derzeit ist DeerFlow offiziell in das [FaaS-Anwendungszentrum von Volcengine](https://console.volcengine.com/vefaas/region:vefaas+cn-beijing/market) eingezogen. Benutzer können es über den [Erfahrungslink](https://console.volcengine.com/vefaas/region:vefaas+cn-beijing/market/deerflow/?channel=github&source=deerflow) online erleben, um seine leistungsstarken Funktionen und bequemen Operationen intuitiv zu spüren. Gleichzeitig unterstützt DeerFlow zur Erfüllung der Bereitstellungsanforderungen verschiedener Benutzer die Ein-Klick-Bereitstellung basierend auf Volcengine. Klicken Sie auf den [Bereitstellungslink](https://console.volcengine.com/vefaas/region:vefaas+cn-beijing/application/create?templateId=683adf9e372daa0008aaed5c&channel=github&source=deerflow), um den Bereitstellungsprozess schnell abzuschließen und eine effiziente Forschungsreise zu beginnen.
DeerFlow hat neu die intelligente Such- und Crawling-Toolset von BytePlus integriert - [InfoQuest (unterstützt kostenlose Online-Erfahrung)](https://docs.byteplus.com/en/docs/InfoQuest/What_is_Info_Quest)
<a href="https://docs.byteplus.com/en/docs/InfoQuest/What_is_Info_Quest" target="_blank">
<img
src="https://sf16-sg.tiktokcdn.com/obj/eden-sg/hubseh7bsbps/20251208-160108.png" alt="infoquest_bannar"
/>
</a>
Besuchen Sie [unsere offizielle Website](https://deerflow.tech/) für weitere Details.
## Demo
### Video
<https://github.com/user-attachments/assets/f3786598-1f2a-4d07-919e-8b99dfa1de3e>
In dieser Demo zeigen wir, wie man DeerFlow nutzt, um:
- Nahtlos mit MCP-Diensten zu integrieren
- Den Prozess der tiefgehenden Recherche durchzuführen und einen umfassenden Bericht mit Bildern zu erstellen
- Podcast-Audio basierend auf dem generierten Bericht zu erstellen
### Wiedergaben
- [Wie hoch ist der Eiffelturm im Vergleich zum höchsten Gebäude?](https://deerflow.tech/chat?replay=eiffel-tower-vs-tallest-building)
- [Was sind die angesagtesten Repositories auf GitHub?](https://deerflow.tech/chat?replay=github-top-trending-repo)
- [Einen Artikel über traditionelle Gerichte aus Nanjing schreiben](https://deerflow.tech/chat?replay=nanjing-traditional-dishes)
- [Wie dekoriert man eine Mietwohnung?](https://deerflow.tech/chat?replay=rental-apartment-decoration)
- [Besuchen Sie unsere offizielle Website, um weitere Wiedergaben zu entdecken.](https://deerflow.tech/#case-studies)
---
## 📑 Inhaltsverzeichnis
- [🚀 Schnellstart](#schnellstart)
- [🌟 Funktionen](#funktionen)
- [🏗️ Architektur](#architektur)
- [🛠️ Entwicklung](#entwicklung)
- [🐳 Docker](#docker)
- [🗣️ Text-zu-Sprache-Integration](#text-zu-sprache-integration)
- [📚 Beispiele](#beispiele)
- [❓ FAQ](#faq)
- [📜 Lizenz](#lizenz)
- [💖 Danksagungen](#danksagungen)
- [⭐ Star-Verlauf](#star-verlauf)
## Schnellstart
DeerFlow ist in Python entwickelt und kommt mit einer in Node.js geschriebenen Web-UI. Um einen reibungslosen Einrichtungsprozess zu gewährleisten, empfehlen wir die Verwendung der folgenden Tools:
### Empfohlene Tools
- **[`uv`](https://docs.astral.sh/uv/getting-started/installation/):**
Vereinfacht die Verwaltung von Python-Umgebungen und Abhängigkeiten. `uv` erstellt automatisch eine virtuelle Umgebung im Stammverzeichnis und installiert alle erforderlichen Pakete für Sie—keine manuelle Installation von Python-Umgebungen notwendig.
- **[`nvm`](https://github.com/nvm-sh/nvm):**
Verwalten Sie mühelos mehrere Versionen der Node.js-Laufzeit.
- **[`pnpm`](https://pnpm.io/installation):**
Installieren und verwalten Sie Abhängigkeiten des Node.js-Projekts.
### Umgebungsanforderungen
Stellen Sie sicher, dass Ihr System die folgenden Mindestanforderungen erfüllt:
- **[Python](https://www.python.org/downloads/):** Version `3.12+`
- **[Node.js](https://nodejs.org/en/download/):** Version `22+`
### Installation
```bash
# Repository klonen
git clone https://github.com/bytedance/deer-flow.git
cd deer-flow
# Abhängigkeiten installieren, uv kümmert sich um den Python-Interpreter und die Erstellung der venv sowie die Installation der erforderlichen Pakete
uv sync
# Konfigurieren Sie .env mit Ihren API-Schlüsseln
# Tavily: https://app.tavily.com/home
# Brave_SEARCH: https://brave.com/search/api/
# volcengine TTS: Fügen Sie Ihre TTS-Anmeldedaten hinzu, falls vorhanden
cp .env.example .env
# Siehe die Abschnitte 'Unterstützte Suchmaschinen' und 'Text-zu-Sprache-Integration' unten für alle verfügbaren Optionen
# Konfigurieren Sie conf.yaml für Ihr LLM-Modell und API-Schlüssel
# Weitere Details finden Sie unter 'docs/configuration_guide.md'
cp conf.yaml.example conf.yaml
# Installieren Sie marp für PPT-Generierung
# https://github.com/marp-team/marp-cli?tab=readme-ov-file#use-package-manager
brew install marp-cli
```
Optional können Sie Web-UI-Abhängigkeiten über [pnpm](https://pnpm.io/installation) installieren:
```bash
cd deer-flow/web
pnpm install
```
### Konfigurationen
Weitere Informationen finden Sie im [Konfigurationsleitfaden](docs/configuration_guide.md).
> [!HINWEIS]
> Lesen Sie den Leitfaden sorgfältig, bevor Sie das Projekt starten, und aktualisieren Sie die Konfigurationen entsprechend Ihren spezifischen Einstellungen und Anforderungen.
### Konsolen-UI
Der schnellste Weg, um das Projekt auszuführen, ist die Verwendung der Konsolen-UI.
```bash
# Führen Sie das Projekt in einer bash-ähnlichen Shell aus
uv run main.py
```
### Web-UI
Dieses Projekt enthält auch eine Web-UI, die ein dynamischeres und ansprechenderes interaktives Erlebnis bietet.
> [!HINWEIS]
> Sie müssen zuerst die Abhängigkeiten der Web-UI installieren.
```bash
# Führen Sie sowohl den Backend- als auch den Frontend-Server im Entwicklungsmodus aus
# Unter macOS/Linux
./bootstrap.sh -d
# Unter Windows
bootstrap.bat -d
```
> [!HINWEIS]
> Standardmäßig bindet sich der Backend-Server aus Sicherheitsgründen an 127.0.0.1 (localhost). Wenn Sie externe Verbindungen zulassen müssen (z. B. bei der Bereitstellung auf einem Linux-Server), können Sie den Server-Host im Bootstrap-Skript auf 0.0.0.0 ändern (uv run server.py --host 0.0.0.0).
> Bitte stellen Sie sicher, dass Ihre Umgebung ordnungsgemäß gesichert ist, bevor Sie den Service externen Netzwerken aussetzen.
Öffnen Sie Ihren Browser und besuchen Sie [`http://localhost:3000`](http://localhost:3000), um die Web-UI zu erkunden.
Weitere Details finden Sie im Verzeichnis [`web`](./web/).
## Unterstützte Suchmaschinen
### Websuche
DeerFlow unterstützt mehrere Suchmaschinen, die in Ihrer `.env`-Datei über die Variable `SEARCH_API` konfiguriert werden können:
- **Tavily** (Standard): Eine spezialisierte Such-API für KI-Anwendungen
- Erfordert `TAVILY_API_KEY` in Ihrer `.env`-Datei
- Registrieren Sie sich unter: https://app.tavily.com/home
- **InfoQuest** (empfohlen): Ein KI-optimiertes intelligentes Such- und Crawling-Toolset, entwickelt von BytePlus
- Erfordert `INFOQUEST_API_KEY` in Ihrer `.env`-Datei
- Unterstützung für Zeitbereichsfilterung und Seitenfilterung
- Bietet qualitativ hochwertige Suchergebnisse und Inhaltsextraktion
- Registrieren Sie sich unter: https://console.byteplus.com/infoquest/infoquests
- Besuchen Sie https://docs.byteplus.com/de/docs/InfoQuest/What_is_Info_Quest für weitere Informationen
- **DuckDuckGo**: Datenschutzorientierte Suchmaschine
- Kein API-Schlüssel erforderlich
- **Brave Search**: Datenschutzorientierte Suchmaschine mit erweiterten Funktionen
- Erfordert `BRAVE_SEARCH_API_KEY` in Ihrer `.env`-Datei
- Registrieren Sie sich unter: https://brave.com/search/api/
- **Arxiv**: Wissenschaftliche Papiersuche für akademische Forschung
- Kein API-Schlüssel erforderlich
- Spezialisiert auf wissenschaftliche und akademische Papiere
- **Searx/SearxNG**: Selbstgehostete Metasuchmaschine
- Erfordert `SEARX_HOST` in Ihrer `.env`-Datei
- Unterstützt die Anbindung an Searx oder SearxNG
Um Ihre bevorzugte Suchmaschine zu konfigurieren, setzen Sie die Variable `SEARCH_API` in Ihrer `.env`-Datei:
```bash
# Wählen Sie eine: tavily, infoquest, duckduckgo, brave_search, arxiv
SEARCH_API=tavily
```
### Crawling-Tools
- **Jina** (Standard): Kostenloses, zugängliches Webinhalts-Crawling-Tool
- Kein API-Schlüssel erforderlich für grundlegende Funktionen
- Mit API-Schlüssel erhalten Sie höhere Zugriffsraten
- Weitere Informationen unter <https://jina.ai/reader>
- **InfoQuest** (empfohlen): KI-optimiertes intelligentes Such- und Crawling-Toolset, entwickelt von BytePlus
- Erfordert `INFOQUEST_API_KEY` in Ihrer `.env`-Datei
- Bietet konfigurierbare Crawling-Parameter
- Unterstützt benutzerdefinierte Timeout-Einstellungen
- Bietet stärkere Inhaltsextraktionsfähigkeiten
- Weitere Informationen unter <https://docs.byteplus.com/de/docs/InfoQuest/What_is_Info_Quest>
Um Ihr bevorzugtes Crawling-Tool zu konfigurieren, setzen Sie Folgendes in Ihrer `conf.yaml`-Datei:
```yaml
CRAWLER_ENGINE:
# Engine-Typ: "jina" (Standard) oder "infoquest"
engine: infoquest
```
### Private Wissensbasis
DeerFlow unterstützt private Wissensbasen wie RAGFlow und VikingDB, sodass Sie Ihre privaten Dokumente zur Beantwortung von Fragen verwenden können.
- **[RAGFlow](https://ragflow.io/docs/dev/)**Open-Source-RAG-Engine
```
# Beispiele in .env.example
RAG_PROVIDER=ragflow
RAGFLOW_API_URL="http://localhost:9388"
RAGFLOW_API_KEY="ragflow-xxx"
RAGFLOW_RETRIEVAL_SIZE=10
RAGFLOW_CROSS_LANGUAGES=English,Chinese,Spanish,French,German,Japanese,Korean
```
## Funktionen
### Kernfähigkeiten
- 🤖 **LLM-Integration**
- Unterstützt die Integration der meisten Modelle über [litellm](https://docs.litellm.ai/docs/providers).
- Unterstützung für Open-Source-Modelle wie Qwen
- OpenAI-kompatible API-Schnittstelle
- Mehrstufiges LLM-System für unterschiedliche Aufgabenkomplexitäten
### Tools und MCP-Integrationen
- 🔍 **Suche und Abruf**
- Websuche über Tavily, InfoQuest, Brave Search und mehr
- Crawling mit Jina und InfoQuest
- Fortgeschrittene Inhaltsextraktion
- Unterstützung für private Wissensbasis
- 📃 **RAG-Integration**
- Unterstützt die Erwähnung von Dateien aus [RAGFlow](https://github.com/infiniflow/ragflow) innerhalb der Eingabebox. [RAGFlow-Server starten](https://ragflow.io/docs/dev/).
- 🔗 **MCP Nahtlose Integration**
- Erweiterte Fähigkeiten für privaten Domänenzugriff, Wissensgraphen, Webbrowsing und mehr
- Erleichtert die Integration verschiedener Forschungswerkzeuge und -methoden
### Menschliche Zusammenarbeit
- 🧠 **Mensch-in-der-Schleife**
- Unterstützt interaktive Modifikation von Forschungsplänen mit natürlicher Sprache
- Unterstützt automatische Akzeptanz von Forschungsplänen
- 📝 **Bericht-Nachbearbeitung**
- Unterstützt Notion-ähnliche Blockbearbeitung
- Ermöglicht KI-Verfeinerungen, einschließlich KI-unterstützter Polierung, Satzkürzung und -erweiterung
- Angetrieben von [tiptap](https://tiptap.dev/)
### Inhaltserstellung
- 🎙️ **Podcast- und Präsentationserstellung**
- KI-gestützte Podcast-Skripterstellung und Audiosynthese
- Automatisierte Erstellung einfacher PowerPoint-Präsentationen
- Anpassbare Vorlagen für maßgeschneiderte Inhalte
## Architektur
DeerFlow implementiert eine modulare Multi-Agenten-Systemarchitektur, die für automatisierte Forschung und Codeanalyse konzipiert ist. Das System basiert auf LangGraph und ermöglicht einen flexiblen zustandsbasierten Workflow, bei dem Komponenten über ein klar definiertes Nachrichtenübermittlungssystem kommunizieren.
![Architekturdiagramm](./assets/architecture.png)
> Sehen Sie es live auf [deerflow.tech](https://deerflow.tech/#multi-agent-architecture)
Das System verwendet einen optimierten Workflow mit den folgenden Komponenten:
1. **Koordinator**: Der Einstiegspunkt, der den Workflow-Lebenszyklus verwaltet
- Initiiert den Forschungsprozess basierend auf Benutzereingaben
- Delegiert Aufgaben bei Bedarf an den Planer
- Fungiert als primäre Schnittstelle zwischen dem Benutzer und dem System
2. **Planer**: Strategische Komponente für Aufgabenzerlegung und -planung
- Analysiert Forschungsziele und erstellt strukturierte Ausführungspläne
- Bestimmt, ob ausreichend Kontext verfügbar ist oder ob weitere Forschung benötigt wird
- Verwaltet den Forschungsablauf und entscheidet, wann der endgültige Bericht erstellt wird
3. **Forschungsteam**: Eine Sammlung spezialisierter Agenten, die den Plan ausführen:
- **Forscher**: Führt Websuchen und Informationssammlung mit Tools wie Websuchmaschinen, Crawling und sogar MCP-Diensten durch.
- **Codierer**: Behandelt Codeanalyse, -ausführung und technische Aufgaben mit dem Python REPL Tool.
Jeder Agent hat Zugriff auf spezifische Tools, die für seine Rolle optimiert sind, und operiert innerhalb des LangGraph-Frameworks
4. **Reporter**: Endphasenprozessor für Forschungsergebnisse
- Aggregiert Erkenntnisse vom Forschungsteam
- Verarbeitet und strukturiert die gesammelten Informationen
- Erstellt umfassende Forschungsberichte
## Text-zu-Sprache-Integration
DeerFlow enthält jetzt eine Text-zu-Sprache (TTS)-Funktion, mit der Sie Forschungsberichte in Sprache umwandeln können. Diese Funktion verwendet die volcengine TTS API, um hochwertige Audios aus Text zu generieren. Funktionen wie Geschwindigkeit, Lautstärke und Tonhöhe können ebenfalls angepasst werden.
### Verwendung der TTS API
Sie können auf die TTS-Funktionalität über den Endpunkt `/api/tts` zugreifen:
```bash
# Beispiel API-Aufruf mit curl
curl --location 'http://localhost:8000/api/tts' \
--header 'Content-Type: application/json' \
--data '{
"text": "Dies ist ein Test der Text-zu-Sprache-Funktionalität.",
"speed_ratio": 1.0,
"volume_ratio": 1.0,
"pitch_ratio": 1.0
}' \
--output speech.mp3
```
## Entwicklung
### Testen
Führen Sie die Testsuite aus:
```bash
# Alle Tests ausführen
make test
# Spezifische Testdatei ausführen
pytest tests/integration/test_workflow.py
# Mit Abdeckung ausführen
make coverage
```
### Codequalität
```bash
# Lint ausführen
make lint
# Code formatieren
make format
```
### Debugging mit LangGraph Studio
DeerFlow verwendet LangGraph für seine Workflow-Architektur. Sie können LangGraph Studio verwenden, um den Workflow in Echtzeit zu debuggen und zu visualisieren.
#### LangGraph Studio lokal ausführen
DeerFlow enthält eine `langgraph.json`-Konfigurationsdatei, die die Graphstruktur und Abhängigkeiten für das LangGraph Studio definiert. Diese Datei verweist auf die im Projekt definierten Workflow-Graphen und lädt automatisch Umgebungsvariablen aus der `.env`-Datei.
##### Mac
```bash
# Installieren Sie den uv-Paketmanager, wenn Sie ihn noch nicht haben
curl -LsSf https://astral.sh/uv/install.sh | sh
# Installieren Sie Abhängigkeiten und starten Sie den LangGraph-Server
uvx --refresh --from "langgraph-cli[inmem]" --with-editable . --python 3.12 langgraph dev --allow-blocking
```
##### Windows / Linux
```bash
# Abhängigkeiten installieren
pip install -e .
pip install -U "langgraph-cli[inmem]"
# LangGraph-Server starten
langgraph dev
```
Nach dem Start des LangGraph-Servers sehen Sie mehrere URLs im Terminal:
- API: http://127.0.0.1:2024
- Studio UI: https://smith.langchain.com/studio/?baseUrl=http://127.0.0.1:2024
- API-Dokumentation: http://127.0.0.1:2024/docs
Öffnen Sie den Studio UI-Link in Ihrem Browser, um auf die Debugging-Schnittstelle zuzugreifen.
#### Verwendung von LangGraph Studio
In der Studio UI können Sie:
1. Den Workflow-Graphen visualisieren und sehen, wie Komponenten verbunden sind
2. Die Ausführung in Echtzeit verfolgen, um zu sehen, wie Daten durch das System fließen
3. Den Zustand in jedem Schritt des Workflows inspizieren
4. Probleme durch Untersuchung von Ein- und Ausgaben jeder Komponente debuggen
5. Feedback während der Planungsphase geben, um Forschungspläne zu verfeinern
Wenn Sie ein Forschungsthema in der Studio UI einreichen, können Sie die gesamte Workflow-Ausführung sehen, einschließlich:
- Die Planungsphase, in der der Forschungsplan erstellt wird
- Die Feedback-Schleife, in der Sie den Plan ändern können
- Die Forschungs- und Schreibphasen für jeden Abschnitt
- Die Erstellung des endgültigen Berichts
### Aktivieren von LangSmith-Tracing
DeerFlow unterstützt LangSmith-Tracing, um Ihnen beim Debuggen und Überwachen Ihrer Workflows zu helfen. Um LangSmith-Tracing zu aktivieren:
1. Stellen Sie sicher, dass Ihre `.env`-Datei die folgenden Konfigurationen enthält (siehe `.env.example`):
```bash
LANGSMITH_TRACING=true
LANGSMITH_ENDPOINT="https://api.smith.langchain.com"
LANGSMITH_API_KEY="xxx"
LANGSMITH_PROJECT="xxx"
```
2. Starten Sie das Tracing mit LangSmith lokal, indem Sie folgenden Befehl ausführen:
```bash
langgraph dev
```
Dies aktiviert die Trace-Visualisierung in LangGraph Studio und sendet Ihre Traces zur Überwachung und Analyse an LangSmith.
## Docker
Sie können dieses Projekt auch mit Docker ausführen.
Zuerst müssen Sie die [Konfiguration](docs/configuration_guide.md) unten lesen. Stellen Sie sicher, dass die Dateien `.env` und `.conf.yaml` bereit sind.
Zweitens, um ein Docker-Image Ihres eigenen Webservers zu erstellen:
```bash
docker build -t deer-flow-api .
```
Schließlich starten Sie einen Docker-Container, der den Webserver ausführt:
```bash
# Ersetzen Sie deer-flow-api-app durch Ihren bevorzugten Container-Namen
# Starten Sie den Server und binden Sie ihn an localhost:8000
docker run -d -t -p 127.0.0.1:8000:8000 --env-file .env --name deer-flow-api-app deer-flow-api
# Server stoppen
docker stop deer-flow-api-app
```
### Docker Compose (umfasst sowohl Backend als auch Frontend)
DeerFlow bietet ein docker-compose-Setup, um sowohl das Backend als auch das Frontend einfach zusammen auszuführen:
```bash
# Docker-Image erstellen
docker compose build
# Server starten
docker compose up
```
> [!WARNING]
> Wenn Sie DeerFlow in Produktionsumgebungen bereitstellen möchten, fügen Sie bitte Authentifizierung zur Website hinzu und bewerten Sie Ihre Sicherheitsüberprüfung des MCPServer und Python Repl.
## Beispiele
Die folgenden Beispiele demonstrieren die Fähigkeiten von DeerFlow:
### Forschungsberichte
1. **OpenAI Sora Bericht** - Analyse von OpenAIs Sora KI-Tool
- Diskutiert Funktionen, Zugang, Prompt-Engineering, Einschränkungen und ethische Überlegungen
- [Vollständigen Bericht ansehen](examples/openai_sora_report.md)
2. **Googles Agent-to-Agent-Protokoll Bericht** - Überblick über Googles Agent-to-Agent (A2A)-Protokoll
- Diskutiert seine Rolle in der KI-Agentenkommunikation und seine Beziehung zum Model Context Protocol (MCP) von Anthropic
- [Vollständigen Bericht ansehen](examples/what_is_agent_to_agent_protocol.md)
3. **Was ist MCP?** - Eine umfassende Analyse des Begriffs "MCP" in mehreren Kontexten
- Untersucht Model Context Protocol in KI, Monocalciumphosphat in der Chemie und Micro-channel Plate in der Elektronik
- [Vollständigen Bericht ansehen](examples/what_is_mcp.md)
4. **Bitcoin-Preisschwankungen** - Analyse der jüngsten Bitcoin-Preisbewegungen
- Untersucht Markttrends, regulatorische Einflüsse und technische Indikatoren
- Bietet Empfehlungen basierend auf historischen Daten
- [Vollständigen Bericht ansehen](examples/bitcoin_price_fluctuation.md)
5. **Was ist LLM?** - Eine eingehende Erforschung großer Sprachmodelle
- Diskutiert Architektur, Training, Anwendungen und ethische Überlegungen
- [Vollständigen Bericht ansehen](examples/what_is_llm.md)
6. **Wie nutzt man Claude für tiefgehende Recherche?** - Best Practices und Workflows für die Verwendung von Claude in der tiefgehenden Forschung
- Behandelt Prompt-Engineering, Datenanalyse und Integration mit anderen Tools
- [Vollständigen Bericht ansehen](examples/how_to_use_claude_deep_research.md)
7. **KI-Adoption im Gesundheitswesen: Einflussfaktoren** - Analyse der Faktoren, die die KI-Adoption im Gesundheitswesen vorantreiben
- Diskutiert KI-Technologien, Datenqualität, ethische Überlegungen, wirtschaftliche Bewertungen, organisatorische Bereitschaft und digitale Infrastruktur
- [Vollständigen Bericht ansehen](examples/AI_adoption_in_healthcare.md)
8. **Auswirkungen des Quantencomputing auf die Kryptographie** - Analyse der Auswirkungen des Quantencomputing auf die Kryptographie
- Diskutiert Schwachstellen der klassischen Kryptographie, Post-Quanten-Kryptographie und quantenresistente kryptographische Lösungen
- [Vollständigen Bericht ansehen](examples/Quantum_Computing_Impact_on_Cryptography.md)
9. **Cristiano Ronaldos Leistungshöhepunkte** - Analyse der Leistungshöhepunkte von Cristiano Ronaldo
- Diskutiert seine Karriereerfolge, internationalen Tore und Leistungen in verschiedenen Spielen
- [Vollständigen Bericht ansehen](examples/Cristiano_Ronaldo's_Performance_Highlights.md)
Um diese Beispiele auszuführen oder Ihre eigenen Forschungsberichte zu erstellen, können Sie die folgenden Befehle verwenden:
```bash
# Mit einer spezifischen Anfrage ausführen
uv run main.py "Welche Faktoren beeinflussen die KI-Adoption im Gesundheitswesen?"
# Mit benutzerdefinierten Planungsparametern ausführen
uv run main.py --max_plan_iterations 3 "Wie wirkt sich Quantencomputing auf die Kryptographie aus?"
# Im interaktiven Modus mit eingebauten Fragen ausführen
uv run main.py --interactive
# Oder mit grundlegendem interaktiven Prompt ausführen
uv run main.py
# Alle verfügbaren Optionen anzeigen
uv run main.py --help
```
### Interaktiver Modus
Die Anwendung unterstützt jetzt einen interaktiven Modus mit eingebauten Fragen in Englisch und Chinesisch:
1. Starten Sie den interaktiven Modus:
```bash
uv run main.py --interactive
```
2. Wählen Sie Ihre bevorzugte Sprache (English oder 中文)
3. Wählen Sie aus einer Liste von eingebauten Fragen oder wählen Sie die Option, Ihre eigene Frage zu stellen
4. Das System wird Ihre Frage verarbeiten und einen umfassenden Forschungsbericht generieren
### Mensch-in-der-Schleife
DeerFlow enthält einen Mensch-in-der-Schleife-Mechanismus, der es Ihnen ermöglicht, Forschungspläne vor ihrer Ausführung zu überprüfen, zu bearbeiten und zu genehmigen:
1. **Planüberprüfung**: Wenn Mensch-in-der-Schleife aktiviert ist, präsentiert das System den generierten Forschungsplan zur Überprüfung vor der Ausführung
2. **Feedback geben**: Sie können:
- Den Plan akzeptieren, indem Sie mit `[ACCEPTED]` antworten
- Den Plan bearbeiten, indem Sie Feedback geben (z.B., `[EDIT PLAN] Fügen Sie mehr Schritte zur technischen Implementierung hinzu`)
- Das System wird Ihr Feedback einarbeiten und einen überarbeiteten Plan generieren
3. **Automatische Akzeptanz**: Sie können die automatische Akzeptanz aktivieren, um den Überprüfungsprozess zu überspringen:
- Über API: Setzen Sie `auto_accepted_plan: true` in Ihrer Anfrage
4. **API-Integration**: Bei Verwendung der API können Sie Feedback über den Parameter `feedback` geben:
```json
{
"messages": [{"role": "user", "content": "Was ist Quantencomputing?"}],
"thread_id": "my_thread_id",
"auto_accepted_plan": false,
"feedback": "[EDIT PLAN] Mehr über Quantenalgorithmen aufnehmen"
}
```
### Kommandozeilenargumente
Die Anwendung unterstützt mehrere Kommandozeilenargumente, um ihr Verhalten anzupassen:
- **query**: Die zu verarbeitende Forschungsanfrage (kann mehrere Wörter umfassen)
- **--interactive**: Im interaktiven Modus mit eingebauten Fragen ausführen
- **--max_plan_iterations**: Maximale Anzahl von Planungszyklen (Standard: 1)
- **--max_step_num**: Maximale Anzahl von Schritten in einem Forschungsplan (Standard: 3)
- **--debug**: Detaillierte Debug-Protokollierung aktivieren
## FAQ
Weitere Informationen finden Sie in der [FAQ.md](docs/FAQ.md).
## Lizenz
Dieses Projekt ist Open Source und unter der [MIT-Lizenz](./LICENSE) verfügbar.
## Danksagungen
DeerFlow baut auf der unglaublichen Arbeit der Open-Source-Community auf. Wir sind allen Projekten und Mitwirkenden zutiefst dankbar, deren Bemühungen DeerFlow möglich gemacht haben. Wahrhaftig stehen wir auf den Schultern von Riesen.
Wir möchten unsere aufrichtige Wertschätzung den folgenden Projekten für ihre unschätzbaren Beiträge aussprechen:
- **[LangChain](https://github.com/langchain-ai/langchain)**: Ihr außergewöhnliches Framework unterstützt unsere LLM-Interaktionen und -Ketten und ermöglicht nahtlose Integration und Funktionalität.
- **[LangGraph](https://github.com/langchain-ai/langgraph)**: Ihr innovativer Ansatz zur Multi-Agenten-Orchestrierung war maßgeblich für die Ermöglichung der ausgeklügelten Workflows von DeerFlow.
- **[Novel](https://github.com/steven-tey/novel)**: Ihr Notion-artiger WYSIWYG-Editor unterstützt unsere Berichtbearbeitung und KI-unterstützte Umschreibung.
- **[RAGFlow](https://github.com/infiniflow/ragflow)**: Wir haben durch die Integration mit RAGFlow die Unterstützung für Forschung auf privaten Wissensdatenbanken der Benutzer erreicht.
Diese Projekte veranschaulichen die transformative Kraft der Open-Source-Zusammenarbeit, und wir sind stolz darauf, auf ihren Grundlagen aufzubauen.
### Hauptmitwirkende
Ein herzliches Dankeschön geht an die Hauptautoren von `DeerFlow`, deren Vision, Leidenschaft und Engagement dieses Projekt zum Leben erweckt haben:
- **[Daniel Walnut](https://github.com/hetaoBackend/)**
- **[Henry Li](https://github.com/magiccube/)**
Ihr unerschütterliches Engagement und Fachwissen waren die treibende Kraft hinter dem Erfolg von DeerFlow. Wir fühlen uns geehrt, Sie an der Spitze dieser Reise zu haben.
## Star-Verlauf
[![Star History Chart](https://api.star-history.com/svg?repos=bytedance/deer-flow&type=Date)](https://star-history.com/#bytedance/deer-flow&Date)
-607
View File
@@ -1,607 +0,0 @@
# 🦌 DeerFlow
[![Python 3.12+](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![DeepWiki](https://img.shields.io/badge/DeepWiki-bytedance%2Fdeer--flow-blue.svg?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACwAAAAyCAYAAAAnWDnqAAAAAXNSR0IArs4c6QAAA05JREFUaEPtmUtyEzEQhtWTQyQLHNak2AB7ZnyXZMEjXMGeK/AIi+QuHrMnbChYY7MIh8g01fJoopFb0uhhEqqcbWTp06/uv1saEDv4O3n3dV60RfP947Mm9/SQc0ICFQgzfc4CYZoTPAswgSJCCUJUnAAoRHOAUOcATwbmVLWdGoH//PB8mnKqScAhsD0kYP3j/Yt5LPQe2KvcXmGvRHcDnpxfL2zOYJ1mFwrryWTz0advv1Ut4CJgf5uhDuDj5eUcAUoahrdY/56ebRWeraTjMt/00Sh3UDtjgHtQNHwcRGOC98BJEAEymycmYcWwOprTgcB6VZ5JK5TAJ+fXGLBm3FDAmn6oPPjR4rKCAoJCal2eAiQp2x0vxTPB3ALO2CRkwmDy5WohzBDwSEFKRwPbknEggCPB/imwrycgxX2NzoMCHhPkDwqYMr9tRcP5qNrMZHkVnOjRMWwLCcr8ohBVb1OMjxLwGCvjTikrsBOiA6fNyCrm8V1rP93iVPpwaE+gO0SsWmPiXB+jikdf6SizrT5qKasx5j8ABbHpFTx+vFXp9EnYQmLx02h1QTTrl6eDqxLnGjporxl3NL3agEvXdT0WmEost648sQOYAeJS9Q7bfUVoMGnjo4AZdUMQku50McCcMWcBPvr0SzbTAFDfvJqwLzgxwATnCgnp4wDl6Aa+Ax283gghmj+vj7feE2KBBRMW3FzOpLOADl0Isb5587h/U4gGvkt5v60Z1VLG8BhYjbzRwyQZemwAd6cCR5/XFWLYZRIMpX39AR0tjaGGiGzLVyhse5C9RKC6ai42ppWPKiBagOvaYk8lO7DajerabOZP46Lby5wKjw1HCRx7p9sVMOWGzb/vA1hwiWc6jm3MvQDTogQkiqIhJV0nBQBTU+3okKCFDy9WwferkHjtxib7t3xIUQtHxnIwtx4mpg26/HfwVNVDb4oI9RHmx5WGelRVlrtiw43zboCLaxv46AZeB3IlTkwouebTr1y2NjSpHz68WNFjHvupy3q8TFn3Hos2IAk4Ju5dCo8B3wP7VPr/FGaKiG+T+v+TQqIrOqMTL1VdWV1DdmcbO8KXBz6esmYWYKPwDL5b5FA1a0hwapHiom0r/cKaoqr+27/XcrS5UwSMbQAAAABJRU5ErkJggg==)](https://deepwiki.com/bytedance/deer-flow)
<!-- DeepWiki badge generated by https://deepwiki.ryoppippi.com/ -->
[English](./README.md) | [简体中文](./README_zh.md) | [日本語](./README_ja.md) | [Deutsch](./README_de.md) | [Español](./README_es.md) | [Русский](./README_ru.md) | [Portuguese](./README_pt.md)
> Originado del código abierto, retribuido al código abierto.
**DeerFlow** (**D**eep **E**xploration and **E**fficient **R**esearch **Flow**) es un marco de Investigación Profunda impulsado por la comunidad que se basa en el increíble trabajo de la comunidad de código abierto. Nuestro objetivo es combinar modelos de lenguaje con herramientas especializadas para tareas como búsqueda web, rastreo y ejecución de código Python, mientras devolvemos a la comunidad que hizo esto posible.
Actualmente, DeerFlow ha ingresado oficialmente al Centro de Aplicaciones FaaS de Volcengine. Los usuarios pueden experimentarlo en línea a través del enlace de experiencia para sentir intuitivamente sus potentes funciones y operaciones convenientes. Al mismo tiempo, para satisfacer las necesidades de implementación de diferentes usuarios, DeerFlow admite la implementación con un clic basada en Volcengine. Haga clic en el enlace de implementación para completar rápidamente el proceso de implementación y comenzar un viaje de investigación eficiente.
DeerFlow ha integrado recientemente el conjunto de herramientas de búsqueda y rastreo inteligente desarrollado independientemente por BytePlus - [InfoQuest (admite experiencia gratuita en línea)](https://docs.byteplus.com/en/docs/InfoQuest/What_is_Info_Quest)
<a href="https://docs.byteplus.com/en/docs/InfoQuest/What_is_Info_Quest" target="_blank">
<img
src="https://sf16-sg.tiktokcdn.com/obj/eden-sg/hubseh7bsbps/20251208-160108.png" alt="infoquest_bannar"
/>
</a>
Por favor, visita [nuestra página web oficial](https://deerflow.tech/) para más detalles.
## Demostración
### Video
<https://github.com/user-attachments/assets/f3786598-1f2a-4d07-919e-8b99dfa1de3e>
En esta demostración, mostramos cómo usar DeerFlow para:
- Integrar perfectamente con servicios MCP
- Realizar el proceso de Investigación Profunda y producir un informe completo con imágenes
- Crear audio de podcast basado en el informe generado
### Repeticiones
- [¿Qué altura tiene la Torre Eiffel comparada con el edificio más alto?](https://deerflow.tech/chat?replay=eiffel-tower-vs-tallest-building)
- [¿Cuáles son los repositorios más populares en GitHub?](https://deerflow.tech/chat?replay=github-top-trending-repo)
- [Escribir un artículo sobre los platos tradicionales de Nanjing](https://deerflow.tech/chat?replay=nanjing-traditional-dishes)
- [¿Cómo decorar un apartamento de alquiler?](https://deerflow.tech/chat?replay=rental-apartment-decoration)
- [Visita nuestra página web oficial para explorar más repeticiones.](https://deerflow.tech/#case-studies)
---
## 📑 Tabla de Contenidos
- [🚀 Inicio Rápido](#inicio-rápido)
- [🌟 Características](#características)
- [🏗️ Arquitectura](#arquitectura)
- [🛠️ Desarrollo](#desarrollo)
- [🐳 Docker](#docker)
- [🗣️ Integración de Texto a Voz](#integración-de-texto-a-voz)
- [📚 Ejemplos](#ejemplos)
- [❓ Preguntas Frecuentes](#preguntas-frecuentes)
- [📜 Licencia](#licencia)
- [💖 Agradecimientos](#agradecimientos)
- [⭐ Historial de Estrellas](#historial-de-estrellas)
## Inicio Rápido
DeerFlow está desarrollado en Python y viene con una interfaz web escrita en Node.js. Para garantizar un proceso de configuración sin problemas, recomendamos utilizar las siguientes herramientas:
### Herramientas Recomendadas
- **[`uv`](https://docs.astral.sh/uv/getting-started/installation/):**
Simplifica la gestión del entorno Python y las dependencias. `uv` crea automáticamente un entorno virtual en el directorio raíz e instala todos los paquetes necesarios por ti—sin necesidad de instalar entornos Python manualmente.
- **[`nvm`](https://github.com/nvm-sh/nvm):**
Gestiona múltiples versiones del entorno de ejecución Node.js sin esfuerzo.
- **[`pnpm`](https://pnpm.io/installation):**
Instala y gestiona dependencias del proyecto Node.js.
### Requisitos del Entorno
Asegúrate de que tu sistema cumple con los siguientes requisitos mínimos:
- **[Python](https://www.python.org/downloads/):** Versión `3.12+`
- **[Node.js](https://nodejs.org/en/download/):** Versión `22+`
### Instalación
```bash
# Clonar el repositorio
git clone https://github.com/bytedance/deer-flow.git
cd deer-flow
# Instalar dependencias, uv se encargará del intérprete de python, la creación del entorno virtual y la instalación de los paquetes necesarios
uv sync
# Configurar .env con tus claves API
# Tavily: https://app.tavily.com/home
# Brave_SEARCH: https://brave.com/search/api/
# volcengine TTS: Añade tus credenciales TTS si las tienes
cp .env.example .env
# Ver las secciones 'Motores de Búsqueda Compatibles' e 'Integración de Texto a Voz' a continuación para todas las opciones disponibles
# Configurar conf.yaml para tu modelo LLM y claves API
# Por favor, consulta 'docs/configuration_guide.md' para más detalles
cp conf.yaml.example conf.yaml
# Instalar marp para la generación de presentaciones
# https://github.com/marp-team/marp-cli?tab=readme-ov-file#use-package-manager
brew install marp-cli
```
Opcionalmente, instala las dependencias de la interfaz web vía [pnpm](https://pnpm.io/installation):
```bash
cd deer-flow/web
pnpm install
```
### Configuraciones
Por favor, consulta la [Guía de Configuración](docs/configuration_guide.md) para más detalles.
> [!NOTA]
> Antes de iniciar el proyecto, lee la guía cuidadosamente y actualiza las configuraciones para que coincidan con tus ajustes y requisitos específicos.
### Interfaz de Consola
La forma más rápida de ejecutar el proyecto es utilizar la interfaz de consola.
```bash
# Ejecutar el proyecto en un shell tipo bash
uv run main.py
```
### Interfaz Web
Este proyecto también incluye una Interfaz Web, que ofrece una experiencia interactiva más dinámica y atractiva.
> [!NOTA]
> Necesitas instalar primero las dependencias de la interfaz web.
```bash
# Ejecutar tanto el servidor backend como el frontend en modo desarrollo
# En macOS/Linux
./bootstrap.sh -d
# En Windows
bootstrap.bat -d
```
> [!NOTA]
> Por defecto, el servidor backend se enlaza a 127.0.0.1 (localhost) por razones de seguridad. Si necesitas permitir conexiones externas (por ejemplo, al desplegar en un servidor Linux), puedes modificar el host del servidor a 0.0.0.0 en el script de arranque (uv run server.py --host 0.0.0.0).
> Por favor, asegúrate de que tu entorno esté correctamente protegido antes de exponer el servicio a redes externas.
Abre tu navegador y visita [`http://localhost:3000`](http://localhost:3000) para explorar la interfaz web.
Explora más detalles en el directorio [`web`](./web/).
## Motores de Búsqueda Compatibles
DeerFlow soporta múltiples motores de búsqueda que pueden configurarse en tu archivo `.env` usando la variable `SEARCH_API`:
- **Tavily** (predeterminado): Una API de búsqueda especializada para aplicaciones de IA
- Requiere `TAVILY_API_KEY` en tu archivo `.env`
- Regístrate en: <https://app.tavily.com/home>
- **InfoQuest** (recomendado): Un conjunto de herramientas inteligentes de búsqueda y rastreo optimizadas para IA, desarrollado por BytePlus
- Requiere `INFOQUEST_API_KEY` en tu archivo `.env`
- Soporte para filtrado por rango de fecha y filtrado de sitios web
- Proporciona resultados de búsqueda y extracción de contenido de alta calidad
- Regístrate en: <https://console.byteplus.com/infoquest/infoquests>
- Visita https://docs.byteplus.com/es/docs/InfoQuest/What_is_Info_Quest para obtener más información
- **DuckDuckGo**: Motor de búsqueda centrado en la privacidad
- No requiere clave API
- **Brave Search**: Motor de búsqueda centrado en la privacidad con características avanzadas
- Requiere `BRAVE_SEARCH_API_KEY` en tu archivo `.env`
- Regístrate en: <https://brave.com/search/api/>
- **Arxiv**: Búsqueda de artículos científicos para investigación académica
- No requiere clave API
- Especializado en artículos científicos y académicos
- **Searx/SearxNG**: Motor de metabúsqueda autoalojado
- Requiere `SEARX_HOST` en tu archivo `.env`
- Compatible con Searx o SearxNG
Para configurar tu motor de búsqueda preferido, establece la variable `SEARCH_API` en tu archivo `.env`:
```bash
# Elige uno: tavily, infoquest, duckduckgo, brave_search, arxiv
SEARCH_API=tavily
```
### Herramientas de Rastreo
- **Jina** (predeterminado): Herramienta gratuita de rastreo de contenido web accesible
- No se requiere clave API para usar funciones básicas
- Al usar una clave API, se obtienen límites de tasa de acceso más altos
- Visite <https://jina.ai/reader> para obtener más información
- **InfoQuest** (recomendado): Conjunto de herramientas inteligentes de búsqueda y rastreo optimizadas para IA, desarrollado por BytePlus
- Requiere `INFOQUEST_API_KEY` en tu archivo `.env`
- Proporciona parámetros de rastreo configurables
- Admite configuración de tiempo de espera personalizada
- Ofrece capacidades más potentes de extracción de contenido
- Visita <https://docs.byteplus.com/es/docs/InfoQuest/What_is_Info_Quest> para obtener más información
Para configurar su herramienta de rastreo preferida, establezca lo siguiente en su archivo `conf.yaml`:
```yaml
CRAWLER_ENGINE:
# Tipo de motor: "jina" (predeterminado) o "infoquest"
engine: infoquest
```
## Características
### Capacidades Principales
- 🤖 **Integración de LLM**
- Soporta la integración de la mayoría de los modelos a través de [litellm](https://docs.litellm.ai/docs/providers).
- Soporte para modelos de código abierto como Qwen
- Interfaz API compatible con OpenAI
- Sistema LLM de múltiples niveles para diferentes complejidades de tareas
### Herramientas e Integraciones MCP
- 🔍 **Búsqueda y Recuperación**
- Búsqueda web a través de Tavily, InfoQuest, Brave Search y más
- Rastreo con Jina e InfoQuest
- Extracción avanzada de contenido
- 🔗 **Integración Perfecta con MCP**
- Amplía capacidades para acceso a dominio privado, gráfico de conocimiento, navegación web y más
- Facilita la integración de diversas herramientas y metodologías de investigación
### Colaboración Humana
- 🧠 **Humano en el Bucle**
- Soporta modificación interactiva de planes de investigación usando lenguaje natural
- Soporta aceptación automática de planes de investigación
- 📝 **Post-Edición de Informes**
- Soporta edición de bloques tipo Notion
- Permite refinamientos por IA, incluyendo pulido asistido por IA, acortamiento y expansión de oraciones
- Impulsado por [tiptap](https://tiptap.dev/)
### Creación de Contenido
- 🎙️ **Generación de Podcasts y Presentaciones**
- Generación de guiones de podcast y síntesis de audio impulsadas por IA
- Creación automatizada de presentaciones PowerPoint simples
- Plantillas personalizables para contenido a medida
## Arquitectura
DeerFlow implementa una arquitectura modular de sistema multi-agente diseñada para investigación automatizada y análisis de código. El sistema está construido sobre LangGraph, permitiendo un flujo de trabajo flexible basado en estados donde los componentes se comunican a través de un sistema de paso de mensajes bien definido.
![Diagrama de Arquitectura](./assets/architecture.png)
> Vélo en vivo en [deerflow.tech](https://deerflow.tech/#multi-agent-architecture)
El sistema emplea un flujo de trabajo racionalizado con los siguientes componentes:
1. **Coordinador**: El punto de entrada que gestiona el ciclo de vida del flujo de trabajo
- Inicia el proceso de investigación basado en la entrada del usuario
- Delega tareas al planificador cuando corresponde
- Actúa como la interfaz principal entre el usuario y el sistema
2. **Planificador**: Componente estratégico para descomposición y planificación de tareas
- Analiza objetivos de investigación y crea planes de ejecución estructurados
- Determina si hay suficiente contexto disponible o si se necesita más investigación
- Gestiona el flujo de investigación y decide cuándo generar el informe final
3. **Equipo de Investigación**: Una colección de agentes especializados que ejecutan el plan:
- **Investigador**: Realiza búsquedas web y recopilación de información utilizando herramientas como motores de búsqueda web, rastreo e incluso servicios MCP.
- **Programador**: Maneja análisis de código, ejecución y tareas técnicas utilizando la herramienta Python REPL.
Cada agente tiene acceso a herramientas específicas optimizadas para su rol y opera dentro del marco LangGraph
4. **Reportero**: Procesador de etapa final para los resultados de la investigación
- Agrega hallazgos del equipo de investigación
- Procesa y estructura la información recopilada
- Genera informes de investigación completos
## Integración de Texto a Voz
DeerFlow ahora incluye una función de Texto a Voz (TTS) que te permite convertir informes de investigación a voz. Esta función utiliza la API TTS de volcengine para generar audio de alta calidad a partir de texto. Características como velocidad, volumen y tono también son personalizables.
### Usando la API TTS
Puedes acceder a la funcionalidad TTS a través del punto final `/api/tts`:
```bash
# Ejemplo de llamada API usando curl
curl --location 'http://localhost:8000/api/tts' \
--header 'Content-Type: application/json' \
--data '{
"text": "Esto es una prueba de la funcionalidad de texto a voz.",
"speed_ratio": 1.0,
"volume_ratio": 1.0,
"pitch_ratio": 1.0
}' \
--output speech.mp3
```
## Desarrollo
### Pruebas
Ejecuta el conjunto de pruebas:
```bash
# Ejecutar todas las pruebas
make test
# Ejecutar archivo de prueba específico
pytest tests/integration/test_workflow.py
# Ejecutar con cobertura
make coverage
```
### Calidad del Código
```bash
# Ejecutar linting
make lint
# Formatear código
make format
```
### Depuración con LangGraph Studio
DeerFlow utiliza LangGraph para su arquitectura de flujo de trabajo. Puedes usar LangGraph Studio para depurar y visualizar el flujo de trabajo en tiempo real.
#### Ejecutando LangGraph Studio Localmente
DeerFlow incluye un archivo de configuración `langgraph.json` que define la estructura del grafo y las dependencias para LangGraph Studio. Este archivo apunta a los grafos de flujo de trabajo definidos en el proyecto y carga automáticamente variables de entorno desde el archivo `.env`.
##### Mac
```bash
# Instala el gestor de paquetes uv si no lo tienes
curl -LsSf https://astral.sh/uv/install.sh | sh
# Instala dependencias e inicia el servidor LangGraph
uvx --refresh --from "langgraph-cli[inmem]" --with-editable . --python 3.12 langgraph dev --allow-blocking
```
##### Windows / Linux
```bash
# Instalar dependencias
pip install -e .
pip install -U "langgraph-cli[inmem]"
# Iniciar el servidor LangGraph
langgraph dev
```
Después de iniciar el servidor LangGraph, verás varias URLs en la terminal:
- API: <http://127.0.0.1:2024>
- UI de Studio: <https://smith.langchain.com/studio/?baseUrl=http://127.0.0.1:2024>
- Docs de API: <http://127.0.0.1:2024/docs>
Abre el enlace de UI de Studio en tu navegador para acceder a la interfaz de depuración.
#### Usando LangGraph Studio
En la UI de Studio, puedes:
1. Visualizar el grafo de flujo de trabajo y ver cómo se conectan los componentes
2. Rastrear la ejecución en tiempo real para ver cómo fluyen los datos a través del sistema
3. Inspeccionar el estado en cada paso del flujo de trabajo
4. Depurar problemas examinando entradas y salidas de cada componente
5. Proporcionar retroalimentación durante la fase de planificación para refinar planes de investigación
Cuando envías un tema de investigación en la UI de Studio, podrás ver toda la ejecución del flujo de trabajo, incluyendo:
- La fase de planificación donde se crea el plan de investigación
- El bucle de retroalimentación donde puedes modificar el plan
- Las fases de investigación y escritura para cada sección
- La generación del informe final
### Habilitando el Rastreo de LangSmith
DeerFlow soporta el rastreo de LangSmith para ayudarte a depurar y monitorear tus flujos de trabajo. Para habilitar el rastreo de LangSmith:
1. Asegúrate de que tu archivo `.env` tenga las siguientes configuraciones (ver `.env.example`):
```bash
LANGSMITH_TRACING=true
LANGSMITH_ENDPOINT="https://api.smith.langchain.com"
LANGSMITH_API_KEY="xxx"
LANGSMITH_PROJECT="xxx"
```
2. Inicia el rastreo y visualiza el grafo localmente con LangSmith ejecutando:
```bash
langgraph dev
```
Esto habilitará la visualización de rastros en LangGraph Studio y enviará tus rastros a LangSmith para monitoreo y análisis.
## Docker
También puedes ejecutar este proyecto con Docker.
Primero, necesitas leer la [configuración](docs/configuration_guide.md) a continuación. Asegúrate de que los archivos `.env` y `.conf.yaml` estén listos.
Segundo, para construir una imagen Docker de tu propio servidor web:
```bash
docker build -t deer-flow-api .
```
Finalmente, inicia un contenedor Docker que ejecute el servidor web:
```bash
# Reemplaza deer-flow-api-app con tu nombre de contenedor preferido
# Inicia el servidor y enlázalo a localhost:8000
docker run -d -t -p 127.0.0.1:8000:8000 --env-file .env --name deer-flow-api-app deer-flow-api
# detener el servidor
docker stop deer-flow-api-app
```
### Docker Compose (incluye tanto backend como frontend)
DeerFlow proporciona una configuración docker-compose para ejecutar fácilmente tanto el backend como el frontend juntos:
```bash
# construir imagen docker
docker compose build
# iniciar el servidor
docker compose up
```
> [!WARNING]
> Si desea implementar DeerFlow en entornos de producción, agregue autenticación al sitio web y evalúe su verificación de seguridad del MCPServer y Python Repl.
## Ejemplos
Los siguientes ejemplos demuestran las capacidades de DeerFlow:
### Informes de Investigación
1. **Informe sobre OpenAI Sora** - Análisis de la herramienta IA Sora de OpenAI
- Discute características, acceso, ingeniería de prompts, limitaciones y consideraciones éticas
- [Ver informe completo](examples/openai_sora_report.md)
2. **Informe sobre el Protocolo Agent to Agent de Google** - Visión general del protocolo Agent to Agent (A2A) de Google
- Discute su papel en la comunicación de agentes IA y su relación con el Model Context Protocol (MCP) de Anthropic
- [Ver informe completo](examples/what_is_agent_to_agent_protocol.md)
3. **¿Qué es MCP?** - Un análisis completo del término "MCP" en múltiples contextos
- Explora Model Context Protocol en IA, Fosfato Monocálcico en química y Placa de Microcanales en electrónica
- [Ver informe completo](examples/what_is_mcp.md)
4. **Fluctuaciones del Precio de Bitcoin** - Análisis de los movimientos recientes del precio de Bitcoin
- Examina tendencias del mercado, influencias regulatorias e indicadores técnicos
- Proporciona recomendaciones basadas en datos históricos
- [Ver informe completo](examples/bitcoin_price_fluctuation.md)
5. **¿Qué es LLM?** - Una exploración en profundidad de los Modelos de Lenguaje Grandes
- Discute arquitectura, entrenamiento, aplicaciones y consideraciones éticas
- [Ver informe completo](examples/what_is_llm.md)
6. **¿Cómo usar Claude para Investigación Profunda?** - Mejores prácticas y flujos de trabajo para usar Claude en investigación profunda
- Cubre ingeniería de prompts, análisis de datos e integración con otras herramientas
- [Ver informe completo](examples/how_to_use_claude_deep_research.md)
7. **Adopción de IA en Salud: Factores de Influencia** - Análisis de factores que impulsan la adopción de IA en salud
- Discute tecnologías IA, calidad de datos, consideraciones éticas, evaluaciones económicas, preparación organizativa e infraestructura digital
- [Ver informe completo](examples/AI_adoption_in_healthcare.md)
8. **Impacto de la Computación Cuántica en la Criptografía** - Análisis del impacto de la computación cuántica en la criptografía
- Discute vulnerabilidades de la criptografía clásica, criptografía post-cuántica y soluciones criptográficas resistentes a la cuántica
- [Ver informe completo](examples/Quantum_Computing_Impact_on_Cryptography.md)
9. **Aspectos Destacados del Rendimiento de Cristiano Ronaldo** - Análisis de los aspectos destacados del rendimiento de Cristiano Ronaldo
- Discute sus logros profesionales, goles internacionales y rendimiento en varios partidos
- [Ver informe completo](examples/Cristiano_Ronaldo's_Performance_Highlights.md)
Para ejecutar estos ejemplos o crear tus propios informes de investigación, puedes usar los siguientes comandos:
```bash
# Ejecutar con una consulta específica
uv run main.py "¿Qué factores están influyendo en la adopción de IA en salud?"
# Ejecutar con parámetros de planificación personalizados
uv run main.py --max_plan_iterations 3 "¿Cómo impacta la computación cuántica en la criptografía?"
# Ejecutar en modo interactivo con preguntas integradas
uv run main.py --interactive
# O ejecutar con prompt interactivo básico
uv run main.py
# Ver todas las opciones disponibles
uv run main.py --help
```
### Modo Interactivo
La aplicación ahora soporta un modo interactivo con preguntas integradas tanto en inglés como en chino:
1. Lanza el modo interactivo:
```bash
uv run main.py --interactive
```
2. Selecciona tu idioma preferido (English o 中文)
3. Elige de una lista de preguntas integradas o selecciona la opción para hacer tu propia pregunta
4. El sistema procesará tu pregunta y generará un informe de investigación completo
### Humano en el Bucle
DeerFlow incluye un mecanismo de humano en el bucle que te permite revisar, editar y aprobar planes de investigación antes de que sean ejecutados:
1. **Revisión del Plan**: Cuando el humano en el bucle está habilitado, el sistema presentará el plan de investigación generado para tu revisión antes de la ejecución
2. **Proporcionando Retroalimentación**: Puedes:
- Aceptar el plan respondiendo con `[ACCEPTED]`
- Editar el plan proporcionando retroalimentación (p.ej., `[EDIT PLAN] Añadir más pasos sobre implementación técnica`)
- El sistema incorporará tu retroalimentación y generará un plan revisado
3. **Auto-aceptación**: Puedes habilitar la auto-aceptación para omitir el proceso de revisión:
- Vía API: Establece `auto_accepted_plan: true` en tu solicitud
4. **Integración API**: Cuando uses la API, puedes proporcionar retroalimentación a través del parámetro `feedback`:
```json
{
"messages": [{ "role": "user", "content": "¿Qué es la computación cuántica?" }],
"thread_id": "my_thread_id",
"auto_accepted_plan": false,
"feedback": "[EDIT PLAN] Incluir más sobre algoritmos cuánticos"
}
```
### Argumentos de Línea de Comandos
La aplicación soporta varios argumentos de línea de comandos para personalizar su comportamiento:
- **query**: La consulta de investigación a procesar (puede ser múltiples palabras)
- **--interactive**: Ejecutar en modo interactivo con preguntas integradas
- **--max_plan_iterations**: Número máximo de ciclos de planificación (predeterminado: 1)
- **--max_step_num**: Número máximo de pasos en un plan de investigación (predeterminado: 3)
- **--debug**: Habilitar registro detallado de depuración
## Preguntas Frecuentes
Por favor, consulta [FAQ.md](docs/FAQ.md) para más detalles.
## Licencia
Este proyecto es de código abierto y está disponible bajo la [Licencia MIT](./LICENSE).
## Agradecimientos
DeerFlow está construido sobre el increíble trabajo de la comunidad de código abierto. Estamos profundamente agradecidos a todos los proyectos y contribuyentes cuyos esfuerzos han hecho posible DeerFlow. Verdaderamente, nos apoyamos en hombros de gigantes.
Nos gustaría extender nuestro sincero agradecimiento a los siguientes proyectos por sus invaluables contribuciones:
- **[LangChain](https://github.com/langchain-ai/langchain)**: Su excepcional marco impulsa nuestras interacciones y cadenas LLM, permitiendo integración y funcionalidad sin problemas.
- **[LangGraph](https://github.com/langchain-ai/langgraph)**: Su enfoque innovador para la orquestación multi-agente ha sido instrumental en permitir los sofisticados flujos de trabajo de DeerFlow.
Estos proyectos ejemplifican el poder transformador de la colaboración de código abierto, y estamos orgullosos de construir sobre sus cimientos.
### Contribuyentes Clave
Un sentido agradecimiento va para los autores principales de `DeerFlow`, cuya visión, pasión y dedicación han dado vida a este proyecto:
- **[Daniel Walnut](https://github.com/hetaoBackend/)**
- **[Henry Li](https://github.com/magiccube/)**
Su compromiso inquebrantable y experiencia han sido la fuerza impulsora detrás del éxito de DeerFlow. Nos sentimos honrados de tenerlos al timón de este viaje.
## Historial de Estrellas
[![Gráfico de Historial de Estrellas](https://api.star-history.com/svg?repos=bytedance/deer-flow&type=Date)](https://star-history.com/#bytedance/deer-flow&Date)
+610
View File
@@ -0,0 +1,610 @@
# 🦌 DeerFlow - 2.0
[English](./README.md) | [中文](./README_zh.md) | [日本語](./README_ja.md) | Français | [Русский](./README_ru.md)
[![Python](https://img.shields.io/badge/Python-3.12%2B-3776AB?logo=python&logoColor=white)](./backend/pyproject.toml)
[![Node.js](https://img.shields.io/badge/Node.js-22%2B-339933?logo=node.js&logoColor=white)](./Makefile)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](./LICENSE)
<a href="https://trendshift.io/repositories/14699" target="_blank"><img src="https://trendshift.io/api/badge/repositories/14699" alt="bytedance%2Fdeer-flow | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
> Le 28 février 2026, DeerFlow a décroché la 🏆 1re place sur GitHub Trending suite au lancement de la version 2. Un immense merci à notre incroyable communauté — c'est grâce à vous ! 💪🔥
DeerFlow (**D**eep **E**xploration and **E**fficient **R**esearch **Flow**) est un **super agent harness** open source qui orchestre des **sub-agents**, de la **mémoire** et des **sandboxes** pour accomplir pratiquement n'importe quelle tâche — le tout propulsé par des **skills extensibles**.
https://github.com/user-attachments/assets/a8bcadc4-e040-4cf2-8fda-dd768b999c18
> [!NOTE]
> **DeerFlow 2.0 est une réécriture complète.** Il ne partage aucun code avec la v1. Si vous cherchez le framework Deep Research original, il est maintenu sur la [branche `1.x`](https://github.com/bytedance/deer-flow/tree/main-1.x) — les contributions y sont toujours les bienvenues. Le développement actif a migré vers la 2.0.
## Site officiel
[<img width="2880" height="1600" alt="image" src="https://github.com/user-attachments/assets/a598c49f-3b2f-41ea-a052-05e21349188a" />](https://deerflow.tech)
Découvrez-en plus et regardez des **démos réelles** sur notre [**site officiel**](https://deerflow.tech).
## Coding Plan de ByteDance Volcengine
<img width="4808" height="2400" alt="英文方舟" src="https://github.com/user-attachments/assets/2ecc7b9d-50be-4185-b1f7-5542d222fb2d" />
- Nous recommandons fortement d'utiliser Doubao-Seed-2.0-Code, DeepSeek v3.2 et Kimi 2.5 pour exécuter DeerFlow
- [En savoir plus](https://www.byteplus.com/en/activity/codingplan?utm_campaign=deer_flow&utm_content=deer_flow&utm_medium=devrel&utm_source=OWO&utm_term=deer_flow)
- [Développeurs en Chine continentale, cliquez ici](https://www.volcengine.com/activity/codingplan?utm_campaign=deer_flow&utm_content=deer_flow&utm_medium=devrel&utm_source=OWO&utm_term=deer_flow)
## InfoQuest
DeerFlow intègre désormais le toolkit de recherche et de crawling intelligent développé par BytePlus — [InfoQuest (essai gratuit en ligne)](https://docs.byteplus.com/en/docs/InfoQuest/What_is_Info_Quest)
<a href="https://docs.byteplus.com/en/docs/InfoQuest/What_is_Info_Quest" target="_blank">
<img
src="https://sf16-sg.tiktokcdn.com/obj/eden-sg/hubseh7bsbps/20251208-160108.png" alt="InfoQuest_banner"
/>
</a>
---
## Table des matières
- [🦌 DeerFlow - 2.0](#-deerflow---20)
- [Site officiel](#site-officiel)
- [InfoQuest](#infoquest)
- [Table des matières](#table-des-matières)
- [Installation en une phrase pour un coding agent](#installation-en-une-phrase-pour-un-coding-agent)
- [Démarrage rapide](#démarrage-rapide)
- [Configuration](#configuration)
- [Lancer l'application](#lancer-lapplication)
- [Option 1 : Docker (recommandé)](#option-1--docker-recommandé)
- [Option 2 : Développement local](#option-2--développement-local)
- [Avancé](#avancé)
- [Mode Sandbox](#mode-sandbox)
- [Serveur MCP](#serveur-mcp)
- [Canaux de messagerie](#canaux-de-messagerie)
- [Traçage LangSmith](#traçage-langsmith)
- [Du Deep Research au Super Agent Harness](#du-deep-research-au-super-agent-harness)
- [Fonctionnalités principales](#fonctionnalités-principales)
- [Skills et outils](#skills-et-outils)
- [Intégration Claude Code](#intégration-claude-code)
- [Sub-Agents](#sub-agents)
- [Sandbox et système de fichiers](#sandbox-et-système-de-fichiers)
- [Context Engineering](#context-engineering)
- [Mémoire à long terme](#mémoire-à-long-terme)
- [Modèles recommandés](#modèles-recommandés)
- [Client Python intégré](#client-python-intégré)
- [Documentation](#documentation)
- [⚠️ Avertissement de sécurité](#-avertissement-de-sécurité)
- [Contribuer](#contribuer)
- [Licence](#licence)
- [Remerciements](#remerciements)
- [Contributeurs principaux](#contributeurs-principaux)
- [Star History](#star-history)
## Installation en une phrase pour un coding agent
Si vous utilisez Claude Code, Codex, Cursor, Windsurf ou un autre coding agent, vous pouvez simplement lui envoyer cette phrase :
```text
Aide-moi à cloner DeerFlow si nécessaire, puis à initialiser son environnement de développement local en suivant https://raw.githubusercontent.com/bytedance/deer-flow/main/Install.md
```
Ce prompt est destiné aux coding agents. Il leur demande de cloner le dépôt si nécessaire, de privilégier Docker quand il est disponible, puis de s'arrêter avec la commande exacte pour lancer DeerFlow et la liste des configurations encore manquantes.
## Démarrage rapide
### Configuration
1. **Cloner le dépôt DeerFlow**
```bash
git clone https://github.com/bytedance/deer-flow.git
cd deer-flow
```
2. **Générer les fichiers de configuration locaux**
Depuis le répertoire racine du projet (`deer-flow/`), exécutez :
```bash
make config
```
Cette commande crée les fichiers de configuration locaux à partir des templates fournis.
3. **Configurer le(s) modèle(s) de votre choix**
Éditez `config.yaml` et définissez au moins un modèle :
```yaml
models:
- name: gpt-4 # Internal identifier
display_name: GPT-4 # Human-readable name
use: langchain_openai:ChatOpenAI # LangChain class path
model: gpt-4 # Model identifier for API
api_key: $OPENAI_API_KEY # API key (recommended: use env var)
max_tokens: 4096 # Maximum tokens per request
temperature: 0.7 # Sampling temperature
- name: openrouter-gemini-2.5-flash
display_name: Gemini 2.5 Flash (OpenRouter)
use: langchain_openai:ChatOpenAI
model: google/gemini-2.5-flash-preview
api_key: $OPENAI_API_KEY # OpenRouter still uses the OpenAI-compatible field name here
base_url: https://openrouter.ai/api/v1
- name: gpt-5-responses
display_name: GPT-5 (Responses API)
use: langchain_openai:ChatOpenAI
model: gpt-5
api_key: $OPENAI_API_KEY
use_responses_api: true
output_version: responses/v1
```
OpenRouter et les passerelles compatibles OpenAI similaires doivent être configurés avec `langchain_openai:ChatOpenAI` et `base_url`. Si vous préférez utiliser un nom de variable d'environnement propre au fournisseur, pointez `api_key` vers cette variable explicitement (par exemple `api_key: $OPENROUTER_API_KEY`).
Pour router les modèles OpenAI via `/v1/responses`, continuez d'utiliser `langchain_openai:ChatOpenAI` et définissez `use_responses_api: true` avec `output_version: responses/v1`.
Exemples de providers basés sur un CLI :
```yaml
models:
- name: gpt-5.4
display_name: GPT-5.4 (Codex CLI)
use: deerflow.models.openai_codex_provider:CodexChatModel
model: gpt-5.4
supports_thinking: true
supports_reasoning_effort: true
- name: claude-sonnet-4.6
display_name: Claude Sonnet 4.6 (Claude Code OAuth)
use: deerflow.models.claude_provider:ClaudeChatModel
model: claude-sonnet-4-6
max_tokens: 4096
supports_thinking: true
```
- Codex CLI lit `~/.codex/auth.json`
- L'endpoint Responses de Codex rejette actuellement `max_tokens` et `max_output_tokens`, donc `CodexChatModel` n'expose pas de limite de tokens par requête
- Claude Code accepte `CLAUDE_CODE_OAUTH_TOKEN`, `ANTHROPIC_AUTH_TOKEN`, `CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR`, `CLAUDE_CODE_CREDENTIALS_PATH`, ou en clair `~/.claude/.credentials.json`
- Sur macOS, DeerFlow ne sonde pas le Keychain automatiquement. Exportez l'auth Claude Code explicitement si nécessaire :
```bash
eval "$(python3 scripts/export_claude_code_oauth.py --print-export)"
```
4. **Définir les clés API pour le(s) modèle(s) configuré(s)**
Choisissez l'une des méthodes suivantes :
- Option A : Éditer le fichier `.env` à la racine du projet (recommandé)
```bash
TAVILY_API_KEY=your-tavily-api-key
OPENAI_API_KEY=your-openai-api-key
# OpenRouter also uses OPENAI_API_KEY when your config uses langchain_openai:ChatOpenAI + base_url.
# Add other provider keys as needed
INFOQUEST_API_KEY=your-infoquest-api-key
```
- Option B : Exporter les variables d'environnement dans votre shell
```bash
export OPENAI_API_KEY=your-openai-api-key
```
Pour les providers basés sur un CLI :
- Codex CLI : `~/.codex/auth.json`
- Claude Code OAuth : handoff explicite via env/fichier ou `~/.claude/.credentials.json`
- Option C : Éditer `config.yaml` directement (non recommandé en production)
```yaml
models:
- name: gpt-4
api_key: your-actual-api-key-here # Replace placeholder
```
### Lancer l'application
#### Option 1 : Docker (recommandé)
**Développement** (hot-reload, montage des sources) :
```bash
make docker-init # Pull sandbox image (only once or when image updates)
make docker-start # Start services (auto-detects sandbox mode from config.yaml)
```
`make docker-start` ne lance `provisioner` que si `config.yaml` utilise le mode provisioner (`sandbox.use: deerflow.community.aio_sandbox:AioSandboxProvider` avec `provisioner_url`).
Les processus backend récupèrent automatiquement les changements dans `config.yaml` au prochain accès à la configuration, donc les mises à jour de métadonnées des modèles ne nécessitent pas de redémarrage manuel en développement.
> [!TIP]
> Sous Linux, si les commandes Docker échouent avec `permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock`, ajoutez votre utilisateur au groupe `docker` et reconnectez-vous avant de réessayer. Voir [CONTRIBUTING.md](CONTRIBUTING.md#linux-docker-daemon-permission-denied) pour la solution complète.
**Production** (build des images en local, montage de la config et des données) :
```bash
make up # Build images and start all production services
make down # Stop and remove containers
```
> [!NOTE]
> Le serveur d'agents LangGraph fonctionne actuellement via `langgraph dev` (le serveur CLI open source).
Accès : http://localhost:2026
Voir [CONTRIBUTING.md](CONTRIBUTING.md) pour le guide complet de développement avec Docker.
#### Option 2 : Développement local
Si vous préférez lancer les services en local :
Prérequis : complétez d'abord les étapes de « Configuration » ci-dessus (`make config` et clés API des modèles). `make dev` nécessite un fichier de configuration valide (par défaut `config.yaml` à la racine du projet ; modifiable via `DEER_FLOW_CONFIG_PATH`).
1. **Vérifier les prérequis** :
```bash
make check # Verifies Node.js 22+, pnpm, uv, nginx
```
2. **Installer les dépendances** :
```bash
make install # Install backend + frontend dependencies
```
3. **(Optionnel) Pré-télécharger l'image sandbox** :
```bash
# Recommended if using Docker/Container-based sandbox
make setup-sandbox
```
4. **Démarrer les services** :
```bash
make dev
```
5. **Accès** : http://localhost:2026
### Avancé
#### Mode Sandbox
DeerFlow supporte plusieurs modes d'exécution sandbox :
- **Exécution locale** (exécute le code sandbox directement sur la machine hôte)
- **Exécution Docker** (exécute le code sandbox dans des conteneurs Docker isolés)
- **Exécution Docker avec Kubernetes** (exécute le code sandbox dans des pods Kubernetes via le service provisioner)
En développement Docker, le démarrage des services suit le mode sandbox défini dans `config.yaml`. En mode Local/Docker, `provisioner` n'est pas démarré.
Voir le [Guide de configuration Sandbox](backend/docs/CONFIGURATION.md#sandbox) pour configurer le mode de votre choix.
#### Serveur MCP
DeerFlow supporte des serveurs MCP et des skills configurables pour étendre ses capacités.
Pour les serveurs MCP HTTP/SSE, les flux de tokens OAuth sont supportés (`client_credentials`, `refresh_token`).
Voir le [Guide MCP Server](backend/docs/MCP_SERVER.md) pour les instructions détaillées.
#### Canaux de messagerie
DeerFlow peut recevoir des tâches depuis des applications de messagerie. Les canaux démarrent automatiquement une fois configurés — aucune IP publique n'est requise.
| Canal | Transport | Difficulté |
|---------|-----------|------------|
| Telegram | Bot API (long-polling) | Facile |
| Slack | Socket Mode | Modérée |
| Feishu / Lark | WebSocket | Modérée |
**Configuration dans `config.yaml` :**
```yaml
channels:
# LangGraph Server URL (default: http://localhost:2024)
langgraph_url: http://localhost:2024
# Gateway API URL (default: http://localhost:8001)
gateway_url: http://localhost:8001
# Optional: global session defaults for all mobile channels
session:
assistant_id: lead_agent
config:
recursion_limit: 100
context:
thinking_enabled: true
is_plan_mode: false
subagent_enabled: false
feishu:
enabled: true
app_id: $FEISHU_APP_ID
app_secret: $FEISHU_APP_SECRET
# domain: https://open.feishu.cn # China (default)
# domain: https://open.larksuite.com # International
slack:
enabled: true
bot_token: $SLACK_BOT_TOKEN # xoxb-...
app_token: $SLACK_APP_TOKEN # xapp-... (Socket Mode)
allowed_users: [] # empty = allow all
telegram:
enabled: true
bot_token: $TELEGRAM_BOT_TOKEN
allowed_users: [] # empty = allow all
# Optional: per-channel / per-user session settings
session:
assistant_id: mobile_agent
context:
thinking_enabled: false
users:
"123456789":
assistant_id: vip_agent
config:
recursion_limit: 150
context:
thinking_enabled: true
subagent_enabled: true
```
Définissez les clés API correspondantes dans votre fichier `.env` :
```bash
# Telegram
TELEGRAM_BOT_TOKEN=123456789:ABCdefGHIjklMNOpqrSTUvwxYZ
# Slack
SLACK_BOT_TOKEN=xoxb-...
SLACK_APP_TOKEN=xapp-...
# Feishu / Lark
FEISHU_APP_ID=cli_xxxx
FEISHU_APP_SECRET=your_app_secret
```
**Configuration Telegram**
1. Ouvrez une conversation avec [@BotFather](https://t.me/BotFather), envoyez `/newbot`, et copiez le token HTTP API.
2. Définissez `TELEGRAM_BOT_TOKEN` dans `.env` et activez le canal dans `config.yaml`.
**Configuration Slack**
1. Créez une Slack App sur [api.slack.com/apps](https://api.slack.com/apps) → Create New App → From scratch.
2. Dans **OAuth & Permissions**, ajoutez les Bot Token Scopes : `app_mentions:read`, `chat:write`, `im:history`, `im:read`, `im:write`, `files:write`.
3. Activez le **Socket Mode** → générez un App-Level Token (`xapp-…`) avec le scope `connections:write`.
4. Dans **Event Subscriptions**, abonnez-vous aux bot events : `app_mention`, `message.im`.
5. Définissez `SLACK_BOT_TOKEN` et `SLACK_APP_TOKEN` dans `.env` et activez le canal dans `config.yaml`.
**Configuration Feishu / Lark**
1. Créez une application sur [Feishu Open Platform](https://open.feishu.cn/) → activez la capacité **Bot**.
2. Ajoutez les permissions : `im:message`, `im:message.p2p_msg:readonly`, `im:resource`.
3. Dans **Events**, abonnez-vous à `im.message.receive_v1` et sélectionnez le mode **Long Connection**.
4. Copiez l'App ID et l'App Secret. Définissez `FEISHU_APP_ID` et `FEISHU_APP_SECRET` dans `.env` et activez le canal dans `config.yaml`.
**Commandes**
Une fois un canal connecté, vous pouvez interagir avec DeerFlow directement depuis le chat :
| Commande | Description |
|---------|-------------|
| `/new` | Démarrer une nouvelle conversation |
| `/status` | Afficher les infos du thread en cours |
| `/models` | Lister les modèles disponibles |
| `/memory` | Consulter la mémoire |
| `/help` | Afficher l'aide |
> Les messages sans préfixe de commande sont traités comme du chat classique — DeerFlow crée un thread et répond de manière conversationnelle.
#### Traçage LangSmith
DeerFlow intègre nativement [LangSmith](https://smith.langchain.com) pour l'observabilité. Une fois activé, tous les appels LLM, les exécutions d'agents et les exécutions d'outils sont tracés et visibles dans le tableau de bord LangSmith.
Ajoutez les lignes suivantes à votre fichier `.env` :
```bash
LANGSMITH_TRACING=true
LANGSMITH_ENDPOINT=https://api.smith.langchain.com
LANGSMITH_API_KEY=lsv2_pt_xxxxxxxxxxxxxxxx
LANGSMITH_PROJECT=xxx
```
Pour les déploiements Docker, le traçage est désactivé par défaut. Définissez `LANGSMITH_TRACING=true` et `LANGSMITH_API_KEY` dans votre `.env` pour l'activer.
## Du Deep Research au Super Agent Harness
DeerFlow a démarré comme un framework de Deep Research — et la communauté s'en est emparée. Depuis le lancement, les développeurs l'ont poussé bien au-delà de la recherche : construction de pipelines de données, génération de présentations, mise en place de dashboards, automatisation de workflows de contenu. Des usages qu'on n'avait jamais anticipés.
Ça nous a révélé quelque chose d'important : DeerFlow n'était pas qu'un simple outil de recherche. C'était un **harness** — un runtime qui donne aux agents l'infrastructure nécessaire pour vraiment accomplir du travail.
On l'a donc reconstruit de zéro.
DeerFlow 2.0 n'est plus un framework à assembler soi-même. C'est un super agent harness — clé en main et entièrement extensible. Construit sur LangGraph et LangChain, il embarque tout ce dont un agent a besoin out of the box : un système de fichiers, de la mémoire, des skills, une exécution sandboxée, et la capacité de planifier et de lancer des sub-agents pour les tâches complexes et multi-étapes.
Utilisez-le tel quel. Ou démontez-le et faites-en le vôtre.
## Fonctionnalités principales
### Skills et outils
Les skills sont ce qui permet à DeerFlow de faire *pratiquement n'importe quoi*.
Un Agent Skill standard est un module de capacité structuré — un fichier Markdown qui définit un workflow, des bonnes pratiques et des références vers des ressources associées. DeerFlow est livré avec des skills intégrés pour la recherche, la génération de rapports, la création de présentations, les pages web, la génération d'images et de vidéos, et bien plus. Mais la vraie force réside dans l'extensibilité : ajoutez vos propres skills, remplacez ceux fournis, ou combinez-les en workflows composites.
Les skills sont chargés progressivement — uniquement quand la tâche le nécessite, pas tous en même temps. Ça permet de garder la fenêtre de contexte légère et de bien fonctionner même avec des modèles sensibles au nombre de tokens.
Quand vous installez des archives `.skill` via le Gateway, DeerFlow accepte les métadonnées frontmatter optionnelles standard comme `version`, `author` et `compatibility`, plutôt que de rejeter des skills externes par ailleurs valides.
Les outils suivent la même philosophie. DeerFlow est livré avec un ensemble d'outils de base — recherche web, fetch de pages web, opérations sur les fichiers, exécution bash — et supporte les outils custom via des serveurs MCP et des fonctions Python. Remplacez n'importe quoi. Ajoutez n'importe quoi.
Les suggestions de suivi générées par le Gateway normalisent désormais aussi bien la sortie texte brut du modèle que le contenu riche au format bloc/liste avant de parser la réponse en tableau JSON, de sorte que les wrappers de contenu propres à chaque provider ne suppriment plus silencieusement les suggestions.
```
# Paths inside the sandbox container
/mnt/skills/public
├── research/SKILL.md
├── report-generation/SKILL.md
├── slide-creation/SKILL.md
├── web-page/SKILL.md
└── image-generation/SKILL.md
/mnt/skills/custom
└── your-custom-skill/SKILL.md ← yours
```
#### Intégration Claude Code
Le skill `claude-to-deerflow` vous permet d'interagir avec une instance DeerFlow en cours d'exécution directement depuis [Claude Code](https://docs.anthropic.com/en/docs/claude-code). Envoyez des tâches de recherche, vérifiez le statut, gérez les threads — le tout sans quitter le terminal.
**Installer le skill** :
```bash
npx skills add https://github.com/bytedance/deer-flow --skill claude-to-deerflow
```
Assurez-vous ensuite que DeerFlow tourne (par défaut sur `http://localhost:2026`) et utilisez la commande `/claude-to-deerflow` dans Claude Code.
**Ce que vous pouvez faire** :
- Envoyer des messages à DeerFlow et recevoir des réponses en streaming
- Choisir le mode d'exécution : flash (rapide), standard, pro (planification), ultra (sub-agents)
- Vérifier la santé de DeerFlow, lister les modèles/skills/agents
- Gérer les threads et l'historique des conversations
- Upload des fichiers pour analyse
**Variables d'environnement** (optionnel, pour des endpoints custom) :
```bash
DEERFLOW_URL=http://localhost:2026 # Unified proxy base URL
DEERFLOW_GATEWAY_URL=http://localhost:2026 # Gateway API
DEERFLOW_LANGGRAPH_URL=http://localhost:2026/api/langgraph # LangGraph API
```
Voir [`skills/public/claude-to-deerflow/SKILL.md`](skills/public/claude-to-deerflow/SKILL.md) pour la référence API complète.
### Sub-Agents
Les tâches complexes tiennent rarement en une seule passe. DeerFlow les décompose.
L'agent principal peut lancer des sub-agents à la volée — chacun avec son propre contexte délimité, ses outils et ses conditions d'arrêt. Les sub-agents s'exécutent en parallèle quand c'est possible, remontent des résultats structurés, et l'agent principal synthétise le tout en une sortie cohérente.
C'est comme ça que DeerFlow gère les tâches qui prennent de quelques minutes à plusieurs heures : une tâche de recherche peut se déployer en une dizaine de sub-agents, chacun explorant un angle différent, puis converger vers un seul rapport — ou un site web — ou un jeu de slides avec des visuels générés. Un seul harness, de nombreuses mains.
### Sandbox et système de fichiers
DeerFlow ne se contente pas de *parler* de faire les choses. Il dispose de son propre ordinateur.
Chaque tâche s'exécute dans un conteneur Docker isolé avec un système de fichiers complet — skills, workspace, uploads, outputs. L'agent lit, écrit et édite des fichiers. Il exécute des commandes bash et du code. Il visualise des images. Le tout sandboxé, le tout auditable, zéro contamination entre les sessions.
C'est la différence entre un chatbot avec accès à des outils et un agent doté d'un véritable environnement d'exécution.
```
# Paths inside the sandbox container
/mnt/user-data/
├── uploads/ ← your files
├── workspace/ ← agents' working directory
└── outputs/ ← final deliverables
```
### Context Engineering
**Contexte isolé des Sub-Agents** : chaque sub-agent s'exécute dans son propre contexte isolé. Il ne peut voir ni le contexte de l'agent principal, ni celui des autres sub-agents. L'objectif est de garantir que chaque sub-agent reste concentré sur sa tâche sans être parasité par des informations non pertinentes.
**Résumé** : au sein d'une session, DeerFlow gère le contexte de manière agressive — en résumant les sous-tâches terminées, en déchargeant les résultats intermédiaires vers le système de fichiers, en compressant ce qui n'est plus immédiatement pertinent. Ça lui permet de rester efficace sur des tâches longues et multi-étapes sans faire exploser la fenêtre de contexte.
### Mémoire à long terme
La plupart des agents oublient tout dès qu'une conversation se termine. DeerFlow, lui, se souvient.
D'une session à l'autre, DeerFlow construit une mémoire persistante de votre profil, de vos préférences et de vos connaissances accumulées. Plus vous l'utilisez, mieux il vous connaît — votre style d'écriture, votre stack technique, vos workflows récurrents. La mémoire est stockée localement et reste sous votre contrôle.
Les mises à jour de la mémoire ignorent désormais les entrées de faits en double au moment de l'application, de sorte que les préférences et le contexte répétés ne s'accumulent plus indéfiniment entre les sessions.
## Modèles recommandés
DeerFlow est agnostique en termes de modèle — il fonctionne avec n'importe quel LLM implémentant l'API compatible OpenAI. Cela dit, il offre de meilleures performances avec des modèles qui supportent :
- **De longues fenêtres de contexte** (100k+ tokens) pour la recherche approfondie et les tâches multi-étapes
- **Des capacités de raisonnement** pour la planification adaptative et la décomposition de tâches complexes
- **Des entrées multimodales** pour la compréhension d'images et de vidéos
- **Un usage fiable des outils (tool use)** pour des appels de fonctions et des sorties structurées fiables
## Client Python intégré
DeerFlow peut être utilisé comme bibliothèque Python intégrée sans lancer l'ensemble des services HTTP. Le `DeerFlowClient` fournit un accès direct in-process à toutes les capacités d'agent et de Gateway, en retournant les mêmes schémas de réponse que l'API HTTP Gateway. Le HTTP Gateway expose également `DELETE /api/threads/{thread_id}` pour supprimer les données de thread locales gérées par DeerFlow après la suppression du thread LangGraph :
```python
from deerflow.client import DeerFlowClient
client = DeerFlowClient()
# Chat
response = client.chat("Analyze this paper for me", thread_id="my-thread")
# Streaming (LangGraph SSE protocol: values, messages-tuple, end)
for event in client.stream("hello"):
if event.type == "messages-tuple" and event.data.get("type") == "ai":
print(event.data["content"])
# Configuration & management — returns Gateway-aligned dicts
models = client.list_models() # {"models": [...]}
skills = client.list_skills() # {"skills": [...]}
client.update_skill("web-search", enabled=True)
client.upload_files("thread-1", ["./report.pdf"]) # {"success": True, "files": [...]}
```
Toutes les méthodes retournant des dicts sont validées en CI contre les modèles de réponse Pydantic du Gateway (`TestGatewayConformance`), garantissant que le client intégré reste synchronisé avec les schémas de l'API HTTP. Voir `backend/packages/harness/deerflow/client.py` pour la documentation API complète.
## Documentation
- [Guide de contribution](CONTRIBUTING.md) - Mise en place de l'environnement de développement et workflow
- [Guide de configuration](backend/docs/CONFIGURATION.md) - Instructions d'installation et de configuration
- [Vue d'ensemble de l'architecture](backend/CLAUDE.md) - Détails de l'architecture technique
- [Architecture backend](backend/README.md) - Architecture backend et référence API
## ⚠️ Avertissement de sécurité
### Un déploiement inapproprié peut introduire des risques de sécurité
DeerFlow dispose de capacités clés à hauts privilèges, notamment **l'exécution de commandes système, les opérations sur les ressources et l'invocation de logique métier**. Il est conçu par défaut pour être **déployé dans un environnement local de confiance (accessible uniquement via l'interface de loopback 127.0.0.1)**. Si vous déployez l'agent dans des environnements non fiables — tels que des réseaux LAN, des serveurs cloud publics ou d'autres environnements accessibles depuis plusieurs terminaux — sans mesures de sécurité strictes, cela peut introduire des risques, notamment :
- **Invocation non autorisée** : les fonctionnalités de l'agent pourraient être découvertes par des tiers non autorisés ou des scanners malveillants, déclenchant des requêtes non autorisées en masse qui exécutent des opérations à haut risque (commandes système, lecture/écriture de fichiers), pouvant causer de graves conséquences.
- **Risques juridiques et de conformité** : si l'agent est utilisé illégalement pour mener des cyberattaques, du vol de données ou d'autres activités illicites, cela peut entraîner des responsabilités juridiques et des risques de conformité.
### Recommandations de sécurité
**Note : nous recommandons fortement de déployer DeerFlow dans un environnement réseau local de confiance.** Si vous avez besoin d'un déploiement multi-appareils ou multi-réseaux, vous devez mettre en place des mesures de sécurité strictes, par exemple :
- **Liste blanche d'IP** : utilisez `iptables`, ou déployez des pare-feux matériels / commutateurs avec ACL, pour **configurer des règles de liste blanche d'IP** et refuser l'accès à toutes les autres adresses IP.
- **Passerelle d'authentification** : configurez un proxy inverse (ex. nginx) et **activez une authentification forte en amont**, bloquant tout accès non authentifié.
- **Isolation réseau** : si possible, placez l'agent et les appareils de confiance dans le **même VLAN dédié**, isolé des autres équipements réseau.
- **Restez informé** : continuez à suivre les mises à jour de sécurité du projet DeerFlow.
## Contribuer
Les contributions sont les bienvenues ! Consultez [CONTRIBUTING.md](CONTRIBUTING.md) pour la mise en place de l'environnement de développement, le workflow et les conventions.
La couverture de tests de régression inclut la détection du mode sandbox Docker et les tests de gestion du kubeconfig-path du provisioner dans `backend/tests/`.
## Licence
Ce projet est open source et disponible sous la [Licence MIT](./LICENSE).
## Remerciements
DeerFlow est construit sur le travail remarquable de la communauté open source. Nous sommes profondément reconnaissants envers tous les projets et contributeurs dont les efforts ont rendu DeerFlow possible. Nous nous tenons véritablement sur les épaules de géants.
Nous tenons à exprimer notre sincère gratitude aux projets suivants pour leurs contributions inestimables :
- **[LangChain](https://github.com/langchain-ai/langchain)** : leur excellent framework propulse nos interactions LLM et nos chaînes, permettant une intégration et des fonctionnalités fluides.
- **[LangGraph](https://github.com/langchain-ai/langgraph)** : leur approche innovante de l'orchestration multi-agents a été déterminante pour les workflows sophistiqués de DeerFlow.
Ces projets illustrent le pouvoir transformateur de la collaboration open source, et nous sommes fiers de bâtir sur leurs fondations.
### Contributeurs principaux
Un grand merci aux auteurs principaux de `DeerFlow`, dont la vision, la passion et le dévouement ont donné vie à ce projet :
- **[Daniel Walnut](https://github.com/hetaoBackend/)**
- **[Henry Li](https://github.com/magiccube/)**
Votre engagement sans faille et votre expertise sont le moteur du succès de DeerFlow. Nous sommes honorés de vous avoir à la barre de cette aventure.
## Star History
[![Star History Chart](https://api.star-history.com/svg?repos=bytedance/deer-flow&type=Date)](https://star-history.com/#bytedance/deer-flow&Date)
+441 -502
View File
File diff suppressed because it is too large Load Diff
-593
View File
@@ -1,593 +0,0 @@
# 🦌 DeerFlow
[![Python 3.12+](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![DeepWiki](https://img.shields.io/badge/DeepWiki-bytedance%2Fdeer--flow-blue.svg?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACwAAAAyCAYAAAAnWDnqAAAAAXNSR0IArs4c6QAAA05JREFUaEPtmUtyEzEQhtWTQyQLHNak2AB7ZnyXZMEjXMGeK/AIi+QuHrMnbChYY7MIh8g01fJoopFb0uhhEqqcbWTp06/uv1saEDv4O3n3dV60RfP947Mm9/SQc0ICFQgzfc4CYZoTPAswgSJCCUJUnAAoRHOAUOcATwbmVLWdGoH//PB8mnKqScAhsD0kYP3j/Yt5LPQe2KvcXmGvRHcDnpxfL2zOYJ1mFwrryWTz0advv1Ut4CJgf5uhDuDj5eUcAUoahrdY/56ebRWeraTjMt/00Sh3UDtjgHtQNHwcRGOC98BJEAEymycmYcWwOprTgcB6VZ5JK5TAJ+fXGLBm3FDAmn6oPPjR4rKCAoJCal2eAiQp2x0vxTPB3ALO2CRkwmDy5WohzBDwSEFKRwPbknEggCPB/imwrycgxX2NzoMCHhPkDwqYMr9tRcP5qNrMZHkVnOjRMWwLCcr8ohBVb1OMjxLwGCvjTikrsBOiA6fNyCrm8V1rP93iVPpwaE+gO0SsWmPiXB+jikdf6SizrT5qKasx5j8ABbHpFTx+vFXp9EnYQmLx02h1QTTrl6eDqxLnGjporxl3NL3agEvXdT0WmEost648sQOYAeJS9Q7bfUVoMGnjo4AZdUMQku50McDcMWcBPvr0SzbTAFDfvJqwLzgxwATnCgnp4wDl6Aa+Ax283gghmj+vj7feE2KBBRMW3FzOpLOADl0Isb5587h/U4gGvkt5v60Z1VLG8BhYjbzRwyQZemwAd6cCR5/XFWLYZRIMpX39AR0tjaGGiGzLVyhse5C9RKC6ai42ppWPKiBagOvaYk8lO7DajerabOZP46Lby5wKjw1HCRx7p9sVMOWGzb/vA1hwiWc6jm3MvQDTogQkiqIhJV0nBQBTU+3okKCFDy9WwferkHjtxib7t3xIUQtHxnIwtx4mpg26/HfwVNVDb4oI9RHmx5WGelRVlrtiw43zboCLaxv46AZeB3IlTkwouebTr1y2NjSpHz68WNFjHvupy3q8TFn3Hos2IAk4Ju5dCo8B3wP7VPr/FGaKiG+T+v+TQqIrOqMTL1VdWV1DdmcbO8KXBz6esmYWYKPwDL5b5FA1a0hwapHiom0r/cKaoqr+27/XcrS5UwSMbQAAAABJRU5ErkJggg==)](https://deepwiki.com/bytedance/deer-flow)
<!-- DeepWiki badge generated by https://deepwiki.ryoppippi.com/ -->
[English](./README.md) | [简体中文](./README_zh.md) | [日本語](./README_ja.md) | [Deutsch](./README_de.md) | [Español](./README_es.md) | [Русский](./README_ru.md) | [Portuguese](./README_pt.md)
> Originado do Open Source, de volta ao Open Source
**DeerFlow** (**D**eep **E**xploration and **E**fficient **R**esearch **Flow**) é um framework de Pesquisa Profunda orientado-a-comunidade que baseia-se em um íncrivel trabalho da comunidade open source. Nosso objetivo é combinar modelos de linguagem com ferramentas especializadas para tarefas como busca na web, crawling, e execução de código Python, enquanto retribui com a comunidade que o tornou possível.
Atualmente, o DeerFlow entrou oficialmente no Centro de Aplicações FaaS da Volcengine. Os usuários podem experimentá-lo online através do link de experiência para sentir intuitivamente suas funções poderosas e operações convenientes. Ao mesmo tempo, para atender às necessidades de implantação de diferentes usuários, o DeerFlow suporta implantação com um clique baseada na Volcengine. Clique no link de implantação para completar rapidamente o processo de implantação e iniciar uma jornada de pesquisa eficiente.
O DeerFlow recentemente integrou o conjunto de ferramentas de busca e rastreamento inteligente desenvolvido independentemente pela BytePlus — [InfoQuest (oferece experiência gratuita online)](https://docs.byteplus.com/en/docs/InfoQuest/What_is_Info_Quest)
<a href="https://docs.byteplus.com/en/docs/InfoQuest/What_is_Info_Quest" target="_blank">
<img
src="https://sf16-sg.tiktokcdn.com/obj/eden-sg/hubseh7bsbps/20251208-160108.png" alt="infoquest_bannar"
/>
</a>
Por favor, visite [Nosso Site Oficial](https://deerflow.tech/) para maiores detalhes.
## Demo
### Video
<https://github.com/user-attachments/assets/f3786598-1f2a-4d07-919e-8b99dfa1de3e>
Nesse demo, nós demonstramos como usar o DeerFlow para:
In this demo, we showcase how to use DeerFlow to:
- Integração fácil com serviços MCP
- Conduzir o processo de Pesquisa Profunda e produzir um relatório abrangente com imagens
- Criar um áudio podcast baseado no relatório gerado
### Replays
- [Quão alta é a Torre Eiffel comparada ao prédio mais alto?](https://deerflow.tech/chat?replay=eiffel-tower-vs-tallest-building)
- [Quais são os top repositórios tendência no GitHub?](https://deerflow.tech/chat?replay=github-top-trending-repo)
- [Escreva um artigo sobre os pratos tradicionais de Nanjing's](https://deerflow.tech/chat?replay=nanjing-traditional-dishes)
- [Como decorar um apartamento alugado?](https://deerflow.tech/chat?replay=rental-apartment-decoration)
- [Visite nosso site oficial para explorar mais replays.](https://deerflow.tech/#case-studies)
---
## 📑 Tabela de Conteúdos
- [🚀 Início Rápido](#Início-Rápido)
- [🌟 Funcionalidades](#funcionalidades)
- [🏗️ Arquitetura](#arquitetura)
- [🛠️ Desenvolvimento](#desenvolvimento)
- [🐳 Docker](#docker)
- [🗣️ Texto-para-fala Integração](#texto-para-fala-integração)
- [📚 Exemplos](#exemplos)
- [❓ FAQ](#faq)
- [📜 Licença](#licença)
- [💖 Agradecimentos](#agradecimentos)
- [🏆 Contribuidores-Chave](#contribuidores-chave)
- [⭐ Histórico de Estrelas](#Histórico-Estrelas)
## Início-Rápido
DeerFlow é desenvolvido em Python, e vem com uma IU web escrita em Node.js. Para garantir um processo de configuração fácil, nós recomendamos o uso das seguintes ferramentas:
### Ferramentas Recomendadas
- **[`uv`](https://docs.astral.sh/uv/getting-started/installation/):**
Simplifica o gerenciamento de dependência de ambientes Python. `uv` automaticamente cria um ambiente virtual no diretório raiz e instala todos os pacotes necessários para não haver a necessidade de instalar ambientes Python manualmente
- **[`nvm`](https://github.com/nvm-sh/nvm):**
Gerencia múltiplas versões do ambiente de execução do Node.js sem esforço.
- **[`pnpm`](https://pnpm.io/installation):**
Instala e gerencia dependências do projeto Node.js.
### Requisitos de Ambiente
Certifique-se de que seu sistema atenda os seguintes requisitos mínimos:
- **[Python](https://www.python.org/downloads/):** Versão `3.12+`
- **[Node.js](https://nodejs.org/en/download/):** Versão `22+`
### Instalação
```bash
# Clone o repositório
git clone https://github.com/bytedance/deer-flow.git
cd deer-flow
# Instale as dependências, uv irá lidar com o interpretador do python e a criação do venv, e instalar os pacotes necessários
uv sync
# Configure .env com suas chaves de API
# Tavily: https://app.tavily.com/home
# Brave_SEARCH: https://brave.com/search/api/
# volcengine TTS: Adicione sua credencial TTS caso você a possua
cp .env.example .env
# Veja as seções abaixo 'Supported Search Engines' and 'Texto-para-Fala Integração' para todas as opções disponíveis
# Configure o conf.yaml para o seu modelo LLM e chaves API
# Por favor, consulte 'docs/configuration_guide.md' para maiores detalhes
cp conf.yaml.example conf.yaml
# Instale marp para geração de ppt
# https://github.com/marp-team/marp-cli?tab=readme-ov-file#use-package-manager
brew install marp-cli
```
Opcionalmente, instale as dependências IU web via [pnpm](https://pnpm.io/installation):
```bash
cd deer-flow/web
pnpm install
```
### Configurações
Por favor, consulte o [Guia de Configuração](docs/configuration_guide.md) para maiores detalhes.
> [!NOTA]
> Antes de iniciar o projeto, leia o guia detalhadamente, e atualize as configurações para baterem com os seus requisitos e configurações específicas.
### Console IU
A maneira mais rápida de rodar o projeto é usar o console IU.
```bash
# Execute o projeto em um shell tipo-bash
uv run main.py
```
### Web IU
Esse projeto também inclui uma IU Web, trazendo uma experiência mais interativa, dinâmica e engajadora.
> [!NOTA]
> Você precisa instalar as dependências do IU web primeiro.
```bash
# Execute ambos os servidores de backend e frontend em modo desenvolvimento
# No macOS/Linux
./bootstrap.sh -d
# No Windows
bootstrap.bat -d
```
> [!NOTA]
> Por padrão, o servidor backend se vincula a 127.0.0.1 (localhost) por motivos de segurança. Se você precisar permitir conexões externas (por exemplo, ao implantar em um servidor Linux), poderá modificar o host do servidor para 0.0.0.0 no script de inicialização (uv run server.py --host 0.0.0.0).
> Certifique-se de que seu ambiente esteja devidamente protegido antes de expor o serviço a redes externas.
Abra seu navegador e visite [`http://localhost:3000`](http://localhost:3000) para explorar a IU web.
Explore mais detalhes no diretório [`web`](./web/) .
## Mecanismos de Busca Suportados
DeerFlow suporta múltiplos mecanismos de busca que podem ser configurados no seu arquivo `.env` usando a variável `SEARCH_API`:
- **Tavily** (padrão): Uma API de busca especializada para aplicações de IA
- Requer `TAVILY_API_KEY` no seu arquivo `.env`
- Inscreva-se em: <https://app.tavily.com/home>
- **InfoQuest** (recomendado): Um conjunto de ferramentas inteligentes de busca e crawling otimizadas para IA, desenvolvido pela BytePlus
- Requer `INFOQUEST_API_KEY` no seu arquivo `.env`
- Suporte para filtragem por intervalo de tempo e filtragem de sites
- Fornece resultados de busca e extração de conteúdo de alta qualidade
- Inscreva-se em: <https://console.byteplus.com/infoquest/infoquests>
- Visite https://docs.byteplus.com/pt/docs/InfoQuest/What_is_Info_Quest para obter mais informações
- **DuckDuckGo**: Mecanismo de busca focado em privacidade
- Não requer chave API
- **Brave Search**: Mecanismo de busca focado em privacidade com funcionalidades avançadas
- Requer `BRAVE_SEARCH_API_KEY` no seu arquivo `.env`
- Inscreva-se em: <https://brave.com/search/api/>
- **Arxiv**: Busca de artigos científicos para pesquisa acadêmica
- Não requer chave API
- Especializado em artigos científicos e acadêmicos
- **Searx/SearxNG**: Mecanismo de metabusca auto-hospedado
- Requer `SEARX_HOST` no seu arquivo `.env`
- Suporta integração com Searx ou SearxNG
Para configurar o seu mecanismo preferido, defina a variável `SEARCH_API` no seu arquivo:
```bash
# Escolha uma: tavily, infoquest, duckduckgo, brave_search, arxiv
SEARCH_API=tavily
```
### Ferramentas de Crawling
- **Jina** (padrão): Ferramenta gratuita de crawling de conteúdo web acessível
- Não é necessária chave API para usar recursos básicos
- Ao usar uma chave API, você obtém limites de taxa de acesso mais altos
- Visite <https://jina.ai/reader> para obter mais informações
- **InfoQuest** (recomendado): Conjunto de ferramentas inteligentes de busca e crawling otimizadas para IA, desenvolvido pela BytePlus
- Requer `INFOQUEST_API_KEY` no seu arquivo `.env`
- Fornece parâmetros de crawling configuráveis
- Suporta configurações de timeout personalizadas
- Oferece capacidades mais poderosas de extração de conteúdo
- Visite <https://docs.byteplus.com/pt/docs/InfoQuest/What_is_Info_Quest> para obter mais informações
Para configurar sua ferramenta de crawling preferida, defina o seguinte em seu arquivo `conf.yaml`:
```yaml
CRAWLER_ENGINE:
# Tipo de mecanismo: "jina" (padrão) ou "infoquest"
engine: infoquest
```
## Funcionalidades
### Principais Funcionalidades
- 🤖 **Integração LLM**
- Suporta a integração da maioria dos modelos através de [litellm](https://docs.litellm.ai/docs/providers).
- Suporte a modelos open source como Qwen
- Interface API compatível com a OpenAI
- Sistema LLM multicamadas para diferentes complexidades de tarefa
### Ferramentas e Integrações MCP
- 🔍 **Busca e Recuperação**
- Busca web com Tavily, InfoQuest, Brave Search e mais
- Crawling com Jina e InfoQuest
- Extração de Conteúdo avançada
- 🔗 **Integração MCP perfeita**
- Expansão de capacidades de acesso para acesso a domínios privados, grafo de conhecimento, navegação web e mais
- Integração facilitdade de diversas ferramentas de pesquisa e metodologias
### Colaboração Humana
- 🧠 **Humano-no-processo**
- Suporta modificação interativa de planos de pesquisa usando linguagem natural
- Suporta auto-aceite de planos de pesquisa
- 📝 **Relatório Pós-Edição**
- Suporta edição de edição de blocos estilo Notion
- Permite refinamentos de IA, incluindo polimento de IA assistida, encurtamento de frase, e expansão
- Distribuído por [tiptap](https://tiptap.dev/)
### Criação de Conteúdo
- 🎙️ **Geração de Podcast e apresentação**
- Script de geração de podcast e síntese de áudio movido por IA
- Criação automatizada de apresentações PowerPoint simples
- Templates customizáveis para conteúdo personalizado
## Arquitetura
DeerFlow implementa uma arquitetura de sistema multi-agente modular designada para pesquisa e análise de código automatizada. O sistema é construído em LangGraph, possibilitando um fluxo de trabalho flexível baseado-em-estado onde os componentes se comunicam através de um sistema de transmissão de mensagens bem-definido.
![Diagrama de Arquitetura](./assets/architecture.png)
> Veja ao vivo em [deerflow.tech](https://deerflow.tech/#multi-agent-architecture)
O sistema emprega um fluxo de trabalho simplificado com os seguintes componentes:
1. **Coordenador**: O ponto de entrada que gerencia o ciclo de vida do fluxo de trabalho
- Inicia o processo de pesquisa baseado na entrada do usuário
- Delega tarefas so planejador quando apropriado
- Atua como a interface primária entre o usuário e o sistema
2. **Planejador**: Componente estratégico para a decomposição e planejamento
- Analisa objetivos de pesquisa e cria planos de execução estruturados
- Determina se há contexto suficiente disponível ou se mais pesquisa é necessária
- Gerencia o fluxo de pesquisa e decide quando gerar o relatório final
3. **Time de Pesquisa**: Uma coleção de agentes especializados que executam o plano:
- **Pesquisador**: Conduz buscas web e coleta informações utilizando ferramentas como mecanismos de busca web, crawling e mesmo serviços MCP.
- **Programador**: Lida com a análise de código, execução e tarefas técnicas como usar a ferramenta Python REPL.
Cada agente tem acesso à ferramentas específicas otimizadas para seu papel e opera dentro do fluxo de trabalho LangGraph.
4. **Repórter**: Estágio final do processador de estágio para saídas de pesquisa
- Resultados agregados do time de pesquisa
- Processa e estrutura as informações coletadas
- Gera relatórios abrangentes de pesquisas
## Texto-para-Fala Integração
DeerFlow agora inclui uma funcionalidade Texto-para-Fala (TTS) que permite que você converta relatórios de busca para voz. Essa funcionalidade usa o mecanismo de voz da API TTS para gerar áudio de alta qualidade a partir do texto. Funcionalidades como velocidade, volume e tom também são customizáveis.
### Usando a API TTS
Você pode acessar a funcionalidade TTS através do endpoint `/api/tts`:
```bash
# Exemplo de chamada da API usando curl
curl --location 'http://localhost:8000/api/tts' \
--header 'Content-Type: application/json' \
--data '{
"text": "This is a test of the text-to-speech functionality.",
"speed_ratio": 1.0,
"volume_ratio": 1.0,
"pitch_ratio": 1.0
}' \
--output speech.mp3
```
## Desenvolvimento
### Testando
Rode o conjunto de testes:
```bash
# Roda todos os testes
make test
# Roda um arquivo de teste específico
pytest tests/integration/test_workflow.py
# Roda com coverage
make coverage
```
### Qualidade de Código
```bash
# Roda o linting
make lint
# Formata de código
make format
```
### Debugando com o LangGraph Studio
DeerFlow usa LangGraph para sua arquitetura de fluxo de trabalho. Nós podemos usar o LangGraph Studio para debugar e visualizar o fluxo de trabalho em tempo real.
#### Rodando o LangGraph Studio Localmente
DeerFlow inclui um arquivo de configuração `langgraph.json` que define a estrutura do grafo e dependências para o LangGraph Studio. Esse arquivo aponta para o grafo do fluxo de trabalho definido no projeto e automaticamente carrega as variáveis de ambiente do arquivo `.env`.
##### Mac
```bash
# Instala o gerenciador de pacote uv caso você não o possua
curl -LsSf https://astral.sh/uv/install.sh | sh
# Instala as dependências e inicia o servidor LangGraph
uvx --refresh --from "langgraph-cli[inmem]" --with-editable . --python 3.12 langgraph dev --allow-blocking
```
##### Windows / Linux
```bash
# Instala as dependências
pip install -e .
pip install -U "langgraph-cli[inmem]"
# Inicia o servidor LangGraph
langgraph dev
```
Após iniciar o servidor LangGraph, você verá diversas URLs no seu terminal:
- API: <http://127.0.0.1:2024>
- Studio UI: <https://smith.langchain.com/studio/?baseUrl=http://127.0.0.1:2024>
- API Docs: <http://127.0.0.1:2024/docs>
Abra o link do Studio UI no seu navegador para acessar a interface de depuração.
#### Usando o LangGraph Studio
No Studio UI, você pode:
1. Visualizar o grafo do fluxo de trabalho e como seus componentes se conectam
2. Rastrear a execução em tempo-real e ver como os dados fluem através do sistema
3. Inspecionar o estado de cada passo do fluxo de trabalho
4. Depurar problemas ao examinar entradas e saídas de cada componente
5. Coletar feedback durante a fase de planejamento para refinar os planos de pesquisa
Quando você envia um tópico de pesquisa ao Studio UI, você será capaz de ver toda a execução do fluxo de trabalho, incluindo:
- A fase de planejamento onde o plano de pesquisa foi criado
- O processo de feedback onde você pode modificar o plano
- As fases de pesquisa e escrita de cada seção
- A geração do relatório final
## Docker
Você também pode executar esse projeto via Docker.
Primeiro, voce deve ler a [configuração](#configuration) below. Make sure `.env`, `.conf.yaml` files are ready.
Segundo, para fazer o build de sua imagem docker em seu próprio servidor:
```bash
docker build -t deer-flow-api .
```
E por fim, inicie um container docker rodando o servidor web:
```bash
# substitua deer-flow-api-app com seu nome de container preferido
# Inicie o servidor e faça o bind com localhost:8000
docker run -d -t -p 127.0.0.1:8000:8000 --env-file .env --name deer-flow-api-app deer-flow-api
# pare o servidor
docker stop deer-flow-api-app
```
### Docker Compose (inclui ambos backend e frontend)
DeerFlow fornece uma estrutura docker-compose para facilmente executar ambos o backend e frontend juntos:
```bash
# building docker image
docker compose build
# start the server
docker compose up
```
> [!WARNING]
> Se você quiser implantar o DeerFlow em ambientes de produção, adicione autenticação ao site e avalie sua verificação de segurança do MCPServer e Python Repl.
## Exemplos
Os seguintes exemplos demonstram as capacidades do DeerFlow:
### Relatórios de Pesquisa
1. **Relatório OpenAI Sora** - Análise da ferramenta Sora da OpenAI
- Discute funcionalidades, acesso, engenharia de prompt, limitações e considerações éticas
- [Veja o relatório completo](examples/openai_sora_report.md)
2. **Relatório Protocolo Agent-to-Agent do Google** - Visão geral do protocolo Agent-to-Agent (A2A) do Google
- Discute o seu papel na comunicação de Agente de IA e seu relacionamento com o Protocolo de Contexto de Modelo ( MCP ) da Anthropic
- [Veja o relatório completo](examples/what_is_agent_to_agent_protocol.md)
3. **O que é MCP?** - Uma análise abrangente to termo "MCP" através de múltiplos contextos
- Explora o Protocolo de Contexto de Modelo em IA, Fosfato Monocálcio em Química, e placa de microcanal em eletrônica
- [Veja o relatório completo](examples/what_is_mcp.md)
4. **Bitcoin Price Fluctuations** - Análise das recentes movimentações de preço do Bitcoin
- Examina tendências de mercado, influências regulatórias, e indicadores técnicos
- Fornece recomendações baseadas nos dados históricos
- [Veja o relatório completo](examples/bitcoin_price_fluctuation.md)
5. **O que é LLM?** - Uma exploração em profundidade de Large Language Models
- Discute arquitetura, treinamento, aplicações, e considerações éticas
- [Veja o relatório completo](examples/what_is_llm.md)
6. **Como usar Claude para Pesquisa Aprofundada?** - Melhores práticas e fluxos de trabalho para usar Claude em pesquisa aprofundada
- Cobre engenharia de prompt, análise de dados, e integração com outras ferramentas
- [Veja o relatório completo](examples/how_to_use_claude_deep_research.md)
7. **Adoção de IA na Área da Saúde: Fatores de Influência** - Análise dos fatores que levam à adoção de IA na área da saúde
- Discute tecnologias de IA, qualidade de dados, considerações éticas, avaliações econômicas, prontidão organizacional, e infraestrutura digital
- [Veja o relatório completo](examples/AI_adoption_in_healthcare.md)
8. **Impacto da Computação Quântica em Criptografia** - Análise dos impactos da computação quântica em criptografia
- Discture vulnerabilidades da criptografia clássica, criptografia pós-quântica, e soluções criptográficas de resistência-quântica
- [Veja o relatório completo](examples/Quantum_Computing_Impact_on_Cryptography.md)
9. **Destaques da Performance do Cristiano Ronaldo** - Análise dos destaques da performance do Cristiano Ronaldo
- Discute as suas conquistas de carreira, objetivos internacionais, e performance em diversas partidas
- [Veja o relatório completo](examples/Cristiano_Ronaldo's_Performance_Highlights.md)
Para executar esses exemplos ou criar seus próprios relatórios de pesquisa, você deve utilizar os seguintes comandos:
```bash
# Executa com uma consulta específica
uv run main.py "Quais fatores estão influenciando a adoção de IA na área da saúde?"
# Executa com parâmetros de planejamento customizados
uv run main.py --max_plan_iterations 3 "Como a computação quântica impacta na criptografia?"
# Executa em modo interativo com questões embutidas
uv run main.py --interactive
# Ou executa com um prompt interativo básico
uv run main.py
# Vê todas as opções disponíveis
uv run main.py --help
```
### Modo Interativo
A aplicação agora suporta um modo interativo com questões embutidas tanto em Inglês quanto Chinês:
1. Inicie o modo interativo:
```bash
uv run main.py --interactive
```
2. Selecione sua linguagem de preferência (English or 中文)
3. Escolha uma das questões embutidas da lista ou selecione a opção para perguntar sua própria questão
4. O sistema irá processar sua questão e gerar um relatório abrangente de pesquisa
### Humano no processo
DeerFlow inclue um mecanismo de humano no processo que permite a você revisar, editar e aprovar planos de pesquisa antes que estes sejam executados:
1. **Revisão de Plano**: Quando o humano no processo está habilitado, o sistema irá apresentar o plano de pesquisa gerado para sua revisão antes da execução
2. **Fornecimento de Feedback**: Você pode:
- Aceitar o plano respondendo com `[ACCEPTED]`
- Edite o plano fornecendo feedback (e.g., `[EDIT PLAN] Adicione mais passos sobre a implementação técnica`)
- O sistema irá incorporar seu feedback e gerar um plano revisado
3. **Auto-aceite**: Você pode habilitar o auto-aceite ou pular o processo de revisão:
- Via API: Defina `auto_accepted_plan: true` na sua requisição
4. **Integração de API**: Quanto usar a API, você pode fornecer um feedback através do parâmetro `feedback`:
```json
{
"messages": [{ "role": "user", "content": "O que é computação quântica?" }],
"thread_id": "my_thread_id",
"auto_accepted_plan": false,
"feedback": "[EDIT PLAN] Inclua mais sobre algoritmos quânticos"
}
```
### Argumentos via Linha de Comando
A aplicação suporta diversos argumentos via linha de comando para customizar o seu comportamento:
- **consulta**: A consulta de pesquisa a ser processada (podem ser múltiplas palavras)
- **--interativo**: Roda no modo interativo com questões embutidas
- **--max_plan_iterations**: Número máximo de ciclos de planejamento (padrão: 1)
- **--max_step_num**: Número máximo de passos em um plano de pesquisa (padrão: 3)
- **--debug**: Habilita Enable um log de depuração detalhado
## FAQ
Por favor consulte a [FAQ.md](docs/FAQ.md) para maiores detalhes.
## Licença
Esse projeto é open source e disponível sob a [MIT License](./LICENSE).
## Agradecimentos
DeerFlow é construído através do incrível trabalho da comunidade open-source. Nós somos profundamente gratos a todos os projetos e contribuidores cujos esforços tornaram o DeerFlow possível. Realmente, nós estamos apoiados nos ombros de gigantes.
Nós gostaríamos de extender nossos sinceros agradecimentos aos seguintes projetos por suas invaloráveis contribuições:
- **[LangChain](https://github.com/langchain-ai/langchain)**: O framework excepcional deles empodera nossas interações via LLM e correntes, permitindo uma integração perfeita e funcional.
- **[LangGraph](https://github.com/langchain-ai/langgraph)**: A abordagem inovativa para orquestração multi-agente deles tem sido foi fundamental em permitir o acesso dos fluxos de trabalho sofisticados do DeerFlow.
Esses projetos exemplificam o poder transformador da colaboração open-source, e nós temos orgulho de construir baseado em suas fundações.
### Contribuidores-Chave
Um sincero muito obrigado vai para os principais autores do `DeerFlow`, cuja visão, paixão, e dedicação trouxe esse projeto à vida:
- **[Daniel Walnut](https://github.com/hetaoBackend/)**
- **[Henry Li](https://github.com/magiccube/)**
O seu compromisso inabalável e experiência tem sido a força por trás do sucesso do DeerFlow. Nós estamos honrados em tê-los no comando dessa trajetória.
## Histórico-Estrelas
[![Gráfico do Histórico de Estrelas](https://api.star-history.com/svg?repos=bytedance/deer-flow&type=Date)](https://star-history.com/#bytedance/deer-flow&Date)
+381 -498
View File
@@ -1,607 +1,490 @@
# 🦌 DeerFlow
# 🦌 DeerFlow - 2.0
[![Python 3.12+](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![DeepWiki](https://img.shields.io/badge/DeepWiki-bytedance%2Fdeer--flow-blue.svg?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACwAAAAyCAYAAAAnWDnqAAAAAXNSR0IArs4c6QAAA05JREFUaEPtmUtyEzEQhtWTQyQLHNak2AB7ZnyXZMEjXMGeK/AIi+QuHrMnbChYY7MIh8g01fJoopFb0uhhEqqcbWTp06/uv1saEDv4O3n3dV60RfP947Mm9/SQc0ICFQgzfc4CYZoTPAswgSJCCUJUnAAoRHOAUOcATwbmVLWdGoH//PB8mnKqScAhsD0kYP3j/Yt5LPQe2KvcXmGvRHcDnpxfL2zOYJ1mFwrryWTz0advv1Ut4CJgf5uhDuDj5eUcAUoahrdY/56ebRWeraTjMt/00Sh3UDtjgHtQNHwcRGOC98BJEAEymycmYcWwOprTgcB6VZ5JK5TAJ+fXGLBm3FDAmn6oPPjR4rKCAoJCal2eAiQp2x0vxTPB3ALO2CRkwmDy5WohzBDwSEFKRwPbknEggCPB/imwrycgxX2NzoMCHhPkDwqYMr9tRcP5qNrMZHkVnOjRMWwLCcr8ohBVb1OMjxLwGCvjTikrsBOiA6fNyCrm8V1rP93iVPpwaE+gO0SsWmPiXB+jikdf6SizrT5qKasx5j8ABbHpFTx+vFXp9EnYQmLx02h1QTTrl6eDqxLnGjporxl3NL3agEvXdT0WmEost648sQOYAeJS9Q7bfUVoMGnjo4AZdUMQku50McCcMWcBPvr0SzbTAFDfvJqwLzgxwATnCgnp4wDl6Aa+Ax283gghmj+vj7feE2KBBRMW3FzOpLOADl0Isb5587h/U4gGvkt5v60Z1VLG8BhYjbzRwyQZemwAd6cCR5/XFWLYZRIMpX39AR0tjaGGiGzLVyhse5C9RKC6ai42ppWPKiBagOvaYk8lO7DajerabOZP46Lby5wKjw1HCRx7p9sVMOWGzb/vA1hwiWc6jm3MvQDTogQkiqIhJV0nBQBTU+3okKCFDy9WwferkHjtxib7t3xIUQtHxnIwtx4mpg26/HfwVNVDb4oI9RHmx5WGelRVlrtiw43zboCLaxv46AZeB3IlTkwouebTr1y2NjSpHz68WNFjHvupy3q8TFn3Hos2IAk4Ju5dCo8B3wP7VPr/FGaKiG+T+v+TQqIrOqMTL1VdWV1DdmcbO8KXBz6esmYWYKPwDL5b5FA1a0hwapHiom0r/cKaoqr+27/XcrS5UwSMbQAAAABJRU5ErkJggg==)](https://deepwiki.com/bytedance/deer-flow)
<!-- DeepWiki badge generated by https://deepwiki.ryoppippi.com/ -->
[English](./README.md) | [中文](./README_zh.md) | [日本語](./README_ja.md) | [Français](./README_fr.md) | Русский
[English](./README.md) | [简体中文](./README_zh.md) | [日本語](./README_ja.md) | [Deutsch](./README_de.md) | [Español](./README_es.md) | [Русский](./README_ru.md) | [Portuguese](./README_pt.md)
[![Python](https://img.shields.io/badge/Python-3.12%2B-3776AB?logo=python&logoColor=white)](./backend/pyproject.toml)
[![Node.js](https://img.shields.io/badge/Node.js-22%2B-339933?logo=node.js&logoColor=white)](./Makefile)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](./LICENSE)
> Создано на базе открытого кода, возвращено в открытый код.
<a href="https://trendshift.io/repositories/14699" target="_blank"><img src="https://trendshift.io/api/badge/repositories/14699" alt="bytedance%2Fdeer-flow | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
**DeerFlow** (**D**eep **E**xploration and **E**fficient **R**esearch **Flow**) - это фреймворк для глубокого исследования, разработанный сообществом и основанный на впечатляющей работе сообщества открытого кода. Наша цель - объединить языковые модели со специализированными инструментами для таких задач, как веб-поиск, сканирование и выполнение кода Python, одновременно возвращая пользу сообществу, которое сделало это возможным.
> 28 февраля 2026 года DeerFlow занял 🏆 #1 в GitHub Trending после релиза версии 2. Спасибо огромное нашему сообществу — всё благодаря вам! 💪🔥
В настоящее время DeerFlow официально вошел в Центр приложений FaaS Volcengine. Пользователи могут испытать его онлайн через ссылку для опыта, чтобы интуитивно почувствовать его мощные функции и удобные операции. В то же время, для удовлетворения потребностей развертывания различных пользователей, DeerFlow поддерживает развертывание одним кликом на основе Volcengine. Нажмите на ссылку развертывания, чтобы быстро завершить процесс развертывания и начать эффективное исследовательское путешествие.
DeerFlow (**D**eep **E**xploration and **E**fficient **R**esearch **Flow**) — open-source **Super Agent Harness**, который управляет **Sub-Agents**, **Memory** и **Sandbox** для решения почти любой задачи. Всё на основе расширяемых **Skills**.
DeerFlow недавно интегрировал интеллектуальный набор инструментов поиска и краулинга, разработанный самостоятельно компанией BytePlus — [InfoQuest (поддерживает бесплатное онлайн-опробование)](https://docs.byteplus.com/en/docs/InfoQuest/What_is_Info_Quest)
https://github.com/user-attachments/assets/a8bcadc4-e040-4cf2-8fda-dd768b999c18
> [!NOTE]
> **DeerFlow 2.0 — проект переписан с нуля.** Общего кода с v1 нет. Если нужен оригинальный Deep Research фреймворк — он живёт в ветке [`1.x`](https://github.com/bytedance/deer-flow/tree/main-1.x), туда тоже принимают контрибьюты. Активная разработка идёт в 2.0.
## Официальный сайт
[<img width="2880" height="1600" alt="image" src="https://github.com/user-attachments/assets/a598c49f-3b2f-41ea-a052-05e21349188a" />](https://deerflow.tech)
Больше информации и живые демо на [**официальном сайте**](https://deerflow.tech).
## Coding Plan от ByteDance Volcengine
<img width="4808" height="2400" alt="英文方舟" src="https://github.com/user-attachments/assets/2ecc7b9d-50be-4185-b1f7-5542d222fb2d" />
- Рекомендуем Doubao-Seed-2.0-Code, DeepSeek v3.2 и Kimi 2.5 для запуска DeerFlow
- [Подробнее](https://www.byteplus.com/en/activity/codingplan?utm_campaign=deer_flow&utm_content=deer_flow&utm_medium=devrel&utm_source=OWO&utm_term=deer_flow)
- [Для разработчиков из материкового Китая](https://www.volcengine.com/activity/codingplan?utm_campaign=deer_flow&utm_content=deer_flow&utm_medium=devrel&utm_source=OWO&utm_term=deer_flow)
## InfoQuest
DeerFlow интегрирован с инструментарием для умного поиска и краулинга от BytePlus — [InfoQuest (есть бесплатный онлайн-доступ)](https://docs.byteplus.com/en/docs/InfoQuest/What_is_Info_Quest)
<a href="https://docs.byteplus.com/en/docs/InfoQuest/What_is_Info_Quest" target="_blank">
<img
src="https://sf16-sg.tiktokcdn.com/obj/eden-sg/hubseh7bsbps/20251208-160108.png" alt="infoquest_bannar"
<img
src="https://sf16-sg.tiktokcdn.com/obj/eden-sg/hubseh7bsbps/20251208-160108.png"
alt="InfoQuest_banner"
/>
</a>
Пожалуйста, посетите [наш официальный сайт](https://deerflow.tech/) для получения дополнительной информации.
## Демонстрация
### Видео
<https://github.com/user-attachments/assets/f3786598-1f2a-4d07-919e-8b99dfa1de3e>
В этой демонстрации мы показываем, как использовать DeerFlow для:
- Бесшовной интеграции с сервисами MCP
- Проведения процесса глубокого исследования и создания комплексного отчета с изображениями
- Создания аудио подкаста на основе сгенерированного отчета
### Повторы
- [Какова высота Эйфелевой башни по сравнению с самым высоким зданием?](https://deerflow.tech/chat?replay=eiffel-tower-vs-tallest-building)
- [Какие репозитории самые популярные на GitHub?](https://deerflow.tech/chat?replay=github-top-trending-repo)
- [Написать статью о традиционных блюдах Нанкина](https://deerflow.tech/chat?replay=nanjing-traditional-dishes)
- [Как украсить съемную квартиру?](https://deerflow.tech/chat?replay=rental-apartment-decoration)
- [Посетите наш официальный сайт, чтобы изучить больше повторов.](https://deerflow.tech/#case-studies)
---
## 📑 Оглавление
## Содержание
- [🚀 Быстрый старт](#быстрый-старт)
- [🌟 Особенности](#особенности)
- [🏗️ Архитектура](#архитектура)
- [🛠️ Разработка](#разработка)
- [🐳 Docker](#docker)
- [🗣️ Интеграция преобразования текста в речь](#интеграция-преобразования-текста-в-речь)
- [📚 Примеры](#примеры)
- [❓ FAQ](#faq)
- [📜 Лицензия](#лицензия)
- [💖 Благодарности](#благодарности)
- [⭐ История звезд](#история-звезд)
- [🦌 DeerFlow - 2.0](#-deerflow---20)
- [Официальный сайт](#официальный-сайт)
- [InfoQuest](#infoquest)
- [Содержание](#содержание)
- [Установка одной фразой для coding agent](#установка-одной-фразой-для-coding-agent)
- [Быстрый старт](#быстрый-старт)
- [Конфигурация](#конфигурация)
- [Запуск](#запуск)
- [Вариант 1: Docker (рекомендуется)](#вариант-1-docker-рекомендуется)
- [Вариант 2: Локальная разработка](#вариант-2-локальная-разработка)
- [Дополнительно](#дополнительно)
- [Режим Sandbox](#режим-sandbox)
- [MCP-сервер](#mcp-сервер)
- [Мессенджеры](#мессенджеры)
- [Трассировка LangSmith](#трассировка-langsmith)
- [От Deep Research к Super Agent Harness](#от-deep-research-к-super-agent-harness)
- [Core Features](#core-features)
- [Skills & Tools](#skills--tools)
- [Интеграция с Claude Code](#интеграция-с-claude-code)
- [Sub-Agents](#sub-agents)
- [Sandbox & файловая система](#sandbox--файловая-система)
- [Context Engineering](#context-engineering)
- [Long-Term Memory](#long-term-memory)
- [Рекомендуемые модели](#рекомендуемые-модели)
- [Встроенный Python-клиент](#встроенный-python-клиент)
- [Документация](#документация)
- [⚠️ Безопасность](#-безопасность)
- [Участие в разработке](#участие-в-разработке)
- [Лицензия](#лицензия)
- [Благодарности](#благодарности)
- [Ключевые контрибьюторы](#ключевые-контрибьюторы)
- [История звёзд](#история-звёзд)
## Установка одной фразой для coding agent
Если вы используете Claude Code, Codex, Cursor, Windsurf или другой coding agent, просто отправьте ему эту фразу:
```text
Если DeerFlow еще не клонирован, сначала клонируй его, а затем подготовь локальное окружение разработки по инструкции https://raw.githubusercontent.com/bytedance/deer-flow/main/Install.md
```
Этот prompt предназначен для coding agent. Он просит агента при необходимости сначала клонировать репозиторий, предпочесть Docker, если он доступен, и в конце вернуть точную команду запуска и список недостающих настроек.
## Быстрый старт
DeerFlow разработан на Python и поставляется с веб-интерфейсом, написанным на Node.js. Для обеспечения плавного процесса настройки мы рекомендуем использовать следующие инструменты:
### Конфигурация
### Рекомендуемые инструменты
1. **Склонировать репозиторий DeerFlow**
- **[`uv`](https://docs.astral.sh/uv/getting-started/installation/):**
Упрощает управление средой Python и зависимостями. `uv` автоматически создает виртуальную среду в корневом каталоге и устанавливает все необходимые пакеты за вас—без необходимости вручную устанавливать среды Python.
```bash
git clone https://github.com/bytedance/deer-flow.git
cd deer-flow
```
- **[`nvm`](https://github.com/nvm-sh/nvm):**
Легко управляйте несколькими версиями среды выполнения Node.js.
2. **Сгенерировать локальные конфиги**
- **[`pnpm`](https://pnpm.io/installation):**
Установка и управление зависимостями проекта Node.js.
Из корня проекта (`deer-flow/`) запустите:
### Требования к среде
```bash
make config
```
Убедитесь, что ваша система соответствует следующим минимальным требованиям:
Команда создаёт локальные конфиги на основе шаблонов.
- **[Python](https://www.python.org/downloads/):** Версия `3.12+`
- **[Node.js](https://nodejs.org/en/download/):** Версия `22+`
3. **Настроить модель**
### Установка
Отредактируйте `config.yaml` и задайте хотя бы одну модель:
```yaml
models:
- name: gpt-4 # Внутренний идентификатор
display_name: GPT-4 # Отображаемое имя
use: langchain_openai:ChatOpenAI # Путь к классу LangChain
model: gpt-4 # Идентификатор модели для API
api_key: $OPENAI_API_KEY # API-ключ (рекомендуется: переменная окружения)
max_tokens: 4096 # Максимальное количество токенов на запрос
temperature: 0.7 # Температура сэмплирования
- name: openrouter-gemini-2.5-flash
display_name: Gemini 2.5 Flash (OpenRouter)
use: langchain_openai:ChatOpenAI
model: google/gemini-2.5-flash-preview
api_key: $OPENAI_API_KEY
base_url: https://openrouter.ai/api/v1
- name: gpt-5-responses
display_name: GPT-5 (Responses API)
use: langchain_openai:ChatOpenAI
model: gpt-5
api_key: $OPENAI_API_KEY
use_responses_api: true
output_version: responses/v1
```
OpenRouter и аналогичные OpenAI-совместимые шлюзы настраиваются через `langchain_openai:ChatOpenAI` с параметром `base_url`. Для CLI-провайдеров:
```yaml
models:
- name: gpt-5.4
display_name: GPT-5.4 (Codex CLI)
use: deerflow.models.openai_codex_provider:CodexChatModel
model: gpt-5.4
supports_thinking: true
supports_reasoning_effort: true
- name: claude-sonnet-4.6
display_name: Claude Sonnet 4.6 (Claude Code OAuth)
use: deerflow.models.claude_provider:ClaudeChatModel
model: claude-sonnet-4-6
max_tokens: 4096
supports_thinking: true
```
- Codex CLI читает `~/.codex/auth.json`
- Claude Code принимает `CLAUDE_CODE_OAUTH_TOKEN`, `ANTHROPIC_AUTH_TOKEN` или `~/.claude/.credentials.json`
- На macOS при необходимости экспортируйте аутентификацию Claude Code явно:
```bash
eval "$(python3 scripts/export_claude_code_oauth.py --print-export)"
```
4. **Указать API-ключи**
- **Вариант А**: файл `.env` в корне проекта (рекомендуется)
```bash
TAVILY_API_KEY=your-tavily-api-key
OPENAI_API_KEY=your-openai-api-key
INFOQUEST_API_KEY=your-infoquest-api-key
```
- **Вариант Б**: переменные окружения в терминале
```bash
export OPENAI_API_KEY=your-openai-api-key
```
- **Вариант В**: напрямую в `config.yaml` (не рекомендуется для продакшена)
### Запуск
#### Вариант 1: Docker (рекомендуется)
**Разработка** (hot-reload, монтирование исходников):
```bash
# Клонировать репозиторий
git clone https://github.com/bytedance/deer-flow.git
cd deer-flow
# Установить зависимости, uv позаботится об интерпретаторе python и создании venv, и установит необходимые пакеты
uv sync
# Настроить .env с вашими API-ключами
# Tavily: https://app.tavily.com/home
# Brave_SEARCH: https://brave.com/search/api/
# volcengine TTS: Добавьте ваши учетные данные TTS, если они у вас есть
cp .env.example .env
# См. разделы 'Поддерживаемые поисковые системы' и 'Интеграция преобразования текста в речь' ниже для всех доступных опций
# Настроить conf.yaml для вашей модели LLM и API-ключей
# Пожалуйста, обратитесь к 'docs/configuration_guide.md' для получения дополнительной информации
cp conf.yaml.example conf.yaml
# Установить marp для генерации презентаций
# https://github.com/marp-team/marp-cli?tab=readme-ov-file#use-package-manager
brew install marp-cli
make docker-init # Загрузить образ Sandbox (один раз или при обновлении)
make docker-start # Запустить сервисы
```
По желанию установите зависимости веб-интерфейса через [pnpm](https://pnpm.io/installation):
**Продакшен** (собирает образы локально):
```bash
cd deer-flow/web
pnpm install
make up # Собрать образы и запустить все сервисы
make down # Остановить и удалить контейнеры
```
### Конфигурации
> [!TIP]
> На Linux при ошибке `permission denied` для Docker daemon добавьте пользователя в группу `docker` и перелогиньтесь. Подробнее в [CONTRIBUTING.md](CONTRIBUTING.md#linux-docker-daemon-permission-denied).
Пожалуйста, обратитесь к [Руководству по конфигурации](docs/configuration_guide.md) для получения дополнительной информации.
Адрес: http://localhost:2026
> [!ПРИМЕЧАНИЕ]
> Прежде чем запустить проект, внимательно прочитайте руководство и обновите конфигурации в соответствии с вашими конкретными настройками и требованиями.
#### Вариант 2: Локальная разработка
### Консольный интерфейс
1. **Проверить зависимости**:
```bash
make check # Проверяет Node.js 22+, pnpm, uv, nginx
```
Самый быстрый способ запустить проект - использовать консольный интерфейс.
2. **Установить зависимости**:
```bash
make install
```
```bash
# Запустить проект в оболочке, похожей на bash
uv run main.py
```
3. **(Опционально) Загрузить образ Sandbox заранее**:
```bash
make setup-sandbox
```
### Веб-интерфейс
4. **Запустить сервисы**:
```bash
make dev
```
Этот проект также включает веб-интерфейс, предлагающий более динамичный и привлекательный интерактивный опыт.
5. **Адрес**: http://localhost:2026
> [!ПРИМЕЧАНИЕ]
> Сначала вам нужно установить зависимости веб-интерфейса.
### Дополнительно
```bash
# Запустить оба сервера, бэкенд и фронтенд, в режиме разработки
# На macOS/Linux
./bootstrap.sh -d
#### Режим Sandbox
# На Windows
bootstrap.bat -d
```
> [!Примечание]
> По умолчанию сервер бэкенда привязывается к 127.0.0.1 (localhost) по соображениям безопасности. Если вам нужно разрешить внешние подключения (например, при развертывании на сервере Linux), вы можете изменить хост сервера на 0.0.0.0 в скрипте загрузки (uv run server.py --host 0.0.0.0).
> Пожалуйста, убедитесь, что ваша среда должным образом защищена, прежде чем подвергать сервис внешним сетям.
DeerFlow поддерживает несколько режимов выполнения:
- **Локальное выполнение** — код запускается прямо на хосте
- **Docker** — код выполняется в изолированных Docker-контейнерах
- **Docker + Kubernetes** — выполнение в Kubernetes-подах через provisioner
Откройте ваш браузер и посетите [`http://localhost:3000`](http://localhost:3000), чтобы исследовать веб-интерфейс.
Подробнее в [руководстве по конфигурации Sandbox](backend/docs/CONFIGURATION.md#sandbox).
Исследуйте больше деталей в каталоге [`web`](./web/).
#### MCP-сервер
## Поддерживаемые поисковые системы
DeerFlow поддерживает настраиваемые MCP-серверы для расширения возможностей. Для HTTP/SSE MCP-серверов поддерживаются OAuth-токены (`client_credentials`, `refresh_token`). Подробнее в [руководстве по MCP-серверу](backend/docs/MCP_SERVER.md).
DeerFlow поддерживает несколько поисковых систем, которые можно настроить в файле `.env` с помощью переменной `SEARCH_API`:
#### Мессенджеры
- **Tavily** (по умолчанию): Специализированный поисковый API для приложений ИИ
DeerFlow принимает задачи прямо из мессенджеров. Каналы запускаются автоматически при настройке, публичный IP не нужен.
- Требуется `TAVILY_API_KEY` в вашем файле `.env`
- Зарегистрируйтесь на: <https://app.tavily.com/home>
| Канал | Транспорт | Сложность |
|-------|-----------|-----------|
| Telegram | Bot API (long-polling) | Просто |
| Slack | Socket Mode | Средне |
| Feishu / Lark | WebSocket | Средне |
- **InfoQuest** (рекомендуется): Набор интеллектуальных инструментов для поиска и сканирования, оптимизированных для ИИ, разработанный компанией BytePlus
- Требуется `INFOQUEST_API_KEY` в вашем файле `.env`
- Поддержка фильтрации по диапазону времени и фильтрации сайтов
- Предоставляет высококачественные результаты поиска и извлечение контента
- Зарегистрируйтесь на: <https://console.byteplus.com/infoquest/infoquests>
- Посетите https://docs.byteplus.com/ru/docs/InfoQuest/What_is_Info_Quest для получения дополнительной информации
- **DuckDuckGo**: Поисковая система, ориентированная на конфиденциальность
- Не требуется API-ключ
- **Brave Search**: Поисковая система, ориентированная на конфиденциальность, с расширенными функциями
- Требуется `BRAVE_SEARCH_API_KEY` в вашем файле `.env`
- Зарегистрируйтесь на: <https://brave.com/search/api/>
- **Arxiv**: Поиск научных статей для академических исследований
- Не требуется API-ключ
- Специализируется на научных и академических статьях
- **Searx/SearxNG**: Самостоятельно размещённая метапоисковая система
- Требуется `SEARX_HOST` в вашем файле `.env`
- Поддерживает подключение к Searx или SearxNG
Чтобы настроить предпочитаемую поисковую систему, установите переменную `SEARCH_API` в вашем файле `.env`:
```bash
# Выберите одно: tavily, infoquest, duckduckgo, brave_search, arxiv
SEARCH_API=tavily
```
### Инструменты сканирования
- **Jina** (по умолчанию): Бесплатный доступный инструмент для сканирования веб-контента
- API-ключ не требуется для использования базовых функций
- При использовании API-ключа вы получаете более высокие лимиты скорости доступа
- Посетите <https://jina.ai/reader> для получения дополнительной информации
- **InfoQuest** (рекомендуется): Набор интеллектуальных инструментов для поиска и сканирования, оптимизированных для ИИ, разработанный компанией BytePlus
- Требуется `INFOQUEST_API_KEY` в вашем файле `.env`
- Предоставляет настраиваемые параметры сканирования
- Поддерживает настройки пользовательских тайм-аутов
- Предоставляет более мощные возможности извлечения контента
- Посетите <https://docs.byteplus.com/ru/docs/InfoQuest/What_is_Info_Quest> для получения дополнительной информации
Чтобы настроить предпочитаемый инструмент сканирования, установите следующее в вашем файле `conf.yaml`:
**Конфигурация в `config.yaml`:**
```yaml
CRAWLER_ENGINE:
# Тип движка: "jina" (по умолчанию) или "infoquest"
engine: infoquest
channels:
feishu:
enabled: true
app_id: $FEISHU_APP_ID
app_secret: $FEISHU_APP_SECRET
# domain: https://open.feishu.cn # China (default)
# domain: https://open.larksuite.com # International
slack:
enabled: true
bot_token: $SLACK_BOT_TOKEN
app_token: $SLACK_APP_TOKEN
allowed_users: []
telegram:
enabled: true
bot_token: $TELEGRAM_BOT_TOKEN
allowed_users: []
```
## Особенности
**Настройка Telegram**
### Ключевые возможности
1. Напишите [@BotFather](https://t.me/BotFather), отправьте `/newbot` и скопируйте HTTP API-токен.
2. Укажите `TELEGRAM_BOT_TOKEN` в `.env` и включите канал в `config.yaml`.
- 🤖 **Интеграция LLM**
- Поддерживает интеграцию большинства моделей через [litellm](https://docs.litellm.ai/docs/providers).
- Поддержка моделей с открытым исходным кодом, таких как Qwen
- API-интерфейс, совместимый с OpenAI
- Многоуровневая система LLM для задач различной сложности
**Доступные команды**
### Инструменты и интеграции MCP
| Команда | Описание |
|---------|----------|
| `/new` | Начать новый диалог |
| `/status` | Показать информацию о текущем треде |
| `/models` | Список доступных моделей |
| `/memory` | Просмотреть память |
| `/help` | Показать справку |
- 🔍 **Поиск и извлечение**
> Сообщения без команды воспринимаются как обычный чат — DeerFlow создаёт тред и отвечает.
- Веб-поиск через Tavily, InfoQuest, Brave Search и другие
- Сканирование с Jina и InfoQuest
- Расширенное извлечение контента
#### Трассировка LangSmith
- 🔗 **Бесшовная интеграция MCP**
- Расширение возможностей для доступа к частным доменам, графам знаний, веб-браузингу и многому другому
- Облегчает интеграцию различных исследовательских инструментов и методологий
DeerFlow имеет встроенную интеграцию с [LangSmith](https://smith.langchain.com) для наблюдаемости. При включении все вызовы LLM, запуски агентов и выполнения инструментов отслеживаются и отображаются в дашборде LangSmith.
### Человеческое взаимодействие
- 🧠 **Человек в контуре**
- Поддерживает интерактивное изменение планов исследования с использованием естественного языка
- Поддерживает автоматическое принятие планов исследования
- 📝 **Пост-редактирование отчетов**
- Поддерживает блочное редактирование в стиле Notion
- Позволяет совершенствовать с помощью ИИ, включая полировку, сокращение и расширение предложений
- Работает на [tiptap](https://tiptap.dev/)
### Создание контента
- 🎙️ **Генерация подкастов и презентаций**
- Генерация сценариев подкастов и синтез аудио с помощью ИИ
- Автоматическое создание простых презентаций PowerPoint
- Настраиваемые шаблоны для индивидуального контента
## Архитектура
DeerFlow реализует модульную архитектуру системы с несколькими агентами, предназначенную для автоматизированных исследований и анализа кода. Система построена на LangGraph, обеспечивающей гибкий рабочий процесс на основе состояний, где компоненты взаимодействуют через четко определенную систему передачи сообщений.
![Диаграмма архитектуры](./assets/architecture.png)
> Посмотрите вживую на [deerflow.tech](https://deerflow.tech/#multi-agent-architecture)
В системе используется оптимизированный рабочий процесс со следующими компонентами:
1. **Координатор**: Точка входа, управляющая жизненным циклом рабочего процесса
- Инициирует процесс исследования на основе пользовательского ввода
- Делегирует задачи планировщику, когда это необходимо
- Выступает в качестве основного интерфейса между пользователем и системой
2. **Планировщик**: Стратегический компонент для декомпозиции и планирования задач
- Анализирует цели исследования и создает структурированные планы выполнения
- Определяет, достаточно ли доступного контекста или требуется дополнительное исследование
- Управляет потоком исследования и решает, когда генерировать итоговый отчет
3. **Исследовательская команда**: Набор специализированных агентов, которые выполняют план:
- **Исследователь**: Проводит веб-поиск и сбор информации с использованием таких инструментов, как поисковые системы, сканирование и даже сервисы MCP.
- **Программист**: Обрабатывает анализ кода, выполнение и технические задачи с помощью инструмента Python REPL.
Каждый агент имеет доступ к определенным инструментам, оптимизированным для его роли, и работает в рамках фреймворка LangGraph
4. **Репортер**: Процессор финальной стадии для результатов исследования
- Агрегирует находки исследовательской команды
- Обрабатывает и структурирует собранную информацию
- Генерирует комплексные исследовательские отчеты
## Интеграция преобразования текста в речь
DeerFlow теперь включает функцию преобразования текста в речь (TTS), которая позволяет конвертировать исследовательские отчеты в речь. Эта функция использует API TTS volcengine для генерации высококачественного аудио из текста. Также можно настраивать такие параметры, как скорость, громкость и тон.
### Использование API TTS
Вы можете получить доступ к функциональности TTS через конечную точку `/api/tts`:
Добавьте в файл `.env` в корне проекта:
```bash
# Пример вызова API с использованием curl
curl --location 'http://localhost:8000/api/tts' \
--header 'Content-Type: application/json' \
--data '{
"text": "Это тест функциональности преобразования текста в речь.",
"speed_ratio": 1.0,
"volume_ratio": 1.0,
"pitch_ratio": 1.0
}' \
--output speech.mp3
LANGSMITH_TRACING=true
LANGSMITH_API_KEY=lsv2_pt_xxxxxxxxxxxxxxxx
LANGSMITH_PROJECT=deer-flow
```
## Разработка
`LANGSMITH_ENDPOINT` по умолчанию `https://api.smith.langchain.com` и может быть переопределён при необходимости. Устаревшие переменные `LANGCHAIN_*` (`LANGCHAIN_TRACING_V2`, `LANGCHAIN_API_KEY` и т.д.) также поддерживаются для обратной совместимости; `LANGSMITH_*` имеет приоритет, когда заданы обе.
### Тестирование
В Docker-развёртываниях трассировка отключена по умолчанию. Установите `LANGSMITH_TRACING=true` и `LANGSMITH_API_KEY` в `.env` для включения.
Запустите набор тестов:
## От Deep Research к Super Agent Harness
DeerFlow начинался как фреймворк для Deep Research, и сообщество вышло далеко за эти рамки. После запуска разработчики строили пайплайны, генерировали презентации, поднимали дашборды, автоматизировали контент. То, чего мы не ожидали.
Стало понятно: DeerFlow не просто research-инструмент. Это **harness**: runtime, который даёт агентам необходимую инфраструктуру.
Поэтому мы переписали всё с нуля.
DeerFlow 2.0 — это Super Agent Harness «из коробки». Batteries included, полностью расширяемый. Построен на LangGraph и LangChain. По умолчанию есть всё, что нужно агенту: файловая система, memory, skills, sandbox-выполнение и возможность планировать и запускать sub-agents для сложных многошаговых задач.
Используйте как есть. Или разберите и переделайте под себя.
## Core Features
### Skills & Tools
Skills — это то, что позволяет DeerFlow делать почти что угодно.
Agent Skill — это структурированный модуль: Markdown-файл с описанием воркфлоу, лучших практик и ссылок на ресурсы. DeerFlow поставляется со встроенными skills для ресёрча, генерации отчётов, слайдов, веб-страниц, изображений и видео. Но главное — расширяемость: добавляйте свои skills, заменяйте встроенные или собирайте из них составные воркфлоу.
Skills загружаются по мере необходимости, только когда задача их требует. Это держит контекстное окно чистым.
```
# Пути внутри контейнера sandbox
/mnt/skills/public
├── research/SKILL.md
├── report-generation/SKILL.md
├── slide-creation/SKILL.md
├── web-page/SKILL.md
└── image-generation/SKILL.md
/mnt/skills/custom
└── your-custom-skill/SKILL.md ← ваш skill
```
#### Интеграция с Claude Code
Skill `claude-to-deerflow` позволяет работать с DeerFlow прямо из [Claude Code](https://docs.anthropic.com/en/docs/claude-code). Отправляйте задачи, проверяйте статус, управляйте тредами, не выходя из терминала.
**Установка скилла**:
```bash
# Запустить все тесты
make test
# Запустить определенный тестовый файл
pytest tests/integration/test_workflow.py
# Запустить с покрытием
make coverage
npx skills add https://github.com/bytedance/deer-flow --skill claude-to-deerflow
```
### Качество кода
**Что можно делать**:
- Отправлять сообщения в DeerFlow и получать потоковые ответы
- Выбирать режимы выполнения: flash (быстро), standard, pro (planning), ultra (sub-agents)
- Проверять статус DeerFlow, просматривать модели, скиллы, агентов
- Управлять тредами и историей диалога
- Загружать файлы для анализа
```bash
# Запустить линтинг
make lint
Полный справочник API в [`skills/public/claude-to-deerflow/SKILL.md`](skills/public/claude-to-deerflow/SKILL.md).
# Форматировать код
make format
### Sub-Agents
Сложные задачи редко решаются за один проход. DeerFlow их декомпозирует.
Lead agent запускает sub-agents на лету, каждый со своим изолированным контекстом, инструментами и условиями завершения. Sub-agents работают параллельно, возвращают структурированные результаты, а lead agent собирает всё в единый итог.
Вот как DeerFlow справляется с задачами на минуты и часы: research-задача разветвляется в дюжину sub-agents, каждый копает свой угол, потом всё сходится в один отчёт, или сайт, или слайддек со сгенерированными визуалами. Один harness, много рук.
### Sandbox & файловая система
DeerFlow не просто *говорит* о том, что умеет что-то делать. У него есть собственный компьютер.
Каждая задача выполняется внутри изолированного Docker-контейнера с полной файловой системой: skills, workspace, uploads, outputs. Агент читает, пишет и редактирует файлы. Выполняет bash-команды и пишет код. Смотрит на изображения. Всё изолировано, всё прозрачно, никакого пересечения между сессиями.
Это разница между чатботом с доступом к инструментам и агентом с реальной средой выполнения.
```
# Пути внутри контейнера sandbox
/mnt/user-data/
├── uploads/ ← ваши файлы
├── workspace/ ← рабочая директория агентов
└── outputs/ ← результаты
```
### Отладка с LangGraph Studio
### Context Engineering
DeerFlow использует LangGraph для своей архитектуры рабочего процесса. Вы можете использовать LangGraph Studio для отладки и визуализации рабочего процесса в реальном времени.
**Изолированный контекст**: каждый sub-agent работает в своём контексте и не видит контекст главного агента или других sub-agents. Агент фокусируется на своей задаче.
#### Запуск LangGraph Studio локально
**Управление контекстом**: внутри сессии DeerFlow агрессивно сжимает контекст и суммирует завершённые подзадачи, выгружает промежуточные результаты в файловую систему, сжимает то, что уже не актуально. На длинных многошаговых задачах контекстное окно не переполняется.
DeerFlow включает конфигурационный файл `langgraph.json`, который определяет структуру графа и зависимости для LangGraph Studio. Этот файл указывает на графы рабочего процесса, определенные в проекте, и автоматически загружает переменные окружения из файла `.env`.
### Long-Term Memory
##### Mac
Большинство агентов забывают всё, когда диалог заканчивается. DeerFlow помнит.
```bash
# Установите менеджер пакетов uv, если у вас его нет
curl -LsSf https://astral.sh/uv/install.sh | sh
DeerFlow сохраняет ваш профиль, предпочтения и накопленные знания между сессиями. Чем больше используете, тем лучше он вас знает: стиль, технологический стек, повторяющиеся воркфлоу. Всё хранится локально и остаётся под вашим контролем.
# Установите зависимости и запустите сервер LangGraph
uvx --refresh --from "langgraph-cli[inmem]" --with-editable . --python 3.12 langgraph dev --allow-blocking
## Рекомендуемые модели
DeerFlow работает с любым LLM через OpenAI-совместимый API. Лучше всего — с моделями, которые поддерживают:
- **Большое контекстное окно** (100k+ токенов) — для deep research и многошаговых задач
- **Reasoning capabilities** — для адаптивного планирования и сложной декомпозиции
- **Multimodal inputs** — для работы с изображениями и видео
- **Strong tool-use** — для надёжного вызова функций и структурированных ответов
## Встроенный Python-клиент
DeerFlow можно использовать как Python-библиотеку прямо в коде — без запуска HTTP-сервисов. `DeerFlowClient` даёт доступ ко всем возможностям агента и Gateway, возвращает те же схемы ответов, что и HTTP Gateway API:
```python
from deerflow.client import DeerFlowClient
client = DeerFlowClient()
# Chat
response = client.chat("Analyze this paper for me", thread_id="my-thread")
# Streaming (LangGraph SSE protocol: values, messages-tuple, end)
for event in client.stream("hello"):
if event.type == "messages-tuple" and event.data.get("type") == "ai":
print(event.data["content"])
# Configuration & management — returns Gateway-aligned dicts
models = client.list_models() # {"models": [...]}
skills = client.list_skills() # {"skills": [...]}
client.update_skill("web-search", enabled=True)
client.upload_files("thread-1", ["./report.pdf"]) # {"success": True, "files": [...]}
```
##### Windows / Linux
## Документация
```bash
# Установить зависимости
pip install -e .
pip install -U "langgraph-cli[inmem]"
- [Руководство по участию](CONTRIBUTING.md) — настройка среды разработки, воркфлоу и гайдлайны
- [Руководство по конфигурации](backend/docs/CONFIGURATION.md) — инструкции по настройке
- [Обзор архитектуры](backend/CLAUDE.md) — технические детали
- [Архитектура бэкенда](backend/README.md) — бэкенд и справочник API
# Запустить сервер LangGraph
langgraph dev
```
## ⚠️ Безопасность
После запуска сервера LangGraph вы увидите несколько URL в терминале:
### Неправильное развёртывание может привести к угрозам безопасности
- API: <http://127.0.0.1:2024>
- Studio UI: <https://smith.langchain.com/studio/?baseUrl=http://127.0.0.1:2024>
- API Docs: <http://127.0.0.1:2024/docs>
DeerFlow обладает ключевыми высокопривилегированными возможностями, включая **выполнение системных команд, операции с ресурсами и вызов бизнес-логики**. По умолчанию он рассчитан на **развёртывание в локальной доверенной среде (доступ только через loopback-адрес 127.0.0.1)**. Если вы разворачиваете агент в недоверенных средах — локальных сетях, публичных облачных серверах или других окружениях, доступных с нескольких устройств — без строгих мер безопасности, это может привести к следующим угрозам:
Откройте ссылку Studio UI в вашем браузере для доступа к интерфейсу отладки.
- **Несанкционированные вызовы**: функциональность агента может быть обнаружена неавторизованными третьими лицами или вредоносными сканерами, что приведёт к массовым несанкционированным запросам с выполнением высокорисковых операций (системные команды, чтение/запись файлов) и серьёзным последствиям для безопасности.
- **Юридические и compliance-риски**: если агент будет незаконно использован для кибератак, кражи данных или других противоправных действий, это может повлечь юридическую ответственность и compliance-риски.
#### Использование LangGraph Studio
### Рекомендации по безопасности
В интерфейсе Studio вы можете:
**Примечание: настоятельно рекомендуем развёртывать DeerFlow только в локальной доверенной сети.** Если вам необходимо развёртывание через несколько устройств или сетей, обязательно реализуйте строгие меры безопасности, например:
1. Визуализировать граф рабочего процесса и видеть, как соединяются компоненты
2. Отслеживать выполнение в реальном времени, чтобы видеть, как данные проходят через систему
3. Исследовать состояние на каждом шаге рабочего процесса
4. Отлаживать проблемы путем изучения входов и выходов каждого компонента
5. Предоставлять обратную связь во время фазы планирования для уточнения планов исследования
- **Белый список IP-адресов**: используйте `iptables` или аппаратные межсетевые экраны / коммутаторы с ACL, чтобы **настроить правила белого списка IP** и заблокировать доступ со всех остальных адресов.
- **Шлюз аутентификации**: настройте обратный прокси (nginx и др.) и **включите строгую предварительную аутентификацию**, запрещающую любой доступ без авторизации.
- **Сетевая изоляция**: по возможности разместите агент и доверенные устройства в **одном выделенном VLAN**, изолированном от остальной сети.
- **Следите за обновлениями**: регулярно отслеживайте обновления безопасности проекта DeerFlow.
Когда вы отправляете тему исследования в интерфейсе Studio, вы сможете увидеть весь процесс выполнения рабочего процесса, включая:
## Участие в разработке
- Фазу планирования, где создается план исследования
- Цикл обратной связи, где вы можете модифицировать план
- Фазы исследования и написания для каждого раздела
- Генерацию итогового отчета
### Включение трассировки LangSmith
DeerFlow поддерживает трассировку LangSmith, чтобы помочь вам отладить и контролировать ваши рабочие процессы. Чтобы включить трассировку LangSmith:
1. Убедитесь, что в вашем файле `.env` есть следующие конфигурации (см. `.env.example`):
```bash
LANGSMITH_TRACING=true
LANGSMITH_ENDPOINT="https://api.smith.langchain.com"
LANGSMITH_API_KEY="xxx"
LANGSMITH_PROJECT="xxx"
```
2. Запустите трассировку и визуализируйте граф локально с LangSmith, выполнив:
```bash
langgraph dev
```
Это включит визуализацию трассировки в LangGraph Studio и отправит ваши трассировки в LangSmith для мониторинга и анализа.
## Docker
Вы также можете запустить этот проект с Docker.
Во-первых, вам нужно прочитать [конфигурацию](docs/configuration_guide.md) ниже. Убедитесь, что файлы `.env`, `.conf.yaml` готовы.
Во-вторых, чтобы построить Docker-образ вашего собственного веб-сервера:
```bash
docker build -t deer-flow-api .
```
Наконец, запустите Docker-контейнер с веб-сервером:
```bash
# Замените deer-flow-api-app на предпочитаемое вами имя контейнера
# Запустите сервер и привяжите к localhost:8000
docker run -d -t -p 127.0.0.1:8000:8000 --env-file .env --name deer-flow-api-app deer-flow-api
# остановить сервер
docker stop deer-flow-api-app
```
### Docker Compose (включает как бэкенд, так и фронтенд)
DeerFlow предоставляет настройку docker-compose для легкого запуска бэкенда и фронтенда вместе:
```bash
# сборка docker-образа
docker compose build
# запуск сервера
docker compose up
```
> [!WARNING]
> Если вы хотите развернуть DeerFlow в производственных средах, пожалуйста, добавьте аутентификацию к веб-сайту и оцените свою проверку безопасности MCPServer и Python Repl.
## Примеры
Следующие примеры демонстрируют возможности DeerFlow:
### Исследовательские отчеты
1. **Отчет о OpenAI Sora** - Анализ инструмента ИИ Sora от OpenAI
- Обсуждаются функции, доступ, инженерия промптов, ограничения и этические соображения
- [Просмотреть полный отчет](examples/openai_sora_report.md)
2. **Отчет о протоколе Agent to Agent от Google** - Обзор протокола Agent to Agent (A2A) от Google
- Обсуждается его роль в коммуникации агентов ИИ и его отношение к протоколу Model Context Protocol (MCP) от Anthropic
- [Просмотреть полный отчет](examples/what_is_agent_to_agent_protocol.md)
3. **Что такое MCP?** - Комплексный анализ термина "MCP" в различных контекстах
- Исследует Model Context Protocol в ИИ, Монокальцийфосфат в химии и Микроканальные пластины в электронике
- [Просмотреть полный отчет](examples/what_is_mcp.md)
4. **Колебания цены Биткоина** - Анализ недавних движений цены Биткоина
- Исследует рыночные тренды, регуляторные влияния и технические индикаторы
- Предоставляет рекомендации на основе исторических данных
- [Просмотреть полный отчет](examples/bitcoin_price_fluctuation.md)
5. **Что такое LLM?** - Углубленное исследование больших языковых моделей
- Обсуждаются архитектура, обучение, приложения и этические соображения
- [Просмотреть полный отчет](examples/what_is_llm.md)
6. **Как использовать Claude для глубокого исследования?** - Лучшие практики и рабочие процессы для использования Claude в глубоком исследовании
- Охватывает инженерию промптов, анализ данных и интеграцию с другими инструментами
- [Просмотреть полный отчет](examples/how_to_use_claude_deep_research.md)
7. **Внедрение ИИ в здравоохранении: Влияющие факторы** - Анализ факторов, движущих внедрением ИИ в здравоохранении
- Обсуждаются технологии ИИ, качество данных, этические соображения, экономические оценки, организационная готовность и цифровая инфраструктура
- [Просмотреть полный отчет](examples/AI_adoption_in_healthcare.md)
8. **Влияние квантовых вычислений на криптографию** - Анализ влияния квантовых вычислений на криптографию
- Обсуждаются уязвимости классической криптографии, пост-квантовая криптография и криптографические решения, устойчивые к квантовым вычислениям
- [Просмотреть полный отчет](examples/Quantum_Computing_Impact_on_Cryptography.md)
9. **Ключевые моменты выступлений Криштиану Роналду** - Анализ выдающихся выступлений Криштиану Роналду
- Обсуждаются его карьерные достижения, международные голы и выступления в различных матчах
- [Просмотреть полный отчет](examples/Cristiano_Ronaldo's_Performance_Highlights.md)
Чтобы запустить эти примеры или создать собственные исследовательские отчеты, вы можете использовать следующие команды:
```bash
# Запустить с определенным запросом
uv run main.py "Какие факторы влияют на внедрение ИИ в здравоохранении?"
# Запустить с пользовательскими параметрами планирования
uv run main.py --max_plan_iterations 3 "Как квантовые вычисления влияют на криптографию?"
# Запустить в интерактивном режиме с встроенными вопросами
uv run main.py --interactive
# Или запустить с базовым интерактивным приглашением
uv run main.py
# Посмотреть все доступные опции
uv run main.py --help
```
### Интерактивный режим
Приложение теперь поддерживает интерактивный режим с встроенными вопросами как на английском, так и на китайском языках:
1. Запустите интерактивный режим:
```bash
uv run main.py --interactive
```
2. Выберите предпочитаемый язык (English или 中文)
3. Выберите из списка встроенных вопросов или выберите опцию задать собственный вопрос
4. Система обработает ваш вопрос и сгенерирует комплексный исследовательский отчет
### Человек в контуре
DeerFlow включает механизм "человек в контуре", который позволяет вам просматривать, редактировать и утверждать планы исследования перед их выполнением:
1. **Просмотр плана**: Когда активирован режим "человек в контуре", система представит сгенерированный план исследования для вашего просмотра перед выполнением
2. **Предоставление обратной связи**: Вы можете:
- Принять план, ответив `[ACCEPTED]`
- Отредактировать план, предоставив обратную связь (например, `[EDIT PLAN] Добавить больше шагов о технической реализации`)
- Система включит вашу обратную связь и сгенерирует пересмотренный план
3. **Автоматическое принятие**: Вы можете включить автоматическое принятие, чтобы пропустить процесс просмотра:
- Через API: Установите `auto_accepted_plan: true` в вашем запросе
4. **Интеграция API**: При использовании API вы можете предоставить обратную связь через параметр `feedback`:
```json
{
"messages": [{ "role": "user", "content": "Что такое квантовые вычисления?" }],
"thread_id": "my_thread_id",
"auto_accepted_plan": false,
"feedback": "[EDIT PLAN] Включить больше о квантовых алгоритмах"
}
```
### Аргументы командной строки
Приложение поддерживает несколько аргументов командной строки для настройки его поведения:
- **query**: Запрос исследования для обработки (может состоять из нескольких слов)
- **--interactive**: Запустить в интерактивном режиме с встроенными вопросами
- **--max_plan_iterations**: Максимальное количество циклов планирования (по умолчанию: 1)
- **--max_step_num**: Максимальное количество шагов в плане исследования (по умолчанию: 3)
- **--debug**: Включить подробное логирование отладки
## FAQ
Пожалуйста, обратитесь к [FAQ.md](docs/FAQ.md) для получения дополнительной информации.
Приветствуем контрибьюторов! Настройка среды разработки, воркфлоу и гайдлайны — в [CONTRIBUTING.md](CONTRIBUTING.md).
## Лицензия
Этот проект имеет открытый исходный код и доступен под [Лицензией MIT](./LICENSE).
Проект распространяется под [лицензией MIT](./LICENSE).
## Благодарности
DeerFlow создан на основе невероятной работы сообщества открытого кода. Мы глубоко благодарны всем проектам и контрибьюторам, чьи усилия сделали DeerFlow возможным. Поистине, мы стоим на плечах гигантов.
DeerFlow стоит на плечах open-source сообщества. Спасибо всем проектам и разработчикам, чья работа сделала его возможным.
Мы хотели бы выразить искреннюю признательность следующим проектам за их неоценимый вклад:
Отдельная благодарность:
- **[LangChain](https://github.com/langchain-ai/langchain)**: Их исключительный фреймворк обеспечивает наши взаимодействия и цепочки LLM, позволяя бесшовную интеграцию и функциональность.
- **[LangGraph](https://github.com/langchain-ai/langgraph)**: Их инновационный подход к оркестровке многоагентных систем сыграл решающую роль в обеспечении сложных рабочих процессов DeerFlow.
Эти проекты являются примером преобразующей силы сотрудничества в области открытого кода, и мы гордимся тем, что строим на их основе.
- **[LangChain](https://github.com/langchain-ai/langchain)** — фреймворк для взаимодействия с LLM и построения цепочек.
- **[LangGraph](https://github.com/langchain-ai/langgraph)** — многоагентная оркестрация, на которой держатся сложные воркфлоу DeerFlow.
### Ключевые контрибьюторы
Сердечная благодарность основным авторам `DeerFlow`, чье видение, страсть и преданность делу вдохнули жизнь в этот проект:
Авторы DeerFlow, без которых проекта бы не было:
- **[Daniel Walnut](https://github.com/hetaoBackend/)**
- **[Henry Li](https://github.com/magiccube/)**
Ваша непоколебимая приверженность и опыт стали движущей силой успеха DeerFlow. Мы считаем за честь иметь вас во главе этого путешествия.
## История звёзд
## История звезд
[![Star History Chart](https://api.star-history.com/svg?repos=bytedance/deer-flow&type=Date)](https://star-history.com/#bytedance/deer-flow&Date)
[![Star History Chart](https://api.star-history.com/svg?repos=bytedance/deer-flow&type=Date)](https://star-history.com/#bytedance/deer-flow&Date)
+492 -608
View File
File diff suppressed because it is too large Load Diff
+4 -1
View File
@@ -2,7 +2,10 @@
## Supported Versions
As deer-flow doesn't provide an offical release yet, please use the latest version for the security updates.
As deer-flow doesn't provide an official release yet, please use the latest version for the security updates.
Currently, we have two branches to maintain:
* main branch for deer-flow 2.x
* main-1.x branch for deer-flow 1.x
## Reporting a Vulnerability
Binary file not shown.

Before

Width:  |  Height:  |  Size: 151 KiB

+28
View File
@@ -0,0 +1,28 @@
# Python-generated files
__pycache__/
*.py[oc]
build/
dist/
wheels/
*.egg-info
.coverage
.coverage.*
.ruff_cache
agent_history.gif
static/browser_history/*.gif
log/
log/*
# Virtual environments
.venv
venv/
# User config file
config.yaml
# Langgraph
.langgraph_api
# Claude Code settings
.claude/settings.local.json
+3
View File
@@ -0,0 +1,3 @@
{
"recommendations": ["charliermarsh.ruff"]
}
+11
View File
@@ -0,0 +1,11 @@
{
"window.title": "${activeEditorShort}${separator}${separator}deer-flow/backend",
"[python]": {
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {
"source.fixAll": "explicit",
"source.organizeImports": "explicit"
},
"editor.defaultFormatter": "charliermarsh.ruff"
}
}
+2
View File
@@ -0,0 +1,2 @@
For the backend architecture and design patterns:
@./CLAUDE.md
+547
View File
@@ -0,0 +1,547 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
DeerFlow is a LangGraph-based AI super agent system with a full-stack architecture. The backend provides a "super agent" with sandbox execution, persistent memory, subagent delegation, and extensible tool integration - all operating in per-thread isolated environments.
**Architecture**:
- **LangGraph Server** (port 2024): Agent runtime and workflow execution
- **Gateway API** (port 8001): REST API for models, MCP, skills, memory, artifacts, uploads, and local thread cleanup
- **Frontend** (port 3000): Next.js web interface
- **Nginx** (port 2026): Unified reverse proxy entry point
- **Provisioner** (port 8002, optional in Docker dev): Started only when sandbox is configured for provisioner/Kubernetes mode
**Runtime Modes**:
- **Standard mode** (`make dev`): LangGraph Server handles agent execution as a separate process. 4 processes total.
- **Gateway mode** (`make dev-pro`, experimental): Agent runtime embedded in Gateway via `RunManager` + `run_agent()` + `StreamBridge` (`packages/harness/deerflow/runtime/`). Service manages its own concurrency via async tasks. 3 processes total, no LangGraph Server.
**Project Structure**:
```
deer-flow/
├── Makefile # Root commands (check, install, dev, stop)
├── config.yaml # Main application configuration
├── extensions_config.json # MCP servers and skills configuration
├── backend/ # Backend application (this directory)
│ ├── Makefile # Backend-only commands (dev, gateway, lint)
│ ├── langgraph.json # LangGraph server configuration
│ ├── packages/
│ │ └── harness/ # deerflow-harness package (import: deerflow.*)
│ │ ├── pyproject.toml
│ │ └── deerflow/
│ │ ├── agents/ # LangGraph agent system
│ │ │ ├── lead_agent/ # Main agent (factory + system prompt)
│ │ │ ├── middlewares/ # 10 middleware components
│ │ │ ├── memory/ # Memory extraction, queue, prompts
│ │ │ └── thread_state.py # ThreadState schema
│ │ ├── sandbox/ # Sandbox execution system
│ │ │ ├── local/ # Local filesystem provider
│ │ │ ├── sandbox.py # Abstract Sandbox interface
│ │ │ ├── tools.py # bash, ls, read/write/str_replace
│ │ │ └── middleware.py # Sandbox lifecycle management
│ │ ├── subagents/ # Subagent delegation system
│ │ │ ├── builtins/ # general-purpose, bash agents
│ │ │ ├── executor.py # Background execution engine
│ │ │ └── registry.py # Agent registry
│ │ ├── tools/builtins/ # Built-in tools (present_files, ask_clarification, view_image)
│ │ ├── mcp/ # MCP integration (tools, cache, client)
│ │ ├── models/ # Model factory with thinking/vision support
│ │ ├── skills/ # Skills discovery, loading, parsing
│ │ ├── config/ # Configuration system (app, model, sandbox, tool, etc.)
│ │ ├── community/ # Community tools (tavily, jina_ai, firecrawl, image_search, aio_sandbox)
│ │ ├── reflection/ # Dynamic module loading (resolve_variable, resolve_class)
│ │ ├── utils/ # Utilities (network, readability)
│ │ └── client.py # Embedded Python client (DeerFlowClient)
│ ├── app/ # Application layer (import: app.*)
│ │ ├── gateway/ # FastAPI Gateway API
│ │ │ ├── app.py # FastAPI application
│ │ │ └── routers/ # FastAPI route modules (models, mcp, memory, skills, uploads, threads, artifacts, agents, suggestions, channels)
│ │ └── channels/ # IM platform integrations
│ ├── tests/ # Test suite
│ └── docs/ # Documentation
├── frontend/ # Next.js frontend application
└── skills/ # Agent skills directory
├── public/ # Public skills (committed)
└── custom/ # Custom skills (gitignored)
```
## Important Development Guidelines
### Documentation Update Policy
**CRITICAL: Always update README.md and CLAUDE.md after every code change**
When making code changes, you MUST update the relevant documentation:
- Update `README.md` for user-facing changes (features, setup, usage instructions)
- Update `CLAUDE.md` for development changes (architecture, commands, workflows, internal systems)
- Keep documentation synchronized with the codebase at all times
- Ensure accuracy and timeliness of all documentation
## Commands
**Root directory** (for full application):
```bash
make check # Check system requirements
make install # Install all dependencies (frontend + backend)
make dev # Start all services (LangGraph + Gateway + Frontend + Nginx), with config.yaml preflight
make dev-pro # Gateway mode (experimental): skip LangGraph, agent runtime embedded in Gateway
make start-pro # Production + Gateway mode (experimental)
make stop # Stop all services
```
**Backend directory** (for backend development only):
```bash
make install # Install backend dependencies
make dev # Run LangGraph server only (port 2024)
make gateway # Run Gateway API only (port 8001)
make test # Run all backend tests
make lint # Lint with ruff
make format # Format code with ruff
```
Regression tests related to Docker/provisioner behavior:
- `tests/test_docker_sandbox_mode_detection.py` (mode detection from `config.yaml`)
- `tests/test_provisioner_kubeconfig.py` (kubeconfig file/directory handling)
Boundary check (harness → app import firewall):
- `tests/test_harness_boundary.py` — ensures `packages/harness/deerflow/` never imports from `app.*`
CI runs these regression tests for every pull request via [.github/workflows/backend-unit-tests.yml](../.github/workflows/backend-unit-tests.yml).
## Architecture
### Harness / App Split
The backend is split into two layers with a strict dependency direction:
- **Harness** (`packages/harness/deerflow/`): Publishable agent framework package (`deerflow-harness`). Import prefix: `deerflow.*`. Contains agent orchestration, tools, sandbox, models, MCP, skills, config — everything needed to build and run agents.
- **App** (`app/`): Unpublished application code. Import prefix: `app.*`. Contains the FastAPI Gateway API and IM channel integrations (Feishu, Slack, Telegram).
**Dependency rule**: App imports deerflow, but deerflow never imports app. This boundary is enforced by `tests/test_harness_boundary.py` which runs in CI.
**Import conventions**:
```python
# Harness internal
from deerflow.agents import make_lead_agent
from deerflow.models import create_chat_model
# App internal
from app.gateway.app import app
from app.channels.service import start_channel_service
# App → Harness (allowed)
from deerflow.config import get_app_config
# Harness → App (FORBIDDEN — enforced by test_harness_boundary.py)
# from app.gateway.routers.uploads import ... # ← will fail CI
```
### Agent System
**Lead Agent** (`packages/harness/deerflow/agents/lead_agent/agent.py`):
- Entry point: `make_lead_agent(config: RunnableConfig)` registered in `langgraph.json`
- Dynamic model selection via `create_chat_model()` with thinking/vision support
- Tools loaded via `get_available_tools()` - combines sandbox, built-in, MCP, community, and subagent tools
- System prompt generated by `apply_prompt_template()` with skills, memory, and subagent instructions
**ThreadState** (`packages/harness/deerflow/agents/thread_state.py`):
- Extends `AgentState` with: `sandbox`, `thread_data`, `title`, `artifacts`, `todos`, `uploaded_files`, `viewed_images`
- Uses custom reducers: `merge_artifacts` (deduplicate), `merge_viewed_images` (merge/clear)
**Runtime Configuration** (via `config.configurable`):
- `thinking_enabled` - Enable model's extended thinking
- `model_name` - Select specific LLM model
- `is_plan_mode` - Enable TodoList middleware
- `subagent_enabled` - Enable task delegation tool
### Middleware Chain
Middlewares execute in strict order in `packages/harness/deerflow/agents/lead_agent/agent.py`:
1. **ThreadDataMiddleware** - Creates per-thread directories (`backend/.deer-flow/threads/{thread_id}/user-data/{workspace,uploads,outputs}`); Web UI thread deletion now follows LangGraph thread removal with Gateway cleanup of the local `.deer-flow/threads/{thread_id}` directory
2. **UploadsMiddleware** - Tracks and injects newly uploaded files into conversation
3. **SandboxMiddleware** - Acquires sandbox, stores `sandbox_id` in state
4. **DanglingToolCallMiddleware** - Injects placeholder ToolMessages for AIMessage tool_calls that lack responses (e.g., due to user interruption)
5. **GuardrailMiddleware** - Pre-tool-call authorization via pluggable `GuardrailProvider` protocol (optional, if `guardrails.enabled` in config). Evaluates each tool call and returns error ToolMessage on deny. Three provider options: built-in `AllowlistProvider` (zero deps), OAP policy providers (e.g. `aport-agent-guardrails`), or custom providers. See [docs/GUARDRAILS.md](docs/GUARDRAILS.md) for setup, usage, and how to implement a provider.
6. **SummarizationMiddleware** - Context reduction when approaching token limits (optional, if enabled)
7. **TodoListMiddleware** - Task tracking with `write_todos` tool (optional, if plan_mode)
8. **TitleMiddleware** - Auto-generates thread title after first complete exchange and normalizes structured message content before prompting the title model
9. **MemoryMiddleware** - Queues conversations for async memory update (filters to user + final AI responses)
10. **ViewImageMiddleware** - Injects base64 image data before LLM call (conditional on vision support)
11. **SubagentLimitMiddleware** - Truncates excess `task` tool calls from model response to enforce `MAX_CONCURRENT_SUBAGENTS` limit (optional, if subagent_enabled)
12. **ClarificationMiddleware** - Intercepts `ask_clarification` tool calls, interrupts via `Command(goto=END)` (must be last)
### Configuration System
**Main Configuration** (`config.yaml`):
Setup: Copy `config.example.yaml` to `config.yaml` in the **project root** directory.
**Config Versioning**: `config.example.yaml` has a `config_version` field. On startup, `AppConfig.from_file()` compares user version vs example version and emits a warning if outdated. Missing `config_version` = version 0. Run `make config-upgrade` to auto-merge missing fields. When changing the config schema, bump `config_version` in `config.example.yaml`.
**Config Caching**: `get_app_config()` caches the parsed config, but automatically reloads it when the resolved config path changes or the file's mtime increases. This keeps Gateway and LangGraph reads aligned with `config.yaml` edits without requiring a manual process restart.
Configuration priority:
1. Explicit `config_path` argument
2. `DEER_FLOW_CONFIG_PATH` environment variable
3. `config.yaml` in current directory (backend/)
4. `config.yaml` in parent directory (project root - **recommended location**)
Config values starting with `$` are resolved as environment variables (e.g., `$OPENAI_API_KEY`).
`ModelConfig` also declares `use_responses_api` and `output_version` so OpenAI `/v1/responses` can be enabled explicitly while still using `langchain_openai:ChatOpenAI`.
**Extensions Configuration** (`extensions_config.json`):
MCP servers and skills are configured together in `extensions_config.json` in project root:
Configuration priority:
1. Explicit `config_path` argument
2. `DEER_FLOW_EXTENSIONS_CONFIG_PATH` environment variable
3. `extensions_config.json` in current directory (backend/)
4. `extensions_config.json` in parent directory (project root - **recommended location**)
### Gateway API (`app/gateway/`)
FastAPI application on port 8001 with health check at `GET /health`.
**Routers**:
| Router | Endpoints |
|--------|-----------|
| **Models** (`/api/models`) | `GET /` - list models; `GET /{name}` - model details |
| **MCP** (`/api/mcp`) | `GET /config` - get config; `PUT /config` - update config (saves to extensions_config.json) |
| **Skills** (`/api/skills`) | `GET /` - list skills; `GET /{name}` - details; `PUT /{name}` - update enabled; `POST /install` - install from .skill archive (accepts standard optional frontmatter like `version`, `author`, `compatibility`) |
| **Memory** (`/api/memory`) | `GET /` - memory data; `POST /reload` - force reload; `GET /config` - config; `GET /status` - config + data |
| **Uploads** (`/api/threads/{id}/uploads`) | `POST /` - upload files (auto-converts PDF/PPT/Excel/Word); `GET /list` - list; `DELETE /{filename}` - delete |
| **Threads** (`/api/threads/{id}`) | `DELETE /` - remove DeerFlow-managed local thread data after LangGraph thread deletion; unexpected failures are logged server-side and return a generic 500 detail |
| **Artifacts** (`/api/threads/{id}/artifacts`) | `GET /{path}` - serve artifacts; active content types (`text/html`, `application/xhtml+xml`, `image/svg+xml`) are always forced as download attachments to reduce XSS risk; `?download=true` still forces download for other file types |
| **Suggestions** (`/api/threads/{id}/suggestions`) | `POST /` - generate follow-up questions; rich list/block model content is normalized before JSON parsing |
Proxied through nginx: `/api/langgraph/*` → LangGraph, all other `/api/*` → Gateway.
### Sandbox System (`packages/harness/deerflow/sandbox/`)
**Interface**: Abstract `Sandbox` with `execute_command`, `read_file`, `write_file`, `list_dir`
**Provider Pattern**: `SandboxProvider` with `acquire`, `get`, `release` lifecycle
**Implementations**:
- `LocalSandboxProvider` - Singleton local filesystem execution with path mappings
- `AioSandboxProvider` (`packages/harness/deerflow/community/`) - Docker-based isolation
**Virtual Path System**:
- Agent sees: `/mnt/user-data/{workspace,uploads,outputs}`, `/mnt/skills`
- Physical: `backend/.deer-flow/threads/{thread_id}/user-data/...`, `deer-flow/skills/`
- Translation: `replace_virtual_path()` / `replace_virtual_paths_in_command()`
- Detection: `is_local_sandbox()` checks `sandbox_id == "local"`
**Sandbox Tools** (in `packages/harness/deerflow/sandbox/tools.py`):
- `bash` - Execute commands with path translation and error handling
- `ls` - Directory listing (tree format, max 2 levels)
- `read_file` - Read file contents with optional line range
- `write_file` - Write/append to files, creates directories
- `str_replace` - Substring replacement (single or all occurrences); same-path serialization is scoped to `(sandbox.id, path)` so isolated sandboxes do not contend on identical virtual paths inside one process
### Subagent System (`packages/harness/deerflow/subagents/`)
**Built-in Agents**: `general-purpose` (all tools except `task`) and `bash` (command specialist)
**Execution**: Dual thread pool - `_scheduler_pool` (3 workers) + `_execution_pool` (3 workers)
**Concurrency**: `MAX_CONCURRENT_SUBAGENTS = 3` enforced by `SubagentLimitMiddleware` (truncates excess tool calls in `after_model`), 15-minute timeout
**Flow**: `task()` tool → `SubagentExecutor` → background thread → poll 5s → SSE events → result
**Events**: `task_started`, `task_running`, `task_completed`/`task_failed`/`task_timed_out`
### Tool System (`packages/harness/deerflow/tools/`)
`get_available_tools(groups, include_mcp, model_name, subagent_enabled)` assembles:
1. **Config-defined tools** - Resolved from `config.yaml` via `resolve_variable()`
2. **MCP tools** - From enabled MCP servers (lazy initialized, cached with mtime invalidation)
3. **Built-in tools**:
- `present_files` - Make output files visible to user (only `/mnt/user-data/outputs`)
- `ask_clarification` - Request clarification (intercepted by ClarificationMiddleware → interrupts)
- `view_image` - Read image as base64 (added only if model supports vision)
4. **Subagent tool** (if enabled):
- `task` - Delegate to subagent (description, prompt, subagent_type, max_turns)
**Community tools** (`packages/harness/deerflow/community/`):
- `tavily/` - Web search (5 results default) and web fetch (4KB limit)
- `jina_ai/` - Web fetch via Jina reader API with readability extraction
- `firecrawl/` - Web scraping via Firecrawl API
**ACP agent tools**:
- `invoke_acp_agent` - Invokes external ACP-compatible agents from `config.yaml`
- ACP launchers must be real ACP adapters. The standard `codex` CLI is not ACP-compatible by itself; configure a wrapper such as `npx -y @zed-industries/codex-acp` or an installed `codex-acp` binary
- Missing ACP executables now return an actionable error message instead of a raw `[Errno 2]`
- Each ACP agent uses a per-thread workspace at `{base_dir}/threads/{thread_id}/acp-workspace/`. The workspace is accessible to the lead agent via the virtual path `/mnt/acp-workspace/` (read-only). In docker sandbox mode, the directory is volume-mounted into the container at `/mnt/acp-workspace` (read-only); in local sandbox mode, path translation is handled by `tools.py`
- `image_search/` - Image search via DuckDuckGo
### MCP System (`packages/harness/deerflow/mcp/`)
- Uses `langchain-mcp-adapters` `MultiServerMCPClient` for multi-server management
- **Lazy initialization**: Tools loaded on first use via `get_cached_mcp_tools()`
- **Cache invalidation**: Detects config file changes via mtime comparison
- **Transports**: stdio (command-based), SSE, HTTP
- **OAuth (HTTP/SSE)**: Supports token endpoint flows (`client_credentials`, `refresh_token`) with automatic token refresh + Authorization header injection
- **Runtime updates**: Gateway API saves to extensions_config.json; LangGraph detects via mtime
### Skills System (`packages/harness/deerflow/skills/`)
- **Location**: `deer-flow/skills/{public,custom}/`
- **Format**: Directory with `SKILL.md` (YAML frontmatter: name, description, license, allowed-tools)
- **Loading**: `load_skills()` recursively scans `skills/{public,custom}` for `SKILL.md`, parses metadata, and reads enabled state from extensions_config.json
- **Injection**: Enabled skills listed in agent system prompt with container paths
- **Installation**: `POST /api/skills/install` extracts .skill ZIP archive to custom/ directory
### Model Factory (`packages/harness/deerflow/models/factory.py`)
- `create_chat_model(name, thinking_enabled)` instantiates LLM from config via reflection
- Supports `thinking_enabled` flag with per-model `when_thinking_enabled` overrides
- Supports `supports_vision` flag for image understanding models
- Config values starting with `$` resolved as environment variables
- Missing provider modules surface actionable install hints from reflection resolvers (for example `uv add langchain-google-genai`)
### IM Channels System (`app/channels/`)
Bridges external messaging platforms (Feishu, Slack, Telegram) to the DeerFlow agent via the LangGraph Server.
**Architecture**: Channels communicate with the LangGraph Server through `langgraph-sdk` HTTP client (same as the frontend), ensuring threads are created and managed server-side.
**Components**:
- `message_bus.py` - Async pub/sub hub (`InboundMessage` → queue → dispatcher; `OutboundMessage` → callbacks → channels)
- `store.py` - JSON-file persistence mapping `channel_name:chat_id[:topic_id]``thread_id` (keys are `channel:chat` for root conversations and `channel:chat:topic` for threaded conversations)
- `manager.py` - Core dispatcher: creates threads via `client.threads.create()`, routes commands, keeps Slack/Telegram on `client.runs.wait()`, and uses `client.runs.stream(["messages-tuple", "values"])` for Feishu incremental outbound updates
- `base.py` - Abstract `Channel` base class (start/stop/send lifecycle)
- `service.py` - Manages lifecycle of all configured channels from `config.yaml`
- `slack.py` / `feishu.py` / `telegram.py` - Platform-specific implementations (`feishu.py` tracks the running card `message_id` in memory and patches the same card in place)
**Message Flow**:
1. External platform -> Channel impl -> `MessageBus.publish_inbound()`
2. `ChannelManager._dispatch_loop()` consumes from queue
3. For chat: look up/create thread on LangGraph Server
4. Feishu chat: `runs.stream()` → accumulate AI text → publish multiple outbound updates (`is_final=False`) → publish final outbound (`is_final=True`)
5. Slack/Telegram chat: `runs.wait()` → extract final response → publish outbound
6. Feishu channel sends one running reply card up front, then patches the same card for each outbound update (card JSON sets `config.update_multi=true` for Feishu's patch API requirement)
7. For commands (`/new`, `/status`, `/models`, `/memory`, `/help`): handle locally or query Gateway API
8. Outbound → channel callbacks → platform reply
**Configuration** (`config.yaml` -> `channels`):
- `langgraph_url` - LangGraph Server URL (default: `http://localhost:2024`)
- `gateway_url` - Gateway API URL for auxiliary commands (default: `http://localhost:8001`)
- In Docker Compose, IM channels run inside the `gateway` container, so `localhost` points back to that container. Use `http://langgraph:2024` / `http://gateway:8001`, or set `DEER_FLOW_CHANNELS_LANGGRAPH_URL` / `DEER_FLOW_CHANNELS_GATEWAY_URL`.
- Per-channel configs: `feishu` (app_id, app_secret), `slack` (bot_token, app_token), `telegram` (bot_token)
### Memory System (`packages/harness/deerflow/agents/memory/`)
**Components**:
- `updater.py` - LLM-based memory updates with fact extraction, whitespace-normalized fact deduplication (trims leading/trailing whitespace before comparing), and atomic file I/O
- `queue.py` - Debounced update queue (per-thread deduplication, configurable wait time)
- `prompt.py` - Prompt templates for memory updates
**Data Structure** (stored in `backend/.deer-flow/memory.json`):
- **User Context**: `workContext`, `personalContext`, `topOfMind` (1-3 sentence summaries)
- **History**: `recentMonths`, `earlierContext`, `longTermBackground`
- **Facts**: Discrete facts with `id`, `content`, `category` (preference/knowledge/context/behavior/goal), `confidence` (0-1), `createdAt`, `source`
**Workflow**:
1. `MemoryMiddleware` filters messages (user inputs + final AI responses) and queues conversation
2. Queue debounces (30s default), batches updates, deduplicates per-thread
3. Background thread invokes LLM to extract context updates and facts
4. Applies updates atomically (temp file + rename) with cache invalidation, skipping duplicate fact content before append
5. Next interaction injects top 15 facts + context into `<memory>` tags in system prompt
Focused regression coverage for the updater lives in `backend/tests/test_memory_updater.py`.
**Configuration** (`config.yaml``memory`):
- `enabled` / `injection_enabled` - Master switches
- `storage_path` - Path to memory.json
- `debounce_seconds` - Wait time before processing (default: 30)
- `model_name` - LLM for updates (null = default model)
- `max_facts` / `fact_confidence_threshold` - Fact storage limits (100 / 0.7)
- `max_injection_tokens` - Token limit for prompt injection (2000)
### Reflection System (`packages/harness/deerflow/reflection/`)
- `resolve_variable(path)` - Import module and return variable (e.g., `module.path:variable_name`)
- `resolve_class(path, base_class)` - Import and validate class against base class
### Config Schema
**`config.yaml`** key sections:
- `models[]` - LLM configs with `use` class path, `supports_thinking`, `supports_vision`, provider-specific fields
- `tools[]` - Tool configs with `use` variable path and `group`
- `tool_groups[]` - Logical groupings for tools
- `sandbox.use` - Sandbox provider class path
- `skills.path` / `skills.container_path` - Host and container paths to skills directory
- `title` - Auto-title generation (enabled, max_words, max_chars, prompt_template)
- `summarization` - Context summarization (enabled, trigger conditions, keep policy)
- `subagents.enabled` - Master switch for subagent delegation
- `memory` - Memory system (enabled, storage_path, debounce_seconds, model_name, max_facts, fact_confidence_threshold, injection_enabled, max_injection_tokens)
**`extensions_config.json`**:
- `mcpServers` - Map of server name → config (enabled, type, command, args, env, url, headers, oauth, description)
- `skills` - Map of skill name → state (enabled)
Both can be modified at runtime via Gateway API endpoints or `DeerFlowClient` methods.
### Embedded Client (`packages/harness/deerflow/client.py`)
`DeerFlowClient` provides direct in-process access to all DeerFlow capabilities without HTTP services. All return types align with the Gateway API response schemas, so consumer code works identically in HTTP and embedded modes.
**Architecture**: Imports the same `deerflow` modules that LangGraph Server and Gateway API use. Shares the same config files and data directories. No FastAPI dependency.
**Agent Conversation** (replaces LangGraph Server):
- `chat(message, thread_id)` — synchronous, returns final text
- `stream(message, thread_id)` — yields `StreamEvent` aligned with LangGraph SSE protocol:
- `"values"` — full state snapshot (title, messages, artifacts)
- `"messages-tuple"` — per-message update (AI text, tool calls, tool results)
- `"end"` — stream finished
- Agent created lazily via `create_agent()` + `_build_middlewares()`, same as `make_lead_agent`
- Supports `checkpointer` parameter for state persistence across turns
- `reset_agent()` forces agent recreation (e.g. after memory or skill changes)
**Gateway Equivalent Methods** (replaces Gateway API):
| Category | Methods | Return format |
|----------|---------|---------------|
| Models | `list_models()`, `get_model(name)` | `{"models": [...]}`, `{name, display_name, ...}` |
| MCP | `get_mcp_config()`, `update_mcp_config(servers)` | `{"mcp_servers": {...}}` |
| Skills | `list_skills()`, `get_skill(name)`, `update_skill(name, enabled)`, `install_skill(path)` | `{"skills": [...]}` |
| Memory | `get_memory()`, `reload_memory()`, `get_memory_config()`, `get_memory_status()` | dict |
| Uploads | `upload_files(thread_id, files)`, `list_uploads(thread_id)`, `delete_upload(thread_id, filename)` | `{"success": true, "files": [...]}`, `{"files": [...], "count": N}` |
| Artifacts | `get_artifact(thread_id, path)``(bytes, mime_type)` | tuple |
**Key difference from Gateway**: Upload accepts local `Path` objects instead of HTTP `UploadFile`, rejects directory paths before copying, and reuses a single worker when document conversion must run inside an active event loop. Artifact returns `(bytes, mime_type)` instead of HTTP Response. The new Gateway-only thread cleanup route deletes `.deer-flow/threads/{thread_id}` after LangGraph thread deletion; there is no matching `DeerFlowClient` method yet. `update_mcp_config()` and `update_skill()` automatically invalidate the cached agent.
**Tests**: `tests/test_client.py` (77 unit tests including `TestGatewayConformance`), `tests/test_client_live.py` (live integration tests, requires config.yaml)
**Gateway Conformance Tests** (`TestGatewayConformance`): Validate that every dict-returning client method conforms to the corresponding Gateway Pydantic response model. Each test parses the client output through the Gateway model — if Gateway adds a required field that the client doesn't provide, Pydantic raises `ValidationError` and CI catches the drift. Covers: `ModelsListResponse`, `ModelResponse`, `SkillsListResponse`, `SkillResponse`, `SkillInstallResponse`, `McpConfigResponse`, `UploadResponse`, `MemoryConfigResponse`, `MemoryStatusResponse`.
## Development Workflow
### Test-Driven Development (TDD) — MANDATORY
**Every new feature or bug fix MUST be accompanied by unit tests. No exceptions.**
- Write tests in `backend/tests/` following the existing naming convention `test_<feature>.py`
- Run the full suite before and after your change: `make test`
- Tests must pass before a feature is considered complete
- For lightweight config/utility modules, prefer pure unit tests with no external dependencies
- If a module causes circular import issues in tests, add a `sys.modules` mock in `tests/conftest.py` (see existing example for `deerflow.subagents.executor`)
```bash
# Run all tests
make test
# Run a specific test file
PYTHONPATH=. uv run pytest tests/test_<feature>.py -v
```
### Running the Full Application
From the **project root** directory:
```bash
make dev
```
This starts all services and makes the application available at `http://localhost:2026`.
**All startup modes:**
| | **Local Foreground** | **Local Daemon** | **Docker Dev** | **Docker Prod** |
|---|---|---|---|---|
| **Dev** | `./scripts/serve.sh --dev`<br/>`make dev` | `./scripts/serve.sh --dev --daemon`<br/>`make dev-daemon` | `./scripts/docker.sh start`<br/>`make docker-start` | — |
| **Dev + Gateway** | `./scripts/serve.sh --dev --gateway`<br/>`make dev-pro` | `./scripts/serve.sh --dev --gateway --daemon`<br/>`make dev-daemon-pro` | `./scripts/docker.sh start --gateway`<br/>`make docker-start-pro` | — |
| **Prod** | `./scripts/serve.sh --prod`<br/>`make start` | `./scripts/serve.sh --prod --daemon`<br/>`make start-daemon` | — | `./scripts/deploy.sh`<br/>`make up` |
| **Prod + Gateway** | `./scripts/serve.sh --prod --gateway`<br/>`make start-pro` | `./scripts/serve.sh --prod --gateway --daemon`<br/>`make start-daemon-pro` | — | `./scripts/deploy.sh --gateway`<br/>`make up-pro` |
| Action | Local | Docker Dev | Docker Prod |
|---|---|---|---|
| **Stop** | `./scripts/serve.sh --stop`<br/>`make stop` | `./scripts/docker.sh stop`<br/>`make docker-stop` | `./scripts/deploy.sh down`<br/>`make down` |
| **Restart** | `./scripts/serve.sh --restart [flags]` | `./scripts/docker.sh restart` | — |
Gateway mode embeds the agent runtime in Gateway, no LangGraph server.
**Nginx routing**:
- Standard mode: `/api/langgraph/*` → LangGraph Server (2024)
- Gateway mode: `/api/langgraph/*` → Gateway embedded runtime (8001) (via envsubst)
- `/api/*` (other) → Gateway API (8001)
- `/` (non-API) → Frontend (3000)
### Running Backend Services Separately
From the **backend** directory:
```bash
# Terminal 1: LangGraph server
make dev
# Terminal 2: Gateway API
make gateway
```
Direct access (without nginx):
- LangGraph: `http://localhost:2024`
- Gateway: `http://localhost:8001`
### Frontend Configuration
The frontend uses environment variables to connect to backend services:
- `NEXT_PUBLIC_LANGGRAPH_BASE_URL` - Defaults to `/api/langgraph` (through nginx)
- `NEXT_PUBLIC_BACKEND_BASE_URL` - Defaults to empty string (through nginx)
When using `make dev` from root, the frontend automatically connects through nginx.
## Key Features
### File Upload
Multi-file upload with automatic document conversion:
- Endpoint: `POST /api/threads/{thread_id}/uploads`
- Supports: PDF, PPT, Excel, Word documents (converted via `markitdown`)
- Rejects directory inputs before copying so uploads stay all-or-nothing
- Reuses one conversion worker per request when called from an active event loop
- Files stored in thread-isolated directories
- Agent receives uploaded file list via `UploadsMiddleware`
See [docs/FILE_UPLOAD.md](docs/FILE_UPLOAD.md) for details.
### Plan Mode
TodoList middleware for complex multi-step tasks:
- Controlled via runtime config: `config.configurable.is_plan_mode = True`
- Provides `write_todos` tool for task tracking
- One task in_progress at a time, real-time updates
See [docs/plan_mode_usage.md](docs/plan_mode_usage.md) for details.
### Context Summarization
Automatic conversation summarization when approaching token limits:
- Configured in `config.yaml` under `summarization` key
- Trigger types: tokens, messages, or fraction of max input
- Keeps recent messages while summarizing older ones
See [docs/summarization.md](docs/summarization.md) for details.
### Vision Support
For models with `supports_vision: true`:
- `ViewImageMiddleware` processes images in conversation
- `view_image_tool` added to agent's toolset
- Images automatically converted to base64 and injected into state
## Code Style
- Uses `ruff` for linting and formatting
- Line length: 240 characters
- Python 3.12+ with type hints
- Double quotes, space indentation
## Documentation
See `docs/` directory for detailed documentation:
- [CONFIGURATION.md](docs/CONFIGURATION.md) - Configuration options
- [ARCHITECTURE.md](docs/ARCHITECTURE.md) - Architecture details
- [API.md](docs/API.md) - API reference
- [SETUP.md](docs/SETUP.md) - Setup guide
- [FILE_UPLOAD.md](docs/FILE_UPLOAD.md) - File upload feature
- [PATH_EXAMPLES.md](docs/PATH_EXAMPLES.md) - Path types and usage
- [summarization.md](docs/summarization.md) - Context summarization
- [plan_mode_usage.md](docs/plan_mode_usage.md) - Plan mode with TodoList
+426
View File
@@ -0,0 +1,426 @@
# Contributing to DeerFlow Backend
Thank you for your interest in contributing to DeerFlow! This document provides guidelines and instructions for contributing to the backend codebase.
## Table of Contents
- [Getting Started](#getting-started)
- [Development Setup](#development-setup)
- [Project Structure](#project-structure)
- [Code Style](#code-style)
- [Making Changes](#making-changes)
- [Testing](#testing)
- [Pull Request Process](#pull-request-process)
- [Architecture Guidelines](#architecture-guidelines)
## Getting Started
### Prerequisites
- Python 3.12 or higher
- [uv](https://docs.astral.sh/uv/) package manager
- Git
- Docker (optional, for Docker sandbox testing)
### Fork and Clone
1. Fork the repository on GitHub
2. Clone your fork locally:
```bash
git clone https://github.com/YOUR_USERNAME/deer-flow.git
cd deer-flow
```
## Development Setup
### Install Dependencies
```bash
# From project root
cp config.example.yaml config.yaml
# Install backend dependencies
cd backend
make install
```
### Configure Environment
Set up your API keys for testing:
```bash
export OPENAI_API_KEY="your-api-key"
# Add other keys as needed
```
### Run the Development Server
```bash
# Terminal 1: LangGraph server
make dev
# Terminal 2: Gateway API
make gateway
```
## Project Structure
```
backend/src/
├── agents/ # Agent system
│ ├── lead_agent/ # Main agent implementation
│ │ └── agent.py # Agent factory and creation
│ ├── middlewares/ # Agent middlewares
│ │ ├── thread_data_middleware.py
│ │ ├── sandbox_middleware.py
│ │ ├── title_middleware.py
│ │ ├── uploads_middleware.py
│ │ ├── view_image_middleware.py
│ │ └── clarification_middleware.py
│ └── thread_state.py # Thread state definition
├── gateway/ # FastAPI Gateway
│ ├── app.py # FastAPI application
│ └── routers/ # Route handlers
│ ├── models.py # /api/models endpoints
│ ├── mcp.py # /api/mcp endpoints
│ ├── skills.py # /api/skills endpoints
│ ├── artifacts.py # /api/threads/.../artifacts
│ └── uploads.py # /api/threads/.../uploads
├── sandbox/ # Sandbox execution
│ ├── __init__.py # Sandbox interface
│ ├── local.py # Local sandbox provider
│ └── tools.py # Sandbox tools (bash, file ops)
├── tools/ # Agent tools
│ └── builtins/ # Built-in tools
│ ├── present_file_tool.py
│ ├── ask_clarification_tool.py
│ └── view_image_tool.py
├── mcp/ # MCP integration
│ └── manager.py # MCP server management
├── models/ # Model system
│ └── factory.py # Model factory
├── skills/ # Skills system
│ └── loader.py # Skills loader
├── config/ # Configuration
│ ├── app_config.py # Main app config
│ ├── extensions_config.py # Extensions config
│ └── summarization_config.py
├── community/ # Community tools
│ ├── tavily/ # Tavily web search
│ ├── jina/ # Jina web fetch
│ ├── firecrawl/ # Firecrawl scraping
│ └── aio_sandbox/ # Docker sandbox
├── reflection/ # Dynamic loading
│ └── __init__.py # Module resolution
└── utils/ # Utilities
└── __init__.py
```
## Code Style
### Linting and Formatting
We use `ruff` for both linting and formatting:
```bash
# Check for issues
make lint
# Auto-fix and format
make format
```
### Style Guidelines
- **Line length**: 240 characters maximum
- **Python version**: 3.12+ features allowed
- **Type hints**: Use type hints for function signatures
- **Quotes**: Double quotes for strings
- **Indentation**: 4 spaces (no tabs)
- **Imports**: Group by standard library, third-party, local
### Docstrings
Use docstrings for public functions and classes:
```python
def create_chat_model(name: str, thinking_enabled: bool = False) -> BaseChatModel:
"""Create a chat model instance from configuration.
Args:
name: The model name as defined in config.yaml
thinking_enabled: Whether to enable extended thinking
Returns:
A configured LangChain chat model instance
Raises:
ValueError: If the model name is not found in configuration
"""
...
```
## Making Changes
### Branch Naming
Use descriptive branch names:
- `feature/add-new-tool` - New features
- `fix/sandbox-timeout` - Bug fixes
- `docs/update-readme` - Documentation
- `refactor/config-system` - Code refactoring
### Commit Messages
Write clear, concise commit messages:
```
feat: add support for Claude 3.5 model
- Add model configuration in config.yaml
- Update model factory to handle Claude-specific settings
- Add tests for new model
```
Prefix types:
- `feat:` - New feature
- `fix:` - Bug fix
- `docs:` - Documentation
- `refactor:` - Code refactoring
- `test:` - Tests
- `chore:` - Build/config changes
## Testing
### Running Tests
```bash
uv run pytest
```
### Writing Tests
Place tests in the `tests/` directory mirroring the source structure:
```
tests/
├── test_models/
│ └── test_factory.py
├── test_sandbox/
│ └── test_local.py
└── test_gateway/
└── test_models_router.py
```
Example test:
```python
import pytest
from deerflow.models.factory import create_chat_model
def test_create_chat_model_with_valid_name():
"""Test that a valid model name creates a model instance."""
model = create_chat_model("gpt-4")
assert model is not None
def test_create_chat_model_with_invalid_name():
"""Test that an invalid model name raises ValueError."""
with pytest.raises(ValueError):
create_chat_model("nonexistent-model")
```
## Pull Request Process
### Before Submitting
1. **Ensure tests pass**: `uv run pytest`
2. **Run linter**: `make lint`
3. **Format code**: `make format`
4. **Update documentation** if needed
### PR Description
Include in your PR description:
- **What**: Brief description of changes
- **Why**: Motivation for the change
- **How**: Implementation approach
- **Testing**: How you tested the changes
### Review Process
1. Submit PR with clear description
2. Address review feedback
3. Ensure CI passes
4. Maintainer will merge when approved
## Architecture Guidelines
### Adding New Tools
1. Create tool in `packages/harness/deerflow/tools/builtins/` or `packages/harness/deerflow/community/`:
```python
# packages/harness/deerflow/tools/builtins/my_tool.py
from langchain_core.tools import tool
@tool
def my_tool(param: str) -> str:
"""Tool description for the agent.
Args:
param: Description of the parameter
Returns:
Description of return value
"""
return f"Result: {param}"
```
2. Register in `config.yaml`:
```yaml
tools:
- name: my_tool
group: my_group
use: deerflow.tools.builtins.my_tool:my_tool
```
### Adding New Middleware
1. Create middleware in `packages/harness/deerflow/agents/middlewares/`:
```python
# packages/harness/deerflow/agents/middlewares/my_middleware.py
from langchain.agents.middleware import BaseMiddleware
from langchain_core.runnables import RunnableConfig
class MyMiddleware(BaseMiddleware):
"""Middleware description."""
def transform_state(self, state: dict, config: RunnableConfig) -> dict:
"""Transform the state before agent execution."""
# Modify state as needed
return state
```
2. Register in `packages/harness/deerflow/agents/lead_agent/agent.py`:
```python
middlewares = [
ThreadDataMiddleware(),
SandboxMiddleware(),
MyMiddleware(), # Add your middleware
TitleMiddleware(),
ClarificationMiddleware(),
]
```
### Adding New API Endpoints
1. Create router in `app/gateway/routers/`:
```python
# app/gateway/routers/my_router.py
from fastapi import APIRouter
router = APIRouter(prefix="/my-endpoint", tags=["my-endpoint"])
@router.get("/")
async def get_items():
"""Get all items."""
return {"items": []}
@router.post("/")
async def create_item(data: dict):
"""Create a new item."""
return {"created": data}
```
2. Register in `app/gateway/app.py`:
```python
from app.gateway.routers import my_router
app.include_router(my_router.router)
```
### Configuration Changes
When adding new configuration options:
1. Update `packages/harness/deerflow/config/app_config.py` with new fields
2. Add default values in `config.example.yaml`
3. Document in `docs/CONFIGURATION.md`
### MCP Server Integration
To add support for a new MCP server:
1. Add configuration in `extensions_config.json`:
```json
{
"mcpServers": {
"my-server": {
"enabled": true,
"type": "stdio",
"command": "npx",
"args": ["-y", "@my-org/mcp-server"],
"description": "My MCP Server"
}
}
}
```
2. Update `extensions_config.example.json` with the new server
### Skills Development
To create a new skill:
1. Create directory in `skills/public/` or `skills/custom/`:
```
skills/public/my-skill/
└── SKILL.md
```
2. Write `SKILL.md` with YAML front matter:
```markdown
---
name: My Skill
description: What this skill does
license: MIT
allowed-tools:
- read_file
- write_file
- bash
---
# My Skill
Instructions for the agent when this skill is enabled...
```
## Questions?
If you have questions about contributing:
1. Check existing documentation in `docs/`
2. Look for similar issues or PRs on GitHub
3. Open a discussion or issue on GitHub
Thank you for contributing to DeerFlow!
+100
View File
@@ -0,0 +1,100 @@
# Backend Dockerfile — multi-stage build
# Stage 1 (builder): compiles native Python extensions with build-essential
# Stage 2 (dev): retains toolchain for dev containers (uv sync at startup)
# Stage 3 (runtime): clean image without compiler toolchain for production
# UV source image (override for restricted networks that cannot reach ghcr.io)
ARG UV_IMAGE=ghcr.io/astral-sh/uv:0.7.20
FROM ${UV_IMAGE} AS uv-source
# ── Stage 1: Builder ──────────────────────────────────────────────────────────
FROM python:3.12-slim-bookworm AS builder
ARG NODE_MAJOR=22
ARG NODE_VERSION=22.16.0
ARG APT_MIRROR
ARG UV_INDEX_URL
ARG NODE_DIST_URL
# Optionally override apt mirror for restricted networks (e.g. APT_MIRROR=mirrors.byted.org)
RUN if [ -n "${APT_MIRROR}" ]; then \
sed -i "s|deb.debian.org|${APT_MIRROR}|g" /etc/apt/sources.list.d/debian.sources 2>/dev/null || true; \
sed -i "s|deb.debian.org|${APT_MIRROR}|g" /etc/apt/sources.list 2>/dev/null || true; \
fi
# Install build tools + Node.js (build-essential needed for native Python extensions)
# NODE_DIST_URL: base URL for Node.js binary tarballs in restricted networks.
# npmmirror: https://registry.npmmirror.com/-/binary/node
# official: https://nodejs.org/dist (default, via nodesource apt)
RUN apt-get update && apt-get install -y \
curl \
build-essential \
gnupg \
ca-certificates \
xz-utils \
&& if [ -n "${NODE_DIST_URL}" ]; then \
curl -fsSL "${NODE_DIST_URL}/v${NODE_VERSION}/node-v${NODE_VERSION}-linux-x64.tar.xz" \
| tar -xJ --strip-components=1 -C /usr/local \
&& ln -sf /usr/local/bin/node /usr/bin/node \
&& ln -sf /usr/local/lib/node_modules /usr/lib/node_modules; \
else \
mkdir -p /etc/apt/keyrings \
&& curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg \
&& echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_${NODE_MAJOR}.x nodistro main" > /etc/apt/sources.list.d/nodesource.list \
&& apt-get update \
&& apt-get install -y nodejs; \
fi \
&& rm -rf /var/lib/apt/lists/*
# Install uv (source image overridable via UV_IMAGE build arg)
COPY --from=uv-source /uv /uvx /usr/local/bin/
# Set working directory
WORKDIR /app
# Copy backend source code
COPY backend ./backend
# Install dependencies with cache mount
RUN --mount=type=cache,target=/root/.cache/uv \
sh -c "cd backend && UV_INDEX_URL=${UV_INDEX_URL:-https://pypi.org/simple} uv sync"
# ── Stage 2: Dev ──────────────────────────────────────────────────────────────
# Retains compiler toolchain from builder so startup-time `uv sync` can build
# source distributions in development containers.
FROM builder AS dev
# Install Docker CLI (for DooD: allows starting sandbox containers via host Docker socket)
COPY --from=docker:cli /usr/local/bin/docker /usr/local/bin/docker
EXPOSE 8001 2024
CMD ["sh", "-c", "cd backend && PYTHONPATH=. uv run uvicorn app.gateway.app:app --host 0.0.0.0 --port 8001"]
# ── Stage 3: Runtime ──────────────────────────────────────────────────────────
# Clean image without build-essential — reduces size (~200 MB) and attack surface.
FROM python:3.12-slim-bookworm
# Copy Node.js runtime from builder (provides npx for MCP servers)
COPY --from=builder /usr/bin/node /usr/bin/node
COPY --from=builder /usr/lib/node_modules /usr/lib/node_modules
RUN ln -s ../lib/node_modules/npm/bin/npm-cli.js /usr/bin/npm \
&& ln -s ../lib/node_modules/npm/bin/npx-cli.js /usr/bin/npx
# Install Docker CLI (for DooD: allows starting sandbox containers via host Docker socket)
COPY --from=docker:cli /usr/local/bin/docker /usr/local/bin/docker
# Install uv (source image overridable via UV_IMAGE build arg)
COPY --from=uv-source /uv /uvx /usr/local/bin/
# Set working directory
WORKDIR /app
# Copy backend with pre-built virtualenv from builder
COPY --from=builder /app/backend ./backend
# Expose ports (gateway: 8001, langgraph: 2024)
EXPOSE 8001 2024
# Default command (can be overridden in docker-compose)
CMD ["sh", "-c", "cd backend && PYTHONPATH=. uv run uvicorn app.gateway.app:app --host 0.0.0.0 --port 8001"]
+18
View File
@@ -0,0 +1,18 @@
install:
uv sync
dev:
uv run langgraph dev --no-browser --no-reload --n-jobs-per-worker 10
gateway:
PYTHONPATH=. uv run uvicorn app.gateway.app:app --host 0.0.0.0 --port 8001
test:
PYTHONPATH=. uv run pytest tests/ -v
lint:
uvx ruff check .
uvx ruff format --check .
format:
uvx ruff check . --fix && uvx ruff format .
+418
View File
@@ -0,0 +1,418 @@
# DeerFlow Backend
DeerFlow is a LangGraph-based AI super agent with sandbox execution, persistent memory, and extensible tool integration. The backend enables AI agents to execute code, browse the web, manage files, delegate tasks to subagents, and retain context across conversations - all in isolated, per-thread environments.
---
## Architecture
```
┌──────────────────────────────────────┐
│ Nginx (Port 2026) │
│ Unified reverse proxy │
└───────┬──────────────────┬───────────┘
│ │
/api/langgraph/* │ │ /api/* (other)
▼ ▼
┌────────────────────┐ ┌────────────────────────┐
│ LangGraph Server │ │ Gateway API (8001) │
│ (Port 2024) │ │ FastAPI REST │
│ │ │ │
│ ┌────────────────┐ │ │ Models, MCP, Skills, │
│ │ Lead Agent │ │ │ Memory, Uploads, │
│ │ ┌──────────┐ │ │ │ Artifacts │
│ │ │Middleware│ │ │ └────────────────────────┘
│ │ │ Chain │ │ │
│ │ └──────────┘ │ │
│ │ ┌──────────┐ │ │
│ │ │ Tools │ │ │
│ │ └──────────┘ │ │
│ │ ┌──────────┐ │ │
│ │ │Subagents │ │ │
│ │ └──────────┘ │ │
│ └────────────────┘ │
└────────────────────┘
```
**Request Routing** (via Nginx):
- `/api/langgraph/*` → LangGraph Server - agent interactions, threads, streaming
- `/api/*` (other) → Gateway API - models, MCP, skills, memory, artifacts, uploads, thread-local cleanup
- `/` (non-API) → Frontend - Next.js web interface
---
## Core Components
### Lead Agent
The single LangGraph agent (`lead_agent`) is the runtime entry point, created via `make_lead_agent(config)`. It combines:
- **Dynamic model selection** with thinking and vision support
- **Middleware chain** for cross-cutting concerns (9 middlewares)
- **Tool system** with sandbox, MCP, community, and built-in tools
- **Subagent delegation** for parallel task execution
- **System prompt** with skills injection, memory context, and working directory guidance
### Middleware Chain
Middlewares execute in strict order, each handling a specific concern:
| # | Middleware | Purpose |
|---|-----------|---------|
| 1 | **ThreadDataMiddleware** | Creates per-thread isolated directories (workspace, uploads, outputs) |
| 2 | **UploadsMiddleware** | Injects newly uploaded files into conversation context |
| 3 | **SandboxMiddleware** | Acquires sandbox environment for code execution |
| 4 | **SummarizationMiddleware** | Reduces context when approaching token limits (optional) |
| 5 | **TodoListMiddleware** | Tracks multi-step tasks in plan mode (optional) |
| 6 | **TitleMiddleware** | Auto-generates conversation titles after first exchange |
| 7 | **MemoryMiddleware** | Queues conversations for async memory extraction |
| 8 | **ViewImageMiddleware** | Injects image data for vision-capable models (conditional) |
| 9 | **ClarificationMiddleware** | Intercepts clarification requests and interrupts execution (must be last) |
### Sandbox System
Per-thread isolated execution with virtual path translation:
- **Abstract interface**: `execute_command`, `read_file`, `write_file`, `list_dir`
- **Providers**: `LocalSandboxProvider` (filesystem) and `AioSandboxProvider` (Docker, in community/)
- **Virtual paths**: `/mnt/user-data/{workspace,uploads,outputs}` → thread-specific physical directories
- **Skills path**: `/mnt/skills``deer-flow/skills/` directory
- **Skills loading**: Recursively discovers nested `SKILL.md` files under `skills/{public,custom}` and preserves nested container paths
- **File-write safety**: `str_replace` serializes read-modify-write per `(sandbox.id, path)` so isolated sandboxes keep concurrency even when virtual paths match
- **Tools**: `bash`, `ls`, `read_file`, `write_file`, `str_replace` (`bash` is disabled by default when using `LocalSandboxProvider`; use `AioSandboxProvider` for isolated shell access)
### Subagent System
Async task delegation with concurrent execution:
- **Built-in agents**: `general-purpose` (full toolset) and `bash` (command specialist, exposed only when shell access is available)
- **Concurrency**: Max 3 subagents per turn, 15-minute timeout
- **Execution**: Background thread pools with status tracking and SSE events
- **Flow**: Agent calls `task()` tool → executor runs subagent in background → polls for completion → returns result
### Memory System
LLM-powered persistent context retention across conversations:
- **Automatic extraction**: Analyzes conversations for user context, facts, and preferences
- **Structured storage**: User context (work, personal, top-of-mind), history, and confidence-scored facts
- **Debounced updates**: Batches updates to minimize LLM calls (configurable wait time)
- **System prompt injection**: Top facts + context injected into agent prompts
- **Storage**: JSON file with mtime-based cache invalidation
### Tool Ecosystem
| Category | Tools |
|----------|-------|
| **Sandbox** | `bash`, `ls`, `read_file`, `write_file`, `str_replace` |
| **Built-in** | `present_files`, `ask_clarification`, `view_image`, `task` (subagent) |
| **Community** | Tavily (web search), Jina AI (web fetch), Firecrawl (scraping), DuckDuckGo (image search) |
| **MCP** | Any Model Context Protocol server (stdio, SSE, HTTP transports) |
| **Skills** | Domain-specific workflows injected via system prompt |
### Gateway API
FastAPI application providing REST endpoints for frontend integration:
| Route | Purpose |
|-------|---------|
| `GET /api/models` | List available LLM models |
| `GET/PUT /api/mcp/config` | Manage MCP server configurations |
| `GET/PUT /api/skills` | List and manage skills |
| `POST /api/skills/install` | Install skill from `.skill` archive |
| `GET /api/memory` | Retrieve memory data |
| `POST /api/memory/reload` | Force memory reload |
| `GET /api/memory/config` | Memory configuration |
| `GET /api/memory/status` | Combined config + data |
| `POST /api/threads/{id}/uploads` | Upload files (auto-converts PDF/PPT/Excel/Word to Markdown, rejects directory paths) |
| `GET /api/threads/{id}/uploads/list` | List uploaded files |
| `DELETE /api/threads/{id}` | Delete DeerFlow-managed local thread data after LangGraph thread deletion; unexpected failures are logged server-side and return a generic 500 detail |
| `GET /api/threads/{id}/artifacts/{path}` | Serve generated artifacts |
### IM Channels
The IM bridge supports Feishu, Slack, and Telegram. Slack and Telegram still use the final `runs.wait()` response path, while Feishu now streams through `runs.stream(["messages-tuple", "values"])` and updates a single in-thread card in place.
For Feishu card updates, DeerFlow stores the running card's `message_id` per inbound message and patches that same card until the run finishes, preserving the existing `OK` / `DONE` reaction flow.
---
## Quick Start
### Prerequisites
- Python 3.12+
- [uv](https://docs.astral.sh/uv/) package manager
- API keys for your chosen LLM provider
### Installation
```bash
cd deer-flow
# Copy configuration files
cp config.example.yaml config.yaml
# Install backend dependencies
cd backend
make install
```
### Configuration
Edit `config.yaml` in the project root:
```yaml
models:
- name: gpt-4o
display_name: GPT-4o
use: langchain_openai:ChatOpenAI
model: gpt-4o
api_key: $OPENAI_API_KEY
supports_thinking: false
supports_vision: true
- name: gpt-5-responses
display_name: GPT-5 (Responses API)
use: langchain_openai:ChatOpenAI
model: gpt-5
api_key: $OPENAI_API_KEY
use_responses_api: true
output_version: responses/v1
supports_vision: true
```
Set your API keys:
```bash
export OPENAI_API_KEY="your-api-key-here"
```
### Running
**Full Application** (from project root):
```bash
make dev # Starts LangGraph + Gateway + Frontend + Nginx
```
Access at: http://localhost:2026
**Backend Only** (from backend directory):
```bash
# Terminal 1: LangGraph server
make dev
# Terminal 2: Gateway API
make gateway
```
Direct access: LangGraph at http://localhost:2024, Gateway at http://localhost:8001
---
## Project Structure
```
backend/
├── src/
│ ├── agents/ # Agent system
│ │ ├── lead_agent/ # Main agent (factory, prompts)
│ │ ├── middlewares/ # 9 middleware components
│ │ ├── memory/ # Memory extraction & storage
│ │ └── thread_state.py # ThreadState schema
│ ├── gateway/ # FastAPI Gateway API
│ │ ├── app.py # Application setup
│ │ └── routers/ # 6 route modules
│ ├── sandbox/ # Sandbox execution
│ │ ├── local/ # Local filesystem provider
│ │ ├── sandbox.py # Abstract interface
│ │ ├── tools.py # bash, ls, read/write/str_replace
│ │ └── middleware.py # Sandbox lifecycle
│ ├── subagents/ # Subagent delegation
│ │ ├── builtins/ # general-purpose, bash agents
│ │ ├── executor.py # Background execution engine
│ │ └── registry.py # Agent registry
│ ├── tools/builtins/ # Built-in tools
│ ├── mcp/ # MCP protocol integration
│ ├── models/ # Model factory
│ ├── skills/ # Skill discovery & loading
│ ├── config/ # Configuration system
│ ├── community/ # Community tools & providers
│ ├── reflection/ # Dynamic module loading
│ └── utils/ # Utilities
├── docs/ # Documentation
├── tests/ # Test suite
├── langgraph.json # LangGraph server configuration
├── pyproject.toml # Python dependencies
├── Makefile # Development commands
└── Dockerfile # Container build
```
---
## Configuration
### Main Configuration (`config.yaml`)
Place in project root. Config values starting with `$` resolve as environment variables.
Key sections:
- `models` - LLM configurations with class paths, API keys, thinking/vision flags
- `tools` - Tool definitions with module paths and groups
- `tool_groups` - Logical tool groupings
- `sandbox` - Execution environment provider
- `skills` - Skills directory paths
- `title` - Auto-title generation settings
- `summarization` - Context summarization settings
- `subagents` - Subagent system (enabled/disabled)
- `memory` - Memory system settings (enabled, storage, debounce, facts limits)
Provider note:
- `models[*].use` references provider classes by module path (for example `langchain_openai:ChatOpenAI`).
- If a provider module is missing, DeerFlow now returns an actionable error with install guidance (for example `uv add langchain-google-genai`).
### Extensions Configuration (`extensions_config.json`)
MCP servers and skill states in a single file:
```json
{
"mcpServers": {
"github": {
"enabled": true,
"type": "stdio",
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-github"],
"env": {"GITHUB_TOKEN": "$GITHUB_TOKEN"}
},
"secure-http": {
"enabled": true,
"type": "http",
"url": "https://api.example.com/mcp",
"oauth": {
"enabled": true,
"token_url": "https://auth.example.com/oauth/token",
"grant_type": "client_credentials",
"client_id": "$MCP_OAUTH_CLIENT_ID",
"client_secret": "$MCP_OAUTH_CLIENT_SECRET"
}
}
},
"skills": {
"pdf-processing": {"enabled": true}
}
}
```
### Environment Variables
- `DEER_FLOW_CONFIG_PATH` - Override config.yaml location
- `DEER_FLOW_EXTENSIONS_CONFIG_PATH` - Override extensions_config.json location
- Model API keys: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `DEEPSEEK_API_KEY`, etc.
- Tool API keys: `TAVILY_API_KEY`, `GITHUB_TOKEN`, etc.
### LangSmith Tracing
DeerFlow has built-in [LangSmith](https://smith.langchain.com) integration for observability. When enabled, all LLM calls, agent runs, tool executions, and middleware processing are traced and visible in the LangSmith dashboard.
**Setup:**
1. Sign up at [smith.langchain.com](https://smith.langchain.com) and create a project.
2. Add the following to your `.env` file in the project root:
```bash
LANGSMITH_TRACING=true
LANGSMITH_ENDPOINT=https://api.smith.langchain.com
LANGSMITH_API_KEY=lsv2_pt_xxxxxxxxxxxxxxxx
LANGSMITH_PROJECT=xxx
```
**Legacy variables:** The `LANGCHAIN_TRACING_V2`, `LANGCHAIN_API_KEY`, `LANGCHAIN_PROJECT`, and `LANGCHAIN_ENDPOINT` variables are also supported for backward compatibility. `LANGSMITH_*` variables take precedence when both are set.
### Langfuse Tracing
DeerFlow also supports [Langfuse](https://langfuse.com) observability for LangChain-compatible runs.
Add the following to your `.env` file:
```bash
LANGFUSE_TRACING=true
LANGFUSE_PUBLIC_KEY=pk-lf-xxxxxxxxxxxxxxxx
LANGFUSE_SECRET_KEY=sk-lf-xxxxxxxxxxxxxxxx
LANGFUSE_BASE_URL=https://cloud.langfuse.com
```
If you are using a self-hosted Langfuse deployment, set `LANGFUSE_BASE_URL` to your Langfuse host.
### Dual Provider Behavior
If both LangSmith and Langfuse are enabled, DeerFlow initializes and attaches both callbacks so the same run data is reported to both systems.
If a provider is explicitly enabled but required credentials are missing, or the provider callback cannot be initialized, DeerFlow raises an error when tracing is initialized during model creation instead of silently disabling tracing.
**Docker:** In `docker-compose.yaml`, tracing is disabled by default (`LANGSMITH_TRACING=false`). Set `LANGSMITH_TRACING=true` and/or `LANGFUSE_TRACING=true` in your `.env`, together with the required credentials, to enable tracing in containerized deployments.
---
## Development
### Commands
```bash
make install # Install dependencies
make dev # Run LangGraph server (port 2024)
make gateway # Run Gateway API (port 8001)
make lint # Run linter (ruff)
make format # Format code (ruff)
```
### Code Style
- **Linter/Formatter**: `ruff`
- **Line length**: 240 characters
- **Python**: 3.12+ with type hints
- **Quotes**: Double quotes
- **Indentation**: 4 spaces
### Testing
```bash
uv run pytest
```
---
## Technology Stack
- **LangGraph** (1.0.6+) - Agent framework and multi-agent orchestration
- **LangChain** (1.2.3+) - LLM abstractions and tool system
- **FastAPI** (0.115.0+) - Gateway REST API
- **langchain-mcp-adapters** - Model Context Protocol support
- **agent-sandbox** - Sandboxed code execution
- **markitdown** - Multi-format document conversion
- **tavily-python** / **firecrawl-py** - Web search and scraping
---
## Documentation
- [Configuration Guide](docs/CONFIGURATION.md)
- [Architecture Details](docs/ARCHITECTURE.md)
- [API Reference](docs/API.md)
- [File Upload](docs/FILE_UPLOAD.md)
- [Path Examples](docs/PATH_EXAMPLES.md)
- [Context Summarization](docs/summarization.md)
- [Plan Mode](docs/plan_mode_usage.md)
- [Setup Guide](docs/SETUP.md)
---
## License
See the [LICENSE](../LICENSE) file in the project root.
## Contributing
See [CONTRIBUTING.md](CONTRIBUTING.md) for contribution guidelines.
View File
+16
View File
@@ -0,0 +1,16 @@
"""IM Channel integration for DeerFlow.
Provides a pluggable channel system that connects external messaging platforms
(Feishu/Lark, Slack, Telegram) to the DeerFlow agent via the ChannelManager,
which uses ``langgraph-sdk`` to communicate with the underlying LangGraph Server.
"""
from app.channels.base import Channel
from app.channels.message_bus import InboundMessage, MessageBus, OutboundMessage
__all__ = [
"Channel",
"InboundMessage",
"MessageBus",
"OutboundMessage",
]
+108
View File
@@ -0,0 +1,108 @@
"""Abstract base class for IM channels."""
from __future__ import annotations
import logging
from abc import ABC, abstractmethod
from typing import Any
from app.channels.message_bus import InboundMessage, InboundMessageType, MessageBus, OutboundMessage, ResolvedAttachment
logger = logging.getLogger(__name__)
class Channel(ABC):
"""Base class for all IM channel implementations.
Each channel connects to an external messaging platform and:
1. Receives messages, wraps them as InboundMessage, publishes to the bus.
2. Subscribes to outbound messages and sends replies back to the platform.
Subclasses must implement ``start``, ``stop``, and ``send``.
"""
def __init__(self, name: str, bus: MessageBus, config: dict[str, Any]) -> None:
self.name = name
self.bus = bus
self.config = config
self._running = False
@property
def is_running(self) -> bool:
return self._running
# -- lifecycle ---------------------------------------------------------
@abstractmethod
async def start(self) -> None:
"""Start listening for messages from the external platform."""
@abstractmethod
async def stop(self) -> None:
"""Gracefully stop the channel."""
# -- outbound ----------------------------------------------------------
@abstractmethod
async def send(self, msg: OutboundMessage) -> None:
"""Send a message back to the external platform.
The implementation should use ``msg.chat_id`` and ``msg.thread_ts``
to route the reply to the correct conversation/thread.
"""
async def send_file(self, msg: OutboundMessage, attachment: ResolvedAttachment) -> bool:
"""Upload a single file attachment to the platform.
Returns True if the upload succeeded, False otherwise.
Default implementation returns False (no file upload support).
"""
return False
# -- helpers -----------------------------------------------------------
def _make_inbound(
self,
chat_id: str,
user_id: str,
text: str,
*,
msg_type: InboundMessageType = InboundMessageType.CHAT,
thread_ts: str | None = None,
files: list[dict[str, Any]] | None = None,
metadata: dict[str, Any] | None = None,
) -> InboundMessage:
"""Convenience factory for creating InboundMessage instances."""
return InboundMessage(
channel_name=self.name,
chat_id=chat_id,
user_id=user_id,
text=text,
msg_type=msg_type,
thread_ts=thread_ts,
files=files or [],
metadata=metadata or {},
)
async def _on_outbound(self, msg: OutboundMessage) -> None:
"""Outbound callback registered with the bus.
Only forwards messages targeted at this channel.
Sends the text message first, then uploads any file attachments.
File uploads are skipped entirely when the text send fails to avoid
partial deliveries (files without accompanying text).
"""
if msg.channel_name == self.name:
try:
await self.send(msg)
except Exception:
logger.exception("Failed to send outbound message on channel %s", self.name)
return # Do not attempt file uploads when the text message failed
for attachment in msg.attachments:
try:
success = await self.send_file(msg, attachment)
if not success:
logger.warning("[%s] file upload skipped for %s", self.name, attachment.filename)
except Exception:
logger.exception("[%s] failed to upload file %s", self.name, attachment.filename)
+20
View File
@@ -0,0 +1,20 @@
"""Shared command definitions used by all channel implementations.
Keeping the authoritative command set in one place ensures that channel
parsers (e.g. Feishu) and the ChannelManager dispatcher stay in sync
automatically — adding or removing a command here is the single edit
required.
"""
from __future__ import annotations
KNOWN_CHANNEL_COMMANDS: frozenset[str] = frozenset(
{
"/bootstrap",
"/new",
"/status",
"/models",
"/memory",
"/help",
}
)
+549
View File
@@ -0,0 +1,549 @@
"""Feishu/Lark channel — connects to Feishu via WebSocket (no public IP needed)."""
from __future__ import annotations
import asyncio
import json
import logging
import threading
from typing import Any
from app.channels.base import Channel
from app.channels.commands import KNOWN_CHANNEL_COMMANDS
from app.channels.message_bus import InboundMessageType, MessageBus, OutboundMessage, ResolvedAttachment
logger = logging.getLogger(__name__)
def _is_feishu_command(text: str) -> bool:
if not text.startswith("/"):
return False
return text.split(maxsplit=1)[0].lower() in KNOWN_CHANNEL_COMMANDS
class FeishuChannel(Channel):
"""Feishu/Lark IM channel using the ``lark-oapi`` WebSocket client.
Configuration keys (in ``config.yaml`` under ``channels.feishu``):
- ``app_id``: Feishu app ID.
- ``app_secret``: Feishu app secret.
- ``verification_token``: (optional) Event verification token.
The channel uses WebSocket long-connection mode so no public IP is required.
Message flow:
1. User sends a message → bot adds "OK" emoji reaction
2. Bot replies in thread: "Working on it......"
3. Agent processes the message and returns a result
4. Bot replies in thread with the result
5. Bot adds "DONE" emoji reaction to the original message
"""
def __init__(self, bus: MessageBus, config: dict[str, Any]) -> None:
super().__init__(name="feishu", bus=bus, config=config)
self._thread: threading.Thread | None = None
self._main_loop: asyncio.AbstractEventLoop | None = None
self._api_client = None
self._CreateMessageReactionRequest = None
self._CreateMessageReactionRequestBody = None
self._Emoji = None
self._PatchMessageRequest = None
self._PatchMessageRequestBody = None
self._background_tasks: set[asyncio.Task] = set()
self._running_card_ids: dict[str, str] = {}
self._running_card_tasks: dict[str, asyncio.Task] = {}
self._CreateFileRequest = None
self._CreateFileRequestBody = None
self._CreateImageRequest = None
self._CreateImageRequestBody = None
async def start(self) -> None:
if self._running:
return
try:
import lark_oapi as lark
from lark_oapi.api.im.v1 import (
CreateFileRequest,
CreateFileRequestBody,
CreateImageRequest,
CreateImageRequestBody,
CreateMessageReactionRequest,
CreateMessageReactionRequestBody,
CreateMessageRequest,
CreateMessageRequestBody,
Emoji,
PatchMessageRequest,
PatchMessageRequestBody,
ReplyMessageRequest,
ReplyMessageRequestBody,
)
except ImportError:
logger.error("lark-oapi is not installed. Install it with: uv add lark-oapi")
return
self._lark = lark
self._CreateMessageRequest = CreateMessageRequest
self._CreateMessageRequestBody = CreateMessageRequestBody
self._ReplyMessageRequest = ReplyMessageRequest
self._ReplyMessageRequestBody = ReplyMessageRequestBody
self._CreateMessageReactionRequest = CreateMessageReactionRequest
self._CreateMessageReactionRequestBody = CreateMessageReactionRequestBody
self._Emoji = Emoji
self._PatchMessageRequest = PatchMessageRequest
self._PatchMessageRequestBody = PatchMessageRequestBody
self._CreateFileRequest = CreateFileRequest
self._CreateFileRequestBody = CreateFileRequestBody
self._CreateImageRequest = CreateImageRequest
self._CreateImageRequestBody = CreateImageRequestBody
app_id = self.config.get("app_id", "")
app_secret = self.config.get("app_secret", "")
domain = self.config.get("domain", "https://open.feishu.cn")
if not app_id or not app_secret:
logger.error("Feishu channel requires app_id and app_secret")
return
self._api_client = lark.Client.builder().app_id(app_id).app_secret(app_secret).domain(domain).build()
logger.info("[Feishu] using domain: %s", domain)
self._main_loop = asyncio.get_event_loop()
self._running = True
self.bus.subscribe_outbound(self._on_outbound)
# Both ws.Client construction and start() must happen in a dedicated
# thread with its own event loop. lark-oapi caches the running loop
# at construction time and later calls loop.run_until_complete(),
# which conflicts with an already-running uvloop.
self._thread = threading.Thread(
target=self._run_ws,
args=(app_id, app_secret, domain),
daemon=True,
)
self._thread.start()
logger.info("Feishu channel started")
def _run_ws(self, app_id: str, app_secret: str, domain: str) -> None:
"""Construct and run the lark WS client in a thread with a fresh event loop.
The lark-oapi SDK captures a module-level event loop at import time
(``lark_oapi.ws.client.loop``). When uvicorn uses uvloop, that
captured loop is the *main* thread's uvloop — which is already
running, so ``loop.run_until_complete()`` inside ``Client.start()``
raises ``RuntimeError``.
We work around this by creating a plain asyncio event loop for this
thread and patching the SDK's module-level reference before calling
``start()``.
"""
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
try:
import lark_oapi as lark
import lark_oapi.ws.client as _ws_client_mod
# Replace the SDK's module-level loop so Client.start() uses
# this thread's (non-running) event loop instead of the main
# thread's uvloop.
_ws_client_mod.loop = loop
event_handler = lark.EventDispatcherHandler.builder("", "").register_p2_im_message_receive_v1(self._on_message).build()
ws_client = lark.ws.Client(
app_id=app_id,
app_secret=app_secret,
event_handler=event_handler,
log_level=lark.LogLevel.INFO,
domain=domain,
)
ws_client.start()
except Exception:
if self._running:
logger.exception("Feishu WebSocket error")
async def stop(self) -> None:
self._running = False
self.bus.unsubscribe_outbound(self._on_outbound)
for task in list(self._background_tasks):
task.cancel()
self._background_tasks.clear()
for task in list(self._running_card_tasks.values()):
task.cancel()
self._running_card_tasks.clear()
if self._thread:
self._thread.join(timeout=5)
self._thread = None
logger.info("Feishu channel stopped")
async def send(self, msg: OutboundMessage, *, _max_retries: int = 3) -> None:
if not self._api_client:
logger.warning("[Feishu] send called but no api_client available")
return
logger.info(
"[Feishu] sending reply: chat_id=%s, thread_ts=%s, text_len=%d",
msg.chat_id,
msg.thread_ts,
len(msg.text),
)
last_exc: Exception | None = None
for attempt in range(_max_retries):
try:
await self._send_card_message(msg)
return # success
except Exception as exc:
last_exc = exc
if attempt < _max_retries - 1:
delay = 2**attempt # 1s, 2s
logger.warning(
"[Feishu] send failed (attempt %d/%d), retrying in %ds: %s",
attempt + 1,
_max_retries,
delay,
exc,
)
await asyncio.sleep(delay)
logger.error("[Feishu] send failed after %d attempts: %s", _max_retries, last_exc)
if last_exc is None:
raise RuntimeError("Feishu send failed without an exception from any attempt")
raise last_exc
async def send_file(self, msg: OutboundMessage, attachment: ResolvedAttachment) -> bool:
if not self._api_client:
return False
# Check size limits (image: 10MB, file: 30MB)
if attachment.is_image and attachment.size > 10 * 1024 * 1024:
logger.warning("[Feishu] image too large (%d bytes), skipping: %s", attachment.size, attachment.filename)
return False
if not attachment.is_image and attachment.size > 30 * 1024 * 1024:
logger.warning("[Feishu] file too large (%d bytes), skipping: %s", attachment.size, attachment.filename)
return False
try:
if attachment.is_image:
file_key = await self._upload_image(attachment.actual_path)
msg_type = "image"
content = json.dumps({"image_key": file_key})
else:
file_key = await self._upload_file(attachment.actual_path, attachment.filename)
msg_type = "file"
content = json.dumps({"file_key": file_key})
if msg.thread_ts:
request = self._ReplyMessageRequest.builder().message_id(msg.thread_ts).request_body(self._ReplyMessageRequestBody.builder().msg_type(msg_type).content(content).reply_in_thread(True).build()).build()
await asyncio.to_thread(self._api_client.im.v1.message.reply, request)
else:
request = self._CreateMessageRequest.builder().receive_id_type("chat_id").request_body(self._CreateMessageRequestBody.builder().receive_id(msg.chat_id).msg_type(msg_type).content(content).build()).build()
await asyncio.to_thread(self._api_client.im.v1.message.create, request)
logger.info("[Feishu] file sent: %s (type=%s)", attachment.filename, msg_type)
return True
except Exception:
logger.exception("[Feishu] failed to upload/send file: %s", attachment.filename)
return False
async def _upload_image(self, path) -> str:
"""Upload an image to Feishu and return the image_key."""
with open(str(path), "rb") as f:
request = self._CreateImageRequest.builder().request_body(self._CreateImageRequestBody.builder().image_type("message").image(f).build()).build()
response = await asyncio.to_thread(self._api_client.im.v1.image.create, request)
if not response.success():
raise RuntimeError(f"Feishu image upload failed: code={response.code}, msg={response.msg}")
return response.data.image_key
async def _upload_file(self, path, filename: str) -> str:
"""Upload a file to Feishu and return the file_key."""
suffix = path.suffix.lower() if hasattr(path, "suffix") else ""
if suffix in (".xls", ".xlsx", ".csv"):
file_type = "xls"
elif suffix in (".ppt", ".pptx"):
file_type = "ppt"
elif suffix == ".pdf":
file_type = "pdf"
elif suffix in (".doc", ".docx"):
file_type = "doc"
else:
file_type = "stream"
with open(str(path), "rb") as f:
request = self._CreateFileRequest.builder().request_body(self._CreateFileRequestBody.builder().file_type(file_type).file_name(filename).file(f).build()).build()
response = await asyncio.to_thread(self._api_client.im.v1.file.create, request)
if not response.success():
raise RuntimeError(f"Feishu file upload failed: code={response.code}, msg={response.msg}")
return response.data.file_key
# -- message formatting ------------------------------------------------
@staticmethod
def _build_card_content(text: str) -> str:
"""Build a Feishu interactive card with markdown content.
Feishu's interactive card format natively renders markdown, including
headers, bold/italic, code blocks, lists, and links.
"""
card = {
"config": {"wide_screen_mode": True, "update_multi": True},
"elements": [{"tag": "markdown", "content": text}],
}
return json.dumps(card)
# -- reaction helpers --------------------------------------------------
async def _add_reaction(self, message_id: str, emoji_type: str = "THUMBSUP") -> None:
"""Add an emoji reaction to a message."""
if not self._api_client or not self._CreateMessageReactionRequest:
return
try:
request = self._CreateMessageReactionRequest.builder().message_id(message_id).request_body(self._CreateMessageReactionRequestBody.builder().reaction_type(self._Emoji.builder().emoji_type(emoji_type).build()).build()).build()
await asyncio.to_thread(self._api_client.im.v1.message_reaction.create, request)
logger.info("[Feishu] reaction '%s' added to message %s", emoji_type, message_id)
except Exception:
logger.exception("[Feishu] failed to add reaction '%s' to message %s", emoji_type, message_id)
async def _reply_card(self, message_id: str, text: str) -> str | None:
"""Reply with an interactive card and return the created card message ID."""
if not self._api_client:
return None
content = self._build_card_content(text)
request = self._ReplyMessageRequest.builder().message_id(message_id).request_body(self._ReplyMessageRequestBody.builder().msg_type("interactive").content(content).reply_in_thread(True).build()).build()
response = await asyncio.to_thread(self._api_client.im.v1.message.reply, request)
response_data = getattr(response, "data", None)
return getattr(response_data, "message_id", None)
async def _create_card(self, chat_id: str, text: str) -> None:
"""Create a new card message in the target chat."""
if not self._api_client:
return
content = self._build_card_content(text)
request = self._CreateMessageRequest.builder().receive_id_type("chat_id").request_body(self._CreateMessageRequestBody.builder().receive_id(chat_id).msg_type("interactive").content(content).build()).build()
await asyncio.to_thread(self._api_client.im.v1.message.create, request)
async def _update_card(self, message_id: str, text: str) -> None:
"""Patch an existing card message in place."""
if not self._api_client or not self._PatchMessageRequest:
return
content = self._build_card_content(text)
request = self._PatchMessageRequest.builder().message_id(message_id).request_body(self._PatchMessageRequestBody.builder().content(content).build()).build()
await asyncio.to_thread(self._api_client.im.v1.message.patch, request)
def _track_background_task(self, task: asyncio.Task, *, name: str, msg_id: str) -> None:
"""Keep a strong reference to fire-and-forget tasks and surface errors."""
self._background_tasks.add(task)
task.add_done_callback(lambda done_task, task_name=name, mid=msg_id: self._finalize_background_task(done_task, task_name, mid))
def _finalize_background_task(self, task: asyncio.Task, name: str, msg_id: str) -> None:
self._background_tasks.discard(task)
self._log_task_error(task, name, msg_id)
async def _create_running_card(self, source_message_id: str, text: str) -> str | None:
"""Create the running card and cache its message ID when available."""
running_card_id = await self._reply_card(source_message_id, text)
if running_card_id:
self._running_card_ids[source_message_id] = running_card_id
logger.info("[Feishu] running card created: source=%s card=%s", source_message_id, running_card_id)
else:
logger.warning("[Feishu] running card creation returned no message_id for source=%s, subsequent updates will fall back to new replies", source_message_id)
return running_card_id
def _ensure_running_card_started(self, source_message_id: str, text: str = "Working on it...") -> asyncio.Task | None:
"""Start running-card creation once per source message."""
running_card_id = self._running_card_ids.get(source_message_id)
if running_card_id:
return None
running_card_task = self._running_card_tasks.get(source_message_id)
if running_card_task:
return running_card_task
running_card_task = asyncio.create_task(self._create_running_card(source_message_id, text))
self._running_card_tasks[source_message_id] = running_card_task
running_card_task.add_done_callback(lambda done_task, mid=source_message_id: self._finalize_running_card_task(mid, done_task))
return running_card_task
def _finalize_running_card_task(self, source_message_id: str, task: asyncio.Task) -> None:
if self._running_card_tasks.get(source_message_id) is task:
self._running_card_tasks.pop(source_message_id, None)
self._log_task_error(task, "create_running_card", source_message_id)
async def _ensure_running_card(self, source_message_id: str, text: str = "Working on it...") -> str | None:
"""Ensure the in-thread running card exists and track its message ID."""
running_card_id = self._running_card_ids.get(source_message_id)
if running_card_id:
return running_card_id
running_card_task = self._ensure_running_card_started(source_message_id, text)
if running_card_task is None:
return self._running_card_ids.get(source_message_id)
return await running_card_task
async def _send_running_reply(self, message_id: str) -> None:
"""Reply to a message in-thread with a running card."""
try:
await self._ensure_running_card(message_id)
except Exception:
logger.exception("[Feishu] failed to send running reply for message %s", message_id)
async def _send_card_message(self, msg: OutboundMessage) -> None:
"""Send or update the Feishu card tied to the current request."""
source_message_id = msg.thread_ts
if source_message_id:
running_card_id = self._running_card_ids.get(source_message_id)
awaited_running_card_task = False
if not running_card_id:
running_card_task = self._running_card_tasks.get(source_message_id)
if running_card_task:
awaited_running_card_task = True
running_card_id = await running_card_task
if running_card_id:
try:
await self._update_card(running_card_id, msg.text)
except Exception:
if not msg.is_final:
raise
logger.exception(
"[Feishu] failed to patch running card %s, falling back to final reply",
running_card_id,
)
await self._reply_card(source_message_id, msg.text)
else:
logger.info("[Feishu] running card updated: source=%s card=%s", source_message_id, running_card_id)
elif msg.is_final:
await self._reply_card(source_message_id, msg.text)
elif awaited_running_card_task:
logger.warning(
"[Feishu] running card task finished without message_id for source=%s, skipping duplicate non-final creation",
source_message_id,
)
else:
await self._ensure_running_card(source_message_id, msg.text)
if msg.is_final:
self._running_card_ids.pop(source_message_id, None)
await self._add_reaction(source_message_id, "DONE")
return
await self._create_card(msg.chat_id, msg.text)
# -- internal ----------------------------------------------------------
@staticmethod
def _log_future_error(fut, name: str, msg_id: str) -> None:
"""Callback for run_coroutine_threadsafe futures to surface errors."""
try:
exc = fut.exception()
if exc:
logger.error("[Feishu] %s failed for msg_id=%s: %s", name, msg_id, exc)
except Exception:
pass
@staticmethod
def _log_task_error(task: asyncio.Task, name: str, msg_id: str) -> None:
"""Callback for background asyncio tasks to surface errors."""
try:
exc = task.exception()
if exc:
logger.error("[Feishu] %s failed for msg_id=%s: %s", name, msg_id, exc)
except asyncio.CancelledError:
logger.info("[Feishu] %s cancelled for msg_id=%s", name, msg_id)
except Exception:
pass
async def _prepare_inbound(self, msg_id: str, inbound) -> None:
"""Kick off Feishu side effects without delaying inbound dispatch."""
reaction_task = asyncio.create_task(self._add_reaction(msg_id, "OK"))
self._track_background_task(reaction_task, name="add_reaction", msg_id=msg_id)
self._ensure_running_card_started(msg_id)
await self.bus.publish_inbound(inbound)
def _on_message(self, event) -> None:
"""Called by lark-oapi when a message is received (runs in lark thread)."""
try:
logger.info("[Feishu] raw event received: type=%s", type(event).__name__)
message = event.event.message
chat_id = message.chat_id
msg_id = message.message_id
sender_id = event.event.sender.sender_id.open_id
# root_id is set when the message is a reply within a Feishu thread.
# Use it as topic_id so all replies share the same DeerFlow thread.
root_id = getattr(message, "root_id", None) or None
# Parse message content
content = json.loads(message.content)
if "text" in content:
# Handle plain text messages
text = content["text"]
elif "content" in content and isinstance(content["content"], list):
# Handle rich-text messages with a top-level "content" list (e.g., topic groups/posts)
text_paragraphs: list[str] = []
for paragraph in content["content"]:
if isinstance(paragraph, list):
paragraph_text_parts: list[str] = []
for element in paragraph:
if isinstance(element, dict):
# Include both normal text and @ mentions
if element.get("tag") in ("text", "at"):
text_value = element.get("text", "")
if text_value:
paragraph_text_parts.append(text_value)
if paragraph_text_parts:
# Join text segments within a paragraph with spaces to avoid "helloworld"
text_paragraphs.append(" ".join(paragraph_text_parts))
# Join paragraphs with blank lines to preserve paragraph boundaries
text = "\n\n".join(text_paragraphs)
else:
text = ""
text = text.strip()
logger.info(
"[Feishu] parsed message: chat_id=%s, msg_id=%s, root_id=%s, sender=%s, text=%r",
chat_id,
msg_id,
root_id,
sender_id,
text[:100] if text else "",
)
if not text:
logger.info("[Feishu] empty text, ignoring message")
return
# Only treat known slash commands as commands; absolute paths and
# other slash-prefixed text should be handled as normal chat.
if _is_feishu_command(text):
msg_type = InboundMessageType.COMMAND
else:
msg_type = InboundMessageType.CHAT
# topic_id: use root_id for replies (same topic), msg_id for new messages (new topic)
topic_id = root_id or msg_id
inbound = self._make_inbound(
chat_id=chat_id,
user_id=sender_id,
text=text,
msg_type=msg_type,
thread_ts=msg_id,
metadata={"message_id": msg_id, "root_id": root_id},
)
inbound.topic_id = topic_id
# Schedule on the async event loop
if self._main_loop and self._main_loop.is_running():
logger.info("[Feishu] publishing inbound message to bus (type=%s, msg_id=%s)", msg_type.value, msg_id)
fut = asyncio.run_coroutine_threadsafe(self._prepare_inbound(msg_id, inbound), self._main_loop)
fut.add_done_callback(lambda f, mid=msg_id: self._log_future_error(f, "prepare_inbound", mid))
else:
logger.warning("[Feishu] main loop not running, cannot publish inbound message")
except Exception:
logger.exception("[Feishu] error processing message")
+928
View File
@@ -0,0 +1,928 @@
"""ChannelManager — consumes inbound messages and dispatches them to the DeerFlow agent via LangGraph Server."""
from __future__ import annotations
import asyncio
import logging
import mimetypes
import re
import time
from collections.abc import Awaitable, Callable, Mapping
from typing import Any
import httpx
from langgraph_sdk.errors import ConflictError
from app.channels.commands import KNOWN_CHANNEL_COMMANDS
from app.channels.message_bus import InboundMessage, InboundMessageType, MessageBus, OutboundMessage, ResolvedAttachment
from app.channels.store import ChannelStore
logger = logging.getLogger(__name__)
DEFAULT_LANGGRAPH_URL = "http://localhost:2024"
DEFAULT_GATEWAY_URL = "http://localhost:8001"
DEFAULT_ASSISTANT_ID = "lead_agent"
CUSTOM_AGENT_NAME_PATTERN = re.compile(r"^[A-Za-z0-9-]+$")
DEFAULT_RUN_CONFIG: dict[str, Any] = {"recursion_limit": 100}
DEFAULT_RUN_CONTEXT: dict[str, Any] = {
"thinking_enabled": True,
"is_plan_mode": False,
"subagent_enabled": False,
}
STREAM_UPDATE_MIN_INTERVAL_SECONDS = 0.35
THREAD_BUSY_MESSAGE = "This conversation is already processing another request. Please wait for it to finish and try again."
CHANNEL_CAPABILITIES = {
"feishu": {"supports_streaming": True},
"slack": {"supports_streaming": False},
"telegram": {"supports_streaming": False},
"wecom": {"supports_streaming": True},
}
InboundFileReader = Callable[[dict[str, Any], httpx.AsyncClient], Awaitable[bytes | None]]
INBOUND_FILE_READERS: dict[str, InboundFileReader] = {}
def register_inbound_file_reader(channel_name: str, reader: InboundFileReader) -> None:
INBOUND_FILE_READERS[channel_name] = reader
async def _read_http_inbound_file(file_info: dict[str, Any], client: httpx.AsyncClient) -> bytes | None:
url = file_info.get("url")
if not isinstance(url, str) or not url:
return None
resp = await client.get(url)
resp.raise_for_status()
return resp.content
async def _read_wecom_inbound_file(file_info: dict[str, Any], client: httpx.AsyncClient) -> bytes | None:
data = await _read_http_inbound_file(file_info, client)
if data is None:
return None
aeskey = file_info.get("aeskey") if isinstance(file_info.get("aeskey"), str) else None
if not aeskey:
return data
try:
from aibot.crypto_utils import decrypt_file
except Exception:
logger.exception("[Manager] failed to import WeCom decrypt_file")
return None
return decrypt_file(data, aeskey)
register_inbound_file_reader("wecom", _read_wecom_inbound_file)
class InvalidChannelSessionConfigError(ValueError):
"""Raised when IM channel session overrides contain invalid agent config."""
def _is_thread_busy_error(exc: BaseException | None) -> bool:
if exc is None:
return False
if isinstance(exc, ConflictError):
return True
return "already running a task" in str(exc)
def _as_dict(value: Any) -> dict[str, Any]:
return dict(value) if isinstance(value, Mapping) else {}
def _merge_dicts(*layers: Any) -> dict[str, Any]:
merged: dict[str, Any] = {}
for layer in layers:
if isinstance(layer, Mapping):
merged.update(layer)
return merged
def _normalize_custom_agent_name(raw_value: str) -> str:
"""Normalize legacy channel assistant IDs into valid custom agent names."""
normalized = raw_value.strip().lower().replace("_", "-")
if not normalized:
raise InvalidChannelSessionConfigError("Channel session assistant_id is empty. Use 'lead_agent' or a valid custom agent name.")
if not CUSTOM_AGENT_NAME_PATTERN.fullmatch(normalized):
raise InvalidChannelSessionConfigError(f"Invalid channel session assistant_id {raw_value!r}. Use 'lead_agent' or a custom agent name containing only letters, digits, and hyphens.")
return normalized
def _extract_response_text(result: dict | list) -> str:
"""Extract the last AI message text from a LangGraph runs.wait result.
``runs.wait`` returns the final state dict which contains a ``messages``
list. Each message is a dict with at least ``type`` and ``content``.
Handles special cases:
- Regular AI text responses
- Clarification interrupts (``ask_clarification`` tool messages)
- AI messages with tool_calls but no text content
"""
if isinstance(result, list):
messages = result
elif isinstance(result, dict):
messages = result.get("messages", [])
else:
return ""
# Walk backwards to find usable response text, but stop at the last
# human message to avoid returning text from a previous turn.
for msg in reversed(messages):
if not isinstance(msg, dict):
continue
msg_type = msg.get("type")
# Stop at the last human message — anything before it is a previous turn
if msg_type == "human":
break
# Check for tool messages from ask_clarification (interrupt case)
if msg_type == "tool" and msg.get("name") == "ask_clarification":
content = msg.get("content", "")
if isinstance(content, str) and content:
return content
# Regular AI message with text content
if msg_type == "ai":
content = msg.get("content", "")
if isinstance(content, str) and content:
return content
# content can be a list of content blocks
if isinstance(content, list):
parts = []
for block in content:
if isinstance(block, dict) and block.get("type") == "text":
parts.append(block.get("text", ""))
elif isinstance(block, str):
parts.append(block)
text = "".join(parts)
if text:
return text
return ""
def _extract_text_content(content: Any) -> str:
"""Extract text from a streaming payload content field."""
if isinstance(content, str):
return content
if isinstance(content, list):
parts: list[str] = []
for block in content:
if isinstance(block, str):
parts.append(block)
elif isinstance(block, Mapping):
text = block.get("text")
if isinstance(text, str):
parts.append(text)
else:
nested = block.get("content")
if isinstance(nested, str):
parts.append(nested)
return "".join(parts)
if isinstance(content, Mapping):
for key in ("text", "content"):
value = content.get(key)
if isinstance(value, str):
return value
return ""
def _merge_stream_text(existing: str, chunk: str) -> str:
"""Merge either delta text or cumulative text into a single snapshot."""
if not chunk:
return existing
if not existing or chunk == existing:
return chunk or existing
if chunk.startswith(existing):
return chunk
if existing.endswith(chunk):
return existing
return existing + chunk
def _extract_stream_message_id(payload: Any, metadata: Any) -> str | None:
"""Best-effort extraction of the streamed AI message identifier."""
candidates = [payload, metadata]
if isinstance(payload, Mapping):
candidates.append(payload.get("kwargs"))
for candidate in candidates:
if not isinstance(candidate, Mapping):
continue
for key in ("id", "message_id"):
value = candidate.get(key)
if isinstance(value, str) and value:
return value
return None
def _accumulate_stream_text(
buffers: dict[str, str],
current_message_id: str | None,
event_data: Any,
) -> tuple[str | None, str | None]:
"""Convert a ``messages-tuple`` event into the latest displayable AI text."""
payload = event_data
metadata: Any = None
if isinstance(event_data, (list, tuple)):
if event_data:
payload = event_data[0]
if len(event_data) > 1:
metadata = event_data[1]
if isinstance(payload, str):
message_id = current_message_id or "__default__"
buffers[message_id] = _merge_stream_text(buffers.get(message_id, ""), payload)
return buffers[message_id], message_id
if not isinstance(payload, Mapping):
return None, current_message_id
payload_type = str(payload.get("type", "")).lower()
if "tool" in payload_type:
return None, current_message_id
text = _extract_text_content(payload.get("content"))
if not text and isinstance(payload.get("kwargs"), Mapping):
text = _extract_text_content(payload["kwargs"].get("content"))
if not text:
return None, current_message_id
message_id = _extract_stream_message_id(payload, metadata) or current_message_id or "__default__"
buffers[message_id] = _merge_stream_text(buffers.get(message_id, ""), text)
return buffers[message_id], message_id
def _extract_artifacts(result: dict | list) -> list[str]:
"""Extract artifact paths from the last AI response cycle only.
Instead of reading the full accumulated ``artifacts`` state (which contains
all artifacts ever produced in the thread), this inspects the messages after
the last human message and collects file paths from ``present_files`` tool
calls. This ensures only newly-produced artifacts are returned.
"""
if isinstance(result, list):
messages = result
elif isinstance(result, dict):
messages = result.get("messages", [])
else:
return []
artifacts: list[str] = []
for msg in reversed(messages):
if not isinstance(msg, dict):
continue
# Stop at the last human message — anything before it is a previous turn
if msg.get("type") == "human":
break
# Look for AI messages with present_files tool calls
if msg.get("type") == "ai":
for tc in msg.get("tool_calls", []):
if isinstance(tc, dict) and tc.get("name") == "present_files":
args = tc.get("args", {})
paths = args.get("filepaths", [])
if isinstance(paths, list):
artifacts.extend(p for p in paths if isinstance(p, str))
return artifacts
def _format_artifact_text(artifacts: list[str]) -> str:
"""Format artifact paths into a human-readable text block listing filenames."""
import posixpath
filenames = [posixpath.basename(p) for p in artifacts]
if len(filenames) == 1:
return f"Created File: 📎 {filenames[0]}"
return "Created Files: 📎 " + "".join(filenames)
_OUTPUTS_VIRTUAL_PREFIX = "/mnt/user-data/outputs/"
def _resolve_attachments(thread_id: str, artifacts: list[str]) -> list[ResolvedAttachment]:
"""Resolve virtual artifact paths to host filesystem paths with metadata.
Only paths under ``/mnt/user-data/outputs/`` are accepted; any other
virtual path is rejected with a warning to prevent exfiltrating uploads
or workspace files via IM channels.
Skips artifacts that cannot be resolved (missing files, invalid paths)
and logs warnings for them.
"""
from deerflow.config.paths import get_paths
attachments: list[ResolvedAttachment] = []
paths = get_paths()
outputs_dir = paths.sandbox_outputs_dir(thread_id).resolve()
for virtual_path in artifacts:
# Security: only allow files from the agent outputs directory
if not virtual_path.startswith(_OUTPUTS_VIRTUAL_PREFIX):
logger.warning("[Manager] rejected non-outputs artifact path: %s", virtual_path)
continue
try:
actual = paths.resolve_virtual_path(thread_id, virtual_path)
# Verify the resolved path is actually under the outputs directory
# (guards against path-traversal even after prefix check)
try:
actual.resolve().relative_to(outputs_dir)
except ValueError:
logger.warning("[Manager] artifact path escapes outputs dir: %s -> %s", virtual_path, actual)
continue
if not actual.is_file():
logger.warning("[Manager] artifact not found on disk: %s -> %s", virtual_path, actual)
continue
mime, _ = mimetypes.guess_type(str(actual))
mime = mime or "application/octet-stream"
attachments.append(
ResolvedAttachment(
virtual_path=virtual_path,
actual_path=actual,
filename=actual.name,
mime_type=mime,
size=actual.stat().st_size,
is_image=mime.startswith("image/"),
)
)
except (ValueError, OSError) as exc:
logger.warning("[Manager] failed to resolve artifact %s: %s", virtual_path, exc)
return attachments
def _prepare_artifact_delivery(
thread_id: str,
response_text: str,
artifacts: list[str],
) -> tuple[str, list[ResolvedAttachment]]:
"""Resolve attachments and append filename fallbacks to the text response."""
attachments: list[ResolvedAttachment] = []
if not artifacts:
return response_text, attachments
attachments = _resolve_attachments(thread_id, artifacts)
resolved_virtuals = {attachment.virtual_path for attachment in attachments}
unresolved = [path for path in artifacts if path not in resolved_virtuals]
if unresolved:
artifact_text = _format_artifact_text(unresolved)
response_text = (response_text + "\n\n" + artifact_text) if response_text else artifact_text
# Always include resolved attachment filenames as a text fallback so files
# remain discoverable even when the upload is skipped or fails.
if attachments:
resolved_text = _format_artifact_text([attachment.virtual_path for attachment in attachments])
response_text = (response_text + "\n\n" + resolved_text) if response_text else resolved_text
return response_text, attachments
async def _ingest_inbound_files(thread_id: str, msg: InboundMessage) -> list[dict[str, Any]]:
if not msg.files:
return []
from deerflow.uploads.manager import claim_unique_filename, ensure_uploads_dir, normalize_filename
uploads_dir = ensure_uploads_dir(thread_id)
seen_names = {entry.name for entry in uploads_dir.iterdir() if entry.is_file()}
created: list[dict[str, Any]] = []
file_reader = INBOUND_FILE_READERS.get(msg.channel_name, _read_http_inbound_file)
async with httpx.AsyncClient(timeout=httpx.Timeout(20.0)) as client:
for idx, f in enumerate(msg.files):
if not isinstance(f, dict):
continue
ftype = f.get("type") if isinstance(f.get("type"), str) else "file"
filename = f.get("filename") if isinstance(f.get("filename"), str) else ""
try:
data = await file_reader(f, client)
except Exception:
logger.exception(
"[Manager] failed to read inbound file: channel=%s, file=%s",
msg.channel_name,
f.get("url") or filename or idx,
)
continue
if data is None:
logger.warning(
"[Manager] inbound file reader returned no data: channel=%s, file=%s",
msg.channel_name,
f.get("url") or filename or idx,
)
continue
if not filename:
ext = ".bin"
if ftype == "image":
ext = ".png"
filename = f"{msg.thread_ts or 'msg'}_{idx}{ext}"
try:
safe_name = claim_unique_filename(normalize_filename(filename), seen_names)
except ValueError:
logger.warning(
"[Manager] skipping inbound file with unsafe filename: channel=%s, file=%r",
msg.channel_name,
filename,
)
continue
dest = uploads_dir / safe_name
try:
dest.write_bytes(data)
except Exception:
logger.exception("[Manager] failed to write inbound file: %s", dest)
continue
created.append(
{
"filename": safe_name,
"size": len(data),
"path": f"/mnt/user-data/uploads/{safe_name}",
"is_image": ftype == "image",
}
)
return created
def _format_uploaded_files_block(files: list[dict[str, Any]]) -> str:
lines = [
"<uploaded_files>",
"The following files were uploaded in this message:",
"",
]
if not files:
lines.append("(empty)")
else:
for f in files:
filename = f.get("filename", "")
size = int(f.get("size") or 0)
size_kb = size / 1024 if size else 0
size_str = f"{size_kb:.1f} KB" if size_kb < 1024 else f"{size_kb / 1024:.1f} MB"
path = f.get("path", "")
is_image = bool(f.get("is_image"))
file_kind = "image" if is_image else "file"
lines.append(f"- {filename} ({size_str})")
lines.append(f" Type: {file_kind}")
lines.append(f" Path: {path}")
lines.append("")
lines.append("Use `read_file` for text-based files and documents.")
lines.append("Use `view_image` for image files (jpg, jpeg, png, webp) so the model can inspect the image content.")
lines.append("</uploaded_files>")
return "\n".join(lines)
class ChannelManager:
"""Core dispatcher that bridges IM channels to the DeerFlow agent.
It reads from the MessageBus inbound queue, creates/reuses threads on
the LangGraph Server, sends messages via ``runs.wait``, and publishes
outbound responses back through the bus.
"""
def __init__(
self,
bus: MessageBus,
store: ChannelStore,
*,
max_concurrency: int = 5,
langgraph_url: str = DEFAULT_LANGGRAPH_URL,
gateway_url: str = DEFAULT_GATEWAY_URL,
assistant_id: str = DEFAULT_ASSISTANT_ID,
default_session: dict[str, Any] | None = None,
channel_sessions: dict[str, Any] | None = None,
) -> None:
self.bus = bus
self.store = store
self._max_concurrency = max_concurrency
self._langgraph_url = langgraph_url
self._gateway_url = gateway_url
self._assistant_id = assistant_id
self._default_session = _as_dict(default_session)
self._channel_sessions = dict(channel_sessions or {})
self._client = None # lazy init — langgraph_sdk async client
self._semaphore: asyncio.Semaphore | None = None
self._running = False
self._task: asyncio.Task | None = None
@staticmethod
def _channel_supports_streaming(channel_name: str) -> bool:
return CHANNEL_CAPABILITIES.get(channel_name, {}).get("supports_streaming", False)
def _resolve_session_layer(self, msg: InboundMessage) -> tuple[dict[str, Any], dict[str, Any]]:
channel_layer = _as_dict(self._channel_sessions.get(msg.channel_name))
users_layer = _as_dict(channel_layer.get("users"))
user_layer = _as_dict(users_layer.get(msg.user_id))
return channel_layer, user_layer
def _resolve_run_params(self, msg: InboundMessage, thread_id: str) -> tuple[str, dict[str, Any], dict[str, Any]]:
channel_layer, user_layer = self._resolve_session_layer(msg)
assistant_id = user_layer.get("assistant_id") or channel_layer.get("assistant_id") or self._default_session.get("assistant_id") or self._assistant_id
if not isinstance(assistant_id, str) or not assistant_id.strip():
assistant_id = self._assistant_id
run_config = _merge_dicts(
DEFAULT_RUN_CONFIG,
self._default_session.get("config"),
channel_layer.get("config"),
user_layer.get("config"),
)
run_context = _merge_dicts(
DEFAULT_RUN_CONTEXT,
self._default_session.get("context"),
channel_layer.get("context"),
user_layer.get("context"),
{"thread_id": thread_id},
)
# Custom agents are implemented as lead_agent + agent_name context.
# Keep backward compatibility for channel configs that set
# assistant_id: <custom-agent-name> by routing through lead_agent.
if assistant_id != DEFAULT_ASSISTANT_ID:
run_context.setdefault("agent_name", _normalize_custom_agent_name(assistant_id))
assistant_id = DEFAULT_ASSISTANT_ID
return assistant_id, run_config, run_context
# -- LangGraph SDK client (lazy) ----------------------------------------
def _get_client(self):
"""Return the ``langgraph_sdk`` async client, creating it on first use."""
if self._client is None:
from langgraph_sdk import get_client
self._client = get_client(url=self._langgraph_url)
return self._client
# -- lifecycle ---------------------------------------------------------
async def start(self) -> None:
"""Start the dispatch loop."""
if self._running:
return
self._running = True
self._semaphore = asyncio.Semaphore(self._max_concurrency)
self._task = asyncio.create_task(self._dispatch_loop())
logger.info("ChannelManager started (max_concurrency=%d)", self._max_concurrency)
async def stop(self) -> None:
"""Stop the dispatch loop."""
self._running = False
if self._task:
self._task.cancel()
try:
await self._task
except asyncio.CancelledError:
pass
self._task = None
logger.info("ChannelManager stopped")
# -- dispatch loop -----------------------------------------------------
async def _dispatch_loop(self) -> None:
logger.info("[Manager] dispatch loop started, waiting for inbound messages")
while self._running:
try:
msg = await asyncio.wait_for(self.bus.get_inbound(), timeout=1.0)
except TimeoutError:
continue
except asyncio.CancelledError:
break
logger.info(
"[Manager] received inbound: channel=%s, chat_id=%s, type=%s, text=%r",
msg.channel_name,
msg.chat_id,
msg.msg_type.value,
msg.text[:100] if msg.text else "",
)
task = asyncio.create_task(self._handle_message(msg))
task.add_done_callback(self._log_task_error)
@staticmethod
def _log_task_error(task: asyncio.Task) -> None:
"""Surface unhandled exceptions from background tasks."""
if task.cancelled():
return
exc = task.exception()
if exc:
logger.error("[Manager] unhandled error in message task: %s", exc, exc_info=exc)
async def _handle_message(self, msg: InboundMessage) -> None:
async with self._semaphore:
try:
if msg.msg_type == InboundMessageType.COMMAND:
await self._handle_command(msg)
else:
await self._handle_chat(msg)
except InvalidChannelSessionConfigError as exc:
logger.warning(
"Invalid channel session config for %s (chat=%s): %s",
msg.channel_name,
msg.chat_id,
exc,
)
await self._send_error(msg, str(exc))
except Exception:
logger.exception(
"Error handling message from %s (chat=%s)",
msg.channel_name,
msg.chat_id,
)
await self._send_error(msg, "An internal error occurred. Please try again.")
# -- chat handling -----------------------------------------------------
async def _create_thread(self, client, msg: InboundMessage) -> str:
"""Create a new thread on the LangGraph Server and store the mapping."""
thread = await client.threads.create()
thread_id = thread["thread_id"]
self.store.set_thread_id(
msg.channel_name,
msg.chat_id,
thread_id,
topic_id=msg.topic_id,
user_id=msg.user_id,
)
logger.info("[Manager] new thread created on LangGraph Server: thread_id=%s for chat_id=%s topic_id=%s", thread_id, msg.chat_id, msg.topic_id)
return thread_id
async def _handle_chat(self, msg: InboundMessage, extra_context: dict[str, Any] | None = None) -> None:
client = self._get_client()
# Look up existing DeerFlow thread.
# topic_id may be None (e.g. Telegram private chats) — the store
# handles this by using the "channel:chat_id" key without a topic suffix.
thread_id = self.store.get_thread_id(msg.channel_name, msg.chat_id, topic_id=msg.topic_id)
if thread_id:
logger.info("[Manager] reusing thread: thread_id=%s for topic_id=%s", thread_id, msg.topic_id)
# No existing thread found — create a new one
if thread_id is None:
thread_id = await self._create_thread(client, msg)
assistant_id, run_config, run_context = self._resolve_run_params(msg, thread_id)
if extra_context:
run_context.update(extra_context)
uploaded = await _ingest_inbound_files(thread_id, msg)
if uploaded:
msg.text = f"{_format_uploaded_files_block(uploaded)}\n\n{msg.text}".strip()
if self._channel_supports_streaming(msg.channel_name):
await self._handle_streaming_chat(
client,
msg,
thread_id,
assistant_id,
run_config,
run_context,
)
return
logger.info("[Manager] invoking runs.wait(thread_id=%s, text=%r)", thread_id, msg.text[:100])
result = await client.runs.wait(
thread_id,
assistant_id,
input={"messages": [{"role": "human", "content": msg.text}]},
config=run_config,
context=run_context,
)
response_text = _extract_response_text(result)
artifacts = _extract_artifacts(result)
logger.info(
"[Manager] agent response received: thread_id=%s, response_len=%d, artifacts=%d",
thread_id,
len(response_text) if response_text else 0,
len(artifacts),
)
response_text, attachments = _prepare_artifact_delivery(thread_id, response_text, artifacts)
if not response_text:
if attachments:
response_text = _format_artifact_text([a.virtual_path for a in attachments])
else:
response_text = "(No response from agent)"
outbound = OutboundMessage(
channel_name=msg.channel_name,
chat_id=msg.chat_id,
thread_id=thread_id,
text=response_text,
artifacts=artifacts,
attachments=attachments,
thread_ts=msg.thread_ts,
)
logger.info("[Manager] publishing outbound message to bus: channel=%s, chat_id=%s", msg.channel_name, msg.chat_id)
await self.bus.publish_outbound(outbound)
async def _handle_streaming_chat(
self,
client,
msg: InboundMessage,
thread_id: str,
assistant_id: str,
run_config: dict[str, Any],
run_context: dict[str, Any],
) -> None:
logger.info("[Manager] invoking runs.stream(thread_id=%s, text=%r)", thread_id, msg.text[:100])
last_values: dict[str, Any] | list | None = None
streamed_buffers: dict[str, str] = {}
current_message_id: str | None = None
latest_text = ""
last_published_text = ""
last_publish_at = 0.0
stream_error: BaseException | None = None
try:
async for chunk in client.runs.stream(
thread_id,
assistant_id,
input={"messages": [{"role": "human", "content": msg.text}]},
config=run_config,
context=run_context,
stream_mode=["messages-tuple", "values"],
multitask_strategy="reject",
):
event = getattr(chunk, "event", "")
data = getattr(chunk, "data", None)
if event == "messages-tuple":
accumulated_text, current_message_id = _accumulate_stream_text(streamed_buffers, current_message_id, data)
if accumulated_text:
latest_text = accumulated_text
elif event == "values" and isinstance(data, (dict, list)):
last_values = data
snapshot_text = _extract_response_text(data)
if snapshot_text:
latest_text = snapshot_text
if not latest_text or latest_text == last_published_text:
continue
now = time.monotonic()
if last_published_text and now - last_publish_at < STREAM_UPDATE_MIN_INTERVAL_SECONDS:
continue
await self.bus.publish_outbound(
OutboundMessage(
channel_name=msg.channel_name,
chat_id=msg.chat_id,
thread_id=thread_id,
text=latest_text,
is_final=False,
thread_ts=msg.thread_ts,
)
)
last_published_text = latest_text
last_publish_at = now
except Exception as exc:
stream_error = exc
if _is_thread_busy_error(exc):
logger.warning("[Manager] thread busy (concurrent run rejected): thread_id=%s", thread_id)
else:
logger.exception("[Manager] streaming error: thread_id=%s", thread_id)
finally:
result = last_values if last_values is not None else {"messages": [{"type": "ai", "content": latest_text}]}
response_text = _extract_response_text(result)
artifacts = _extract_artifacts(result)
response_text, attachments = _prepare_artifact_delivery(thread_id, response_text, artifacts)
if not response_text:
if attachments:
response_text = _format_artifact_text([attachment.virtual_path for attachment in attachments])
elif stream_error:
if _is_thread_busy_error(stream_error):
response_text = THREAD_BUSY_MESSAGE
else:
response_text = "An error occurred while processing your request. Please try again."
else:
response_text = latest_text or "(No response from agent)"
logger.info(
"[Manager] streaming response completed: thread_id=%s, response_len=%d, artifacts=%d, error=%s",
thread_id,
len(response_text),
len(artifacts),
stream_error,
)
await self.bus.publish_outbound(
OutboundMessage(
channel_name=msg.channel_name,
chat_id=msg.chat_id,
thread_id=thread_id,
text=response_text,
artifacts=artifacts,
attachments=attachments,
is_final=True,
thread_ts=msg.thread_ts,
)
)
# -- command handling --------------------------------------------------
async def _handle_command(self, msg: InboundMessage) -> None:
text = msg.text.strip()
parts = text.split(maxsplit=1)
command = parts[0].lower().lstrip("/")
if command == "bootstrap":
from dataclasses import replace as _dc_replace
chat_text = parts[1] if len(parts) > 1 else "Initialize workspace"
chat_msg = _dc_replace(msg, text=chat_text, msg_type=InboundMessageType.CHAT)
await self._handle_chat(chat_msg, extra_context={"is_bootstrap": True})
return
if command == "new":
# Create a new thread on the LangGraph Server
client = self._get_client()
thread = await client.threads.create()
new_thread_id = thread["thread_id"]
self.store.set_thread_id(
msg.channel_name,
msg.chat_id,
new_thread_id,
topic_id=msg.topic_id,
user_id=msg.user_id,
)
reply = "New conversation started."
elif command == "status":
thread_id = self.store.get_thread_id(msg.channel_name, msg.chat_id, topic_id=msg.topic_id)
reply = f"Active thread: {thread_id}" if thread_id else "No active conversation."
elif command == "models":
reply = await self._fetch_gateway("/api/models", "models")
elif command == "memory":
reply = await self._fetch_gateway("/api/memory", "memory")
elif command == "help":
reply = (
"Available commands:\n"
"/bootstrap — Start a bootstrap session (enables agent setup)\n"
"/new — Start a new conversation\n"
"/status — Show current thread info\n"
"/models — List available models\n"
"/memory — Show memory status\n"
"/help — Show this help"
)
else:
available = " | ".join(sorted(KNOWN_CHANNEL_COMMANDS))
reply = f"Unknown command: /{command}. Available commands: {available}"
outbound = OutboundMessage(
channel_name=msg.channel_name,
chat_id=msg.chat_id,
thread_id=self.store.get_thread_id(msg.channel_name, msg.chat_id) or "",
text=reply,
thread_ts=msg.thread_ts,
)
await self.bus.publish_outbound(outbound)
async def _fetch_gateway(self, path: str, kind: str) -> str:
"""Fetch data from the Gateway API for command responses."""
import httpx
try:
async with httpx.AsyncClient() as http:
resp = await http.get(f"{self._gateway_url}{path}", timeout=10)
resp.raise_for_status()
data = resp.json()
except Exception:
logger.exception("Failed to fetch %s from gateway", kind)
return f"Failed to fetch {kind} information."
if kind == "models":
names = [m["name"] for m in data.get("models", [])]
return ("Available models:\n" + "\n".join(f"{n}" for n in names)) if names else "No models configured."
elif kind == "memory":
facts = data.get("facts", [])
return f"Memory contains {len(facts)} fact(s)."
return str(data)
# -- error helper ------------------------------------------------------
async def _send_error(self, msg: InboundMessage, error_text: str) -> None:
outbound = OutboundMessage(
channel_name=msg.channel_name,
chat_id=msg.chat_id,
thread_id=self.store.get_thread_id(msg.channel_name, msg.chat_id) or "",
text=error_text,
thread_ts=msg.thread_ts,
)
await self.bus.publish_outbound(outbound)
+173
View File
@@ -0,0 +1,173 @@
"""MessageBus — async pub/sub hub that decouples channels from the agent dispatcher."""
from __future__ import annotations
import asyncio
import logging
import time
from collections.abc import Callable, Coroutine
from dataclasses import dataclass, field
from enum import StrEnum
from pathlib import Path
from typing import Any
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Message types
# ---------------------------------------------------------------------------
class InboundMessageType(StrEnum):
"""Types of messages arriving from IM channels."""
CHAT = "chat"
COMMAND = "command"
@dataclass
class InboundMessage:
"""A message arriving from an IM channel toward the agent dispatcher.
Attributes:
channel_name: Name of the source channel (e.g. "feishu", "slack").
chat_id: Platform-specific chat/conversation identifier.
user_id: Platform-specific user identifier.
text: The message text.
msg_type: Whether this is a regular chat message or a command.
thread_ts: Optional platform thread identifier (for threaded replies).
topic_id: Conversation topic identifier used to map to a DeerFlow thread.
Messages sharing the same ``topic_id`` within a ``chat_id`` will
reuse the same DeerFlow thread. When ``None``, each message
creates a new thread (one-shot Q&A).
files: Optional list of file attachments (platform-specific dicts).
metadata: Arbitrary extra data from the channel.
created_at: Unix timestamp when the message was created.
"""
channel_name: str
chat_id: str
user_id: str
text: str
msg_type: InboundMessageType = InboundMessageType.CHAT
thread_ts: str | None = None
topic_id: str | None = None
files: list[dict[str, Any]] = field(default_factory=list)
metadata: dict[str, Any] = field(default_factory=dict)
created_at: float = field(default_factory=time.time)
@dataclass
class ResolvedAttachment:
"""A file attachment resolved to a host filesystem path, ready for upload.
Attributes:
virtual_path: Original virtual path (e.g. /mnt/user-data/outputs/report.pdf).
actual_path: Resolved host filesystem path.
filename: Basename of the file.
mime_type: MIME type (e.g. "application/pdf").
size: File size in bytes.
is_image: True for image/* MIME types (platforms may handle images differently).
"""
virtual_path: str
actual_path: Path
filename: str
mime_type: str
size: int
is_image: bool
@dataclass
class OutboundMessage:
"""A message from the agent dispatcher back to a channel.
Attributes:
channel_name: Target channel name (used for routing).
chat_id: Target chat/conversation identifier.
thread_id: DeerFlow thread ID that produced this response.
text: The response text.
artifacts: List of artifact paths produced by the agent.
is_final: Whether this is the final message in the response stream.
thread_ts: Optional platform thread identifier for threaded replies.
metadata: Arbitrary extra data.
created_at: Unix timestamp.
"""
channel_name: str
chat_id: str
thread_id: str
text: str
artifacts: list[str] = field(default_factory=list)
attachments: list[ResolvedAttachment] = field(default_factory=list)
is_final: bool = True
thread_ts: str | None = None
metadata: dict[str, Any] = field(default_factory=dict)
created_at: float = field(default_factory=time.time)
# ---------------------------------------------------------------------------
# MessageBus
# ---------------------------------------------------------------------------
OutboundCallback = Callable[[OutboundMessage], Coroutine[Any, Any, None]]
class MessageBus:
"""Async pub/sub hub connecting channels and the agent dispatcher.
Channels publish inbound messages; the dispatcher consumes them.
The dispatcher publishes outbound messages; channels receive them
via registered callbacks.
"""
def __init__(self) -> None:
self._inbound_queue: asyncio.Queue[InboundMessage] = asyncio.Queue()
self._outbound_listeners: list[OutboundCallback] = []
# -- inbound -----------------------------------------------------------
async def publish_inbound(self, msg: InboundMessage) -> None:
"""Enqueue an inbound message from a channel."""
await self._inbound_queue.put(msg)
logger.info(
"[Bus] inbound enqueued: channel=%s, chat_id=%s, type=%s, queue_size=%d",
msg.channel_name,
msg.chat_id,
msg.msg_type.value,
self._inbound_queue.qsize(),
)
async def get_inbound(self) -> InboundMessage:
"""Block until the next inbound message is available."""
return await self._inbound_queue.get()
@property
def inbound_queue(self) -> asyncio.Queue[InboundMessage]:
return self._inbound_queue
# -- outbound ----------------------------------------------------------
def subscribe_outbound(self, callback: OutboundCallback) -> None:
"""Register an async callback for outbound messages."""
self._outbound_listeners.append(callback)
def unsubscribe_outbound(self, callback: OutboundCallback) -> None:
"""Remove a previously registered outbound callback."""
self._outbound_listeners = [cb for cb in self._outbound_listeners if cb is not callback]
async def publish_outbound(self, msg: OutboundMessage) -> None:
"""Dispatch an outbound message to all registered listeners."""
logger.info(
"[Bus] outbound dispatching: channel=%s, chat_id=%s, listeners=%d, text_len=%d",
msg.channel_name,
msg.chat_id,
len(self._outbound_listeners),
len(msg.text),
)
for callback in self._outbound_listeners:
try:
await callback(msg)
except Exception:
logger.exception("Error in outbound callback for channel=%s", msg.channel_name)
+193
View File
@@ -0,0 +1,193 @@
"""ChannelService — manages the lifecycle of all IM channels."""
from __future__ import annotations
import logging
import os
from typing import Any
from app.channels.manager import DEFAULT_GATEWAY_URL, DEFAULT_LANGGRAPH_URL, ChannelManager
from app.channels.message_bus import MessageBus
from app.channels.store import ChannelStore
logger = logging.getLogger(__name__)
# Channel name → import path for lazy loading
_CHANNEL_REGISTRY: dict[str, str] = {
"feishu": "app.channels.feishu:FeishuChannel",
"slack": "app.channels.slack:SlackChannel",
"telegram": "app.channels.telegram:TelegramChannel",
"wecom": "app.channels.wecom:WeComChannel",
}
_CHANNELS_LANGGRAPH_URL_ENV = "DEER_FLOW_CHANNELS_LANGGRAPH_URL"
_CHANNELS_GATEWAY_URL_ENV = "DEER_FLOW_CHANNELS_GATEWAY_URL"
def _resolve_service_url(config: dict[str, Any], config_key: str, env_key: str, default: str) -> str:
value = config.pop(config_key, None)
if isinstance(value, str) and value.strip():
return value
env_value = os.getenv(env_key, "").strip()
if env_value:
return env_value
return default
class ChannelService:
"""Manages the lifecycle of all configured IM channels.
Reads configuration from ``config.yaml`` under the ``channels`` key,
instantiates enabled channels, and starts the ChannelManager dispatcher.
"""
def __init__(self, channels_config: dict[str, Any] | None = None) -> None:
self.bus = MessageBus()
self.store = ChannelStore()
config = dict(channels_config or {})
langgraph_url = _resolve_service_url(config, "langgraph_url", _CHANNELS_LANGGRAPH_URL_ENV, DEFAULT_LANGGRAPH_URL)
gateway_url = _resolve_service_url(config, "gateway_url", _CHANNELS_GATEWAY_URL_ENV, DEFAULT_GATEWAY_URL)
default_session = config.pop("session", None)
channel_sessions = {name: channel_config.get("session") for name, channel_config in config.items() if isinstance(channel_config, dict)}
self.manager = ChannelManager(
bus=self.bus,
store=self.store,
langgraph_url=langgraph_url,
gateway_url=gateway_url,
default_session=default_session if isinstance(default_session, dict) else None,
channel_sessions=channel_sessions,
)
self._channels: dict[str, Any] = {} # name -> Channel instance
self._config = config
self._running = False
@classmethod
def from_app_config(cls) -> ChannelService:
"""Create a ChannelService from the application config."""
from deerflow.config.app_config import get_app_config
config = get_app_config()
channels_config = {}
# extra fields are allowed by AppConfig (extra="allow")
extra = config.model_extra or {}
if "channels" in extra:
channels_config = extra["channels"]
return cls(channels_config=channels_config)
async def start(self) -> None:
"""Start the manager and all enabled channels."""
if self._running:
return
await self.manager.start()
for name, channel_config in self._config.items():
if not isinstance(channel_config, dict):
continue
if not channel_config.get("enabled", False):
logger.info("Channel %s is disabled, skipping", name)
continue
await self._start_channel(name, channel_config)
self._running = True
logger.info("ChannelService started with channels: %s", list(self._channels.keys()))
async def stop(self) -> None:
"""Stop all channels and the manager."""
for name, channel in list(self._channels.items()):
try:
await channel.stop()
logger.info("Channel %s stopped", name)
except Exception:
logger.exception("Error stopping channel %s", name)
self._channels.clear()
await self.manager.stop()
self._running = False
logger.info("ChannelService stopped")
async def restart_channel(self, name: str) -> bool:
"""Restart a specific channel. Returns True if successful."""
if name in self._channels:
try:
await self._channels[name].stop()
except Exception:
logger.exception("Error stopping channel %s for restart", name)
del self._channels[name]
config = self._config.get(name)
if not config or not isinstance(config, dict):
logger.warning("No config for channel %s", name)
return False
return await self._start_channel(name, config)
async def _start_channel(self, name: str, config: dict[str, Any]) -> bool:
"""Instantiate and start a single channel."""
import_path = _CHANNEL_REGISTRY.get(name)
if not import_path:
logger.warning("Unknown channel type: %s", name)
return False
try:
from deerflow.reflection import resolve_class
channel_cls = resolve_class(import_path, base_class=None)
except Exception:
logger.exception("Failed to import channel class for %s", name)
return False
try:
channel = channel_cls(bus=self.bus, config=config)
await channel.start()
self._channels[name] = channel
logger.info("Channel %s started", name)
return True
except Exception:
logger.exception("Failed to start channel %s", name)
return False
def get_status(self) -> dict[str, Any]:
"""Return status information for all channels."""
channels_status = {}
for name in _CHANNEL_REGISTRY:
config = self._config.get(name, {})
enabled = isinstance(config, dict) and config.get("enabled", False)
running = name in self._channels and self._channels[name].is_running
channels_status[name] = {
"enabled": enabled,
"running": running,
}
return {
"service_running": self._running,
"channels": channels_status,
}
# -- singleton access -------------------------------------------------------
_channel_service: ChannelService | None = None
def get_channel_service() -> ChannelService | None:
"""Get the singleton ChannelService instance (if started)."""
return _channel_service
async def start_channel_service() -> ChannelService:
"""Create and start the global ChannelService from app config."""
global _channel_service
if _channel_service is not None:
return _channel_service
_channel_service = ChannelService.from_app_config()
await _channel_service.start()
return _channel_service
async def stop_channel_service() -> None:
"""Stop the global ChannelService."""
global _channel_service
if _channel_service is not None:
await _channel_service.stop()
_channel_service = None
+246
View File
@@ -0,0 +1,246 @@
"""Slack channel — connects via Socket Mode (no public IP needed)."""
from __future__ import annotations
import asyncio
import logging
from typing import Any
from markdown_to_mrkdwn import SlackMarkdownConverter
from app.channels.base import Channel
from app.channels.message_bus import InboundMessageType, MessageBus, OutboundMessage, ResolvedAttachment
logger = logging.getLogger(__name__)
_slack_md_converter = SlackMarkdownConverter()
class SlackChannel(Channel):
"""Slack IM channel using Socket Mode (WebSocket, no public IP).
Configuration keys (in ``config.yaml`` under ``channels.slack``):
- ``bot_token``: Slack Bot User OAuth Token (xoxb-...).
- ``app_token``: Slack App-Level Token (xapp-...) for Socket Mode.
- ``allowed_users``: (optional) List of allowed Slack user IDs. Empty = allow all.
"""
def __init__(self, bus: MessageBus, config: dict[str, Any]) -> None:
super().__init__(name="slack", bus=bus, config=config)
self._socket_client = None
self._web_client = None
self._loop: asyncio.AbstractEventLoop | None = None
self._allowed_users: set[str] = {str(user_id) for user_id in config.get("allowed_users", [])}
async def start(self) -> None:
if self._running:
return
try:
from slack_sdk import WebClient
from slack_sdk.socket_mode import SocketModeClient
from slack_sdk.socket_mode.response import SocketModeResponse
except ImportError:
logger.error("slack-sdk is not installed. Install it with: uv add slack-sdk")
return
self._SocketModeResponse = SocketModeResponse
bot_token = self.config.get("bot_token", "")
app_token = self.config.get("app_token", "")
if not bot_token or not app_token:
logger.error("Slack channel requires bot_token and app_token")
return
self._web_client = WebClient(token=bot_token)
self._socket_client = SocketModeClient(
app_token=app_token,
web_client=self._web_client,
)
self._loop = asyncio.get_event_loop()
self._socket_client.socket_mode_request_listeners.append(self._on_socket_event)
self._running = True
self.bus.subscribe_outbound(self._on_outbound)
# Start socket mode in background thread
asyncio.get_event_loop().run_in_executor(None, self._socket_client.connect)
logger.info("Slack channel started")
async def stop(self) -> None:
self._running = False
self.bus.unsubscribe_outbound(self._on_outbound)
if self._socket_client:
self._socket_client.close()
self._socket_client = None
logger.info("Slack channel stopped")
async def send(self, msg: OutboundMessage, *, _max_retries: int = 3) -> None:
if not self._web_client:
return
kwargs: dict[str, Any] = {
"channel": msg.chat_id,
"text": _slack_md_converter.convert(msg.text),
}
if msg.thread_ts:
kwargs["thread_ts"] = msg.thread_ts
last_exc: Exception | None = None
for attempt in range(_max_retries):
try:
await asyncio.to_thread(self._web_client.chat_postMessage, **kwargs)
# Add a completion reaction to the thread root
if msg.thread_ts:
await asyncio.to_thread(
self._add_reaction,
msg.chat_id,
msg.thread_ts,
"white_check_mark",
)
return
except Exception as exc:
last_exc = exc
if attempt < _max_retries - 1:
delay = 2**attempt # 1s, 2s
logger.warning(
"[Slack] send failed (attempt %d/%d), retrying in %ds: %s",
attempt + 1,
_max_retries,
delay,
exc,
)
await asyncio.sleep(delay)
logger.error("[Slack] send failed after %d attempts: %s", _max_retries, last_exc)
# Add failure reaction on error
if msg.thread_ts:
try:
await asyncio.to_thread(
self._add_reaction,
msg.chat_id,
msg.thread_ts,
"x",
)
except Exception:
pass
if last_exc is None:
raise RuntimeError("Slack send failed without an exception from any attempt")
raise last_exc
async def send_file(self, msg: OutboundMessage, attachment: ResolvedAttachment) -> bool:
if not self._web_client:
return False
try:
kwargs: dict[str, Any] = {
"channel": msg.chat_id,
"file": str(attachment.actual_path),
"filename": attachment.filename,
"title": attachment.filename,
}
if msg.thread_ts:
kwargs["thread_ts"] = msg.thread_ts
await asyncio.to_thread(self._web_client.files_upload_v2, **kwargs)
logger.info("[Slack] file uploaded: %s to channel=%s", attachment.filename, msg.chat_id)
return True
except Exception:
logger.exception("[Slack] failed to upload file: %s", attachment.filename)
return False
# -- internal ----------------------------------------------------------
def _add_reaction(self, channel_id: str, timestamp: str, emoji: str) -> None:
"""Add an emoji reaction to a message (best-effort, non-blocking)."""
if not self._web_client:
return
try:
self._web_client.reactions_add(
channel=channel_id,
timestamp=timestamp,
name=emoji,
)
except Exception as exc:
if "already_reacted" not in str(exc):
logger.warning("[Slack] failed to add reaction %s: %s", emoji, exc)
def _send_running_reply(self, channel_id: str, thread_ts: str) -> None:
"""Send a 'Working on it......' reply in the thread (called from SDK thread)."""
if not self._web_client:
return
try:
self._web_client.chat_postMessage(
channel=channel_id,
text=":hourglass_flowing_sand: Working on it...",
thread_ts=thread_ts,
)
logger.info("[Slack] 'Working on it...' reply sent in channel=%s, thread_ts=%s", channel_id, thread_ts)
except Exception:
logger.exception("[Slack] failed to send running reply in channel=%s", channel_id)
def _on_socket_event(self, client, req) -> None:
"""Called by slack-sdk for each Socket Mode event."""
try:
# Acknowledge the event
response = self._SocketModeResponse(envelope_id=req.envelope_id)
client.send_socket_mode_response(response)
event_type = req.type
if event_type != "events_api":
return
event = req.payload.get("event", {})
etype = event.get("type", "")
# Handle message events (DM or @mention)
if etype in ("message", "app_mention"):
self._handle_message_event(event)
except Exception:
logger.exception("Error processing Slack event")
def _handle_message_event(self, event: dict) -> None:
# Ignore bot messages
if event.get("bot_id") or event.get("subtype"):
return
user_id = event.get("user", "")
# Check allowed users
if self._allowed_users and user_id not in self._allowed_users:
logger.debug("Ignoring message from non-allowed user: %s", user_id)
return
text = event.get("text", "").strip()
if not text:
return
channel_id = event.get("channel", "")
thread_ts = event.get("thread_ts") or event.get("ts", "")
if text.startswith("/"):
msg_type = InboundMessageType.COMMAND
else:
msg_type = InboundMessageType.CHAT
# topic_id: use thread_ts as the topic identifier.
# For threaded messages, thread_ts is the root message ts (shared topic).
# For non-threaded messages, thread_ts is the message's own ts (new topic).
inbound = self._make_inbound(
chat_id=channel_id,
user_id=user_id,
text=text,
msg_type=msg_type,
thread_ts=thread_ts,
)
inbound.topic_id = thread_ts
if self._loop and self._loop.is_running():
# Acknowledge with an eyes reaction
self._add_reaction(channel_id, event.get("ts", thread_ts), "eyes")
# Send "running" reply first (fire-and-forget from SDK thread)
self._send_running_reply(channel_id, thread_ts)
asyncio.run_coroutine_threadsafe(self.bus.publish_inbound(inbound), self._loop)
+153
View File
@@ -0,0 +1,153 @@
"""ChannelStore — persists IM chat-to-DeerFlow thread mappings."""
from __future__ import annotations
import json
import logging
import tempfile
import threading
import time
from pathlib import Path
from typing import Any
logger = logging.getLogger(__name__)
class ChannelStore:
"""JSON-file-backed store that maps IM conversations to DeerFlow threads.
Data layout (on disk)::
{
"<channel_name>:<chat_id>": {
"thread_id": "<uuid>",
"user_id": "<platform_user>",
"created_at": 1700000000.0,
"updated_at": 1700000000.0
},
...
}
The store is intentionally simple — a single JSON file that is atomically
rewritten on every mutation. For production workloads with high concurrency,
this can be swapped for a proper database backend.
"""
def __init__(self, path: str | Path | None = None) -> None:
if path is None:
from deerflow.config.paths import get_paths
path = Path(get_paths().base_dir) / "channels" / "store.json"
self._path = Path(path)
self._path.parent.mkdir(parents=True, exist_ok=True)
self._data: dict[str, dict[str, Any]] = self._load()
self._lock = threading.Lock()
# -- persistence -------------------------------------------------------
def _load(self) -> dict[str, dict[str, Any]]:
if self._path.exists():
try:
return json.loads(self._path.read_text(encoding="utf-8"))
except (json.JSONDecodeError, OSError):
logger.warning("Corrupt channel store at %s, starting fresh", self._path)
return {}
def _save(self) -> None:
fd = tempfile.NamedTemporaryFile(
mode="w",
dir=self._path.parent,
suffix=".tmp",
delete=False,
)
try:
json.dump(self._data, fd, indent=2)
fd.close()
Path(fd.name).replace(self._path)
except BaseException:
fd.close()
Path(fd.name).unlink(missing_ok=True)
raise
# -- key helpers -------------------------------------------------------
@staticmethod
def _key(channel_name: str, chat_id: str, topic_id: str | None = None) -> str:
if topic_id:
return f"{channel_name}:{chat_id}:{topic_id}"
return f"{channel_name}:{chat_id}"
# -- public API --------------------------------------------------------
def get_thread_id(self, channel_name: str, chat_id: str, topic_id: str | None = None) -> str | None:
"""Look up the DeerFlow thread_id for a given IM conversation/topic."""
entry = self._data.get(self._key(channel_name, chat_id, topic_id))
return entry["thread_id"] if entry else None
def set_thread_id(
self,
channel_name: str,
chat_id: str,
thread_id: str,
*,
topic_id: str | None = None,
user_id: str = "",
) -> None:
"""Create or update the mapping for an IM conversation/topic."""
with self._lock:
key = self._key(channel_name, chat_id, topic_id)
now = time.time()
existing = self._data.get(key)
self._data[key] = {
"thread_id": thread_id,
"user_id": user_id,
"created_at": existing["created_at"] if existing else now,
"updated_at": now,
}
self._save()
def remove(self, channel_name: str, chat_id: str, topic_id: str | None = None) -> bool:
"""Remove a mapping.
If ``topic_id`` is provided, only that specific conversation/topic mapping is removed.
If ``topic_id`` is omitted, all mappings whose key starts with
``"<channel_name>:<chat_id>"`` (including topic-specific ones) are removed.
Returns True if at least one mapping was removed.
"""
with self._lock:
# Remove a specific conversation/topic mapping.
if topic_id is not None:
key = self._key(channel_name, chat_id, topic_id)
if key in self._data:
del self._data[key]
self._save()
return True
return False
# Remove all mappings for this channel/chat_id (base and any topic-specific keys).
prefix = self._key(channel_name, chat_id)
keys_to_delete = [k for k in self._data if k == prefix or k.startswith(prefix + ":")]
if not keys_to_delete:
return False
for k in keys_to_delete:
del self._data[k]
self._save()
return True
def list_entries(self, channel_name: str | None = None) -> list[dict[str, Any]]:
"""List all stored mappings, optionally filtered by channel."""
results = []
for key, entry in self._data.items():
parts = key.split(":", 2)
ch = parts[0]
chat = parts[1] if len(parts) > 1 else ""
topic = parts[2] if len(parts) > 2 else None
if channel_name and ch != channel_name:
continue
item: dict[str, Any] = {"channel_name": ch, "chat_id": chat, **entry}
if topic is not None:
item["topic_id"] = topic
results.append(item)
return results
+317
View File
@@ -0,0 +1,317 @@
"""Telegram channel — connects via long-polling (no public IP needed)."""
from __future__ import annotations
import asyncio
import logging
import threading
from typing import Any
from app.channels.base import Channel
from app.channels.message_bus import InboundMessage, InboundMessageType, MessageBus, OutboundMessage, ResolvedAttachment
logger = logging.getLogger(__name__)
class TelegramChannel(Channel):
"""Telegram bot channel using long-polling.
Configuration keys (in ``config.yaml`` under ``channels.telegram``):
- ``bot_token``: Telegram Bot API token (from @BotFather).
- ``allowed_users``: (optional) List of allowed Telegram user IDs. Empty = allow all.
"""
def __init__(self, bus: MessageBus, config: dict[str, Any]) -> None:
super().__init__(name="telegram", bus=bus, config=config)
self._application = None
self._thread: threading.Thread | None = None
self._tg_loop: asyncio.AbstractEventLoop | None = None
self._main_loop: asyncio.AbstractEventLoop | None = None
self._allowed_users: set[int] = set()
for uid in config.get("allowed_users", []):
try:
self._allowed_users.add(int(uid))
except (ValueError, TypeError):
pass
# chat_id -> last sent message_id for threaded replies
self._last_bot_message: dict[str, int] = {}
async def start(self) -> None:
if self._running:
return
try:
from telegram.ext import ApplicationBuilder, CommandHandler, MessageHandler, filters
except ImportError:
logger.error("python-telegram-bot is not installed. Install it with: uv add python-telegram-bot")
return
bot_token = self.config.get("bot_token", "")
if not bot_token:
logger.error("Telegram channel requires bot_token")
return
self._main_loop = asyncio.get_event_loop()
self._running = True
self.bus.subscribe_outbound(self._on_outbound)
# Build the application
app = ApplicationBuilder().token(bot_token).build()
# Command handlers
app.add_handler(CommandHandler("start", self._cmd_start))
app.add_handler(CommandHandler("new", self._cmd_generic))
app.add_handler(CommandHandler("status", self._cmd_generic))
app.add_handler(CommandHandler("models", self._cmd_generic))
app.add_handler(CommandHandler("memory", self._cmd_generic))
app.add_handler(CommandHandler("help", self._cmd_generic))
# General message handler
app.add_handler(MessageHandler(filters.TEXT & ~filters.COMMAND, self._on_text))
self._application = app
# Run polling in a dedicated thread with its own event loop
self._thread = threading.Thread(target=self._run_polling, daemon=True)
self._thread.start()
logger.info("Telegram channel started")
async def stop(self) -> None:
self._running = False
self.bus.unsubscribe_outbound(self._on_outbound)
if self._tg_loop and self._tg_loop.is_running():
self._tg_loop.call_soon_threadsafe(self._tg_loop.stop)
if self._thread:
self._thread.join(timeout=10)
self._thread = None
self._application = None
logger.info("Telegram channel stopped")
async def send(self, msg: OutboundMessage, *, _max_retries: int = 3) -> None:
if not self._application:
return
try:
chat_id = int(msg.chat_id)
except (ValueError, TypeError):
logger.error("Invalid Telegram chat_id: %s", msg.chat_id)
return
kwargs: dict[str, Any] = {"chat_id": chat_id, "text": msg.text}
# Reply to the last bot message in this chat for threading
reply_to = self._last_bot_message.get(msg.chat_id)
if reply_to:
kwargs["reply_to_message_id"] = reply_to
bot = self._application.bot
last_exc: Exception | None = None
for attempt in range(_max_retries):
try:
sent = await bot.send_message(**kwargs)
self._last_bot_message[msg.chat_id] = sent.message_id
return
except Exception as exc:
last_exc = exc
if attempt < _max_retries - 1:
delay = 2**attempt # 1s, 2s
logger.warning(
"[Telegram] send failed (attempt %d/%d), retrying in %ds: %s",
attempt + 1,
_max_retries,
delay,
exc,
)
await asyncio.sleep(delay)
logger.error("[Telegram] send failed after %d attempts: %s", _max_retries, last_exc)
if last_exc is None:
raise RuntimeError("Telegram send failed without an exception from any attempt")
raise last_exc
async def send_file(self, msg: OutboundMessage, attachment: ResolvedAttachment) -> bool:
if not self._application:
return False
try:
chat_id = int(msg.chat_id)
except (ValueError, TypeError):
logger.error("[Telegram] Invalid chat_id: %s", msg.chat_id)
return False
# Telegram limits: 10MB for photos, 50MB for documents
if attachment.size > 50 * 1024 * 1024:
logger.warning("[Telegram] file too large (%d bytes), skipping: %s", attachment.size, attachment.filename)
return False
bot = self._application.bot
reply_to = self._last_bot_message.get(msg.chat_id)
try:
if attachment.is_image and attachment.size <= 10 * 1024 * 1024:
with open(attachment.actual_path, "rb") as f:
kwargs: dict[str, Any] = {"chat_id": chat_id, "photo": f}
if reply_to:
kwargs["reply_to_message_id"] = reply_to
sent = await bot.send_photo(**kwargs)
else:
from telegram import InputFile
with open(attachment.actual_path, "rb") as f:
input_file = InputFile(f, filename=attachment.filename)
kwargs = {"chat_id": chat_id, "document": input_file}
if reply_to:
kwargs["reply_to_message_id"] = reply_to
sent = await bot.send_document(**kwargs)
self._last_bot_message[msg.chat_id] = sent.message_id
logger.info("[Telegram] file sent: %s to chat=%s", attachment.filename, msg.chat_id)
return True
except Exception:
logger.exception("[Telegram] failed to send file: %s", attachment.filename)
return False
# -- helpers -----------------------------------------------------------
async def _send_running_reply(self, chat_id: str, reply_to_message_id: int) -> None:
"""Send a 'Working on it...' reply to the user's message."""
if not self._application:
return
try:
bot = self._application.bot
await bot.send_message(
chat_id=int(chat_id),
text="Working on it...",
reply_to_message_id=reply_to_message_id,
)
logger.info("[Telegram] 'Working on it...' reply sent in chat=%s", chat_id)
except Exception:
logger.exception("[Telegram] failed to send running reply in chat=%s", chat_id)
# -- internal ----------------------------------------------------------
@staticmethod
def _log_future_error(fut, name: str, msg_id: str):
try:
exc = fut.exception()
if exc:
logger.error("[Telegram] %s failed for msg_id=%s: %s", name, msg_id, exc)
except Exception:
logger.exception("[Telegram] Failed to inspect future for %s (msg_id=%s)", name, msg_id)
def _run_polling(self) -> None:
"""Run telegram polling in a dedicated thread."""
self._tg_loop = asyncio.new_event_loop()
asyncio.set_event_loop(self._tg_loop)
try:
# Cannot use run_polling() because it calls add_signal_handler(),
# which only works in the main thread. Instead, manually
# initialize the application and start the updater.
self._tg_loop.run_until_complete(self._application.initialize())
self._tg_loop.run_until_complete(self._application.start())
self._tg_loop.run_until_complete(self._application.updater.start_polling())
self._tg_loop.run_forever()
except Exception:
if self._running:
logger.exception("Telegram polling error")
finally:
# Graceful shutdown
try:
if self._application.updater.running:
self._tg_loop.run_until_complete(self._application.updater.stop())
self._tg_loop.run_until_complete(self._application.stop())
self._tg_loop.run_until_complete(self._application.shutdown())
except Exception:
logger.exception("Error during Telegram shutdown")
def _check_user(self, user_id: int) -> bool:
if not self._allowed_users:
return True
return user_id in self._allowed_users
async def _cmd_start(self, update, context) -> None:
"""Handle /start command."""
if not self._check_user(update.effective_user.id):
return
await update.message.reply_text("Welcome to DeerFlow! Send me a message to start a conversation.\nType /help for available commands.")
async def _process_incoming_with_reply(self, chat_id: str, msg_id: int, inbound: InboundMessage) -> None:
await self._send_running_reply(chat_id, msg_id)
await self.bus.publish_inbound(inbound)
async def _cmd_generic(self, update, context) -> None:
"""Forward slash commands to the channel manager."""
if not self._check_user(update.effective_user.id):
return
text = update.message.text
chat_id = str(update.effective_chat.id)
user_id = str(update.effective_user.id)
msg_id = str(update.message.message_id)
# Use the same topic_id logic as _on_text so that commands
# like /new target the correct thread mapping.
if update.effective_chat.type == "private":
topic_id = None
else:
reply_to = update.message.reply_to_message
if reply_to:
topic_id = str(reply_to.message_id)
else:
topic_id = msg_id
inbound = self._make_inbound(
chat_id=chat_id,
user_id=user_id,
text=text,
msg_type=InboundMessageType.COMMAND,
thread_ts=msg_id,
)
inbound.topic_id = topic_id
if self._main_loop and self._main_loop.is_running():
fut = asyncio.run_coroutine_threadsafe(self._process_incoming_with_reply(chat_id, update.message.message_id, inbound), self._main_loop)
fut.add_done_callback(lambda f: self._log_future_error(f, "process_incoming_with_reply", update.message.message_id))
else:
logger.warning("[Telegram] Main loop not running. Cannot publish inbound message.")
async def _on_text(self, update, context) -> None:
"""Handle regular text messages."""
if not self._check_user(update.effective_user.id):
return
text = update.message.text.strip()
if not text:
return
chat_id = str(update.effective_chat.id)
user_id = str(update.effective_user.id)
msg_id = str(update.message.message_id)
# topic_id determines which DeerFlow thread the message maps to.
# In private chats, use None so that all messages share a single
# thread (the store key becomes "channel:chat_id").
# In group chats, use the reply-to message id or the current
# message id to keep separate conversation threads.
if update.effective_chat.type == "private":
topic_id = None
else:
reply_to = update.message.reply_to_message
if reply_to:
topic_id = str(reply_to.message_id)
else:
topic_id = msg_id
inbound = self._make_inbound(
chat_id=chat_id,
user_id=user_id,
text=text,
msg_type=InboundMessageType.CHAT,
thread_ts=msg_id,
)
inbound.topic_id = topic_id
if self._main_loop and self._main_loop.is_running():
fut = asyncio.run_coroutine_threadsafe(self._process_incoming_with_reply(chat_id, update.message.message_id, inbound), self._main_loop)
fut.add_done_callback(lambda f: self._log_future_error(f, "process_incoming_with_reply", update.message.message_id))
else:
logger.warning("[Telegram] Main loop not running. Cannot publish inbound message.")
+394
View File
@@ -0,0 +1,394 @@
from __future__ import annotations
import asyncio
import base64
import hashlib
import logging
from collections.abc import Awaitable, Callable
from typing import Any, cast
from app.channels.base import Channel
from app.channels.message_bus import (
InboundMessageType,
MessageBus,
OutboundMessage,
ResolvedAttachment,
)
logger = logging.getLogger(__name__)
class WeComChannel(Channel):
def __init__(self, bus: MessageBus, config: dict[str, Any]) -> None:
super().__init__(name="wecom", bus=bus, config=config)
self._bot_id: str | None = None
self._bot_secret: str | None = None
self._ws_client = None
self._ws_task: asyncio.Task | None = None
self._ws_frames: dict[str, dict[str, Any]] = {}
self._ws_stream_ids: dict[str, str] = {}
self._working_message = "Working on it..."
def _clear_ws_context(self, thread_ts: str | None) -> None:
if not thread_ts:
return
self._ws_frames.pop(thread_ts, None)
self._ws_stream_ids.pop(thread_ts, None)
async def _send_ws_upload_command(self, req_id: str, body: dict[str, Any], cmd: str) -> dict[str, Any]:
if not self._ws_client:
raise RuntimeError("WeCom WebSocket client is not available")
ws_manager = getattr(self._ws_client, "_ws_manager", None)
send_reply = getattr(ws_manager, "send_reply", None)
if not callable(send_reply):
raise RuntimeError("Installed wecom-aibot-python-sdk does not expose the WebSocket media upload API expected by DeerFlow. Use wecom-aibot-python-sdk==0.1.6 or update the adapter.")
send_reply_async = cast(Callable[[str, dict[str, Any], str], Awaitable[dict[str, Any]]], send_reply)
return await send_reply_async(req_id, body, cmd)
async def start(self) -> None:
if self._running:
return
bot_id = self.config.get("bot_id")
bot_secret = self.config.get("bot_secret")
working_message = self.config.get("working_message")
self._bot_id = bot_id if isinstance(bot_id, str) and bot_id else None
self._bot_secret = bot_secret if isinstance(bot_secret, str) and bot_secret else None
self._working_message = working_message if isinstance(working_message, str) and working_message else "Working on it..."
if not self._bot_id or not self._bot_secret:
logger.error("WeCom channel requires bot_id and bot_secret")
return
try:
from aibot import WSClient, WSClientOptions
except ImportError:
logger.error("wecom-aibot-python-sdk is not installed. Install it with: uv add wecom-aibot-python-sdk")
return
else:
self._ws_client = WSClient(WSClientOptions(bot_id=self._bot_id, secret=self._bot_secret, logger=logger))
self._ws_client.on("message.text", self._on_ws_text)
self._ws_client.on("message.mixed", self._on_ws_mixed)
self._ws_client.on("message.image", self._on_ws_image)
self._ws_client.on("message.file", self._on_ws_file)
self._ws_task = asyncio.create_task(self._ws_client.connect())
self._running = True
self.bus.subscribe_outbound(self._on_outbound)
logger.info("WeCom channel started")
async def stop(self) -> None:
self._running = False
self.bus.unsubscribe_outbound(self._on_outbound)
if self._ws_task:
try:
self._ws_task.cancel()
except Exception:
pass
self._ws_task = None
if self._ws_client:
try:
self._ws_client.disconnect()
except Exception:
pass
self._ws_client = None
self._ws_frames.clear()
self._ws_stream_ids.clear()
logger.info("WeCom channel stopped")
async def send(self, msg: OutboundMessage, *, _max_retries: int = 3) -> None:
if self._ws_client:
await self._send_ws(msg, _max_retries=_max_retries)
return
logger.warning("[WeCom] send called but WebSocket client is not available")
async def _on_outbound(self, msg: OutboundMessage) -> None:
if msg.channel_name != self.name:
return
try:
await self.send(msg)
except Exception:
logger.exception("Failed to send outbound message on channel %s", self.name)
if msg.is_final:
self._clear_ws_context(msg.thread_ts)
return
for attachment in msg.attachments:
try:
success = await self.send_file(msg, attachment)
if not success:
logger.warning("[%s] file upload skipped for %s", self.name, attachment.filename)
except Exception:
logger.exception("[%s] failed to upload file %s", self.name, attachment.filename)
if msg.is_final:
self._clear_ws_context(msg.thread_ts)
async def send_file(self, msg: OutboundMessage, attachment: ResolvedAttachment) -> bool:
if not msg.is_final:
return True
if not self._ws_client:
return False
if not msg.thread_ts:
return False
frame = self._ws_frames.get(msg.thread_ts)
if not frame:
return False
media_type = "image" if attachment.is_image else "file"
size_limit = 2 * 1024 * 1024 if attachment.is_image else 20 * 1024 * 1024
if attachment.size > size_limit:
logger.warning(
"[WeCom] %s too large (%d bytes), skipping: %s",
media_type,
attachment.size,
attachment.filename,
)
return False
try:
media_id = await self._upload_media_ws(
media_type=media_type,
filename=attachment.filename,
path=str(attachment.actual_path),
size=attachment.size,
)
if not media_id:
return False
body = {media_type: {"media_id": media_id}, "msgtype": media_type}
await self._ws_client.reply(frame, body)
logger.debug("[WeCom] %s sent via ws: %s", media_type, attachment.filename)
return True
except Exception:
logger.exception("[WeCom] failed to upload/send file via ws: %s", attachment.filename)
return False
async def _on_ws_text(self, frame: dict[str, Any]) -> None:
body = frame.get("body", {}) or {}
text = ((body.get("text") or {}).get("content") or "").strip()
quote = body.get("quote", {}).get("text", {}).get("content", "").strip()
if not text and not quote:
return
await self._publish_ws_inbound(frame, text + (f"\nQuote message: {quote}" if quote else ""))
async def _on_ws_mixed(self, frame: dict[str, Any]) -> None:
body = frame.get("body", {}) or {}
mixed = body.get("mixed") or {}
items = mixed.get("msg_item") or []
parts: list[str] = []
files: list[dict[str, Any]] = []
for item in items:
item_type = (item or {}).get("msgtype")
if item_type == "text":
content = (((item or {}).get("text") or {}).get("content") or "").strip()
if content:
parts.append(content)
elif item_type in ("image", "file"):
payload = (item or {}).get(item_type) or {}
url = payload.get("url")
aeskey = payload.get("aeskey")
if isinstance(url, str) and url:
files.append(
{
"type": item_type,
"url": url,
"aeskey": (aeskey if isinstance(aeskey, str) and aeskey else None),
}
)
text = "\n\n".join(parts).strip()
if not text and not files:
return
if not text:
text = "receive image/file"
await self._publish_ws_inbound(frame, text, files=files)
async def _on_ws_image(self, frame: dict[str, Any]) -> None:
body = frame.get("body", {}) or {}
image = body.get("image") or {}
url = image.get("url")
aeskey = image.get("aeskey")
if not isinstance(url, str) or not url:
return
await self._publish_ws_inbound(
frame,
"receive image ",
files=[
{
"type": "image",
"url": url,
"aeskey": aeskey if isinstance(aeskey, str) and aeskey else None,
}
],
)
async def _on_ws_file(self, frame: dict[str, Any]) -> None:
body = frame.get("body", {}) or {}
file_obj = body.get("file") or {}
url = file_obj.get("url")
aeskey = file_obj.get("aeskey")
if not isinstance(url, str) or not url:
return
await self._publish_ws_inbound(
frame,
"receive file",
files=[
{
"type": "file",
"url": url,
"aeskey": aeskey if isinstance(aeskey, str) and aeskey else None,
}
],
)
async def _publish_ws_inbound(
self,
frame: dict[str, Any],
text: str,
*,
files: list[dict[str, Any]] | None = None,
) -> None:
if not self._ws_client:
return
try:
from aibot import generate_req_id
except Exception:
return
body = frame.get("body", {}) or {}
msg_id = body.get("msgid")
if not msg_id:
return
user_id = (body.get("from") or {}).get("userid")
inbound_type = InboundMessageType.COMMAND if text.startswith("/") else InboundMessageType.CHAT
inbound = self._make_inbound(
chat_id=user_id, # keep user's conversation in memory
user_id=user_id,
text=text,
msg_type=inbound_type,
thread_ts=msg_id,
files=files or [],
metadata={"aibotid": body.get("aibotid"), "chattype": body.get("chattype")},
)
inbound.topic_id = user_id # keep the same thread
stream_id = generate_req_id("stream")
self._ws_frames[msg_id] = frame
self._ws_stream_ids[msg_id] = stream_id
try:
await self._ws_client.reply_stream(frame, stream_id, self._working_message, False)
except Exception:
pass
await self.bus.publish_inbound(inbound)
async def _send_ws(self, msg: OutboundMessage, *, _max_retries: int = 3) -> None:
if not self._ws_client:
return
try:
from aibot import generate_req_id
except Exception:
generate_req_id = None
if msg.thread_ts and msg.thread_ts in self._ws_frames:
frame = self._ws_frames[msg.thread_ts]
stream_id = self._ws_stream_ids.get(msg.thread_ts)
if not stream_id and generate_req_id:
stream_id = generate_req_id("stream")
self._ws_stream_ids[msg.thread_ts] = stream_id
if not stream_id:
return
last_exc: Exception | None = None
for attempt in range(_max_retries):
try:
await self._ws_client.reply_stream(frame, stream_id, msg.text, bool(msg.is_final))
return
except Exception as exc:
last_exc = exc
if attempt < _max_retries - 1:
await asyncio.sleep(2**attempt)
if last_exc:
raise last_exc
body = {"msgtype": "markdown", "markdown": {"content": msg.text}}
last_exc = None
for attempt in range(_max_retries):
try:
await self._ws_client.send_message(msg.chat_id, body)
return
except Exception as exc:
last_exc = exc
if attempt < _max_retries - 1:
await asyncio.sleep(2**attempt)
if last_exc:
raise last_exc
async def _upload_media_ws(
self,
*,
media_type: str,
filename: str,
path: str,
size: int,
) -> str | None:
if not self._ws_client:
return None
try:
from aibot import generate_req_id
except Exception:
return None
chunk_size = 512 * 1024
total_chunks = (size + chunk_size - 1) // chunk_size
if total_chunks < 1 or total_chunks > 100:
logger.warning("[WeCom] invalid total_chunks=%d for %s", total_chunks, filename)
return None
md5_hasher = hashlib.md5()
with open(path, "rb") as f:
for chunk in iter(lambda: f.read(1024 * 1024), b""):
md5_hasher.update(chunk)
md5 = md5_hasher.hexdigest()
init_req_id = generate_req_id("aibot_upload_media_init")
init_body = {
"type": media_type,
"filename": filename,
"total_size": int(size),
"total_chunks": int(total_chunks),
"md5": md5,
}
init_ack = await self._send_ws_upload_command(init_req_id, init_body, "aibot_upload_media_init")
upload_id = (init_ack.get("body") or {}).get("upload_id")
if not upload_id:
logger.warning("[WeCom] upload init returned no upload_id: %s", init_ack)
return None
with open(path, "rb") as f:
for idx in range(total_chunks):
data = f.read(chunk_size)
if not data:
break
chunk_req_id = generate_req_id("aibot_upload_media_chunk")
chunk_body = {
"upload_id": upload_id,
"chunk_index": int(idx),
"base64_data": base64.b64encode(data).decode("utf-8"),
}
await self._send_ws_upload_command(chunk_req_id, chunk_body, "aibot_upload_media_chunk")
finish_req_id = generate_req_id("aibot_upload_media_finish")
finish_ack = await self._send_ws_upload_command(finish_req_id, {"upload_id": upload_id}, "aibot_upload_media_finish")
media_id = (finish_ack.get("body") or {}).get("media_id")
if not media_id:
logger.warning("[WeCom] upload finish returned no media_id: %s", finish_ack)
return None
return media_id
+4
View File
@@ -0,0 +1,4 @@
from .app import app, create_app
from .config import GatewayConfig, get_gateway_config
__all__ = ["app", "create_app", "GatewayConfig", "get_gateway_config"]
+339
View File
@@ -0,0 +1,339 @@
import logging
import os
from collections.abc import AsyncGenerator
from contextlib import asynccontextmanager
from datetime import UTC
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from app.gateway.auth_middleware import AuthMiddleware
from app.gateway.config import get_gateway_config
from app.gateway.csrf_middleware import CSRFMiddleware
from app.gateway.deps import langgraph_runtime
from app.gateway.routers import (
agents,
artifacts,
assistants_compat,
auth,
channels,
mcp,
memory,
models,
runs,
skills,
suggestions,
thread_runs,
threads,
uploads,
)
from deerflow.config.app_config import get_app_config
# Configure logging
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
)
logger = logging.getLogger(__name__)
async def _ensure_admin_user(app: FastAPI) -> None:
"""Auto-create the admin user on first boot if no users exist.
Prints the generated password to stdout so the operator can log in.
On subsequent boots, warns if any user still needs setup.
Multi-worker safe: relies on SQLite UNIQUE constraint to resolve races.
Only the worker that successfully creates/updates the admin prints the
password; losers silently skip.
"""
import secrets
from app.gateway.deps import get_local_provider
provider = get_local_provider()
user_count = await provider.count_users()
if user_count == 0:
password = secrets.token_urlsafe(16)
try:
admin = await provider.create_user(email="admin@deerflow.dev", password=password, system_role="admin", needs_setup=True)
except ValueError:
return # Another worker already created the admin.
# Migrate orphaned threads (no user_id) to this admin
store = getattr(app.state, "store", None)
if store is not None:
await _migrate_orphaned_threads(store, str(admin.id))
logger.info("=" * 60)
logger.info(" Admin account created on first boot")
logger.info(" Email: %s", admin.email)
logger.info(" Password: %s", password)
logger.info(" Change it after login: Settings -> Account")
logger.info("=" * 60)
return
# Admin exists but setup never completed — reset password so operator
# can always find it in the console without needing the CLI.
# Multi-worker guard: if admin was created less than 5s ago, another
# worker just created it and will print the password — skip reset.
admin = await provider.get_user_by_email("admin@deerflow.dev")
if admin and admin.needs_setup:
import time
age = time.time() - admin.created_at.replace(tzinfo=UTC).timestamp()
if age < 30:
return # Just created by another worker in this startup; its password is still valid.
from app.gateway.auth.password import hash_password_async
password = secrets.token_urlsafe(16)
admin.password_hash = await hash_password_async(password)
admin.token_version += 1
await provider.update_user(admin)
logger.info("=" * 60)
logger.info(" Admin account setup incomplete — password reset")
logger.info(" Email: %s", admin.email)
logger.info(" Password: %s", password)
logger.info(" Change it after login: Settings -> Account")
logger.info("=" * 60)
async def _migrate_orphaned_threads(store, admin_user_id: str) -> None:
"""Migrate threads with no user_id to the given admin."""
try:
migrated = 0
results = await store.asearch(("threads",), limit=1000)
for item in results:
metadata = item.value.get("metadata", {})
if not metadata.get("user_id"):
metadata["user_id"] = admin_user_id
item.value["metadata"] = metadata
await store.aput(("threads",), item.key, item.value)
migrated += 1
if migrated:
logger.info("Migrated %d orphaned thread(s) to admin", migrated)
except Exception:
logger.exception("Thread migration failed (non-fatal)")
@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
"""Application lifespan handler."""
# Load config and check necessary environment variables at startup
try:
get_app_config()
logger.info("Configuration loaded successfully")
except Exception as e:
error_msg = f"Failed to load configuration during gateway startup: {e}"
logger.exception(error_msg)
raise RuntimeError(error_msg) from e
config = get_gateway_config()
logger.info(f"Starting API Gateway on {config.host}:{config.port}")
# Initialize LangGraph runtime components (StreamBridge, RunManager, checkpointer, store)
async with langgraph_runtime(app):
logger.info("LangGraph runtime initialised")
# Ensure admin user exists (auto-create on first boot)
# Must run AFTER langgraph_runtime so app.state.store is available for thread migration
await _ensure_admin_user(app)
# Start IM channel service if any channels are configured
try:
from app.channels.service import start_channel_service
channel_service = await start_channel_service()
logger.info("Channel service started: %s", channel_service.get_status())
except Exception:
logger.exception("No IM channels configured or channel service failed to start")
yield
# Stop channel service on shutdown
try:
from app.channels.service import stop_channel_service
await stop_channel_service()
except Exception:
logger.exception("Failed to stop channel service")
logger.info("Shutting down API Gateway")
def create_app() -> FastAPI:
"""Create and configure the FastAPI application.
Returns:
Configured FastAPI application instance.
"""
app = FastAPI(
title="DeerFlow API Gateway",
description="""
## DeerFlow API Gateway
API Gateway for DeerFlow - A LangGraph-based AI agent backend with sandbox execution capabilities.
### Features
- **Models Management**: Query and retrieve available AI models
- **MCP Configuration**: Manage Model Context Protocol (MCP) server configurations
- **Memory Management**: Access and manage global memory data for personalized conversations
- **Skills Management**: Query and manage skills and their enabled status
- **Artifacts**: Access thread artifacts and generated files
- **Health Monitoring**: System health check endpoints
### Architecture
LangGraph requests are handled by nginx reverse proxy.
This gateway provides custom endpoints for models, MCP configuration, skills, and artifacts.
""",
version="0.1.0",
lifespan=lifespan,
docs_url="/docs",
redoc_url="/redoc",
openapi_url="/openapi.json",
openapi_tags=[
{
"name": "models",
"description": "Operations for querying available AI models and their configurations",
},
{
"name": "mcp",
"description": "Manage Model Context Protocol (MCP) server configurations",
},
{
"name": "memory",
"description": "Access and manage global memory data for personalized conversations",
},
{
"name": "skills",
"description": "Manage skills and their configurations",
},
{
"name": "artifacts",
"description": "Access and download thread artifacts and generated files",
},
{
"name": "uploads",
"description": "Upload and manage user files for threads",
},
{
"name": "threads",
"description": "Manage DeerFlow thread-local filesystem data",
},
{
"name": "agents",
"description": "Create and manage custom agents with per-agent config and prompts",
},
{
"name": "suggestions",
"description": "Generate follow-up question suggestions for conversations",
},
{
"name": "channels",
"description": "Manage IM channel integrations (Feishu, Slack, Telegram)",
},
{
"name": "assistants-compat",
"description": "LangGraph Platform-compatible assistants API (stub)",
},
{
"name": "runs",
"description": "LangGraph Platform-compatible runs lifecycle (create, stream, cancel)",
},
{
"name": "health",
"description": "Health check and system status endpoints",
},
],
)
# Auth: reject unauthenticated requests to non-public paths (fail-closed safety net)
app.add_middleware(AuthMiddleware)
# CSRF: Double Submit Cookie pattern for state-changing requests
app.add_middleware(CSRFMiddleware)
# CORS: when GATEWAY_CORS_ORIGINS is set (dev without nginx), add CORS middleware
cors_origins_env = os.environ.get("GATEWAY_CORS_ORIGINS", "")
if cors_origins_env:
cors_origins = [o.strip() for o in cors_origins_env.split(",") if o.strip()]
# Validate: wildcard origin with credentials is a security misconfiguration
for origin in cors_origins:
if origin == "*":
logger.error("GATEWAY_CORS_ORIGINS contains wildcard '*' with allow_credentials=True. This is a security misconfiguration — browsers will reject the response. Use explicit scheme://host:port origins instead.")
cors_origins = [o for o in cors_origins if o != "*"]
break
if cors_origins:
app.add_middleware(
CORSMiddleware,
allow_origins=cors_origins,
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
# Include routers
# Models API is mounted at /api/models
app.include_router(models.router)
# MCP API is mounted at /api/mcp
app.include_router(mcp.router)
# Memory API is mounted at /api/memory
app.include_router(memory.router)
# Skills API is mounted at /api/skills
app.include_router(skills.router)
# Artifacts API is mounted at /api/threads/{thread_id}/artifacts
app.include_router(artifacts.router)
# Uploads API is mounted at /api/threads/{thread_id}/uploads
app.include_router(uploads.router)
# Thread cleanup API is mounted at /api/threads/{thread_id}
app.include_router(threads.router)
# Agents API is mounted at /api/agents
app.include_router(agents.router)
# Suggestions API is mounted at /api/threads/{thread_id}/suggestions
app.include_router(suggestions.router)
# Channels API is mounted at /api/channels
app.include_router(channels.router)
# Assistants compatibility API (LangGraph Platform stub)
app.include_router(assistants_compat.router)
# Auth API is mounted at /api/v1/auth
app.include_router(auth.router)
# Thread Runs API (LangGraph Platform-compatible runs lifecycle)
app.include_router(thread_runs.router)
# Stateless Runs API (stream/wait without a pre-existing thread)
app.include_router(runs.router)
@app.get("/health", tags=["health"])
async def health_check() -> dict:
"""Health check endpoint.
Returns:
Service health status information.
"""
return {"status": "healthy", "service": "deer-flow-gateway"}
return app
# Create app instance for uvicorn
app = create_app()
+42
View File
@@ -0,0 +1,42 @@
"""Authentication module for DeerFlow.
This module provides:
- JWT-based authentication
- Provider Factory pattern for extensible auth methods
- UserRepository interface for storage backends (SQLite)
"""
from app.gateway.auth.config import AuthConfig, get_auth_config, set_auth_config
from app.gateway.auth.errors import AuthErrorCode, AuthErrorResponse, TokenError
from app.gateway.auth.jwt import TokenPayload, create_access_token, decode_token
from app.gateway.auth.local_provider import LocalAuthProvider
from app.gateway.auth.models import User, UserResponse
from app.gateway.auth.password import hash_password, verify_password
from app.gateway.auth.providers import AuthProvider
from app.gateway.auth.repositories.base import UserRepository
__all__ = [
# Config
"AuthConfig",
"get_auth_config",
"set_auth_config",
# Errors
"AuthErrorCode",
"AuthErrorResponse",
"TokenError",
# JWT
"TokenPayload",
"create_access_token",
"decode_token",
# Password
"hash_password",
"verify_password",
# Models
"User",
"UserResponse",
# Providers
"AuthProvider",
"LocalAuthProvider",
# Repository
"UserRepository",
]
+55
View File
@@ -0,0 +1,55 @@
"""Authentication configuration for DeerFlow."""
import logging
import os
import secrets
from dotenv import load_dotenv
from pydantic import BaseModel, Field
load_dotenv()
logger = logging.getLogger(__name__)
class AuthConfig(BaseModel):
"""JWT and auth-related configuration. Parsed once at startup."""
jwt_secret: str = Field(
...,
description="Secret key for JWT signing. MUST be set via AUTH_JWT_SECRET.",
)
token_expiry_days: int = Field(default=7, ge=1, le=30)
users_db_path: str | None = Field(
default=None,
description="Path to users SQLite DB. Defaults to .deer-flow/users.db",
)
oauth_github_client_id: str | None = Field(default=None)
oauth_github_client_secret: str | None = Field(default=None)
_auth_config: AuthConfig | None = None
def get_auth_config() -> AuthConfig:
"""Get the global AuthConfig instance. Parses from env on first call."""
global _auth_config
if _auth_config is None:
jwt_secret = os.environ.get("AUTH_JWT_SECRET")
if not jwt_secret:
jwt_secret = secrets.token_urlsafe(32)
os.environ["AUTH_JWT_SECRET"] = jwt_secret
logger.warning(
"⚠ AUTH_JWT_SECRET is not set — using an auto-generated ephemeral secret. "
"Sessions will be invalidated on restart. "
"For production, add AUTH_JWT_SECRET to your .env file: "
'python -c "import secrets; print(secrets.token_urlsafe(32))"'
)
_auth_config = AuthConfig(jwt_secret=jwt_secret)
return _auth_config
def set_auth_config(config: AuthConfig) -> None:
"""Set the global AuthConfig instance (for testing)."""
global _auth_config
_auth_config = config
+44
View File
@@ -0,0 +1,44 @@
"""Typed error definitions for auth module.
AuthErrorCode: exhaustive enum of all auth failure conditions.
TokenError: exhaustive enum of JWT decode failures.
AuthErrorResponse: structured error payload for HTTP responses.
"""
from enum import StrEnum
from pydantic import BaseModel
class AuthErrorCode(StrEnum):
"""Exhaustive list of auth error conditions."""
INVALID_CREDENTIALS = "invalid_credentials"
TOKEN_EXPIRED = "token_expired"
TOKEN_INVALID = "token_invalid"
USER_NOT_FOUND = "user_not_found"
EMAIL_ALREADY_EXISTS = "email_already_exists"
PROVIDER_NOT_FOUND = "provider_not_found"
NOT_AUTHENTICATED = "not_authenticated"
class TokenError(StrEnum):
"""Exhaustive list of JWT decode failure reasons."""
EXPIRED = "expired"
INVALID_SIGNATURE = "invalid_signature"
MALFORMED = "malformed"
class AuthErrorResponse(BaseModel):
"""Structured error response — replaces bare `detail` strings."""
code: AuthErrorCode
message: str
def token_error_to_code(err: TokenError) -> AuthErrorCode:
"""Map TokenError to AuthErrorCode — single source of truth."""
if err == TokenError.EXPIRED:
return AuthErrorCode.TOKEN_EXPIRED
return AuthErrorCode.TOKEN_INVALID
+55
View File
@@ -0,0 +1,55 @@
"""JWT token creation and verification."""
from datetime import UTC, datetime, timedelta
import jwt
from pydantic import BaseModel
from app.gateway.auth.config import get_auth_config
from app.gateway.auth.errors import TokenError
class TokenPayload(BaseModel):
"""JWT token payload."""
sub: str # user_id
exp: datetime
iat: datetime | None = None
ver: int = 0 # token_version — must match User.token_version
def create_access_token(user_id: str, expires_delta: timedelta | None = None, token_version: int = 0) -> str:
"""Create a JWT access token.
Args:
user_id: The user's UUID as string
expires_delta: Optional custom expiry, defaults to 7 days
token_version: User's current token_version for invalidation
Returns:
Encoded JWT string
"""
config = get_auth_config()
expiry = expires_delta or timedelta(days=config.token_expiry_days)
now = datetime.now(UTC)
payload = {"sub": user_id, "exp": now + expiry, "iat": now, "ver": token_version}
return jwt.encode(payload, config.jwt_secret, algorithm="HS256")
def decode_token(token: str) -> TokenPayload | TokenError:
"""Decode and validate a JWT token.
Returns:
TokenPayload if valid, or a specific TokenError variant.
"""
config = get_auth_config()
try:
payload = jwt.decode(token, config.jwt_secret, algorithms=["HS256"])
return TokenPayload(**payload)
except jwt.ExpiredSignatureError:
return TokenError.EXPIRED
except jwt.InvalidSignatureError:
return TokenError.INVALID_SIGNATURE
except jwt.PyJWTError:
return TokenError.MALFORMED
@@ -0,0 +1,87 @@
"""Local email/password authentication provider."""
from app.gateway.auth.models import User
from app.gateway.auth.password import hash_password_async, verify_password_async
from app.gateway.auth.providers import AuthProvider
from app.gateway.auth.repositories.base import UserRepository
class LocalAuthProvider(AuthProvider):
"""Email/password authentication provider using local database."""
def __init__(self, repository: UserRepository):
"""Initialize with a UserRepository.
Args:
repository: UserRepository implementation (SQLite)
"""
self._repo = repository
async def authenticate(self, credentials: dict) -> User | None:
"""Authenticate with email and password.
Args:
credentials: dict with 'email' and 'password' keys
Returns:
User if authentication succeeds, None otherwise
"""
email = credentials.get("email")
password = credentials.get("password")
if not email or not password:
return None
user = await self._repo.get_user_by_email(email)
if user is None:
return None
if user.password_hash is None:
# OAuth user without local password
return None
if not await verify_password_async(password, user.password_hash):
return None
return user
async def get_user(self, user_id: str) -> User | None:
"""Get user by ID."""
return await self._repo.get_user_by_id(user_id)
async def create_user(self, email: str, password: str | None = None, system_role: str = "user", needs_setup: bool = False) -> User:
"""Create a new local user.
Args:
email: User email address
password: Plain text password (will be hashed)
system_role: Role to assign ("admin" or "user")
needs_setup: If True, user must complete setup on first login
Returns:
Created User instance
"""
password_hash = await hash_password_async(password) if password else None
user = User(
email=email,
password_hash=password_hash,
system_role=system_role,
needs_setup=needs_setup,
)
return await self._repo.create_user(user)
async def get_user_by_oauth(self, provider: str, oauth_id: str) -> User | None:
"""Get user by OAuth provider and ID."""
return await self._repo.get_user_by_oauth(provider, oauth_id)
async def count_users(self) -> int:
"""Return total number of registered users."""
return await self._repo.count_users()
async def update_user(self, user: User) -> User:
"""Update an existing user."""
return await self._repo.update_user(user)
async def get_user_by_email(self, email: str) -> User | None:
"""Get user by email."""
return await self._repo.get_user_by_email(email)
+41
View File
@@ -0,0 +1,41 @@
"""User Pydantic models for authentication."""
from datetime import UTC, datetime
from typing import Literal
from uuid import UUID, uuid4
from pydantic import BaseModel, ConfigDict, EmailStr, Field
def _utc_now() -> datetime:
"""Return current UTC time (timezone-aware)."""
return datetime.now(UTC)
class User(BaseModel):
"""Internal user representation."""
model_config = ConfigDict(from_attributes=True)
id: UUID = Field(default_factory=uuid4, description="Primary key")
email: EmailStr = Field(..., description="Unique email address")
password_hash: str | None = Field(None, description="bcrypt hash, nullable for OAuth users")
system_role: Literal["admin", "user"] = Field(default="user")
created_at: datetime = Field(default_factory=_utc_now)
# OAuth linkage (optional)
oauth_provider: str | None = Field(None, description="e.g. 'github', 'google'")
oauth_id: str | None = Field(None, description="User ID from OAuth provider")
# Auth lifecycle
needs_setup: bool = Field(default=False, description="True for auto-created admin until setup completes")
token_version: int = Field(default=0, description="Incremented on password change to invalidate old JWTs")
class UserResponse(BaseModel):
"""Response model for user info endpoint."""
id: str
email: str
system_role: Literal["admin", "user"]
needs_setup: bool = False
+33
View File
@@ -0,0 +1,33 @@
"""Password hashing utilities using bcrypt directly."""
import asyncio
import bcrypt
def hash_password(password: str) -> str:
"""Hash a password using bcrypt."""
return bcrypt.hashpw(password.encode("utf-8"), bcrypt.gensalt()).decode("utf-8")
def verify_password(plain_password: str, hashed_password: str) -> bool:
"""Verify a password against its hash."""
return bcrypt.checkpw(plain_password.encode("utf-8"), hashed_password.encode("utf-8"))
async def hash_password_async(password: str) -> str:
"""Hash a password using bcrypt (non-blocking).
Wraps the blocking bcrypt operation in a thread pool to avoid
blocking the event loop during password hashing.
"""
return await asyncio.to_thread(hash_password, password)
async def verify_password_async(plain_password: str, hashed_password: str) -> bool:
"""Verify a password against its hash (non-blocking).
Wraps the blocking bcrypt operation in a thread pool to avoid
blocking the event loop during password verification.
"""
return await asyncio.to_thread(verify_password, plain_password, hashed_password)
+24
View File
@@ -0,0 +1,24 @@
"""Auth provider abstraction."""
from abc import ABC, abstractmethod
class AuthProvider(ABC):
"""Abstract base class for authentication providers."""
@abstractmethod
async def authenticate(self, credentials: dict) -> "User | None":
"""Authenticate user with given credentials.
Returns User if authentication succeeds, None otherwise.
"""
...
@abstractmethod
async def get_user(self, user_id: str) -> "User | None":
"""Retrieve user by ID."""
...
# Import User at runtime to avoid circular imports
from app.gateway.auth.models import User # noqa: E402
@@ -0,0 +1,82 @@
"""User repository interface for abstracting database operations."""
from abc import ABC, abstractmethod
from app.gateway.auth.models import User
class UserRepository(ABC):
"""Abstract interface for user data storage.
Implement this interface to support different storage backends
(SQLite)
"""
@abstractmethod
async def create_user(self, user: User) -> User:
"""Create a new user.
Args:
user: User object to create
Returns:
Created User with ID assigned
Raises:
ValueError: If email already exists
"""
...
@abstractmethod
async def get_user_by_id(self, user_id: str) -> User | None:
"""Get user by ID.
Args:
user_id: User UUID as string
Returns:
User if found, None otherwise
"""
...
@abstractmethod
async def get_user_by_email(self, email: str) -> User | None:
"""Get user by email.
Args:
email: User email address
Returns:
User if found, None otherwise
"""
...
@abstractmethod
async def update_user(self, user: User) -> User:
"""Update an existing user.
Args:
user: User object with updated fields
Returns:
Updated User
"""
...
@abstractmethod
async def count_users(self) -> int:
"""Return total number of registered users."""
...
@abstractmethod
async def get_user_by_oauth(self, provider: str, oauth_id: str) -> User | None:
"""Get user by OAuth provider and ID.
Args:
provider: OAuth provider name (e.g. 'github', 'google')
oauth_id: User ID from the OAuth provider
Returns:
User if found, None otherwise
"""
...
@@ -0,0 +1,196 @@
"""SQLite implementation of UserRepository."""
import asyncio
import sqlite3
from contextlib import contextmanager
from datetime import UTC, datetime
from pathlib import Path
from typing import Any
from uuid import UUID
from app.gateway.auth.config import get_auth_config
from app.gateway.auth.models import User
from app.gateway.auth.repositories.base import UserRepository
_resolved_db_path: Path | None = None
_table_initialized: bool = False
def _get_users_db_path() -> Path:
"""Get the users database path (resolved and cached once)."""
global _resolved_db_path
if _resolved_db_path is not None:
return _resolved_db_path
config = get_auth_config()
if config.users_db_path:
_resolved_db_path = Path(config.users_db_path)
else:
_resolved_db_path = Path(".deer-flow/users.db")
_resolved_db_path.parent.mkdir(parents=True, exist_ok=True)
return _resolved_db_path
def _get_connection() -> sqlite3.Connection:
"""Get a SQLite connection for the users database."""
db_path = _get_users_db_path()
conn = sqlite3.connect(str(db_path))
conn.row_factory = sqlite3.Row
return conn
def _init_users_table(conn: sqlite3.Connection) -> None:
"""Initialize the users table if it doesn't exist."""
conn.execute("PRAGMA journal_mode=WAL")
conn.execute(
"""
CREATE TABLE IF NOT EXISTS users (
id TEXT PRIMARY KEY,
email TEXT UNIQUE NOT NULL,
password_hash TEXT,
system_role TEXT NOT NULL DEFAULT 'user',
created_at REAL NOT NULL,
oauth_provider TEXT,
oauth_id TEXT,
needs_setup INTEGER NOT NULL DEFAULT 0,
token_version INTEGER NOT NULL DEFAULT 0
)
"""
)
# Add unique constraint for OAuth identity to prevent duplicate social logins
conn.execute(
"""
CREATE UNIQUE INDEX IF NOT EXISTS idx_users_oauth_identity
ON users(oauth_provider, oauth_id)
WHERE oauth_provider IS NOT NULL AND oauth_id IS NOT NULL
"""
)
conn.commit()
@contextmanager
def _get_users_conn():
"""Context manager for users database connection."""
global _table_initialized
conn = _get_connection()
try:
if not _table_initialized:
_init_users_table(conn)
_table_initialized = True
yield conn
finally:
conn.close()
class SQLiteUserRepository(UserRepository):
"""SQLite implementation of UserRepository."""
async def create_user(self, user: User) -> User:
"""Create a new user in SQLite."""
return await asyncio.to_thread(self._create_user_sync, user)
def _create_user_sync(self, user: User) -> User:
"""Synchronous user creation (runs in thread pool)."""
with _get_users_conn() as conn:
try:
conn.execute(
"""
INSERT INTO users (id, email, password_hash, system_role, created_at, oauth_provider, oauth_id, needs_setup, token_version)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
(
str(user.id),
user.email,
user.password_hash,
user.system_role,
datetime.now(UTC).timestamp(),
user.oauth_provider,
user.oauth_id,
int(user.needs_setup),
user.token_version,
),
)
conn.commit()
except sqlite3.IntegrityError as e:
if "UNIQUE constraint failed: users.email" in str(e):
raise ValueError(f"Email already registered: {user.email}") from e
raise
return user
async def get_user_by_id(self, user_id: str) -> User | None:
"""Get user by ID from SQLite."""
return await asyncio.to_thread(self._get_user_by_id_sync, user_id)
def _get_user_by_id_sync(self, user_id: str) -> User | None:
"""Synchronous get by ID (runs in thread pool)."""
with _get_users_conn() as conn:
cursor = conn.execute("SELECT * FROM users WHERE id = ?", (user_id,))
row = cursor.fetchone()
if row is None:
return None
return self._row_to_user(dict(row))
async def get_user_by_email(self, email: str) -> User | None:
"""Get user by email from SQLite."""
return await asyncio.to_thread(self._get_user_by_email_sync, email)
def _get_user_by_email_sync(self, email: str) -> User | None:
"""Synchronous get by email (runs in thread pool)."""
with _get_users_conn() as conn:
cursor = conn.execute("SELECT * FROM users WHERE email = ?", (email,))
row = cursor.fetchone()
if row is None:
return None
return self._row_to_user(dict(row))
async def update_user(self, user: User) -> User:
"""Update an existing user in SQLite."""
return await asyncio.to_thread(self._update_user_sync, user)
def _update_user_sync(self, user: User) -> User:
with _get_users_conn() as conn:
conn.execute(
"UPDATE users SET email = ?, password_hash = ?, system_role = ?, oauth_provider = ?, oauth_id = ?, needs_setup = ?, token_version = ? WHERE id = ?",
(user.email, user.password_hash, user.system_role, user.oauth_provider, user.oauth_id, int(user.needs_setup), user.token_version, str(user.id)),
)
conn.commit()
return user
async def count_users(self) -> int:
"""Return total number of registered users."""
return await asyncio.to_thread(self._count_users_sync)
def _count_users_sync(self) -> int:
with _get_users_conn() as conn:
cursor = conn.execute("SELECT COUNT(*) FROM users")
return cursor.fetchone()[0]
async def get_user_by_oauth(self, provider: str, oauth_id: str) -> User | None:
"""Get user by OAuth provider and ID from SQLite."""
return await asyncio.to_thread(self._get_user_by_oauth_sync, provider, oauth_id)
def _get_user_by_oauth_sync(self, provider: str, oauth_id: str) -> User | None:
"""Synchronous get by OAuth (runs in thread pool)."""
with _get_users_conn() as conn:
cursor = conn.execute(
"SELECT * FROM users WHERE oauth_provider = ? AND oauth_id = ?",
(provider, oauth_id),
)
row = cursor.fetchone()
if row is None:
return None
return self._row_to_user(dict(row))
@staticmethod
def _row_to_user(row: dict[str, Any]) -> User:
"""Convert a database row to a User model."""
return User(
id=UUID(row["id"]),
email=row["email"],
password_hash=row["password_hash"],
system_role=row["system_role"],
created_at=datetime.fromtimestamp(row["created_at"], tz=UTC),
oauth_provider=row.get("oauth_provider"),
oauth_id=row.get("oauth_id"),
needs_setup=bool(row["needs_setup"]),
token_version=int(row["token_version"]),
)
+66
View File
@@ -0,0 +1,66 @@
"""CLI tool to reset admin password.
Usage:
python -m app.gateway.auth.reset_admin
python -m app.gateway.auth.reset_admin --email admin@example.com
"""
import argparse
import secrets
import sys
from app.gateway.auth.password import hash_password
from app.gateway.auth.repositories.sqlite import SQLiteUserRepository
def main() -> None:
parser = argparse.ArgumentParser(description="Reset admin password")
parser.add_argument("--email", help="Admin email (default: first admin found)")
args = parser.parse_args()
repo = SQLiteUserRepository()
# Find admin user synchronously (CLI context, no event loop)
import asyncio
user = asyncio.run(_find_admin(repo, args.email))
if user is None:
if args.email:
print(f"Error: user '{args.email}' not found.", file=sys.stderr)
else:
print("Error: no admin user found.", file=sys.stderr)
sys.exit(1)
new_password = secrets.token_urlsafe(16)
user.password_hash = hash_password(new_password)
user.token_version += 1
user.needs_setup = True
asyncio.run(repo.update_user(user))
print(f"Password reset for: {user.email}")
print(f"New password: {new_password}")
print("Next login will require setup (new email + password).")
async def _find_admin(repo: SQLiteUserRepository, email: str | None):
if email:
return await repo.get_user_by_email(email)
# Find first admin
import asyncio
from app.gateway.auth.repositories.sqlite import _get_users_conn
def _find_sync():
with _get_users_conn() as conn:
cursor = conn.execute("SELECT id FROM users WHERE system_role = 'admin' LIMIT 1")
row = cursor.fetchone()
return dict(row)["id"] if row else None
admin_id = await asyncio.to_thread(_find_sync)
if admin_id:
return await repo.get_user_by_id(admin_id)
return None
if __name__ == "__main__":
main()
+71
View File
@@ -0,0 +1,71 @@
"""Global authentication middleware — fail-closed safety net.
Rejects unauthenticated requests to non-public paths with 401.
Fine-grained permission checks remain in authz.py decorators.
"""
from collections.abc import Callable
from fastapi import Request, Response
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.responses import JSONResponse
from starlette.types import ASGIApp
from app.gateway.auth.errors import AuthErrorCode
# Paths that never require authentication.
_PUBLIC_PATH_PREFIXES: tuple[str, ...] = (
"/health",
"/docs",
"/redoc",
"/openapi.json",
)
# Exact auth paths that are public (login/register/status check).
# /api/v1/auth/me, /api/v1/auth/change-password etc. are NOT public.
_PUBLIC_EXACT_PATHS: frozenset[str] = frozenset(
{
"/api/v1/auth/login/local",
"/api/v1/auth/register",
"/api/v1/auth/logout",
"/api/v1/auth/setup-status",
}
)
def _is_public(path: str) -> bool:
stripped = path.rstrip("/")
if stripped in _PUBLIC_EXACT_PATHS:
return True
return any(path.startswith(prefix) for prefix in _PUBLIC_PATH_PREFIXES)
class AuthMiddleware(BaseHTTPMiddleware):
"""Coarse-grained auth gate: reject requests without a valid session cookie.
This does NOT verify JWT signature or user existence — that is the job of
``get_current_user_from_request`` in deps.py (called by ``@require_auth``).
The middleware only checks *presence* of the cookie so that new endpoints
that forget ``@require_auth`` are not completely exposed.
"""
def __init__(self, app: ASGIApp) -> None:
super().__init__(app)
async def dispatch(self, request: Request, call_next: Callable) -> Response:
if _is_public(request.url.path):
return await call_next(request)
# Non-public path: require session cookie
if not request.cookies.get("access_token"):
return JSONResponse(
status_code=401,
content={
"detail": {
"code": AuthErrorCode.NOT_AUTHENTICATED,
"message": "Authentication required",
}
},
)
return await call_next(request)
+261
View File
@@ -0,0 +1,261 @@
"""Authorization decorators and context for DeerFlow.
Inspired by LangGraph Auth system: https://github.com/langchain-ai/langgraph/blob/main/libs/sdk-py/langgraph_sdk/auth/__init__.py
**Usage:**
1. Use ``@require_auth`` on routes that need authentication
2. Use ``@require_permission("resource", "action", filter_key=...)`` for permission checks
3. The decorator chain processes from bottom to top
**Example:**
@router.get("/{thread_id}")
@require_auth
@require_permission("threads", "read", owner_check=True)
async def get_thread(thread_id: str, request: Request):
# User is authenticated and has threads:read permission
...
**Permission Model:**
- threads:read - View thread
- threads:write - Create/update thread
- threads:delete - Delete thread
- runs:create - Run agent
- runs:read - View run
- runs:cancel - Cancel run
"""
from __future__ import annotations
import functools
from collections.abc import Callable
from typing import TYPE_CHECKING, Any, ParamSpec, TypeVar
from fastapi import HTTPException, Request
if TYPE_CHECKING:
from app.gateway.auth.models import User
P = ParamSpec("P")
T = TypeVar("T")
# Permission constants
class Permissions:
"""Permission constants for resource:action format."""
# Threads
THREADS_READ = "threads:read"
THREADS_WRITE = "threads:write"
THREADS_DELETE = "threads:delete"
# Runs
RUNS_CREATE = "runs:create"
RUNS_READ = "runs:read"
RUNS_CANCEL = "runs:cancel"
class AuthContext:
"""Authentication context for the current request.
Stored in request.state.auth after require_auth decoration.
Attributes:
user: The authenticated user, or None if anonymous
permissions: List of permission strings (e.g., "threads:read")
"""
__slots__ = ("user", "permissions")
def __init__(self, user: User | None = None, permissions: list[str] | None = None):
self.user = user
self.permissions = permissions or []
@property
def is_authenticated(self) -> bool:
"""Check if user is authenticated."""
return self.user is not None
def has_permission(self, resource: str, action: str) -> bool:
"""Check if context has permission for resource:action.
Args:
resource: Resource name (e.g., "threads")
action: Action name (e.g., "read")
Returns:
True if user has permission
"""
permission = f"{resource}:{action}"
return permission in self.permissions
def require_user(self) -> User:
"""Get user or raise 401.
Raises:
HTTPException 401 if not authenticated
"""
if not self.user:
raise HTTPException(status_code=401, detail="Authentication required")
return self.user
def get_auth_context(request: Request) -> AuthContext | None:
"""Get AuthContext from request state."""
return getattr(request.state, "auth", None)
_ALL_PERMISSIONS: list[str] = [
Permissions.THREADS_READ,
Permissions.THREADS_WRITE,
Permissions.THREADS_DELETE,
Permissions.RUNS_CREATE,
Permissions.RUNS_READ,
Permissions.RUNS_CANCEL,
]
async def _authenticate(request: Request) -> AuthContext:
"""Authenticate request and return AuthContext.
Delegates to deps.get_optional_user_from_request() for the JWT→User pipeline.
Returns AuthContext with user=None for anonymous requests.
"""
from app.gateway.deps import get_optional_user_from_request
user = await get_optional_user_from_request(request)
if user is None:
return AuthContext(user=None, permissions=[])
# In future, permissions could be stored in user record
return AuthContext(user=user, permissions=_ALL_PERMISSIONS)
def require_auth[**P, T](func: Callable[P, T]) -> Callable[P, T]:
"""Decorator that authenticates the request and sets AuthContext.
Must be placed ABOVE other decorators (executes after them).
Usage:
@router.get("/{thread_id}")
@require_auth # Bottom decorator (executes first after permission check)
@require_permission("threads", "read")
async def get_thread(thread_id: str, request: Request):
auth: AuthContext = request.state.auth
...
Raises:
ValueError: If 'request' parameter is missing
"""
@functools.wraps(func)
async def wrapper(*args: Any, **kwargs: Any) -> Any:
request = kwargs.get("request")
if request is None:
raise ValueError("require_auth decorator requires 'request' parameter")
# Authenticate and set context
auth_context = await _authenticate(request)
request.state.auth = auth_context
return await func(*args, **kwargs)
return wrapper
def require_permission(
resource: str,
action: str,
owner_check: bool = False,
owner_filter_key: str = "user_id",
inject_record: bool = False,
) -> Callable[[Callable[P, T]], Callable[P, T]]:
"""Decorator that checks permission for resource:action.
Must be used AFTER @require_auth.
Args:
resource: Resource name (e.g., "threads", "runs")
action: Action name (e.g., "read", "write", "delete")
owner_check: If True, validates that the current user owns the resource.
Requires 'thread_id' path parameter and performs ownership check.
owner_filter_key: Field name for ownership filter (default: "user_id")
inject_record: If True and owner_check is True, injects the thread record
into kwargs['thread_record'] for use in the handler.
Usage:
# Simple permission check
@require_permission("threads", "read")
async def get_thread(thread_id: str, request: Request):
...
# With ownership check (for /threads/{thread_id} endpoints)
@require_permission("threads", "delete", owner_check=True)
async def delete_thread(thread_id: str, request: Request):
...
# With ownership check and record injection
@require_permission("threads", "delete", owner_check=True, inject_record=True)
async def delete_thread(thread_id: str, request: Request, thread_record: dict = None):
# thread_record is injected if found
...
Raises:
HTTPException 401: If authentication required but user is anonymous
HTTPException 403: If user lacks permission
HTTPException 404: If owner_check=True but user doesn't own the thread
ValueError: If owner_check=True but 'thread_id' parameter is missing
"""
def decorator(func: Callable[P, T]) -> Callable[P, T]:
@functools.wraps(func)
async def wrapper(*args: Any, **kwargs: Any) -> Any:
request = kwargs.get("request")
if request is None:
raise ValueError("require_permission decorator requires 'request' parameter")
auth: AuthContext = getattr(request.state, "auth", None)
if auth is None:
auth = await _authenticate(request)
request.state.auth = auth
if not auth.is_authenticated:
raise HTTPException(status_code=401, detail="Authentication required")
# Check permission
if not auth.has_permission(resource, action):
raise HTTPException(
status_code=403,
detail=f"Permission denied: {resource}:{action}",
)
# Owner check for thread-specific resources
if owner_check:
thread_id = kwargs.get("thread_id")
if thread_id is None:
raise ValueError("require_permission with owner_check=True requires 'thread_id' parameter")
# Get thread and verify ownership
from app.gateway.routers.threads import _store_get, get_store
store = get_store(request)
if store is not None:
record = await _store_get(store, thread_id)
if record:
owner_id = record.get("metadata", {}).get(owner_filter_key)
if owner_id and owner_id != str(auth.user.id):
raise HTTPException(
status_code=404,
detail=f"Thread {thread_id} not found",
)
# Inject record if requested
if inject_record:
kwargs["thread_record"] = record
return await func(*args, **kwargs)
return wrapper
return decorator
+27
View File
@@ -0,0 +1,27 @@
import os
from pydantic import BaseModel, Field
class GatewayConfig(BaseModel):
"""Configuration for the API Gateway."""
host: str = Field(default="0.0.0.0", description="Host to bind the gateway server")
port: int = Field(default=8001, description="Port to bind the gateway server")
cors_origins: list[str] = Field(default_factory=lambda: ["http://localhost:3000"], description="Allowed CORS origins")
_gateway_config: GatewayConfig | None = None
def get_gateway_config() -> GatewayConfig:
"""Get gateway config, loading from environment if available."""
global _gateway_config
if _gateway_config is None:
cors_origins_str = os.getenv("CORS_ORIGINS", "http://localhost:3000")
_gateway_config = GatewayConfig(
host=os.getenv("GATEWAY_HOST", "0.0.0.0"),
port=int(os.getenv("GATEWAY_PORT", "8001")),
cors_origins=cors_origins_str.split(","),
)
return _gateway_config
+112
View File
@@ -0,0 +1,112 @@
"""CSRF protection middleware for FastAPI.
Per RFC-001:
State-changing operations require CSRF protection.
"""
import secrets
from collections.abc import Callable
from fastapi import Request, Response
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.responses import JSONResponse
from starlette.types import ASGIApp
CSRF_COOKIE_NAME = "csrf_token"
CSRF_HEADER_NAME = "X-CSRF-Token"
CSRF_TOKEN_LENGTH = 64 # bytes
def is_secure_request(request: Request) -> bool:
"""Detect whether the original client request was made over HTTPS."""
return request.headers.get("x-forwarded-proto", request.url.scheme) == "https"
def generate_csrf_token() -> str:
"""Generate a secure random CSRF token."""
return secrets.token_urlsafe(CSRF_TOKEN_LENGTH)
def should_check_csrf(request: Request) -> bool:
"""Determine if a request needs CSRF validation.
CSRF is checked for state-changing methods (POST, PUT, DELETE, PATCH).
GET, HEAD, OPTIONS, and TRACE are exempt per RFC 7231.
"""
if request.method not in ("POST", "PUT", "DELETE", "PATCH"):
return False
path = request.url.path.rstrip("/")
# Exempt /api/v1/auth/me endpoint
if path == "/api/v1/auth/me":
return False
return True
_AUTH_EXEMPT_PATHS: frozenset[str] = frozenset(
{
"/api/v1/auth/login/local",
"/api/v1/auth/logout",
"/api/v1/auth/register",
}
)
def is_auth_endpoint(request: Request) -> bool:
"""Check if the request is to an auth endpoint.
Auth endpoints don't need CSRF validation on first call (no token).
"""
return request.url.path.rstrip("/") in _AUTH_EXEMPT_PATHS
class CSRFMiddleware(BaseHTTPMiddleware):
"""Middleware that implements CSRF protection using Double Submit Cookie pattern."""
def __init__(self, app: ASGIApp) -> None:
super().__init__(app)
async def dispatch(self, request: Request, call_next: Callable) -> Response:
_is_auth = is_auth_endpoint(request)
if should_check_csrf(request) and not _is_auth:
cookie_token = request.cookies.get(CSRF_COOKIE_NAME)
header_token = request.headers.get(CSRF_HEADER_NAME)
if not cookie_token or not header_token:
return JSONResponse(
status_code=403,
content={"detail": "CSRF token missing. Include X-CSRF-Token header."},
)
if not secrets.compare_digest(cookie_token, header_token):
return JSONResponse(
status_code=403,
content={"detail": "CSRF token mismatch."},
)
response = await call_next(request)
# For auth endpoints that set up session, also set CSRF cookie
if _is_auth and request.method == "POST":
# Generate a new CSRF token for the session
csrf_token = generate_csrf_token()
is_https = is_secure_request(request)
response.set_cookie(
key=CSRF_COOKIE_NAME,
value=csrf_token,
httponly=False, # Must be JS-readable for Double Submit Cookie pattern
secure=is_https,
samesite="strict",
)
return response
def get_csrf_token(request: Request) -> str | None:
"""Get the CSRF token from the current request's cookies.
This is useful for server-side rendering where you need to embed
token in forms or headers.
"""
return request.cookies.get(CSRF_COOKIE_NAME)
+153
View File
@@ -0,0 +1,153 @@
"""Centralized accessors for singleton objects stored on ``app.state``.
**Getters** (used by routers): raise 503 when a required dependency is
missing, except ``get_store`` which returns ``None``.
Initialization is handled directly in ``app.py`` via :class:`AsyncExitStack``.
"""
from __future__ import annotations
from collections.abc import AsyncGenerator
from contextlib import AsyncExitStack, asynccontextmanager
from typing import TYPE_CHECKING
from fastapi import FastAPI, HTTPException, Request
from deerflow.runtime import RunManager, StreamBridge
if TYPE_CHECKING:
from app.gateway.auth.local_provider import LocalAuthProvider
from app.gateway.auth.repositories.sqlite import SQLiteUserRepository
# ---------------------------------------------------------------------------
# Getters called by routers per-request
# ---------------------------------------------------------------------------
def get_stream_bridge(request: Request) -> StreamBridge:
"""Return the global :class:`StreamBridge`, or 503."""
bridge = getattr(request.app.state, "stream_bridge", None)
if bridge is None:
raise HTTPException(status_code=503, detail="Stream bridge not available")
return bridge
def get_run_manager(request: Request) -> RunManager:
"""Return the global :class:`RunManager`, or 503."""
mgr = getattr(request.app.state, "run_manager", None)
if mgr is None:
raise HTTPException(status_code=503, detail="Run manager not available")
return mgr
def get_checkpointer(request: Request):
"""Return the global checkpointer, or 503."""
cp = getattr(request.app.state, "checkpointer", None)
if cp is None:
raise HTTPException(status_code=503, detail="Checkpointer not available")
return cp
def get_store(request: Request):
"""Return the global store (may be ``None`` if not configured)."""
return getattr(request.app.state, "store", None)
# ---------------------------------------------------------------------------
# Auth helpers (used by authz.py)
# ---------------------------------------------------------------------------
# Cached singletons to avoid repeated instantiation per request
_cached_local_provider: LocalAuthProvider | None = None
_cached_repo: SQLiteUserRepository | None = None
def get_local_provider() -> LocalAuthProvider:
"""Get or create the cached LocalAuthProvider singleton."""
global _cached_local_provider, _cached_repo
if _cached_repo is None:
from app.gateway.auth.repositories.sqlite import SQLiteUserRepository
_cached_repo = SQLiteUserRepository()
if _cached_local_provider is None:
from app.gateway.auth.local_provider import LocalAuthProvider
_cached_local_provider = LocalAuthProvider(repository=_cached_repo)
return _cached_local_provider
async def get_current_user_from_request(request: Request):
"""Get the current authenticated user from the request cookie.
Raises HTTPException 401 if not authenticated.
"""
from app.gateway.auth import decode_token
from app.gateway.auth.errors import AuthErrorCode, AuthErrorResponse, TokenError, token_error_to_code
access_token = request.cookies.get("access_token")
if not access_token:
raise HTTPException(
status_code=401,
detail=AuthErrorResponse(code=AuthErrorCode.NOT_AUTHENTICATED, message="Not authenticated").model_dump(),
)
payload = decode_token(access_token)
if isinstance(payload, TokenError):
raise HTTPException(
status_code=401,
detail=AuthErrorResponse(code=token_error_to_code(payload), message=f"Token error: {payload.value}").model_dump(),
)
provider = get_local_provider()
user = await provider.get_user(payload.sub)
if user is None:
raise HTTPException(
status_code=401,
detail=AuthErrorResponse(code=AuthErrorCode.USER_NOT_FOUND, message="User not found").model_dump(),
)
# Token version mismatch → password was changed, token is stale
if user.token_version != payload.ver:
raise HTTPException(
status_code=401,
detail=AuthErrorResponse(code=AuthErrorCode.TOKEN_INVALID, message="Token revoked (password changed)").model_dump(),
)
return user
async def get_optional_user_from_request(request: Request):
"""Get optional authenticated user from request.
Returns None if not authenticated.
"""
try:
return await get_current_user_from_request(request)
except HTTPException:
return None
# ---------------------------------------------------------------------------
# Runtime bootstrap
# ---------------------------------------------------------------------------
@asynccontextmanager
async def langgraph_runtime(app: FastAPI) -> AsyncGenerator[None, None]:
"""Bootstrap and tear down all LangGraph runtime singletons.
Usage in ``app.py``::
async with langgraph_runtime(app):
yield
"""
from deerflow.agents.checkpointer.async_provider import make_checkpointer
from deerflow.runtime import make_store, make_stream_bridge
async with AsyncExitStack() as stack:
app.state.stream_bridge = await stack.enter_async_context(make_stream_bridge())
app.state.checkpointer = await stack.enter_async_context(make_checkpointer())
app.state.store = await stack.enter_async_context(make_store())
app.state.run_manager = RunManager()
yield
+106
View File
@@ -0,0 +1,106 @@
"""LangGraph Server auth handler — shares JWT logic with Gateway.
Loaded by LangGraph Server via langgraph.json ``auth.path``.
Reuses the same ``decode_token`` / ``get_auth_config`` as Gateway,
so both modes validate tokens with the same secret and rules.
Two layers:
1. @auth.authenticate validates JWT cookie, extracts user_id,
and enforces CSRF on state-changing methods (POST/PUT/DELETE/PATCH)
2. @auth.on returns metadata filter so each user only sees own threads
"""
import secrets
from langgraph_sdk import Auth
from app.gateway.auth.errors import TokenError
from app.gateway.auth.jwt import decode_token
from app.gateway.deps import get_local_provider
auth = Auth()
# Methods that require CSRF validation (state-changing per RFC 7231).
_CSRF_METHODS = frozenset({"POST", "PUT", "DELETE", "PATCH"})
def _check_csrf(request) -> None:
"""Enforce Double Submit Cookie CSRF check for state-changing requests.
Mirrors Gateway's CSRFMiddleware logic so that LangGraph routes
proxied directly by nginx have the same CSRF protection.
"""
method = getattr(request, "method", "") or ""
if method.upper() not in _CSRF_METHODS:
return
cookie_token = request.cookies.get("csrf_token")
header_token = request.headers.get("x-csrf-token")
if not cookie_token or not header_token:
raise Auth.exceptions.HTTPException(
status_code=403,
detail="CSRF token missing. Include X-CSRF-Token header.",
)
if not secrets.compare_digest(cookie_token, header_token):
raise Auth.exceptions.HTTPException(
status_code=403,
detail="CSRF token mismatch.",
)
@auth.authenticate
async def authenticate(request):
"""Validate the session cookie, decode JWT, and check token_version.
Same validation chain as Gateway's get_current_user_from_request:
cookie decode JWT DB lookup token_version match
Also enforces CSRF on state-changing methods.
"""
# CSRF check before authentication so forged cross-site requests
# are rejected early, even if the cookie carries a valid JWT.
_check_csrf(request)
token = request.cookies.get("access_token")
if not token:
raise Auth.exceptions.HTTPException(
status_code=401,
detail="Not authenticated",
)
payload = decode_token(token)
if isinstance(payload, TokenError):
raise Auth.exceptions.HTTPException(
status_code=401,
detail=f"Token error: {payload.value}",
)
user = await get_local_provider().get_user(payload.sub)
if user is None:
raise Auth.exceptions.HTTPException(
status_code=401,
detail="User not found",
)
if user.token_version != payload.ver:
raise Auth.exceptions.HTTPException(
status_code=401,
detail="Token revoked (password changed)",
)
return payload.sub
@auth.on
async def add_owner_filter(ctx: Auth.types.AuthContext, value: dict):
"""Inject user_id metadata on writes; filter by user_id on reads.
Gateway stores thread ownership as ``metadata.user_id``.
This handler ensures LangGraph Server enforces the same isolation.
"""
# On create/update: stamp user_id into metadata
metadata = value.setdefault("metadata", {})
metadata["user_id"] = ctx.user.identity
# Return filter dict — LangGraph applies it to search/read/delete
return {"user_id": ctx.user.identity}
+28
View File
@@ -0,0 +1,28 @@
"""Shared path resolution for thread virtual paths (e.g. mnt/user-data/outputs/...)."""
from pathlib import Path
from fastapi import HTTPException
from deerflow.config.paths import get_paths
def resolve_thread_virtual_path(thread_id: str, virtual_path: str) -> Path:
"""Resolve a virtual path to the actual filesystem path under thread user-data.
Args:
thread_id: The thread ID.
virtual_path: The virtual path as seen inside the sandbox
(e.g., /mnt/user-data/outputs/file.txt).
Returns:
The resolved filesystem path.
Raises:
HTTPException: If the path is invalid or outside allowed directories.
"""
try:
return get_paths().resolve_virtual_path(thread_id, virtual_path)
except ValueError as e:
status = 403 if "traversal" in str(e) else 400
raise HTTPException(status_code=status, detail=str(e))
+3
View File
@@ -0,0 +1,3 @@
from . import artifacts, assistants_compat, auth, mcp, models, skills, suggestions, thread_runs, threads, uploads
__all__ = ["artifacts", "assistants_compat", "auth", "mcp", "models", "skills", "suggestions", "threads", "thread_runs", "uploads"]
+383
View File
@@ -0,0 +1,383 @@
"""CRUD API for custom agents."""
import logging
import re
import shutil
import yaml
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel, Field
from deerflow.config.agents_config import AgentConfig, list_custom_agents, load_agent_config, load_agent_soul
from deerflow.config.paths import get_paths
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api", tags=["agents"])
AGENT_NAME_PATTERN = re.compile(r"^[A-Za-z0-9-]+$")
class AgentResponse(BaseModel):
"""Response model for a custom agent."""
name: str = Field(..., description="Agent name (hyphen-case)")
description: str = Field(default="", description="Agent description")
model: str | None = Field(default=None, description="Optional model override")
tool_groups: list[str] | None = Field(default=None, description="Optional tool group whitelist")
soul: str | None = Field(default=None, description="SOUL.md content")
class AgentsListResponse(BaseModel):
"""Response model for listing all custom agents."""
agents: list[AgentResponse]
class AgentCreateRequest(BaseModel):
"""Request body for creating a custom agent."""
name: str = Field(..., description="Agent name (must match ^[A-Za-z0-9-]+$, stored as lowercase)")
description: str = Field(default="", description="Agent description")
model: str | None = Field(default=None, description="Optional model override")
tool_groups: list[str] | None = Field(default=None, description="Optional tool group whitelist")
soul: str = Field(default="", description="SOUL.md content — agent personality and behavioral guardrails")
class AgentUpdateRequest(BaseModel):
"""Request body for updating a custom agent."""
description: str | None = Field(default=None, description="Updated description")
model: str | None = Field(default=None, description="Updated model override")
tool_groups: list[str] | None = Field(default=None, description="Updated tool group whitelist")
soul: str | None = Field(default=None, description="Updated SOUL.md content")
def _validate_agent_name(name: str) -> None:
"""Validate agent name against allowed pattern.
Args:
name: The agent name to validate.
Raises:
HTTPException: 422 if the name is invalid.
"""
if not AGENT_NAME_PATTERN.match(name):
raise HTTPException(
status_code=422,
detail=f"Invalid agent name '{name}'. Must match ^[A-Za-z0-9-]+$ (letters, digits, and hyphens only).",
)
def _normalize_agent_name(name: str) -> str:
"""Normalize agent name to lowercase for filesystem storage."""
return name.lower()
def _agent_config_to_response(agent_cfg: AgentConfig, include_soul: bool = False) -> AgentResponse:
"""Convert AgentConfig to AgentResponse."""
soul: str | None = None
if include_soul:
soul = load_agent_soul(agent_cfg.name) or ""
return AgentResponse(
name=agent_cfg.name,
description=agent_cfg.description,
model=agent_cfg.model,
tool_groups=agent_cfg.tool_groups,
soul=soul,
)
@router.get(
"/agents",
response_model=AgentsListResponse,
summary="List Custom Agents",
description="List all custom agents available in the agents directory, including their soul content.",
)
async def list_agents() -> AgentsListResponse:
"""List all custom agents.
Returns:
List of all custom agents with their metadata and soul content.
"""
try:
agents = list_custom_agents()
return AgentsListResponse(agents=[_agent_config_to_response(a, include_soul=True) for a in agents])
except Exception as e:
logger.error(f"Failed to list agents: {e}", exc_info=True)
raise HTTPException(status_code=500, detail=f"Failed to list agents: {str(e)}")
@router.get(
"/agents/check",
summary="Check Agent Name",
description="Validate an agent name and check if it is available (case-insensitive).",
)
async def check_agent_name(name: str) -> dict:
"""Check whether an agent name is valid and not yet taken.
Args:
name: The agent name to check.
Returns:
``{"available": true/false, "name": "<normalized>"}``
Raises:
HTTPException: 422 if the name is invalid.
"""
_validate_agent_name(name)
normalized = _normalize_agent_name(name)
available = not get_paths().agent_dir(normalized).exists()
return {"available": available, "name": normalized}
@router.get(
"/agents/{name}",
response_model=AgentResponse,
summary="Get Custom Agent",
description="Retrieve details and SOUL.md content for a specific custom agent.",
)
async def get_agent(name: str) -> AgentResponse:
"""Get a specific custom agent by name.
Args:
name: The agent name.
Returns:
Agent details including SOUL.md content.
Raises:
HTTPException: 404 if agent not found.
"""
_validate_agent_name(name)
name = _normalize_agent_name(name)
try:
agent_cfg = load_agent_config(name)
return _agent_config_to_response(agent_cfg, include_soul=True)
except FileNotFoundError:
raise HTTPException(status_code=404, detail=f"Agent '{name}' not found")
except Exception as e:
logger.error(f"Failed to get agent '{name}': {e}", exc_info=True)
raise HTTPException(status_code=500, detail=f"Failed to get agent: {str(e)}")
@router.post(
"/agents",
response_model=AgentResponse,
status_code=201,
summary="Create Custom Agent",
description="Create a new custom agent with its config and SOUL.md.",
)
async def create_agent_endpoint(request: AgentCreateRequest) -> AgentResponse:
"""Create a new custom agent.
Args:
request: The agent creation request.
Returns:
The created agent details.
Raises:
HTTPException: 409 if agent already exists, 422 if name is invalid.
"""
_validate_agent_name(request.name)
normalized_name = _normalize_agent_name(request.name)
agent_dir = get_paths().agent_dir(normalized_name)
if agent_dir.exists():
raise HTTPException(status_code=409, detail=f"Agent '{normalized_name}' already exists")
try:
agent_dir.mkdir(parents=True, exist_ok=True)
# Write config.yaml
config_data: dict = {"name": normalized_name}
if request.description:
config_data["description"] = request.description
if request.model is not None:
config_data["model"] = request.model
if request.tool_groups is not None:
config_data["tool_groups"] = request.tool_groups
config_file = agent_dir / "config.yaml"
with open(config_file, "w", encoding="utf-8") as f:
yaml.dump(config_data, f, default_flow_style=False, allow_unicode=True)
# Write SOUL.md
soul_file = agent_dir / "SOUL.md"
soul_file.write_text(request.soul, encoding="utf-8")
logger.info(f"Created agent '{normalized_name}' at {agent_dir}")
agent_cfg = load_agent_config(normalized_name)
return _agent_config_to_response(agent_cfg, include_soul=True)
except HTTPException:
raise
except Exception as e:
# Clean up on failure
if agent_dir.exists():
shutil.rmtree(agent_dir)
logger.error(f"Failed to create agent '{request.name}': {e}", exc_info=True)
raise HTTPException(status_code=500, detail=f"Failed to create agent: {str(e)}")
@router.put(
"/agents/{name}",
response_model=AgentResponse,
summary="Update Custom Agent",
description="Update an existing custom agent's config and/or SOUL.md.",
)
async def update_agent(name: str, request: AgentUpdateRequest) -> AgentResponse:
"""Update an existing custom agent.
Args:
name: The agent name.
request: The update request (all fields optional).
Returns:
The updated agent details.
Raises:
HTTPException: 404 if agent not found.
"""
_validate_agent_name(name)
name = _normalize_agent_name(name)
try:
agent_cfg = load_agent_config(name)
except FileNotFoundError:
raise HTTPException(status_code=404, detail=f"Agent '{name}' not found")
agent_dir = get_paths().agent_dir(name)
try:
# Update config if any config fields changed
config_changed = any(v is not None for v in [request.description, request.model, request.tool_groups])
if config_changed:
updated: dict = {
"name": agent_cfg.name,
"description": request.description if request.description is not None else agent_cfg.description,
}
new_model = request.model if request.model is not None else agent_cfg.model
if new_model is not None:
updated["model"] = new_model
new_tool_groups = request.tool_groups if request.tool_groups is not None else agent_cfg.tool_groups
if new_tool_groups is not None:
updated["tool_groups"] = new_tool_groups
config_file = agent_dir / "config.yaml"
with open(config_file, "w", encoding="utf-8") as f:
yaml.dump(updated, f, default_flow_style=False, allow_unicode=True)
# Update SOUL.md if provided
if request.soul is not None:
soul_path = agent_dir / "SOUL.md"
soul_path.write_text(request.soul, encoding="utf-8")
logger.info(f"Updated agent '{name}'")
refreshed_cfg = load_agent_config(name)
return _agent_config_to_response(refreshed_cfg, include_soul=True)
except HTTPException:
raise
except Exception as e:
logger.error(f"Failed to update agent '{name}': {e}", exc_info=True)
raise HTTPException(status_code=500, detail=f"Failed to update agent: {str(e)}")
class UserProfileResponse(BaseModel):
"""Response model for the global user profile (USER.md)."""
content: str | None = Field(default=None, description="USER.md content, or null if not yet created")
class UserProfileUpdateRequest(BaseModel):
"""Request body for setting the global user profile."""
content: str = Field(default="", description="USER.md content — describes the user's background and preferences")
@router.get(
"/user-profile",
response_model=UserProfileResponse,
summary="Get User Profile",
description="Read the global USER.md file that is injected into all custom agents.",
)
async def get_user_profile() -> UserProfileResponse:
"""Return the current USER.md content.
Returns:
UserProfileResponse with content=None if USER.md does not exist yet.
"""
try:
user_md_path = get_paths().user_md_file
if not user_md_path.exists():
return UserProfileResponse(content=None)
raw = user_md_path.read_text(encoding="utf-8").strip()
return UserProfileResponse(content=raw or None)
except Exception as e:
logger.error(f"Failed to read user profile: {e}", exc_info=True)
raise HTTPException(status_code=500, detail=f"Failed to read user profile: {str(e)}")
@router.put(
"/user-profile",
response_model=UserProfileResponse,
summary="Update User Profile",
description="Write the global USER.md file that is injected into all custom agents.",
)
async def update_user_profile(request: UserProfileUpdateRequest) -> UserProfileResponse:
"""Create or overwrite the global USER.md.
Args:
request: The update request with the new USER.md content.
Returns:
UserProfileResponse with the saved content.
"""
try:
paths = get_paths()
paths.base_dir.mkdir(parents=True, exist_ok=True)
paths.user_md_file.write_text(request.content, encoding="utf-8")
logger.info(f"Updated USER.md at {paths.user_md_file}")
return UserProfileResponse(content=request.content or None)
except Exception as e:
logger.error(f"Failed to update user profile: {e}", exc_info=True)
raise HTTPException(status_code=500, detail=f"Failed to update user profile: {str(e)}")
@router.delete(
"/agents/{name}",
status_code=204,
summary="Delete Custom Agent",
description="Delete a custom agent and all its files (config, SOUL.md, memory).",
)
async def delete_agent(name: str) -> None:
"""Delete a custom agent.
Args:
name: The agent name.
Raises:
HTTPException: 404 if agent not found.
"""
_validate_agent_name(name)
name = _normalize_agent_name(name)
agent_dir = get_paths().agent_dir(name)
if not agent_dir.exists():
raise HTTPException(status_code=404, detail=f"Agent '{name}' not found")
try:
shutil.rmtree(agent_dir)
logger.info(f"Deleted agent '{name}' from {agent_dir}")
except Exception as e:
logger.error(f"Failed to delete agent '{name}': {e}", exc_info=True)
raise HTTPException(status_code=500, detail=f"Failed to delete agent: {str(e)}")
+181
View File
@@ -0,0 +1,181 @@
import logging
import mimetypes
import zipfile
from pathlib import Path
from urllib.parse import quote
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import FileResponse, PlainTextResponse, Response
from app.gateway.path_utils import resolve_thread_virtual_path
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api", tags=["artifacts"])
ACTIVE_CONTENT_MIME_TYPES = {
"text/html",
"application/xhtml+xml",
"image/svg+xml",
}
def _build_content_disposition(disposition_type: str, filename: str) -> str:
"""Build an RFC 5987 encoded Content-Disposition header value."""
return f"{disposition_type}; filename*=UTF-8''{quote(filename)}"
def _build_attachment_headers(filename: str, extra_headers: dict[str, str] | None = None) -> dict[str, str]:
headers = {"Content-Disposition": _build_content_disposition("attachment", filename)}
if extra_headers:
headers.update(extra_headers)
return headers
def is_text_file_by_content(path: Path, sample_size: int = 8192) -> bool:
"""Check if file is text by examining content for null bytes."""
try:
with open(path, "rb") as f:
chunk = f.read(sample_size)
# Text files shouldn't contain null bytes
return b"\x00" not in chunk
except Exception:
return False
def _extract_file_from_skill_archive(zip_path: Path, internal_path: str) -> bytes | None:
"""Extract a file from a .skill ZIP archive.
Args:
zip_path: Path to the .skill file (ZIP archive).
internal_path: Path to the file inside the archive (e.g., "SKILL.md").
Returns:
The file content as bytes, or None if not found.
"""
if not zipfile.is_zipfile(zip_path):
return None
try:
with zipfile.ZipFile(zip_path, "r") as zip_ref:
# List all files in the archive
namelist = zip_ref.namelist()
# Try direct path first
if internal_path in namelist:
return zip_ref.read(internal_path)
# Try with any top-level directory prefix (e.g., "skill-name/SKILL.md")
for name in namelist:
if name.endswith("/" + internal_path) or name == internal_path:
return zip_ref.read(name)
# Not found
return None
except (zipfile.BadZipFile, KeyError):
return None
@router.get(
"/threads/{thread_id}/artifacts/{path:path}",
summary="Get Artifact File",
description="Retrieve an artifact file generated by the AI agent. Text and binary files can be viewed inline, while active web content is always downloaded.",
)
async def get_artifact(thread_id: str, path: str, request: Request, download: bool = False) -> Response:
"""Get an artifact file by its path.
The endpoint automatically detects file types and returns appropriate content types.
Use the `download` query parameter to force file download for non-active content.
Args:
thread_id: The thread ID.
path: The artifact path with virtual prefix (e.g., mnt/user-data/outputs/file.txt).
request: FastAPI request object (automatically injected).
Returns:
The file content as a FileResponse with appropriate content type:
- Active content (HTML/XHTML/SVG): Served as download attachment
- Text files: Plain text with proper MIME type
- Binary files: Inline display with download option
Raises:
HTTPException:
- 400 if path is invalid or not a file
- 403 if access denied (path traversal detected)
- 404 if file not found
Query Parameters:
download (bool): If true, forces attachment download for file types that are
otherwise returned inline or as plain text. Active HTML/XHTML/SVG content
is always downloaded regardless of this flag.
Example:
- Get text file inline: `/api/threads/abc123/artifacts/mnt/user-data/outputs/notes.txt`
- Download file: `/api/threads/abc123/artifacts/mnt/user-data/outputs/data.csv?download=true`
- Active web content such as `.html`, `.xhtml`, and `.svg` artifacts is always downloaded
"""
# Check if this is a request for a file inside a .skill archive (e.g., xxx.skill/SKILL.md)
if ".skill/" in path:
# Split the path at ".skill/" to get the ZIP file path and internal path
skill_marker = ".skill/"
marker_pos = path.find(skill_marker)
skill_file_path = path[: marker_pos + len(".skill")] # e.g., "mnt/user-data/outputs/my-skill.skill"
internal_path = path[marker_pos + len(skill_marker) :] # e.g., "SKILL.md"
actual_skill_path = resolve_thread_virtual_path(thread_id, skill_file_path)
if not actual_skill_path.exists():
raise HTTPException(status_code=404, detail=f"Skill file not found: {skill_file_path}")
if not actual_skill_path.is_file():
raise HTTPException(status_code=400, detail=f"Path is not a file: {skill_file_path}")
# Extract the file from the .skill archive
content = _extract_file_from_skill_archive(actual_skill_path, internal_path)
if content is None:
raise HTTPException(status_code=404, detail=f"File '{internal_path}' not found in skill archive")
# Determine MIME type based on the internal file
mime_type, _ = mimetypes.guess_type(internal_path)
# Add cache headers to avoid repeated ZIP extraction (cache for 5 minutes)
cache_headers = {"Cache-Control": "private, max-age=300"}
download_name = Path(internal_path).name or actual_skill_path.stem
if download or mime_type in ACTIVE_CONTENT_MIME_TYPES:
return Response(content=content, media_type=mime_type or "application/octet-stream", headers=_build_attachment_headers(download_name, cache_headers))
if mime_type and mime_type.startswith("text/"):
return PlainTextResponse(content=content.decode("utf-8"), media_type=mime_type, headers=cache_headers)
# Default to plain text for unknown types that look like text
try:
return PlainTextResponse(content=content.decode("utf-8"), media_type="text/plain", headers=cache_headers)
except UnicodeDecodeError:
return Response(content=content, media_type=mime_type or "application/octet-stream", headers=cache_headers)
actual_path = resolve_thread_virtual_path(thread_id, path)
logger.info(f"Resolving artifact path: thread_id={thread_id}, requested_path={path}, actual_path={actual_path}")
if not actual_path.exists():
raise HTTPException(status_code=404, detail=f"Artifact not found: {path}")
if not actual_path.is_file():
raise HTTPException(status_code=400, detail=f"Path is not a file: {path}")
mime_type, _ = mimetypes.guess_type(actual_path)
if download:
return FileResponse(path=actual_path, filename=actual_path.name, media_type=mime_type, headers=_build_attachment_headers(actual_path.name))
# Always force download for active content types to prevent script execution
# in the application origin when users open generated artifacts.
if mime_type in ACTIVE_CONTENT_MIME_TYPES:
return FileResponse(path=actual_path, filename=actual_path.name, media_type=mime_type, headers=_build_attachment_headers(actual_path.name))
if mime_type and mime_type.startswith("text/"):
return PlainTextResponse(content=actual_path.read_text(encoding="utf-8"), media_type=mime_type)
if is_text_file_by_content(actual_path):
return PlainTextResponse(content=actual_path.read_text(encoding="utf-8"), media_type=mime_type)
return Response(content=actual_path.read_bytes(), media_type=mime_type, headers={"Content-Disposition": _build_content_disposition("inline", actual_path.name)})
@@ -0,0 +1,149 @@
"""Assistants compatibility endpoints.
Provides LangGraph Platform-compatible assistants API backed by the
``langgraph.json`` graph registry and ``config.yaml`` agent definitions.
This is a minimal stub that satisfies the ``useStream`` React hook's
initialization requirements (``assistants.search()`` and ``assistants.get()``).
"""
from __future__ import annotations
import logging
from datetime import UTC, datetime
from typing import Any
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel, Field
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/assistants", tags=["assistants-compat"])
class AssistantResponse(BaseModel):
assistant_id: str
graph_id: str
name: str
config: dict[str, Any] = Field(default_factory=dict)
metadata: dict[str, Any] = Field(default_factory=dict)
description: str | None = None
created_at: str = ""
updated_at: str = ""
version: int = 1
class AssistantSearchRequest(BaseModel):
graph_id: str | None = None
name: str | None = None
metadata: dict[str, Any] | None = None
limit: int = 10
offset: int = 0
def _get_default_assistant() -> AssistantResponse:
"""Return the default lead_agent assistant."""
now = datetime.now(UTC).isoformat()
return AssistantResponse(
assistant_id="lead_agent",
graph_id="lead_agent",
name="lead_agent",
config={},
metadata={"created_by": "system"},
description="DeerFlow lead agent",
created_at=now,
updated_at=now,
version=1,
)
def _list_assistants() -> list[AssistantResponse]:
"""List all available assistants from config."""
assistants = [_get_default_assistant()]
# Also include custom agents from config.yaml agents directory
try:
from deerflow.config.agents_config import list_custom_agents
for agent_cfg in list_custom_agents():
now = datetime.now(UTC).isoformat()
assistants.append(
AssistantResponse(
assistant_id=agent_cfg.name,
graph_id="lead_agent", # All agents use the same graph
name=agent_cfg.name,
config={},
metadata={"created_by": "user"},
description=agent_cfg.description or "",
created_at=now,
updated_at=now,
version=1,
)
)
except Exception:
logger.debug("Could not load custom agents for assistants list")
return assistants
@router.post("/search", response_model=list[AssistantResponse])
async def search_assistants(body: AssistantSearchRequest | None = None) -> list[AssistantResponse]:
"""Search assistants.
Returns all registered assistants (lead_agent + custom agents from config).
"""
assistants = _list_assistants()
if body and body.graph_id:
assistants = [a for a in assistants if a.graph_id == body.graph_id]
if body and body.name:
assistants = [a for a in assistants if body.name.lower() in a.name.lower()]
offset = body.offset if body else 0
limit = body.limit if body else 10
return assistants[offset : offset + limit]
@router.get("/{assistant_id}", response_model=AssistantResponse)
async def get_assistant_compat(assistant_id: str) -> AssistantResponse:
"""Get an assistant by ID."""
for a in _list_assistants():
if a.assistant_id == assistant_id:
return a
raise HTTPException(status_code=404, detail=f"Assistant {assistant_id} not found")
@router.get("/{assistant_id}/graph")
async def get_assistant_graph(assistant_id: str) -> dict:
"""Get the graph structure for an assistant.
Returns a minimal graph description. Full graph introspection is
not supported in the Gateway this stub satisfies SDK validation.
"""
found = any(a.assistant_id == assistant_id for a in _list_assistants())
if not found:
raise HTTPException(status_code=404, detail=f"Assistant {assistant_id} not found")
return {
"graph_id": "lead_agent",
"nodes": [],
"edges": [],
}
@router.get("/{assistant_id}/schemas")
async def get_assistant_schemas(assistant_id: str) -> dict:
"""Get JSON schemas for an assistant's input/output/state.
Returns empty schemas full introspection not supported in Gateway.
"""
found = any(a.assistant_id == assistant_id for a in _list_assistants())
if not found:
raise HTTPException(status_code=404, detail=f"Assistant {assistant_id} not found")
return {
"graph_id": "lead_agent",
"input_schema": {},
"output_schema": {},
"state_schema": {},
"config_schema": {},
}
+303
View File
@@ -0,0 +1,303 @@
"""Authentication endpoints."""
import logging
import time
from fastapi import APIRouter, Depends, HTTPException, Request, Response, status
from fastapi.security import OAuth2PasswordRequestForm
from pydantic import BaseModel, EmailStr, Field
from app.gateway.auth import (
UserResponse,
create_access_token,
)
from app.gateway.auth.config import get_auth_config
from app.gateway.auth.errors import AuthErrorCode, AuthErrorResponse
from app.gateway.csrf_middleware import is_secure_request
from app.gateway.deps import get_current_user_from_request, get_local_provider
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/v1/auth", tags=["auth"])
# ── Request/Response Models ──────────────────────────────────────────────
class LoginResponse(BaseModel):
"""Response model for login — token only lives in HttpOnly cookie."""
expires_in: int # seconds
needs_setup: bool = False
class RegisterRequest(BaseModel):
"""Request model for user registration."""
email: EmailStr
password: str = Field(..., min_length=8)
class ChangePasswordRequest(BaseModel):
"""Request model for password change (also handles setup flow)."""
current_password: str
new_password: str = Field(..., min_length=8)
new_email: EmailStr | None = None
class MessageResponse(BaseModel):
"""Generic message response."""
message: str
# ── Helpers ───────────────────────────────────────────────────────────────
def _set_session_cookie(response: Response, token: str, request: Request) -> None:
"""Set the access_token HttpOnly cookie on the response."""
config = get_auth_config()
is_https = is_secure_request(request)
response.set_cookie(
key="access_token",
value=token,
httponly=True,
secure=is_https,
samesite="lax",
max_age=config.token_expiry_days * 24 * 3600 if is_https else None,
)
# ── Rate Limiting ────────────────────────────────────────────────────────
# In-process dict — not shared across workers. Sufficient for single-worker deployments.
_MAX_LOGIN_ATTEMPTS = 5
_LOCKOUT_SECONDS = 300 # 5 minutes
# ip → (fail_count, lock_until_timestamp)
_login_attempts: dict[str, tuple[int, float]] = {}
def _get_client_ip(request: Request) -> str:
"""Extract the real client IP for rate limiting.
Uses ``X-Real-IP`` header set by nginx (``proxy_set_header X-Real-IP
$remote_addr``). Nginx unconditionally overwrites any client-supplied
``X-Real-IP``, so the value seen by Gateway is always the TCP peer IP
that nginx observed it cannot be spoofed by the client.
``request.client.host`` is NOT reliable because uvicorn's default
``proxy_headers=True`` replaces it with the *first* entry from
``X-Forwarded-For``, which IS client-spoofable.
``X-Forwarded-For`` is intentionally NOT used for the same reason.
"""
real_ip = request.headers.get("x-real-ip", "").strip()
if real_ip:
return real_ip
# Fallback: direct connection without nginx (e.g. unit tests, dev).
return request.client.host if request.client else "unknown"
def _check_rate_limit(ip: str) -> None:
"""Raise 429 if the IP is currently locked out."""
record = _login_attempts.get(ip)
if record is None:
return
fail_count, lock_until = record
if fail_count >= _MAX_LOGIN_ATTEMPTS:
if time.time() < lock_until:
raise HTTPException(
status_code=429,
detail="Too many login attempts. Try again later.",
)
del _login_attempts[ip]
_MAX_TRACKED_IPS = 10000
def _record_login_failure(ip: str) -> None:
"""Record a failed login attempt for the given IP."""
# Evict expired lockouts when dict grows too large
if len(_login_attempts) >= _MAX_TRACKED_IPS:
now = time.time()
expired = [k for k, (c, t) in _login_attempts.items() if c >= _MAX_LOGIN_ATTEMPTS and now >= t]
for k in expired:
del _login_attempts[k]
# If still too large, evict cheapest-to-lose half: below-threshold
# IPs (lock_until=0.0) sort first, then earliest-expiring lockouts.
if len(_login_attempts) >= _MAX_TRACKED_IPS:
by_time = sorted(_login_attempts.items(), key=lambda kv: kv[1][1])
for k, _ in by_time[: len(by_time) // 2]:
del _login_attempts[k]
record = _login_attempts.get(ip)
if record is None:
_login_attempts[ip] = (1, 0.0)
else:
new_count = record[0] + 1
lock_until = time.time() + _LOCKOUT_SECONDS if new_count >= _MAX_LOGIN_ATTEMPTS else 0.0
_login_attempts[ip] = (new_count, lock_until)
def _record_login_success(ip: str) -> None:
"""Clear failure counter for the given IP on successful login."""
_login_attempts.pop(ip, None)
# ── Endpoints ─────────────────────────────────────────────────────────────
@router.post("/login/local", response_model=LoginResponse)
async def login_local(
request: Request,
response: Response,
form_data: OAuth2PasswordRequestForm = Depends(),
):
"""Local email/password login."""
client_ip = _get_client_ip(request)
_check_rate_limit(client_ip)
user = await get_local_provider().authenticate({"email": form_data.username, "password": form_data.password})
if user is None:
_record_login_failure(client_ip)
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail=AuthErrorResponse(code=AuthErrorCode.INVALID_CREDENTIALS, message="Incorrect email or password").model_dump(),
)
_record_login_success(client_ip)
token = create_access_token(str(user.id), token_version=user.token_version)
_set_session_cookie(response, token, request)
return LoginResponse(
expires_in=get_auth_config().token_expiry_days * 24 * 3600,
needs_setup=user.needs_setup,
)
@router.post("/register", response_model=UserResponse, status_code=status.HTTP_201_CREATED)
async def register(request: Request, response: Response, body: RegisterRequest):
"""Register a new user account (always 'user' role).
Admin is auto-created on first boot. This endpoint creates regular users.
Auto-login by setting the session cookie.
"""
try:
user = await get_local_provider().create_user(email=body.email, password=body.password, system_role="user")
except ValueError:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=AuthErrorResponse(code=AuthErrorCode.EMAIL_ALREADY_EXISTS, message="Email already registered").model_dump(),
)
token = create_access_token(str(user.id), token_version=user.token_version)
_set_session_cookie(response, token, request)
return UserResponse(id=str(user.id), email=user.email, system_role=user.system_role)
@router.post("/logout", response_model=MessageResponse)
async def logout(request: Request, response: Response):
"""Logout current user by clearing the cookie."""
response.delete_cookie(key="access_token", secure=is_secure_request(request), samesite="lax")
return MessageResponse(message="Successfully logged out")
@router.post("/change-password", response_model=MessageResponse)
async def change_password(request: Request, response: Response, body: ChangePasswordRequest):
"""Change password for the currently authenticated user.
Also handles the first-boot setup flow:
- If new_email is provided, updates email (checks uniqueness)
- If user.needs_setup is True and new_email is given, clears needs_setup
- Always increments token_version to invalidate old sessions
- Re-issues session cookie with new token_version
"""
from app.gateway.auth.password import hash_password_async, verify_password_async
user = await get_current_user_from_request(request)
if user.password_hash is None:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=AuthErrorResponse(code=AuthErrorCode.INVALID_CREDENTIALS, message="OAuth users cannot change password").model_dump())
if not await verify_password_async(body.current_password, user.password_hash):
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=AuthErrorResponse(code=AuthErrorCode.INVALID_CREDENTIALS, message="Current password is incorrect").model_dump())
provider = get_local_provider()
# Update email if provided
if body.new_email is not None:
existing = await provider.get_user_by_email(body.new_email)
if existing and str(existing.id) != str(user.id):
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=AuthErrorResponse(code=AuthErrorCode.EMAIL_ALREADY_EXISTS, message="Email already in use").model_dump())
user.email = body.new_email
# Update password + bump version
user.password_hash = await hash_password_async(body.new_password)
user.token_version += 1
# Clear setup flag if this is the setup flow
if user.needs_setup and body.new_email is not None:
user.needs_setup = False
await provider.update_user(user)
# Re-issue cookie with new token_version
token = create_access_token(str(user.id), token_version=user.token_version)
_set_session_cookie(response, token, request)
return MessageResponse(message="Password changed successfully")
@router.get("/me", response_model=UserResponse)
async def get_me(request: Request):
"""Get current authenticated user info."""
user = await get_current_user_from_request(request)
return UserResponse(id=str(user.id), email=user.email, system_role=user.system_role, needs_setup=user.needs_setup)
@router.get("/setup-status")
async def setup_status():
"""Check if admin account exists. Always False after first boot."""
user_count = await get_local_provider().count_users()
return {"needs_setup": user_count == 0}
# ── OAuth Endpoints (Future/Placeholder) ─────────────────────────────────
@router.get("/oauth/{provider}")
async def oauth_login(provider: str):
"""Initiate OAuth login flow.
Redirects to the OAuth provider's authorization URL.
Currently a placeholder - requires OAuth provider implementation.
"""
if provider not in ["github", "google"]:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Unsupported OAuth provider: {provider}",
)
raise HTTPException(
status_code=status.HTTP_501_NOT_IMPLEMENTED,
detail="OAuth login not yet implemented",
)
@router.get("/callback/{provider}")
async def oauth_callback(provider: str, code: str, state: str):
"""OAuth callback endpoint.
Handles the OAuth provider's callback after user authorization.
Currently a placeholder.
"""
raise HTTPException(
status_code=status.HTTP_501_NOT_IMPLEMENTED,
detail="OAuth callback not yet implemented",
)
+52
View File
@@ -0,0 +1,52 @@
"""Gateway router for IM channel management."""
from __future__ import annotations
import logging
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/channels", tags=["channels"])
class ChannelStatusResponse(BaseModel):
service_running: bool
channels: dict[str, dict]
class ChannelRestartResponse(BaseModel):
success: bool
message: str
@router.get("/", response_model=ChannelStatusResponse)
async def get_channels_status() -> ChannelStatusResponse:
"""Get the status of all IM channels."""
from app.channels.service import get_channel_service
service = get_channel_service()
if service is None:
return ChannelStatusResponse(service_running=False, channels={})
status = service.get_status()
return ChannelStatusResponse(**status)
@router.post("/{name}/restart", response_model=ChannelRestartResponse)
async def restart_channel(name: str) -> ChannelRestartResponse:
"""Restart a specific IM channel."""
from app.channels.service import get_channel_service
service = get_channel_service()
if service is None:
raise HTTPException(status_code=503, detail="Channel service is not running")
success = await service.restart_channel(name)
if success:
logger.info("Channel %s restarted successfully", name)
return ChannelRestartResponse(success=True, message=f"Channel {name} restarted successfully")
else:
logger.warning("Failed to restart channel %s", name)
return ChannelRestartResponse(success=False, message=f"Failed to restart channel {name}")
+169
View File
@@ -0,0 +1,169 @@
import json
import logging
from pathlib import Path
from typing import Literal
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel, Field
from deerflow.config.extensions_config import ExtensionsConfig, get_extensions_config, reload_extensions_config
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api", tags=["mcp"])
class McpOAuthConfigResponse(BaseModel):
"""OAuth configuration for an MCP server."""
enabled: bool = Field(default=True, description="Whether OAuth token injection is enabled")
token_url: str = Field(default="", description="OAuth token endpoint URL")
grant_type: Literal["client_credentials", "refresh_token"] = Field(default="client_credentials", description="OAuth grant type")
client_id: str | None = Field(default=None, description="OAuth client ID")
client_secret: str | None = Field(default=None, description="OAuth client secret")
refresh_token: str | None = Field(default=None, description="OAuth refresh token")
scope: str | None = Field(default=None, description="OAuth scope")
audience: str | None = Field(default=None, description="OAuth audience")
token_field: str = Field(default="access_token", description="Token response field containing access token")
token_type_field: str = Field(default="token_type", description="Token response field containing token type")
expires_in_field: str = Field(default="expires_in", description="Token response field containing expires-in seconds")
default_token_type: str = Field(default="Bearer", description="Default token type when response omits token_type")
refresh_skew_seconds: int = Field(default=60, description="Refresh this many seconds before expiry")
extra_token_params: dict[str, str] = Field(default_factory=dict, description="Additional form params sent to token endpoint")
class McpServerConfigResponse(BaseModel):
"""Response model for MCP server configuration."""
enabled: bool = Field(default=True, description="Whether this MCP server is enabled")
type: str = Field(default="stdio", description="Transport type: 'stdio', 'sse', or 'http'")
command: str | None = Field(default=None, description="Command to execute to start the MCP server (for stdio type)")
args: list[str] = Field(default_factory=list, description="Arguments to pass to the command (for stdio type)")
env: dict[str, str] = Field(default_factory=dict, description="Environment variables for the MCP server")
url: str | None = Field(default=None, description="URL of the MCP server (for sse or http type)")
headers: dict[str, str] = Field(default_factory=dict, description="HTTP headers to send (for sse or http type)")
oauth: McpOAuthConfigResponse | None = Field(default=None, description="OAuth configuration for MCP HTTP/SSE servers")
description: str = Field(default="", description="Human-readable description of what this MCP server provides")
class McpConfigResponse(BaseModel):
"""Response model for MCP configuration."""
mcp_servers: dict[str, McpServerConfigResponse] = Field(
default_factory=dict,
description="Map of MCP server name to configuration",
)
class McpConfigUpdateRequest(BaseModel):
"""Request model for updating MCP configuration."""
mcp_servers: dict[str, McpServerConfigResponse] = Field(
...,
description="Map of MCP server name to configuration",
)
@router.get(
"/mcp/config",
response_model=McpConfigResponse,
summary="Get MCP Configuration",
description="Retrieve the current Model Context Protocol (MCP) server configurations.",
)
async def get_mcp_configuration() -> McpConfigResponse:
"""Get the current MCP configuration.
Returns:
The current MCP configuration with all servers.
Example:
```json
{
"mcp_servers": {
"github": {
"enabled": true,
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-github"],
"env": {"GITHUB_TOKEN": "ghp_xxx"},
"description": "GitHub MCP server for repository operations"
}
}
}
```
"""
config = get_extensions_config()
return McpConfigResponse(mcp_servers={name: McpServerConfigResponse(**server.model_dump()) for name, server in config.mcp_servers.items()})
@router.put(
"/mcp/config",
response_model=McpConfigResponse,
summary="Update MCP Configuration",
description="Update Model Context Protocol (MCP) server configurations and save to file.",
)
async def update_mcp_configuration(request: McpConfigUpdateRequest) -> McpConfigResponse:
"""Update the MCP configuration.
This will:
1. Save the new configuration to the mcp_config.json file
2. Reload the configuration cache
3. Reset MCP tools cache to trigger reinitialization
Args:
request: The new MCP configuration to save.
Returns:
The updated MCP configuration.
Raises:
HTTPException: 500 if the configuration file cannot be written.
Example Request:
```json
{
"mcp_servers": {
"github": {
"enabled": true,
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-github"],
"env": {"GITHUB_TOKEN": "$GITHUB_TOKEN"},
"description": "GitHub MCP server for repository operations"
}
}
}
```
"""
try:
# Get the current config path (or determine where to save it)
config_path = ExtensionsConfig.resolve_config_path()
# If no config file exists, create one in the parent directory (project root)
if config_path is None:
config_path = Path.cwd().parent / "extensions_config.json"
logger.info(f"No existing extensions config found. Creating new config at: {config_path}")
# Load current config to preserve skills configuration
current_config = get_extensions_config()
# Convert request to dict format for JSON serialization
config_data = {
"mcpServers": {name: server.model_dump() for name, server in request.mcp_servers.items()},
"skills": {name: {"enabled": skill.enabled} for name, skill in current_config.skills.items()},
}
# Write the configuration to file
with open(config_path, "w", encoding="utf-8") as f:
json.dump(config_data, f, indent=2)
logger.info(f"MCP configuration updated and saved to: {config_path}")
# NOTE: No need to reload/reset cache here - LangGraph Server (separate process)
# will detect config file changes via mtime and reinitialize MCP tools automatically
# Reload the configuration and update the global cache
reloaded_config = reload_extensions_config()
return McpConfigResponse(mcp_servers={name: McpServerConfigResponse(**server.model_dump()) for name, server in reloaded_config.mcp_servers.items()})
except Exception as e:
logger.error(f"Failed to update MCP configuration: {e}", exc_info=True)
raise HTTPException(status_code=500, detail=f"Failed to update MCP configuration: {str(e)}")
+353
View File
@@ -0,0 +1,353 @@
"""Memory API router for retrieving and managing global memory data."""
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel, Field
from deerflow.agents.memory.updater import (
clear_memory_data,
create_memory_fact,
delete_memory_fact,
get_memory_data,
import_memory_data,
reload_memory_data,
update_memory_fact,
)
from deerflow.config.memory_config import get_memory_config
router = APIRouter(prefix="/api", tags=["memory"])
class ContextSection(BaseModel):
"""Model for context sections (user and history)."""
summary: str = Field(default="", description="Summary content")
updatedAt: str = Field(default="", description="Last update timestamp")
class UserContext(BaseModel):
"""Model for user context."""
workContext: ContextSection = Field(default_factory=ContextSection)
personalContext: ContextSection = Field(default_factory=ContextSection)
topOfMind: ContextSection = Field(default_factory=ContextSection)
class HistoryContext(BaseModel):
"""Model for history context."""
recentMonths: ContextSection = Field(default_factory=ContextSection)
earlierContext: ContextSection = Field(default_factory=ContextSection)
longTermBackground: ContextSection = Field(default_factory=ContextSection)
class Fact(BaseModel):
"""Model for a memory fact."""
id: str = Field(..., description="Unique identifier for the fact")
content: str = Field(..., description="Fact content")
category: str = Field(default="context", description="Fact category")
confidence: float = Field(default=0.5, description="Confidence score (0-1)")
createdAt: str = Field(default="", description="Creation timestamp")
source: str = Field(default="unknown", description="Source thread ID")
sourceError: str | None = Field(default=None, description="Optional description of the prior mistake or wrong approach")
class MemoryResponse(BaseModel):
"""Response model for memory data."""
version: str = Field(default="1.0", description="Memory schema version")
lastUpdated: str = Field(default="", description="Last update timestamp")
user: UserContext = Field(default_factory=UserContext)
history: HistoryContext = Field(default_factory=HistoryContext)
facts: list[Fact] = Field(default_factory=list)
def _map_memory_fact_value_error(exc: ValueError) -> HTTPException:
"""Convert updater validation errors into stable API responses."""
if exc.args and exc.args[0] == "confidence":
detail = "Invalid confidence value; must be between 0 and 1."
else:
detail = "Memory fact content cannot be empty."
return HTTPException(status_code=400, detail=detail)
class FactCreateRequest(BaseModel):
"""Request model for creating a memory fact."""
content: str = Field(..., min_length=1, description="Fact content")
category: str = Field(default="context", description="Fact category")
confidence: float = Field(default=0.5, ge=0.0, le=1.0, description="Confidence score (0-1)")
class FactPatchRequest(BaseModel):
"""PATCH request model that preserves existing values for omitted fields."""
content: str | None = Field(default=None, min_length=1, description="Fact content")
category: str | None = Field(default=None, description="Fact category")
confidence: float | None = Field(default=None, ge=0.0, le=1.0, description="Confidence score (0-1)")
class MemoryConfigResponse(BaseModel):
"""Response model for memory configuration."""
enabled: bool = Field(..., description="Whether memory is enabled")
storage_path: str = Field(..., description="Path to memory storage file")
debounce_seconds: int = Field(..., description="Debounce time for memory updates")
max_facts: int = Field(..., description="Maximum number of facts to store")
fact_confidence_threshold: float = Field(..., description="Minimum confidence threshold for facts")
injection_enabled: bool = Field(..., description="Whether memory injection is enabled")
max_injection_tokens: int = Field(..., description="Maximum tokens for memory injection")
class MemoryStatusResponse(BaseModel):
"""Response model for memory status."""
config: MemoryConfigResponse
data: MemoryResponse
@router.get(
"/memory",
response_model=MemoryResponse,
response_model_exclude_none=True,
summary="Get Memory Data",
description="Retrieve the current global memory data including user context, history, and facts.",
)
async def get_memory() -> MemoryResponse:
"""Get the current global memory data.
Returns:
The current memory data with user context, history, and facts.
Example Response:
```json
{
"version": "1.0",
"lastUpdated": "2024-01-15T10:30:00Z",
"user": {
"workContext": {"summary": "Working on DeerFlow project", "updatedAt": "..."},
"personalContext": {"summary": "Prefers concise responses", "updatedAt": "..."},
"topOfMind": {"summary": "Building memory API", "updatedAt": "..."}
},
"history": {
"recentMonths": {"summary": "Recent development activities", "updatedAt": "..."},
"earlierContext": {"summary": "", "updatedAt": ""},
"longTermBackground": {"summary": "", "updatedAt": ""}
},
"facts": [
{
"id": "fact_abc123",
"content": "User prefers TypeScript over JavaScript",
"category": "preference",
"confidence": 0.9,
"createdAt": "2024-01-15T10:30:00Z",
"source": "thread_xyz"
}
]
}
```
"""
memory_data = get_memory_data()
return MemoryResponse(**memory_data)
@router.post(
"/memory/reload",
response_model=MemoryResponse,
response_model_exclude_none=True,
summary="Reload Memory Data",
description="Reload memory data from the storage file, refreshing the in-memory cache.",
)
async def reload_memory() -> MemoryResponse:
"""Reload memory data from file.
This forces a reload of the memory data from the storage file,
useful when the file has been modified externally.
Returns:
The reloaded memory data.
"""
memory_data = reload_memory_data()
return MemoryResponse(**memory_data)
@router.delete(
"/memory",
response_model=MemoryResponse,
response_model_exclude_none=True,
summary="Clear All Memory Data",
description="Delete all saved memory data and reset the memory structure to an empty state.",
)
async def clear_memory() -> MemoryResponse:
"""Clear all persisted memory data."""
try:
memory_data = clear_memory_data()
except OSError as exc:
raise HTTPException(status_code=500, detail="Failed to clear memory data.") from exc
return MemoryResponse(**memory_data)
@router.post(
"/memory/facts",
response_model=MemoryResponse,
response_model_exclude_none=True,
summary="Create Memory Fact",
description="Create a single saved memory fact manually.",
)
async def create_memory_fact_endpoint(request: FactCreateRequest) -> MemoryResponse:
"""Create a single fact manually."""
try:
memory_data = create_memory_fact(
content=request.content,
category=request.category,
confidence=request.confidence,
)
except ValueError as exc:
raise _map_memory_fact_value_error(exc) from exc
except OSError as exc:
raise HTTPException(status_code=500, detail="Failed to create memory fact.") from exc
return MemoryResponse(**memory_data)
@router.delete(
"/memory/facts/{fact_id}",
response_model=MemoryResponse,
response_model_exclude_none=True,
summary="Delete Memory Fact",
description="Delete a single saved memory fact by its fact id.",
)
async def delete_memory_fact_endpoint(fact_id: str) -> MemoryResponse:
"""Delete a single fact from memory by fact id."""
try:
memory_data = delete_memory_fact(fact_id)
except KeyError as exc:
raise HTTPException(status_code=404, detail=f"Memory fact '{fact_id}' not found.") from exc
except OSError as exc:
raise HTTPException(status_code=500, detail="Failed to delete memory fact.") from exc
return MemoryResponse(**memory_data)
@router.patch(
"/memory/facts/{fact_id}",
response_model=MemoryResponse,
response_model_exclude_none=True,
summary="Patch Memory Fact",
description="Partially update a single saved memory fact by its fact id while preserving omitted fields.",
)
async def update_memory_fact_endpoint(fact_id: str, request: FactPatchRequest) -> MemoryResponse:
"""Partially update a single fact manually."""
try:
memory_data = update_memory_fact(
fact_id=fact_id,
content=request.content,
category=request.category,
confidence=request.confidence,
)
except ValueError as exc:
raise _map_memory_fact_value_error(exc) from exc
except KeyError as exc:
raise HTTPException(status_code=404, detail=f"Memory fact '{fact_id}' not found.") from exc
except OSError as exc:
raise HTTPException(status_code=500, detail="Failed to update memory fact.") from exc
return MemoryResponse(**memory_data)
@router.get(
"/memory/export",
response_model=MemoryResponse,
response_model_exclude_none=True,
summary="Export Memory Data",
description="Export the current global memory data as JSON for backup or transfer.",
)
async def export_memory() -> MemoryResponse:
"""Export the current memory data."""
memory_data = get_memory_data()
return MemoryResponse(**memory_data)
@router.post(
"/memory/import",
response_model=MemoryResponse,
response_model_exclude_none=True,
summary="Import Memory Data",
description="Import and overwrite the current global memory data from a JSON payload.",
)
async def import_memory(request: MemoryResponse) -> MemoryResponse:
"""Import and persist memory data."""
try:
memory_data = import_memory_data(request.model_dump())
except OSError as exc:
raise HTTPException(status_code=500, detail="Failed to import memory data.") from exc
return MemoryResponse(**memory_data)
@router.get(
"/memory/config",
response_model=MemoryConfigResponse,
summary="Get Memory Configuration",
description="Retrieve the current memory system configuration.",
)
async def get_memory_config_endpoint() -> MemoryConfigResponse:
"""Get the memory system configuration.
Returns:
The current memory configuration settings.
Example Response:
```json
{
"enabled": true,
"storage_path": ".deer-flow/memory.json",
"debounce_seconds": 30,
"max_facts": 100,
"fact_confidence_threshold": 0.7,
"injection_enabled": true,
"max_injection_tokens": 2000
}
```
"""
config = get_memory_config()
return MemoryConfigResponse(
enabled=config.enabled,
storage_path=config.storage_path,
debounce_seconds=config.debounce_seconds,
max_facts=config.max_facts,
fact_confidence_threshold=config.fact_confidence_threshold,
injection_enabled=config.injection_enabled,
max_injection_tokens=config.max_injection_tokens,
)
@router.get(
"/memory/status",
response_model=MemoryStatusResponse,
response_model_exclude_none=True,
summary="Get Memory Status",
description="Retrieve both memory configuration and current data in a single request.",
)
async def get_memory_status() -> MemoryStatusResponse:
"""Get the memory system status including configuration and data.
Returns:
Combined memory configuration and current data.
"""
config = get_memory_config()
memory_data = get_memory_data()
return MemoryStatusResponse(
config=MemoryConfigResponse(
enabled=config.enabled,
storage_path=config.storage_path,
debounce_seconds=config.debounce_seconds,
max_facts=config.max_facts,
fact_confidence_threshold=config.fact_confidence_threshold,
injection_enabled=config.injection_enabled,
max_injection_tokens=config.max_injection_tokens,
),
data=MemoryResponse(**memory_data),
)
+116
View File
@@ -0,0 +1,116 @@
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel, Field
from deerflow.config import get_app_config
router = APIRouter(prefix="/api", tags=["models"])
class ModelResponse(BaseModel):
"""Response model for model information."""
name: str = Field(..., description="Unique identifier for the model")
model: str = Field(..., description="Actual provider model identifier")
display_name: str | None = Field(None, description="Human-readable name")
description: str | None = Field(None, description="Model description")
supports_thinking: bool = Field(default=False, description="Whether model supports thinking mode")
supports_reasoning_effort: bool = Field(default=False, description="Whether model supports reasoning effort")
class ModelsListResponse(BaseModel):
"""Response model for listing all models."""
models: list[ModelResponse]
@router.get(
"/models",
response_model=ModelsListResponse,
summary="List All Models",
description="Retrieve a list of all available AI models configured in the system.",
)
async def list_models() -> ModelsListResponse:
"""List all available models from configuration.
Returns model information suitable for frontend display,
excluding sensitive fields like API keys and internal configuration.
Returns:
A list of all configured models with their metadata.
Example Response:
```json
{
"models": [
{
"name": "gpt-4",
"display_name": "GPT-4",
"description": "OpenAI GPT-4 model",
"supports_thinking": false
},
{
"name": "claude-3-opus",
"display_name": "Claude 3 Opus",
"description": "Anthropic Claude 3 Opus model",
"supports_thinking": true
}
]
}
```
"""
config = get_app_config()
models = [
ModelResponse(
name=model.name,
model=model.model,
display_name=model.display_name,
description=model.description,
supports_thinking=model.supports_thinking,
supports_reasoning_effort=model.supports_reasoning_effort,
)
for model in config.models
]
return ModelsListResponse(models=models)
@router.get(
"/models/{model_name}",
response_model=ModelResponse,
summary="Get Model Details",
description="Retrieve detailed information about a specific AI model by its name.",
)
async def get_model(model_name: str) -> ModelResponse:
"""Get a specific model by name.
Args:
model_name: The unique name of the model to retrieve.
Returns:
Model information if found.
Raises:
HTTPException: 404 if model not found.
Example Response:
```json
{
"name": "gpt-4",
"display_name": "GPT-4",
"description": "OpenAI GPT-4 model",
"supports_thinking": false
}
```
"""
config = get_app_config()
model = config.get_model_config(model_name)
if model is None:
raise HTTPException(status_code=404, detail=f"Model '{model_name}' not found")
return ModelResponse(
name=model.name,
model=model.model,
display_name=model.display_name,
description=model.description,
supports_thinking=model.supports_thinking,
supports_reasoning_effort=model.supports_reasoning_effort,
)
+86
View File
@@ -0,0 +1,86 @@
"""Stateless runs endpoints -- stream and wait without a pre-existing thread.
These endpoints auto-create a temporary thread when no ``thread_id`` is
supplied in the request body. When a ``thread_id`` **is** provided, it
is reused so that conversation history is preserved across calls.
"""
from __future__ import annotations
import asyncio
import logging
import uuid
from fastapi import APIRouter, Request
from fastapi.responses import StreamingResponse
from app.gateway.deps import get_checkpointer, get_run_manager, get_stream_bridge
from app.gateway.routers.thread_runs import RunCreateRequest
from app.gateway.services import sse_consumer, start_run
from deerflow.runtime import serialize_channel_values
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/runs", tags=["runs"])
def _resolve_thread_id(body: RunCreateRequest) -> str:
"""Return the thread_id from the request body, or generate a new one."""
thread_id = (body.config or {}).get("configurable", {}).get("thread_id")
if thread_id:
return str(thread_id)
return str(uuid.uuid4())
@router.post("/stream")
async def stateless_stream(body: RunCreateRequest, request: Request) -> StreamingResponse:
"""Create a run and stream events via SSE.
If ``config.configurable.thread_id`` is provided, the run is created
on the given thread so that conversation history is preserved.
Otherwise a new temporary thread is created.
"""
thread_id = _resolve_thread_id(body)
bridge = get_stream_bridge(request)
run_mgr = get_run_manager(request)
record = await start_run(body, thread_id, request)
return StreamingResponse(
sse_consumer(bridge, record, request, run_mgr),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"Connection": "keep-alive",
"X-Accel-Buffering": "no",
},
)
@router.post("/wait", response_model=dict)
async def stateless_wait(body: RunCreateRequest, request: Request) -> dict:
"""Create a run and block until completion.
If ``config.configurable.thread_id`` is provided, the run is created
on the given thread so that conversation history is preserved.
Otherwise a new temporary thread is created.
"""
thread_id = _resolve_thread_id(body)
record = await start_run(body, thread_id, request)
if record.task is not None:
try:
await record.task
except asyncio.CancelledError:
pass
checkpointer = get_checkpointer(request)
config = {"configurable": {"thread_id": thread_id}}
try:
checkpoint_tuple = await checkpointer.aget_tuple(config)
if checkpoint_tuple is not None:
checkpoint = getattr(checkpoint_tuple, "checkpoint", {}) or {}
channel_values = checkpoint.get("channel_values", {})
return serialize_channel_values(channel_values)
except Exception:
logger.exception("Failed to fetch final state for run %s", record.run_id)
return {"status": record.status.value, "error": record.error}
+173
View File
@@ -0,0 +1,173 @@
import json
import logging
from pathlib import Path
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel, Field
from app.gateway.path_utils import resolve_thread_virtual_path
from deerflow.config.extensions_config import ExtensionsConfig, SkillStateConfig, get_extensions_config, reload_extensions_config
from deerflow.skills import Skill, load_skills
from deerflow.skills.installer import SkillAlreadyExistsError, install_skill_from_archive
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api", tags=["skills"])
class SkillResponse(BaseModel):
"""Response model for skill information."""
name: str = Field(..., description="Name of the skill")
description: str = Field(..., description="Description of what the skill does")
license: str | None = Field(None, description="License information")
category: str = Field(..., description="Category of the skill (public or custom)")
enabled: bool = Field(default=True, description="Whether this skill is enabled")
class SkillsListResponse(BaseModel):
"""Response model for listing all skills."""
skills: list[SkillResponse]
class SkillUpdateRequest(BaseModel):
"""Request model for updating a skill."""
enabled: bool = Field(..., description="Whether to enable or disable the skill")
class SkillInstallRequest(BaseModel):
"""Request model for installing a skill from a .skill file."""
thread_id: str = Field(..., description="The thread ID where the .skill file is located")
path: str = Field(..., description="Virtual path to the .skill file (e.g., mnt/user-data/outputs/my-skill.skill)")
class SkillInstallResponse(BaseModel):
"""Response model for skill installation."""
success: bool = Field(..., description="Whether the installation was successful")
skill_name: str = Field(..., description="Name of the installed skill")
message: str = Field(..., description="Installation result message")
def _skill_to_response(skill: Skill) -> SkillResponse:
"""Convert a Skill object to a SkillResponse."""
return SkillResponse(
name=skill.name,
description=skill.description,
license=skill.license,
category=skill.category,
enabled=skill.enabled,
)
@router.get(
"/skills",
response_model=SkillsListResponse,
summary="List All Skills",
description="Retrieve a list of all available skills from both public and custom directories.",
)
async def list_skills() -> SkillsListResponse:
try:
skills = load_skills(enabled_only=False)
return SkillsListResponse(skills=[_skill_to_response(skill) for skill in skills])
except Exception as e:
logger.error(f"Failed to load skills: {e}", exc_info=True)
raise HTTPException(status_code=500, detail=f"Failed to load skills: {str(e)}")
@router.get(
"/skills/{skill_name}",
response_model=SkillResponse,
summary="Get Skill Details",
description="Retrieve detailed information about a specific skill by its name.",
)
async def get_skill(skill_name: str) -> SkillResponse:
try:
skills = load_skills(enabled_only=False)
skill = next((s for s in skills if s.name == skill_name), None)
if skill is None:
raise HTTPException(status_code=404, detail=f"Skill '{skill_name}' not found")
return _skill_to_response(skill)
except HTTPException:
raise
except Exception as e:
logger.error(f"Failed to get skill {skill_name}: {e}", exc_info=True)
raise HTTPException(status_code=500, detail=f"Failed to get skill: {str(e)}")
@router.put(
"/skills/{skill_name}",
response_model=SkillResponse,
summary="Update Skill",
description="Update a skill's enabled status by modifying the extensions_config.json file.",
)
async def update_skill(skill_name: str, request: SkillUpdateRequest) -> SkillResponse:
try:
skills = load_skills(enabled_only=False)
skill = next((s for s in skills if s.name == skill_name), None)
if skill is None:
raise HTTPException(status_code=404, detail=f"Skill '{skill_name}' not found")
config_path = ExtensionsConfig.resolve_config_path()
if config_path is None:
config_path = Path.cwd().parent / "extensions_config.json"
logger.info(f"No existing extensions config found. Creating new config at: {config_path}")
extensions_config = get_extensions_config()
extensions_config.skills[skill_name] = SkillStateConfig(enabled=request.enabled)
config_data = {
"mcpServers": {name: server.model_dump() for name, server in extensions_config.mcp_servers.items()},
"skills": {name: {"enabled": skill_config.enabled} for name, skill_config in extensions_config.skills.items()},
}
with open(config_path, "w", encoding="utf-8") as f:
json.dump(config_data, f, indent=2)
logger.info(f"Skills configuration updated and saved to: {config_path}")
reload_extensions_config()
skills = load_skills(enabled_only=False)
updated_skill = next((s for s in skills if s.name == skill_name), None)
if updated_skill is None:
raise HTTPException(status_code=500, detail=f"Failed to reload skill '{skill_name}' after update")
logger.info(f"Skill '{skill_name}' enabled status updated to {request.enabled}")
return _skill_to_response(updated_skill)
except HTTPException:
raise
except Exception as e:
logger.error(f"Failed to update skill {skill_name}: {e}", exc_info=True)
raise HTTPException(status_code=500, detail=f"Failed to update skill: {str(e)}")
@router.post(
"/skills/install",
response_model=SkillInstallResponse,
summary="Install Skill",
description="Install a skill from a .skill file (ZIP archive) located in the thread's user-data directory.",
)
async def install_skill(request: SkillInstallRequest) -> SkillInstallResponse:
try:
skill_file_path = resolve_thread_virtual_path(request.thread_id, request.path)
result = install_skill_from_archive(skill_file_path)
return SkillInstallResponse(**result)
except FileNotFoundError as e:
raise HTTPException(status_code=404, detail=str(e))
except SkillAlreadyExistsError as e:
raise HTTPException(status_code=409, detail=str(e))
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
except HTTPException:
raise
except Exception as e:
logger.error(f"Failed to install skill: {e}", exc_info=True)
raise HTTPException(status_code=500, detail=f"Failed to install skill: {str(e)}")
+132
View File
@@ -0,0 +1,132 @@
import json
import logging
from fastapi import APIRouter
from langchain_core.messages import HumanMessage, SystemMessage
from pydantic import BaseModel, Field
from deerflow.models import create_chat_model
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api", tags=["suggestions"])
class SuggestionMessage(BaseModel):
role: str = Field(..., description="Message role: user|assistant")
content: str = Field(..., description="Message content as plain text")
class SuggestionsRequest(BaseModel):
messages: list[SuggestionMessage] = Field(..., description="Recent conversation messages")
n: int = Field(default=3, ge=1, le=5, description="Number of suggestions to generate")
model_name: str | None = Field(default=None, description="Optional model override")
class SuggestionsResponse(BaseModel):
suggestions: list[str] = Field(default_factory=list, description="Suggested follow-up questions")
def _strip_markdown_code_fence(text: str) -> str:
stripped = text.strip()
if not stripped.startswith("```"):
return stripped
lines = stripped.splitlines()
if len(lines) >= 3 and lines[0].startswith("```") and lines[-1].startswith("```"):
return "\n".join(lines[1:-1]).strip()
return stripped
def _parse_json_string_list(text: str) -> list[str] | None:
candidate = _strip_markdown_code_fence(text)
start = candidate.find("[")
end = candidate.rfind("]")
if start == -1 or end == -1 or end <= start:
return None
candidate = candidate[start : end + 1]
try:
data = json.loads(candidate)
except Exception:
return None
if not isinstance(data, list):
return None
out: list[str] = []
for item in data:
if not isinstance(item, str):
continue
s = item.strip()
if not s:
continue
out.append(s)
return out
def _extract_response_text(content: object) -> str:
if isinstance(content, str):
return content
if isinstance(content, list):
parts: list[str] = []
for block in content:
if isinstance(block, str):
parts.append(block)
elif isinstance(block, dict) and block.get("type") in {"text", "output_text"}:
text = block.get("text")
if isinstance(text, str):
parts.append(text)
return "\n".join(parts) if parts else ""
if content is None:
return ""
return str(content)
def _format_conversation(messages: list[SuggestionMessage]) -> str:
parts: list[str] = []
for m in messages:
role = m.role.strip().lower()
if role in ("user", "human"):
parts.append(f"User: {m.content.strip()}")
elif role in ("assistant", "ai"):
parts.append(f"Assistant: {m.content.strip()}")
else:
parts.append(f"{m.role}: {m.content.strip()}")
return "\n".join(parts).strip()
@router.post(
"/threads/{thread_id}/suggestions",
response_model=SuggestionsResponse,
summary="Generate Follow-up Questions",
description="Generate short follow-up questions a user might ask next, based on recent conversation context.",
)
async def generate_suggestions(thread_id: str, request: SuggestionsRequest) -> SuggestionsResponse:
if not request.messages:
return SuggestionsResponse(suggestions=[])
n = request.n
conversation = _format_conversation(request.messages)
if not conversation:
return SuggestionsResponse(suggestions=[])
system_instruction = (
"You are generating follow-up questions to help the user continue the conversation.\n"
f"Based on the conversation below, produce EXACTLY {n} short questions the user might ask next.\n"
"Requirements:\n"
"- Questions must be relevant to the preceding conversation.\n"
"- Questions must be written in the same language as the user.\n"
"- Keep each question concise (ideally <= 20 words / <= 40 Chinese characters).\n"
"- Do NOT include numbering, markdown, or any extra text.\n"
"- Output MUST be a JSON array of strings only.\n"
)
user_content = f"Conversation Context:\n{conversation}\n\nGenerate {n} follow-up questions"
try:
model = create_chat_model(name=request.model_name, thinking_enabled=False)
response = await model.ainvoke([SystemMessage(content=system_instruction), HumanMessage(content=user_content)])
raw = _extract_response_text(response.content)
suggestions = _parse_json_string_list(raw) or []
cleaned = [s.replace("\n", " ").strip() for s in suggestions if s.strip()]
cleaned = cleaned[:n]
return SuggestionsResponse(suggestions=cleaned)
except Exception as exc:
logger.exception("Failed to generate suggestions: thread_id=%s err=%s", thread_id, exc)
return SuggestionsResponse(suggestions=[])
+300
View File
@@ -0,0 +1,300 @@
"""Runs endpoints — create, stream, wait, cancel.
Implements the LangGraph Platform runs API on top of
:class:`deerflow.agents.runs.RunManager` and
:class:`deerflow.agents.stream_bridge.StreamBridge`.
SSE format is aligned with the LangGraph Platform protocol so that
the ``useStream`` React hook from ``@langchain/langgraph-sdk/react``
works without modification.
"""
from __future__ import annotations
import asyncio
import logging
from typing import Any, Literal
from fastapi import APIRouter, HTTPException, Query, Request
from fastapi.responses import Response, StreamingResponse
from pydantic import BaseModel, Field
from app.gateway.authz import require_auth, require_permission
from app.gateway.deps import get_checkpointer, get_run_manager, get_stream_bridge
from app.gateway.services import sse_consumer, start_run
from deerflow.runtime import RunRecord, serialize_channel_values
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/threads", tags=["runs"])
# ---------------------------------------------------------------------------
# Request / response models
# ---------------------------------------------------------------------------
class RunCreateRequest(BaseModel):
assistant_id: str | None = Field(default=None, description="Agent / assistant to use")
input: dict[str, Any] | None = Field(default=None, description="Graph input (e.g. {messages: [...]})")
command: dict[str, Any] | None = Field(default=None, description="LangGraph Command")
metadata: dict[str, Any] | None = Field(default=None, description="Run metadata")
config: dict[str, Any] | None = Field(default=None, description="RunnableConfig overrides")
context: dict[str, Any] | None = Field(default=None, description="DeerFlow context overrides (model_name, thinking_enabled, etc.)")
webhook: str | None = Field(default=None, description="Completion callback URL")
checkpoint_id: str | None = Field(default=None, description="Resume from checkpoint")
checkpoint: dict[str, Any] | None = Field(default=None, description="Full checkpoint object")
interrupt_before: list[str] | Literal["*"] | None = Field(default=None, description="Nodes to interrupt before")
interrupt_after: list[str] | Literal["*"] | None = Field(default=None, description="Nodes to interrupt after")
stream_mode: list[str] | str | None = Field(default=None, description="Stream mode(s)")
stream_subgraphs: bool = Field(default=False, description="Include subgraph events")
stream_resumable: bool | None = Field(default=None, description="SSE resumable mode")
on_disconnect: Literal["cancel", "continue"] = Field(default="cancel", description="Behaviour on SSE disconnect")
on_completion: Literal["delete", "keep"] = Field(default="keep", description="Delete temp thread on completion")
multitask_strategy: Literal["reject", "rollback", "interrupt", "enqueue"] = Field(default="reject", description="Concurrency strategy")
after_seconds: float | None = Field(default=None, description="Delayed execution")
if_not_exists: Literal["reject", "create"] = Field(default="create", description="Thread creation policy")
feedback_keys: list[str] | None = Field(default=None, description="LangSmith feedback keys")
class RunResponse(BaseModel):
run_id: str
thread_id: str
assistant_id: str | None = None
status: str
metadata: dict[str, Any] = Field(default_factory=dict)
kwargs: dict[str, Any] = Field(default_factory=dict)
multitask_strategy: str = "reject"
created_at: str = ""
updated_at: str = ""
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _record_to_response(record: RunRecord) -> RunResponse:
return RunResponse(
run_id=record.run_id,
thread_id=record.thread_id,
assistant_id=record.assistant_id,
status=record.status.value,
metadata=record.metadata,
kwargs=record.kwargs,
multitask_strategy=record.multitask_strategy,
created_at=record.created_at,
updated_at=record.updated_at,
)
# ---------------------------------------------------------------------------
# Endpoints
# ---------------------------------------------------------------------------
@router.post("/{thread_id}/runs", response_model=RunResponse)
@require_auth
@require_permission("runs", "create", owner_check=True)
async def create_run(thread_id: str, body: RunCreateRequest, request: Request) -> RunResponse:
"""Create a background run (returns immediately).
Multi-tenant isolation: only the thread owner can create runs.
"""
record = await start_run(body, thread_id, request)
return _record_to_response(record)
@router.post("/{thread_id}/runs/stream")
@require_auth
@require_permission("runs", "create", owner_check=True)
async def stream_run(thread_id: str, body: RunCreateRequest, request: Request) -> StreamingResponse:
"""Create a run and stream events via SSE.
The response includes a ``Content-Location`` header with the run's
resource URL, matching the LangGraph Platform protocol. The
``useStream`` React hook uses this to extract run metadata.
Multi-tenant isolation: only the thread owner can stream runs.
"""
bridge = get_stream_bridge(request)
run_mgr = get_run_manager(request)
record = await start_run(body, thread_id, request)
return StreamingResponse(
sse_consumer(bridge, record, request, run_mgr),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"Connection": "keep-alive",
"X-Accel-Buffering": "no",
# LangGraph Platform includes run metadata in this header.
# The SDK's _get_run_metadata_from_response() parses it.
"Content-Location": (f"/api/threads/{thread_id}/runs/{record.run_id}/stream?thread_id={thread_id}&run_id={record.run_id}"),
},
)
@router.post("/{thread_id}/runs/wait", response_model=dict)
@require_auth
@require_permission("runs", "create", owner_check=True)
async def wait_run(thread_id: str, body: RunCreateRequest, request: Request) -> dict:
"""Create a run and block until it completes, returning the final state.
Multi-tenant isolation: only the thread owner can wait for runs.
"""
record = await start_run(body, thread_id, request)
if record.task is not None:
try:
await record.task
except asyncio.CancelledError:
pass
checkpointer = get_checkpointer(request)
config = {"configurable": {"thread_id": thread_id}}
try:
checkpoint_tuple = await checkpointer.aget_tuple(config)
if checkpoint_tuple is not None:
checkpoint = getattr(checkpoint_tuple, "checkpoint", {}) or {}
channel_values = checkpoint.get("channel_values", {})
return serialize_channel_values(channel_values)
except Exception:
logger.exception("Failed to fetch final state for run %s", record.run_id)
return {"status": record.status.value, "error": record.error}
@router.get("/{thread_id}/runs", response_model=list[RunResponse])
@require_auth
@require_permission("runs", "read", owner_check=True)
async def list_runs(thread_id: str, request: Request) -> list[RunResponse]:
"""List all runs for a thread.
Multi-tenant isolation: only the thread owner can list runs.
"""
run_mgr = get_run_manager(request)
records = await run_mgr.list_by_thread(thread_id)
return [_record_to_response(r) for r in records]
@router.get("/{thread_id}/runs/{run_id}", response_model=RunResponse)
@require_auth
@require_permission("runs", "read", owner_check=True)
async def get_run(thread_id: str, run_id: str, request: Request) -> RunResponse:
"""Get details of a specific run.
Multi-tenant isolation: only the thread owner can get runs.
"""
run_mgr = get_run_manager(request)
record = run_mgr.get(run_id)
if record is None or record.thread_id != thread_id:
raise HTTPException(status_code=404, detail=f"Run {run_id} not found")
return _record_to_response(record)
@router.post("/{thread_id}/runs/{run_id}/cancel")
@require_auth
@require_permission("runs", "cancel", owner_check=True)
async def cancel_run(
thread_id: str,
run_id: str,
request: Request,
wait: bool = Query(default=False, description="Block until run completes after cancel"),
action: Literal["interrupt", "rollback"] = Query(default="interrupt", description="Cancel action"),
) -> Response:
"""Cancel a running or pending run.
- action=interrupt: Stop execution, keep current checkpoint (can be resumed)
- action=rollback: Stop execution, revert to pre-run checkpoint state
- wait=true: Block until the run fully stops, return 204
- wait=false: Return immediately with 202
Multi-tenant isolation: only the thread owner can cancel runs.
"""
run_mgr = get_run_manager(request)
record = run_mgr.get(run_id)
if record is None or record.thread_id != thread_id:
raise HTTPException(status_code=404, detail=f"Run {run_id} not found")
cancelled = await run_mgr.cancel(run_id, action=action)
if not cancelled:
raise HTTPException(
status_code=409,
detail=f"Run {run_id} is not cancellable (status: {record.status.value})",
)
if wait and record.task is not None:
try:
await record.task
except asyncio.CancelledError:
pass
return Response(status_code=204)
return Response(status_code=202)
@router.get("/{thread_id}/runs/{run_id}/join")
@require_auth
@require_permission("runs", "read", owner_check=True)
async def join_run(thread_id: str, run_id: str, request: Request) -> StreamingResponse:
"""Join an existing run's SSE stream.
Multi-tenant isolation: only the thread owner can join runs.
"""
bridge = get_stream_bridge(request)
run_mgr = get_run_manager(request)
record = run_mgr.get(run_id)
if record is None or record.thread_id != thread_id:
raise HTTPException(status_code=404, detail=f"Run {run_id} not found")
return StreamingResponse(
sse_consumer(bridge, record, request, run_mgr),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"Connection": "keep-alive",
"X-Accel-Buffering": "no",
},
)
@router.api_route("/{thread_id}/runs/{run_id}/stream", methods=["GET", "POST"], response_model=None)
async def stream_existing_run(
thread_id: str,
run_id: str,
request: Request,
action: Literal["interrupt", "rollback"] | None = Query(default=None, description="Cancel action"),
wait: int = Query(default=0, description="Block until cancelled (1) or return immediately (0)"),
):
"""Join an existing run's SSE stream (GET), or cancel-then-stream (POST).
The LangGraph SDK's ``joinStream`` and ``useStream`` stop button both use
``POST`` to this endpoint. When ``action=interrupt`` or ``action=rollback``
is present the run is cancelled first; the response then streams any
remaining buffered events so the client observes a clean shutdown.
"""
run_mgr = get_run_manager(request)
record = run_mgr.get(run_id)
if record is None or record.thread_id != thread_id:
raise HTTPException(status_code=404, detail=f"Run {run_id} not found")
# Cancel if an action was requested (stop-button / interrupt flow)
if action is not None:
cancelled = await run_mgr.cancel(run_id, action=action)
if cancelled and wait and record.task is not None:
try:
await record.task
except (asyncio.CancelledError, Exception):
pass
return Response(status_code=204)
bridge = get_stream_bridge(request)
return StreamingResponse(
sse_consumer(bridge, record, request, run_mgr),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"Connection": "keep-alive",
"X-Accel-Buffering": "no",
},
)
+751
View File
@@ -0,0 +1,751 @@
"""Thread CRUD, state, and history endpoints.
Combines the existing thread-local filesystem cleanup with LangGraph
Platform-compatible thread management backed by the checkpointer.
Channel values returned in state responses are serialized through
:func:`deerflow.runtime.serialization.serialize_channel_values` to
ensure LangChain message objects are converted to JSON-safe dicts
matching the LangGraph Platform wire format expected by the
``useStream`` React hook.
"""
from __future__ import annotations
import logging
import re
import time
import uuid
from typing import Annotated, Any
from fastapi import APIRouter, HTTPException, Path, Request
from pydantic import BaseModel, Field, field_validator
from app.gateway.authz import require_auth, require_permission
from app.gateway.deps import get_checkpointer, get_store
from deerflow.config.paths import Paths, get_paths
from deerflow.runtime import serialize_channel_values
# ---------------------------------------------------------------------------
# Thread ID validation (prevents log-injection via control characters)
# ---------------------------------------------------------------------------
_UUID_RE = re.compile(r"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$")
ThreadId = Annotated[str, Path(description="Thread UUID", pattern=_UUID_RE.pattern)]
# ---------------------------------------------------------------------------
# Store namespace
# ---------------------------------------------------------------------------
THREADS_NS: tuple[str, ...] = ("threads",)
"""Namespace used by the Store for thread metadata records."""
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/threads", tags=["threads"])
# ---------------------------------------------------------------------------
# Response / request models
# ---------------------------------------------------------------------------
class ThreadDeleteResponse(BaseModel):
"""Response model for thread cleanup."""
success: bool
message: str
class ThreadResponse(BaseModel):
"""Response model for a single thread."""
thread_id: str = Field(description="Unique thread identifier")
status: str = Field(default="idle", description="Thread status: idle, busy, interrupted, error")
created_at: str = Field(default="", description="ISO timestamp")
updated_at: str = Field(default="", description="ISO timestamp")
metadata: dict[str, Any] = Field(default_factory=dict, description="Thread metadata")
values: dict[str, Any] = Field(default_factory=dict, description="Current state channel values")
interrupts: dict[str, Any] = Field(default_factory=dict, description="Pending interrupts")
class ThreadCreateRequest(BaseModel):
"""Request body for creating a thread."""
thread_id: str | None = Field(default=None, description="Optional thread ID (auto-generated if omitted)")
metadata: dict[str, Any] = Field(default_factory=dict, description="Initial metadata")
@field_validator("thread_id")
@classmethod
def _validate_uuid(cls, v: str | None) -> str | None:
if v is not None and not _UUID_RE.match(v):
raise ValueError("thread_id must be a valid UUID")
return v
class ThreadSearchRequest(BaseModel):
"""Request body for searching threads."""
metadata: dict[str, Any] = Field(default_factory=dict, description="Metadata filter (exact match)")
limit: int = Field(default=100, ge=1, le=1000, description="Maximum results")
offset: int = Field(default=0, ge=0, description="Pagination offset")
status: str | None = Field(default=None, description="Filter by thread status")
class ThreadStateResponse(BaseModel):
"""Response model for thread state."""
values: dict[str, Any] = Field(default_factory=dict, description="Current channel values")
next: list[str] = Field(default_factory=list, description="Next tasks to execute")
metadata: dict[str, Any] = Field(default_factory=dict, description="Checkpoint metadata")
checkpoint: dict[str, Any] = Field(default_factory=dict, description="Checkpoint info")
checkpoint_id: str | None = Field(default=None, description="Current checkpoint ID")
parent_checkpoint_id: str | None = Field(default=None, description="Parent checkpoint ID")
created_at: str | None = Field(default=None, description="Checkpoint timestamp")
tasks: list[dict[str, Any]] = Field(default_factory=list, description="Interrupted task details")
class ThreadPatchRequest(BaseModel):
"""Request body for patching thread metadata."""
metadata: dict[str, Any] = Field(default_factory=dict, description="Metadata to merge")
class ThreadStateUpdateRequest(BaseModel):
"""Request body for updating thread state (human-in-the-loop resume)."""
values: dict[str, Any] | None = Field(default=None, description="Channel values to merge")
checkpoint_id: str | None = Field(default=None, description="Checkpoint to branch from")
checkpoint: dict[str, Any] | None = Field(default=None, description="Full checkpoint object")
as_node: str | None = Field(default=None, description="Node identity for the update")
class HistoryEntry(BaseModel):
"""Single checkpoint history entry."""
checkpoint_id: str
parent_checkpoint_id: str | None = None
metadata: dict[str, Any] = Field(default_factory=dict)
values: dict[str, Any] = Field(default_factory=dict)
created_at: str | None = None
next: list[str] = Field(default_factory=list)
class ThreadHistoryRequest(BaseModel):
"""Request body for checkpoint history."""
limit: int = Field(default=10, ge=1, le=100, description="Maximum entries")
before: str | None = Field(default=None, description="Cursor for pagination")
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _delete_thread_data(thread_id: str, paths: Paths | None = None) -> ThreadDeleteResponse:
"""Delete local persisted filesystem data for a thread."""
path_manager = paths or get_paths()
try:
path_manager.delete_thread_dir(thread_id)
except ValueError as exc:
raise HTTPException(status_code=422, detail=str(exc)) from exc
except FileNotFoundError:
# Not critical — thread data may not exist on disk
logger.debug("No local thread data to delete for %s", thread_id)
return ThreadDeleteResponse(success=True, message=f"No local data for {thread_id}")
except Exception as exc:
logger.exception("Failed to delete thread data for %s", thread_id)
raise HTTPException(status_code=500, detail="Failed to delete local thread data.") from exc
logger.info("Deleted local thread data for %s", thread_id)
return ThreadDeleteResponse(success=True, message=f"Deleted local thread data for {thread_id}")
async def _store_get(store, thread_id: str) -> dict | None:
"""Fetch a thread record from the Store; returns ``None`` if absent."""
item = await store.aget(THREADS_NS, thread_id)
return item.value if item is not None else None
async def _store_put(store, record: dict) -> None:
"""Write a thread record to the Store."""
await store.aput(THREADS_NS, record["thread_id"], record)
async def _store_upsert(store, thread_id: str, *, metadata: dict | None = None, values: dict | None = None) -> None:
"""Create or refresh a thread record in the Store.
On creation the record is written with ``status="idle"``. On update only
``updated_at`` (and optionally ``metadata`` / ``values``) are changed so
that existing fields are preserved.
``values`` carries the agent-state snapshot exposed to the frontend
(currently just ``{"title": "..."}``).
"""
now = time.time()
existing = await _store_get(store, thread_id)
if existing is None:
await _store_put(
store,
{
"thread_id": thread_id,
"status": "idle",
"created_at": now,
"updated_at": now,
"metadata": metadata or {},
"values": values or {},
},
)
else:
val = dict(existing)
val["updated_at"] = now
if metadata:
val.setdefault("metadata", {}).update(metadata)
if values:
val.setdefault("values", {}).update(values)
await _store_put(store, val)
def _derive_thread_status(checkpoint_tuple) -> str:
"""Derive thread status from checkpoint metadata."""
if checkpoint_tuple is None:
return "idle"
pending_writes = getattr(checkpoint_tuple, "pending_writes", None) or []
# Check for error in pending writes
for pw in pending_writes:
if len(pw) >= 2 and pw[1] == "__error__":
return "error"
# Check for pending next tasks (indicates interrupt)
tasks = getattr(checkpoint_tuple, "tasks", None)
if tasks:
return "interrupted"
return "idle"
# ---------------------------------------------------------------------------
# Endpoints
# ---------------------------------------------------------------------------
@router.delete("/{thread_id}", response_model=ThreadDeleteResponse)
@require_auth
@require_permission("threads", "delete", owner_check=True)
async def delete_thread_data(thread_id: ThreadId, request: Request) -> ThreadDeleteResponse:
"""Delete local persisted filesystem data for a thread.
Cleans DeerFlow-managed thread directories, removes checkpoint data,
and removes the thread record from the Store.
Multi-tenant isolation: only the thread owner can delete their thread.
"""
store = get_store(request)
checkpointer = get_checkpointer(request)
# Clean local filesystem
response = _delete_thread_data(thread_id)
# Remove from Store (best-effort)
if store is not None:
try:
await store.adelete(THREADS_NS, thread_id)
except Exception:
logger.debug("Could not delete store record for thread %s (not critical)", thread_id)
# Remove checkpoints (best-effort)
if checkpointer is not None:
try:
if hasattr(checkpointer, "adelete_thread"):
await checkpointer.adelete_thread(thread_id)
except Exception:
logger.debug("Could not delete checkpoints for thread %s (not critical)", thread_id)
return response
@router.post("", response_model=ThreadResponse)
async def create_thread(body: ThreadCreateRequest, request: Request) -> ThreadResponse:
"""Create a new thread.
The thread record is written to the Store (for fast listing) and an
empty checkpoint is written to the checkpointer (for state reads).
Idempotent: returns the existing record when ``thread_id`` already exists.
If authenticated, the user's ID is injected into the thread metadata
for multi-tenant isolation.
"""
store = get_store(request)
checkpointer = get_checkpointer(request)
thread_id = body.thread_id or str(uuid.uuid4())
now = time.time()
from app.gateway.deps import get_optional_user_from_request
user = await get_optional_user_from_request(request)
thread_metadata = dict(body.metadata)
if user:
thread_metadata["user_id"] = str(user.id)
# Idempotency: return existing record from Store when already present
if store is not None:
existing_record = await _store_get(store, thread_id)
if existing_record is not None:
return ThreadResponse(
thread_id=thread_id,
status=existing_record.get("status", "idle"),
created_at=str(existing_record.get("created_at", "")),
updated_at=str(existing_record.get("updated_at", "")),
metadata=existing_record.get("metadata", {}),
)
# Write thread record to Store
if store is not None:
try:
await _store_put(
store,
{
"thread_id": thread_id,
"status": "idle",
"created_at": now,
"updated_at": now,
"metadata": thread_metadata,
},
)
except Exception:
logger.exception("Failed to write thread %s to store", thread_id)
raise HTTPException(status_code=500, detail="Failed to create thread")
# Write an empty checkpoint so state endpoints work immediately
config = {"configurable": {"thread_id": thread_id, "checkpoint_ns": ""}}
try:
from langgraph.checkpoint.base import empty_checkpoint
ckpt_metadata = {
"step": -1,
"source": "input",
"writes": None,
"parents": {},
**thread_metadata,
"created_at": now,
}
await checkpointer.aput(config, empty_checkpoint(), ckpt_metadata, {})
except Exception:
logger.exception("Failed to create checkpoint for thread %s", thread_id)
raise HTTPException(status_code=500, detail="Failed to create thread")
logger.info("Thread created: %s (user_id=%s)", thread_id, thread_metadata.get("user_id"))
return ThreadResponse(
thread_id=thread_id,
status="idle",
created_at=str(now),
updated_at=str(now),
metadata=thread_metadata,
)
@router.post("/search", response_model=list[ThreadResponse])
async def search_threads(body: ThreadSearchRequest, request: Request) -> list[ThreadResponse]:
"""Search and list threads.
Two-phase approach:
**Phase 1 Store (fast path, O(threads))**: returns threads that were
created or run through this Gateway. Store records are tiny metadata
dicts so fetching all of them at once is cheap.
**Phase 2 Checkpointer supplement (lazy migration)**: threads that
were created directly by LangGraph Server (and therefore absent from the
Store) are discovered here by iterating the shared checkpointer. Any
newly found thread is immediately written to the Store so that the next
search skips Phase 2 for that thread the Store converges to a full
index over time without a one-shot migration job.
If authenticated, only threads belonging to the current user are returned
(enforced by user_id metadata filter for multi-tenant isolation).
"""
store = get_store(request)
checkpointer = get_checkpointer(request)
from app.gateway.deps import get_optional_user_from_request
user = await get_optional_user_from_request(request)
user_id = str(user.id) if user else None
# -----------------------------------------------------------------------
# Phase 1: Store
# -----------------------------------------------------------------------
merged: dict[str, ThreadResponse] = {}
if store is not None:
try:
items = await store.asearch(THREADS_NS, limit=10_000)
except Exception:
logger.warning("Store search failed — falling back to checkpointer only", exc_info=True)
items = []
for item in items:
val = item.value
merged[val["thread_id"]] = ThreadResponse(
thread_id=val["thread_id"],
status=val.get("status", "idle"),
created_at=str(val.get("created_at", "")),
updated_at=str(val.get("updated_at", "")),
metadata=val.get("metadata", {}),
values=val.get("values", {}),
)
# -----------------------------------------------------------------------
# Phase 2: Checkpointer supplement
# Discovers threads not yet in the Store (e.g. created by LangGraph
# Server) and lazily migrates them so future searches skip this phase.
# -----------------------------------------------------------------------
try:
async for checkpoint_tuple in checkpointer.alist(None):
cfg = getattr(checkpoint_tuple, "config", {})
thread_id = cfg.get("configurable", {}).get("thread_id")
if not thread_id or thread_id in merged:
continue
# Skip sub-graph checkpoints (checkpoint_ns is non-empty for those)
if cfg.get("configurable", {}).get("checkpoint_ns", ""):
continue
ckpt_meta = getattr(checkpoint_tuple, "metadata", {}) or {}
# Strip LangGraph internal keys from the user-visible metadata dict
user_meta = {k: v for k, v in ckpt_meta.items() if k not in ("created_at", "updated_at", "step", "source", "writes", "parents")}
# Extract state values (title) from the checkpoint's channel_values
checkpoint_data = getattr(checkpoint_tuple, "checkpoint", {}) or {}
channel_values = checkpoint_data.get("channel_values", {})
ckpt_values = {}
if title := channel_values.get("title"):
ckpt_values["title"] = title
thread_resp = ThreadResponse(
thread_id=thread_id,
status=_derive_thread_status(checkpoint_tuple),
created_at=str(ckpt_meta.get("created_at", "")),
updated_at=str(ckpt_meta.get("updated_at", ckpt_meta.get("created_at", ""))),
metadata=user_meta,
values=ckpt_values,
)
merged[thread_id] = thread_resp
# Lazy migration — write to Store so the next search finds it there
if store is not None:
try:
await _store_upsert(store, thread_id, metadata=user_meta, values=ckpt_values or None)
except Exception:
logger.debug("Failed to migrate thread %s to store (non-fatal)", thread_id)
except Exception:
logger.exception("Checkpointer scan failed during thread search")
# Don't raise — return whatever was collected from Store + partial scan
# -----------------------------------------------------------------------
# Phase 3: Filter → sort → paginate
# -----------------------------------------------------------------------
results = list(merged.values())
# Multi-tenant isolation: filter by user_id if authenticated
if user_id:
results = [r for r in results if r.metadata.get("user_id") == user_id]
if body.metadata:
results = [r for r in results if all(r.metadata.get(k) == v for k, v in body.metadata.items())]
if body.status:
results = [r for r in results if r.status == body.status]
results.sort(key=lambda r: r.updated_at, reverse=True)
return results[body.offset : body.offset + body.limit]
@router.patch("/{thread_id}", response_model=ThreadResponse)
@require_auth
@require_permission("threads", "write", owner_check=True, inject_record=True)
async def patch_thread(thread_id: ThreadId, request: Request, body: ThreadPatchRequest, thread_record: dict = None) -> ThreadResponse:
"""Merge metadata into a thread record.
Multi-tenant isolation: only the thread owner can patch their thread.
"""
store = get_store(request)
if store is None:
raise HTTPException(status_code=503, detail="Store not available")
record = thread_record
if record is None:
record = await _store_get(store, thread_id)
if record is None:
raise HTTPException(status_code=404, detail=f"Thread {thread_id} not found")
now = time.time()
updated = dict(record)
updated.setdefault("metadata", {}).update(body.metadata)
updated["updated_at"] = now
try:
await _store_put(store, updated)
except Exception:
logger.exception("Failed to patch thread %s", thread_id)
raise HTTPException(status_code=500, detail="Failed to update thread")
return ThreadResponse(
thread_id=thread_id,
status=updated.get("status", "idle"),
created_at=str(updated.get("created_at", "")),
updated_at=str(now),
metadata=updated.get("metadata", {}),
)
@router.get("/{thread_id}", response_model=ThreadResponse)
@require_auth
@require_permission("threads", "read", owner_check=True)
async def get_thread(thread_id: ThreadId, request: Request) -> ThreadResponse:
"""Get thread info.
Reads metadata from the Store and derives the accurate execution
status from the checkpointer. Falls back to the checkpointer alone
for threads that pre-date Store adoption (backward compat).
Multi-tenant isolation: returns 404 if the thread does not belong to
the authenticated user.
"""
store = get_store(request)
checkpointer = get_checkpointer(request)
record: dict | None = None
if store is not None:
record = await _store_get(store, thread_id)
# Derive accurate status from the checkpointer
config = {"configurable": {"thread_id": thread_id, "checkpoint_ns": ""}}
try:
checkpoint_tuple = await checkpointer.aget_tuple(config)
except Exception:
logger.exception("Failed to get checkpoint for thread %s", thread_id)
raise HTTPException(status_code=500, detail="Failed to get thread")
if record is None and checkpoint_tuple is None:
raise HTTPException(status_code=404, detail=f"Thread {thread_id} not found")
# If the thread exists in the checkpointer but not the store (e.g. legacy
# data), synthesize a minimal store record from the checkpoint metadata.
if record is None and checkpoint_tuple is not None:
ckpt_meta = getattr(checkpoint_tuple, "metadata", {}) or {}
record = {
"thread_id": thread_id,
"status": "idle",
"created_at": ckpt_meta.get("created_at", ""),
"updated_at": ckpt_meta.get("updated_at", ckpt_meta.get("created_at", "")),
"metadata": {k: v for k, v in ckpt_meta.items() if k not in ("created_at", "updated_at", "step", "source", "writes", "parents")},
}
if record is None:
raise HTTPException(status_code=404, detail=f"Thread {thread_id} not found")
status = _derive_thread_status(checkpoint_tuple) if checkpoint_tuple is not None else record.get("status", "idle")
checkpoint = getattr(checkpoint_tuple, "checkpoint", {}) or {} if checkpoint_tuple is not None else {}
channel_values = checkpoint.get("channel_values", {})
return ThreadResponse(
thread_id=thread_id,
status=status,
created_at=str(record.get("created_at", "")),
updated_at=str(record.get("updated_at", "")),
metadata=record.get("metadata", {}),
values=serialize_channel_values(channel_values),
)
@router.get("/{thread_id}/state", response_model=ThreadStateResponse)
@require_auth
@require_permission("threads", "read", owner_check=True)
async def get_thread_state(thread_id: ThreadId, request: Request) -> ThreadStateResponse:
"""Get the latest state snapshot for a thread.
Channel values are serialized to ensure LangChain message objects
are converted to JSON-safe dicts.
Multi-tenant isolation: returns 404 if thread does not belong to user.
"""
checkpointer = get_checkpointer(request)
config = {"configurable": {"thread_id": thread_id, "checkpoint_ns": ""}}
try:
checkpoint_tuple = await checkpointer.aget_tuple(config)
except Exception:
logger.exception("Failed to get state for thread %s", thread_id)
raise HTTPException(status_code=500, detail="Failed to get thread state")
if checkpoint_tuple is None:
raise HTTPException(status_code=404, detail=f"Thread {thread_id} not found")
checkpoint = getattr(checkpoint_tuple, "checkpoint", {}) or {}
metadata = getattr(checkpoint_tuple, "metadata", {}) or {}
checkpoint_id = None
ckpt_config = getattr(checkpoint_tuple, "config", {})
if ckpt_config:
checkpoint_id = ckpt_config.get("configurable", {}).get("checkpoint_id")
channel_values = checkpoint.get("channel_values", {})
parent_config = getattr(checkpoint_tuple, "parent_config", None)
parent_checkpoint_id = None
if parent_config:
parent_checkpoint_id = parent_config.get("configurable", {}).get("checkpoint_id")
tasks_raw = getattr(checkpoint_tuple, "tasks", []) or []
next_tasks = [t.name for t in tasks_raw if hasattr(t, "name")]
tasks = [{"id": getattr(t, "id", ""), "name": getattr(t, "name", "")} for t in tasks_raw]
return ThreadStateResponse(
values=serialize_channel_values(channel_values),
next=next_tasks,
metadata=metadata,
checkpoint={"id": checkpoint_id, "ts": str(metadata.get("created_at", ""))},
checkpoint_id=checkpoint_id,
parent_checkpoint_id=parent_checkpoint_id,
created_at=str(metadata.get("created_at", "")),
tasks=tasks,
)
@router.post("/{thread_id}/state", response_model=ThreadStateResponse)
@require_auth
@require_permission("threads", "write", owner_check=True)
async def update_thread_state(thread_id: ThreadId, body: ThreadStateUpdateRequest, request: Request) -> ThreadStateResponse:
"""Update thread state (e.g. for human-in-the-loop resume or title rename).
Writes a new checkpoint that merges *body.values* into the latest
channel values, then syncs any updated ``title`` field back to the Store
so that ``/threads/search`` reflects the change immediately.
Multi-tenant isolation: only the thread owner can update their thread.
"""
checkpointer = get_checkpointer(request)
store = get_store(request)
# checkpoint_ns must be present in the config for aput — default to ""
# (the root graph namespace). checkpoint_id is optional; omitting it
# fetches the latest checkpoint for the thread.
read_config: dict[str, Any] = {
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": "",
}
}
if body.checkpoint_id:
read_config["configurable"]["checkpoint_id"] = body.checkpoint_id
try:
checkpoint_tuple = await checkpointer.aget_tuple(read_config)
except Exception:
logger.exception("Failed to get state for thread %s", thread_id)
raise HTTPException(status_code=500, detail="Failed to get thread state")
if checkpoint_tuple is None:
raise HTTPException(status_code=404, detail=f"Thread {thread_id} not found")
# Work on mutable copies so we don't accidentally mutate cached objects.
checkpoint: dict[str, Any] = dict(getattr(checkpoint_tuple, "checkpoint", {}) or {})
metadata: dict[str, Any] = dict(getattr(checkpoint_tuple, "metadata", {}) or {})
channel_values: dict[str, Any] = dict(checkpoint.get("channel_values", {}))
if body.values:
channel_values.update(body.values)
checkpoint["channel_values"] = channel_values
metadata["updated_at"] = time.time()
if body.as_node:
metadata["source"] = "update"
metadata["step"] = metadata.get("step", 0) + 1
metadata["writes"] = {body.as_node: body.values}
# aput requires checkpoint_ns in the config — use the same config used for the
# read (which always includes checkpoint_ns=""). Do NOT include checkpoint_id
# so that aput generates a fresh checkpoint ID for the new snapshot.
write_config: dict[str, Any] = {
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": "",
}
}
try:
new_config = await checkpointer.aput(write_config, checkpoint, metadata, {})
except Exception:
logger.exception("Failed to update state for thread %s", thread_id)
raise HTTPException(status_code=500, detail="Failed to update thread state")
new_checkpoint_id: str | None = None
if isinstance(new_config, dict):
new_checkpoint_id = new_config.get("configurable", {}).get("checkpoint_id")
# Sync title changes to the Store so /threads/search reflects them immediately.
if store is not None and body.values and "title" in body.values:
try:
await _store_upsert(store, thread_id, values={"title": body.values["title"]})
except Exception:
logger.debug("Failed to sync title to store for thread %s (non-fatal)", thread_id)
return ThreadStateResponse(
values=serialize_channel_values(channel_values),
next=[],
metadata=metadata,
checkpoint_id=new_checkpoint_id,
created_at=str(metadata.get("created_at", "")),
)
@router.post("/{thread_id}/history", response_model=list[HistoryEntry])
@require_auth
@require_permission("threads", "read", owner_check=True)
async def get_thread_history(thread_id: ThreadId, body: ThreadHistoryRequest, request: Request) -> list[HistoryEntry]:
"""Get checkpoint history for a thread.
Multi-tenant isolation: returns 404 if thread does not belong to user.
"""
checkpointer = get_checkpointer(request)
config: dict[str, Any] = {"configurable": {"thread_id": thread_id}}
if body.before:
config["configurable"]["checkpoint_id"] = body.before
entries: list[HistoryEntry] = []
try:
async for checkpoint_tuple in checkpointer.alist(config, limit=body.limit):
ckpt_config = getattr(checkpoint_tuple, "config", {})
parent_config = getattr(checkpoint_tuple, "parent_config", None)
metadata = getattr(checkpoint_tuple, "metadata", {}) or {}
checkpoint = getattr(checkpoint_tuple, "checkpoint", {}) or {}
checkpoint_id = ckpt_config.get("configurable", {}).get("checkpoint_id", "")
parent_id = None
if parent_config:
parent_id = parent_config.get("configurable", {}).get("checkpoint_id")
channel_values = checkpoint.get("channel_values", {})
# Derive next tasks
tasks_raw = getattr(checkpoint_tuple, "tasks", []) or []
next_tasks = [t.name for t in tasks_raw if hasattr(t, "name")]
entries.append(
HistoryEntry(
checkpoint_id=checkpoint_id,
parent_checkpoint_id=parent_id,
metadata=metadata,
values=serialize_channel_values(channel_values),
created_at=str(metadata.get("created_at", "")),
next=next_tasks,
)
)
except Exception:
logger.exception("Failed to get history for thread %s", thread_id)
raise HTTPException(status_code=500, detail="Failed to get thread history")
return entries
+168
View File
@@ -0,0 +1,168 @@
"""Upload router for handling file uploads."""
import logging
import os
import stat
from fastapi import APIRouter, File, HTTPException, UploadFile
from pydantic import BaseModel
from deerflow.config.paths import get_paths
from deerflow.sandbox.sandbox_provider import get_sandbox_provider
from deerflow.uploads.manager import (
PathTraversalError,
delete_file_safe,
enrich_file_listing,
ensure_uploads_dir,
get_uploads_dir,
list_files_in_dir,
normalize_filename,
upload_artifact_url,
upload_virtual_path,
)
from deerflow.utils.file_conversion import CONVERTIBLE_EXTENSIONS, convert_file_to_markdown
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/threads/{thread_id}/uploads", tags=["uploads"])
class UploadResponse(BaseModel):
"""Response model for file upload."""
success: bool
files: list[dict[str, str]]
message: str
def _make_file_sandbox_writable(file_path: os.PathLike[str] | str) -> None:
"""Ensure uploaded files remain writable when mounted into non-local sandboxes.
In AIO sandbox mode, the gateway writes the authoritative host-side file
first, then the sandbox runtime may rewrite the same mounted path. Granting
world-writable access here prevents permission mismatches between the
gateway user and the sandbox runtime user.
"""
file_stat = os.lstat(file_path)
if stat.S_ISLNK(file_stat.st_mode):
logger.warning("Skipping sandbox chmod for symlinked upload path: %s", file_path)
return
writable_mode = stat.S_IMODE(file_stat.st_mode) | stat.S_IWUSR | stat.S_IWGRP | stat.S_IWOTH
chmod_kwargs = {"follow_symlinks": False} if os.chmod in os.supports_follow_symlinks else {}
os.chmod(file_path, writable_mode, **chmod_kwargs)
@router.post("", response_model=UploadResponse)
async def upload_files(
thread_id: str,
files: list[UploadFile] = File(...),
) -> UploadResponse:
"""Upload multiple files to a thread's uploads directory."""
if not files:
raise HTTPException(status_code=400, detail="No files provided")
try:
uploads_dir = ensure_uploads_dir(thread_id)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
sandbox_uploads = get_paths().sandbox_uploads_dir(thread_id)
uploaded_files = []
sandbox_provider = get_sandbox_provider()
sandbox_id = sandbox_provider.acquire(thread_id)
sandbox = sandbox_provider.get(sandbox_id)
for file in files:
if not file.filename:
continue
try:
safe_filename = normalize_filename(file.filename)
except ValueError:
logger.warning(f"Skipping file with unsafe filename: {file.filename!r}")
continue
try:
content = await file.read()
file_path = uploads_dir / safe_filename
file_path.write_bytes(content)
virtual_path = upload_virtual_path(safe_filename)
if sandbox_id != "local":
_make_file_sandbox_writable(file_path)
sandbox.update_file(virtual_path, content)
file_info = {
"filename": safe_filename,
"size": str(len(content)),
"path": str(sandbox_uploads / safe_filename),
"virtual_path": virtual_path,
"artifact_url": upload_artifact_url(thread_id, safe_filename),
}
logger.info(f"Saved file: {safe_filename} ({len(content)} bytes) to {file_info['path']}")
file_ext = file_path.suffix.lower()
if file_ext in CONVERTIBLE_EXTENSIONS:
md_path = await convert_file_to_markdown(file_path)
if md_path:
md_virtual_path = upload_virtual_path(md_path.name)
if sandbox_id != "local":
_make_file_sandbox_writable(md_path)
sandbox.update_file(md_virtual_path, md_path.read_bytes())
file_info["markdown_file"] = md_path.name
file_info["markdown_path"] = str(sandbox_uploads / md_path.name)
file_info["markdown_virtual_path"] = md_virtual_path
file_info["markdown_artifact_url"] = upload_artifact_url(thread_id, md_path.name)
uploaded_files.append(file_info)
except Exception as e:
logger.error(f"Failed to upload {file.filename}: {e}")
raise HTTPException(status_code=500, detail=f"Failed to upload {file.filename}: {str(e)}")
return UploadResponse(
success=True,
files=uploaded_files,
message=f"Successfully uploaded {len(uploaded_files)} file(s)",
)
@router.get("/list", response_model=dict)
async def list_uploaded_files(thread_id: str) -> dict:
"""List all files in a thread's uploads directory."""
try:
uploads_dir = get_uploads_dir(thread_id)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
result = list_files_in_dir(uploads_dir)
enrich_file_listing(result, thread_id)
# Gateway additionally includes the sandbox-relative path.
sandbox_uploads = get_paths().sandbox_uploads_dir(thread_id)
for f in result["files"]:
f["path"] = str(sandbox_uploads / f["filename"])
return result
@router.delete("/{filename}")
async def delete_uploaded_file(thread_id: str, filename: str) -> dict:
"""Delete a file from a thread's uploads directory."""
try:
uploads_dir = get_uploads_dir(thread_id)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
try:
return delete_file_safe(uploads_dir, filename, convertible_extensions=CONVERTIBLE_EXTENSIONS)
except FileNotFoundError:
raise HTTPException(status_code=404, detail=f"File not found: {filename}")
except PathTraversalError:
raise HTTPException(status_code=400, detail="Invalid path")
except Exception as e:
logger.error(f"Failed to delete {filename}: {e}")
raise HTTPException(status_code=500, detail=f"Failed to delete {filename}: {str(e)}")
+385
View File
@@ -0,0 +1,385 @@
"""Run lifecycle service layer.
Centralizes the business logic for creating runs, formatting SSE
frames, and consuming stream bridge events. Router modules
(``thread_runs``, ``runs``) are thin HTTP handlers that delegate here.
"""
from __future__ import annotations
import asyncio
import json
import logging
import re
import time
from typing import Any
from fastapi import HTTPException, Request
from langchain_core.messages import HumanMessage
from app.gateway.deps import get_checkpointer, get_run_manager, get_store, get_stream_bridge
from deerflow.runtime import (
END_SENTINEL,
HEARTBEAT_SENTINEL,
ConflictError,
DisconnectMode,
RunManager,
RunRecord,
RunStatus,
StreamBridge,
UnsupportedStrategyError,
run_agent,
)
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# SSE formatting
# ---------------------------------------------------------------------------
def format_sse(event: str, data: Any, *, event_id: str | None = None) -> str:
"""Format a single SSE frame.
Field order: ``event:`` -> ``data:`` -> ``id:`` (optional) -> blank line.
This matches the LangGraph Platform wire format consumed by the
``useStream`` React hook and the Python ``langgraph-sdk`` SSE decoder.
"""
payload = json.dumps(data, default=str, ensure_ascii=False)
parts = [f"event: {event}", f"data: {payload}"]
if event_id:
parts.append(f"id: {event_id}")
parts.append("")
parts.append("")
return "\n".join(parts)
# ---------------------------------------------------------------------------
# Input / config helpers
# ---------------------------------------------------------------------------
def normalize_stream_modes(raw: list[str] | str | None) -> list[str]:
"""Normalize the stream_mode parameter to a list.
Default matches what ``useStream`` expects: values + messages-tuple.
"""
if raw is None:
return ["values"]
if isinstance(raw, str):
return [raw]
return raw if raw else ["values"]
def normalize_input(raw_input: dict[str, Any] | None) -> dict[str, Any]:
"""Convert LangGraph Platform input format to LangChain state dict."""
if raw_input is None:
return {}
messages = raw_input.get("messages")
if messages and isinstance(messages, list):
converted = []
for msg in messages:
if isinstance(msg, dict):
role = msg.get("role", msg.get("type", "user"))
content = msg.get("content", "")
if role in ("user", "human"):
converted.append(HumanMessage(content=content))
else:
# TODO: handle other message types (system, ai, tool)
converted.append(HumanMessage(content=content))
else:
converted.append(msg)
return {**raw_input, "messages": converted}
return raw_input
_DEFAULT_ASSISTANT_ID = "lead_agent"
def resolve_agent_factory(assistant_id: str | None):
"""Resolve the agent factory callable from config.
Custom agents are implemented as ``lead_agent`` + an ``agent_name``
injected into ``configurable`` see :func:`build_run_config`. All
``assistant_id`` values therefore map to the same factory; the routing
happens inside ``make_lead_agent`` when it reads ``cfg["agent_name"]``.
"""
from deerflow.agents.lead_agent.agent import make_lead_agent
return make_lead_agent
def build_run_config(
thread_id: str,
request_config: dict[str, Any] | None,
metadata: dict[str, Any] | None,
*,
assistant_id: str | None = None,
user_id: str | None = None,
) -> dict[str, Any]:
"""Build a RunnableConfig dict for the agent.
When *assistant_id* refers to a custom agent (anything other than
``"lead_agent"`` / ``None``), the name is forwarded as
``configurable["agent_name"]``. ``make_lead_agent`` reads this key to
load the matching ``agents/<name>/SOUL.md`` and per-agent config
without it the agent silently runs as the default lead agent.
This mirrors the channel manager's ``_resolve_run_params`` logic so that
the LangGraph Platform-compatible HTTP API and the IM channel path behave
identically.
If *user_id* is provided, it is injected into the config metadata for
multi-tenant isolation.
"""
config: dict[str, Any] = {"recursion_limit": 100}
if request_config:
# LangGraph >= 0.6.0 introduced ``context`` as the preferred way to
# pass thread-level data and rejects requests that include both
# ``configurable`` and ``context``. If the caller already sends
# ``context``, honour it and skip our own ``configurable`` dict.
if "context" in request_config:
if "configurable" in request_config:
logger.warning(
"build_run_config: client sent both 'context' and 'configurable'; preferring 'context' (LangGraph >= 0.6.0). thread_id=%s, caller_configurable keys=%s",
thread_id,
list(request_config.get("configurable", {}).keys()),
)
config["context"] = request_config["context"]
else:
configurable = {"thread_id": thread_id}
configurable.update(request_config.get("configurable", {}))
config["configurable"] = configurable
for k, v in request_config.items():
if k not in ("configurable", "context"):
config[k] = v
else:
config["configurable"] = {"thread_id": thread_id}
# Inject custom agent name when the caller specified a non-default assistant.
# Honour an explicit configurable["agent_name"] in the request if already set.
if assistant_id and assistant_id != _DEFAULT_ASSISTANT_ID and "configurable" in config:
if "agent_name" not in config["configurable"]:
normalized = assistant_id.strip().lower().replace("_", "-")
if not normalized or not re.fullmatch(r"[a-z0-9-]+", normalized):
raise ValueError(f"Invalid assistant_id {assistant_id!r}: must contain only letters, digits, and hyphens after normalization.")
config["configurable"]["agent_name"] = normalized
# Multi-tenant isolation: inject user_id into metadata
if user_id:
config.setdefault("metadata", {})["user_id"] = user_id
if metadata:
config.setdefault("metadata", {}).update(metadata)
return config
# ---------------------------------------------------------------------------
# Run lifecycle
# ---------------------------------------------------------------------------
async def _upsert_thread_in_store(store, thread_id: str, metadata: dict | None) -> None:
"""Create or refresh the thread record in the Store.
Called from :func:`start_run` so that threads created via the stateless
``/runs/stream`` endpoint (which never calls ``POST /threads``) still
appear in ``/threads/search`` results.
"""
# Deferred import to avoid circular import with the threads router module.
from app.gateway.routers.threads import _store_upsert
try:
await _store_upsert(store, thread_id, metadata=metadata)
except Exception:
logger.warning("Failed to upsert thread %s in store (non-fatal)", thread_id)
async def _sync_thread_title_after_run(
run_task: asyncio.Task,
thread_id: str,
checkpointer: Any,
store: Any,
) -> None:
"""Wait for *run_task* to finish, then persist the generated title to the Store.
TitleMiddleware writes the generated title to the LangGraph agent state
(checkpointer) but the Gateway's Store record is not updated automatically.
This coroutine closes that gap by reading the final checkpoint after the
run completes and syncing ``values.title`` into the Store record so that
subsequent ``/threads/search`` responses include the correct title.
Runs as a fire-and-forget :func:`asyncio.create_task`; failures are
logged at DEBUG level and never propagate.
"""
# Wait for the background run task to complete (any outcome).
# asyncio.wait does not propagate task exceptions — it just returns
# when the task is done, cancelled, or failed.
await asyncio.wait({run_task})
# Deferred import to avoid circular import with the threads router module.
from app.gateway.routers.threads import _store_get, _store_put
try:
ckpt_config = {"configurable": {"thread_id": thread_id, "checkpoint_ns": ""}}
ckpt_tuple = await checkpointer.aget_tuple(ckpt_config)
if ckpt_tuple is None:
return
channel_values = ckpt_tuple.checkpoint.get("channel_values", {})
title = channel_values.get("title")
if not title:
return
existing = await _store_get(store, thread_id)
if existing is None:
return
updated = dict(existing)
updated.setdefault("values", {})["title"] = title
updated["updated_at"] = time.time()
await _store_put(store, updated)
logger.debug("Synced title %r for thread %s", title, thread_id)
except Exception:
logger.debug("Failed to sync title for thread %s (non-fatal)", thread_id, exc_info=True)
async def start_run(
body: Any,
thread_id: str,
request: Request,
) -> RunRecord:
"""Create a RunRecord and launch the background agent task.
Parameters
----------
body : RunCreateRequest
The validated request body (typed as Any to avoid circular import
with the router module that defines the Pydantic model).
thread_id : str
Target thread.
request : Request
FastAPI request used to retrieve singletons from ``app.state``.
"""
bridge = get_stream_bridge(request)
run_mgr = get_run_manager(request)
checkpointer = get_checkpointer(request)
store = get_store(request)
disconnect = DisconnectMode.cancel if body.on_disconnect == "cancel" else DisconnectMode.continue_
# Reuse auth context set by @require_auth decorator to avoid redundant DB lookup
auth = getattr(request.state, "auth", None)
user_id = str(auth.user.id) if auth and auth.user else None
try:
record = await run_mgr.create_or_reject(
thread_id,
body.assistant_id,
on_disconnect=disconnect,
metadata=body.metadata or {},
kwargs={"input": body.input, "config": body.config},
multitask_strategy=body.multitask_strategy,
)
except ConflictError as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
except UnsupportedStrategyError as exc:
raise HTTPException(status_code=501, detail=str(exc)) from exc
# Ensure the thread is visible in /threads/search, even for threads that
# were never explicitly created via POST /threads (e.g. stateless runs).
store = get_store(request)
if store is not None:
await _upsert_thread_in_store(store, thread_id, body.metadata)
agent_factory = resolve_agent_factory(body.assistant_id)
graph_input = normalize_input(body.input)
config = build_run_config(
thread_id,
body.config,
body.metadata,
assistant_id=body.assistant_id,
user_id=user_id,
)
# Merge DeerFlow-specific context overrides into configurable.
# The ``context`` field is a custom extension for the langgraph-compat layer
# that carries agent configuration (model_name, thinking_enabled, etc.).
# Only agent-relevant keys are forwarded; unknown keys (e.g. thread_id) are ignored.
context = getattr(body, "context", None)
if context:
_CONTEXT_CONFIGURABLE_KEYS = {
"model_name",
"mode",
"thinking_enabled",
"reasoning_effort",
"is_plan_mode",
"subagent_enabled",
"max_concurrent_subagents",
}
configurable = config.setdefault("configurable", {})
for key in _CONTEXT_CONFIGURABLE_KEYS:
if key in context:
configurable.setdefault(key, context[key])
stream_modes = normalize_stream_modes(body.stream_mode)
task = asyncio.create_task(
run_agent(
bridge,
run_mgr,
record,
checkpointer=checkpointer,
store=store,
agent_factory=agent_factory,
graph_input=graph_input,
config=config,
stream_modes=stream_modes,
stream_subgraphs=body.stream_subgraphs,
interrupt_before=body.interrupt_before,
interrupt_after=body.interrupt_after,
)
)
record.task = task
# After the run completes, sync the title generated by TitleMiddleware from
# the checkpointer into the Store record so that /threads/search returns the
# correct title instead of an empty values dict.
if store is not None:
asyncio.create_task(_sync_thread_title_after_run(task, thread_id, checkpointer, store))
return record
async def sse_consumer(
bridge: StreamBridge,
record: RunRecord,
request: Request,
run_mgr: RunManager,
):
"""Async generator that yields SSE frames from the bridge.
The ``finally`` block implements ``on_disconnect`` semantics:
- ``cancel``: abort the background task on client disconnect.
- ``continue``: let the task run; events are discarded.
"""
try:
async for entry in bridge.subscribe(record.run_id):
if await request.is_disconnected():
break
if entry is HEARTBEAT_SENTINEL:
yield ": heartbeat\n\n"
continue
if entry is END_SENTINEL:
yield format_sse("end", None, event_id=entry.id or None)
return
yield format_sse(entry.event, entry.data, event_id=entry.id or None)
finally:
if record.status in (RunStatus.pending, RunStatus.running):
if record.on_disconnect == DisconnectMode.cancel:
await run_mgr.cancel(record.run_id)
+91
View File
@@ -0,0 +1,91 @@
#!/usr/bin/env python
"""
Debug script for lead_agent.
Run this file directly in VS Code with breakpoints.
Requirements:
Run with `uv run` from the backend/ directory so that the uv workspace
resolves deerflow-harness and app packages correctly:
cd backend && PYTHONPATH=. uv run python debug.py
Usage:
1. Set breakpoints in agent.py or other files
2. Press F5 or use "Run and Debug" panel
3. Input messages in the terminal to interact with the agent
"""
import asyncio
import logging
from dotenv import load_dotenv
from langchain_core.messages import HumanMessage
from deerflow.agents import make_lead_agent
load_dotenv()
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
)
async def main():
# Initialize MCP tools at startup
try:
from deerflow.mcp import initialize_mcp_tools
await initialize_mcp_tools()
except Exception as e:
print(f"Warning: Failed to initialize MCP tools: {e}")
# Create agent with default config
config = {
"configurable": {
"thread_id": "debug-thread-001",
"thinking_enabled": True,
"is_plan_mode": True,
# Uncomment to use a specific model
"model_name": "kimi-k2.5",
}
}
agent = make_lead_agent(config)
print("=" * 50)
print("Lead Agent Debug Mode")
print("Type 'quit' or 'exit' to stop")
print("=" * 50)
while True:
try:
user_input = input("\nYou: ").strip()
if not user_input:
continue
if user_input.lower() in ("quit", "exit"):
print("Goodbye!")
break
# Invoke the agent
state = {"messages": [HumanMessage(content=user_input)]}
result = await agent.ainvoke(state, config=config, context={"thread_id": "debug-thread-001"})
# Print the response
if result.get("messages"):
last_message = result["messages"][-1]
print(f"\nAgent: {last_message.content}")
except KeyboardInterrupt:
print("\nInterrupted. Goodbye!")
break
except Exception as e:
print(f"\nError: {e}")
import traceback
traceback.print_exc()
if __name__ == "__main__":
asyncio.run(main())
+631
View File
@@ -0,0 +1,631 @@
# API Reference
This document provides a complete reference for the DeerFlow backend APIs.
## Overview
DeerFlow backend exposes two sets of APIs:
1. **LangGraph API** - Agent interactions, threads, and streaming (`/api/langgraph/*`)
2. **Gateway API** - Models, MCP, skills, uploads, and artifacts (`/api/*`)
All APIs are accessed through the Nginx reverse proxy at port 2026.
## LangGraph API
Base URL: `/api/langgraph`
The LangGraph API is provided by the LangGraph server and follows the LangGraph SDK conventions.
### Threads
#### Create Thread
```http
POST /api/langgraph/threads
Content-Type: application/json
```
**Request Body:**
```json
{
"metadata": {}
}
```
**Response:**
```json
{
"thread_id": "abc123",
"created_at": "2024-01-15T10:30:00Z",
"metadata": {}
}
```
#### Get Thread State
```http
GET /api/langgraph/threads/{thread_id}/state
```
**Response:**
```json
{
"values": {
"messages": [...],
"sandbox": {...},
"artifacts": [...],
"thread_data": {...},
"title": "Conversation Title"
},
"next": [],
"config": {...}
}
```
### Runs
#### Create Run
Execute the agent with input.
```http
POST /api/langgraph/threads/{thread_id}/runs
Content-Type: application/json
```
**Request Body:**
```json
{
"input": {
"messages": [
{
"role": "user",
"content": "Hello, can you help me?"
}
]
},
"config": {
"configurable": {
"model_name": "gpt-4",
"thinking_enabled": false,
"is_plan_mode": false
}
},
"stream_mode": ["values", "messages-tuple", "custom"]
}
```
**Stream Mode Compatibility:**
- Use: `values`, `messages-tuple`, `custom`, `updates`, `events`, `debug`, `tasks`, `checkpoints`
- Do not use: `tools` (deprecated/invalid in current `langgraph-api` and will trigger schema validation errors)
**Configurable Options:**
- `model_name` (string): Override the default model
- `thinking_enabled` (boolean): Enable extended thinking for supported models
- `is_plan_mode` (boolean): Enable TodoList middleware for task tracking
**Response:** Server-Sent Events (SSE) stream
```
event: values
data: {"messages": [...], "title": "..."}
event: messages
data: {"content": "Hello! I'd be happy to help.", "role": "assistant"}
event: end
data: {}
```
#### Get Run History
```http
GET /api/langgraph/threads/{thread_id}/runs
```
**Response:**
```json
{
"runs": [
{
"run_id": "run123",
"status": "success",
"created_at": "2024-01-15T10:30:00Z"
}
]
}
```
#### Stream Run
Stream responses in real-time.
```http
POST /api/langgraph/threads/{thread_id}/runs/stream
Content-Type: application/json
```
Same request body as Create Run. Returns SSE stream.
---
## Gateway API
Base URL: `/api`
### Models
#### List Models
Get all available LLM models from configuration.
```http
GET /api/models
```
**Response:**
```json
{
"models": [
{
"name": "gpt-4",
"display_name": "GPT-4",
"supports_thinking": false,
"supports_vision": true
},
{
"name": "claude-3-opus",
"display_name": "Claude 3 Opus",
"supports_thinking": false,
"supports_vision": true
},
{
"name": "deepseek-v3",
"display_name": "DeepSeek V3",
"supports_thinking": true,
"supports_vision": false
}
]
}
```
#### Get Model Details
```http
GET /api/models/{model_name}
```
**Response:**
```json
{
"name": "gpt-4",
"display_name": "GPT-4",
"model": "gpt-4",
"max_tokens": 4096,
"supports_thinking": false,
"supports_vision": true
}
```
### MCP Configuration
#### Get MCP Config
Get current MCP server configurations.
```http
GET /api/mcp/config
```
**Response:**
```json
{
"mcpServers": {
"github": {
"enabled": true,
"type": "stdio",
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-github"],
"env": {
"GITHUB_TOKEN": "***"
},
"description": "GitHub operations"
},
"filesystem": {
"enabled": false,
"type": "stdio",
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-filesystem"],
"description": "File system access"
}
}
}
```
#### Update MCP Config
Update MCP server configurations.
```http
PUT /api/mcp/config
Content-Type: application/json
```
**Request Body:**
```json
{
"mcpServers": {
"github": {
"enabled": true,
"type": "stdio",
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-github"],
"env": {
"GITHUB_TOKEN": "$GITHUB_TOKEN"
},
"description": "GitHub operations"
}
}
}
```
**Response:**
```json
{
"success": true,
"message": "MCP configuration updated"
}
```
### Skills
#### List Skills
Get all available skills.
```http
GET /api/skills
```
**Response:**
```json
{
"skills": [
{
"name": "pdf-processing",
"display_name": "PDF Processing",
"description": "Handle PDF documents efficiently",
"enabled": true,
"license": "MIT",
"path": "public/pdf-processing"
},
{
"name": "frontend-design",
"display_name": "Frontend Design",
"description": "Design and build frontend interfaces",
"enabled": false,
"license": "MIT",
"path": "public/frontend-design"
}
]
}
```
#### Get Skill Details
```http
GET /api/skills/{skill_name}
```
**Response:**
```json
{
"name": "pdf-processing",
"display_name": "PDF Processing",
"description": "Handle PDF documents efficiently",
"enabled": true,
"license": "MIT",
"path": "public/pdf-processing",
"allowed_tools": ["read_file", "write_file", "bash"],
"content": "# PDF Processing\n\nInstructions for the agent..."
}
```
#### Enable Skill
```http
POST /api/skills/{skill_name}/enable
```
**Response:**
```json
{
"success": true,
"message": "Skill 'pdf-processing' enabled"
}
```
#### Disable Skill
```http
POST /api/skills/{skill_name}/disable
```
**Response:**
```json
{
"success": true,
"message": "Skill 'pdf-processing' disabled"
}
```
#### Install Skill
Install a skill from a `.skill` file.
```http
POST /api/skills/install
Content-Type: multipart/form-data
```
**Request Body:**
- `file`: The `.skill` file to install
**Response:**
```json
{
"success": true,
"message": "Skill 'my-skill' installed successfully",
"skill": {
"name": "my-skill",
"display_name": "My Skill",
"path": "custom/my-skill"
}
}
```
### File Uploads
#### Upload Files
Upload one or more files to a thread.
```http
POST /api/threads/{thread_id}/uploads
Content-Type: multipart/form-data
```
**Request Body:**
- `files`: One or more files to upload
**Response:**
```json
{
"success": true,
"files": [
{
"filename": "document.pdf",
"size": 1234567,
"path": ".deer-flow/threads/abc123/user-data/uploads/document.pdf",
"virtual_path": "/mnt/user-data/uploads/document.pdf",
"artifact_url": "/api/threads/abc123/artifacts/mnt/user-data/uploads/document.pdf",
"markdown_file": "document.md",
"markdown_path": ".deer-flow/threads/abc123/user-data/uploads/document.md",
"markdown_virtual_path": "/mnt/user-data/uploads/document.md",
"markdown_artifact_url": "/api/threads/abc123/artifacts/mnt/user-data/uploads/document.md"
}
],
"message": "Successfully uploaded 1 file(s)"
}
```
**Supported Document Formats** (auto-converted to Markdown):
- PDF (`.pdf`)
- PowerPoint (`.ppt`, `.pptx`)
- Excel (`.xls`, `.xlsx`)
- Word (`.doc`, `.docx`)
#### List Uploaded Files
```http
GET /api/threads/{thread_id}/uploads/list
```
**Response:**
```json
{
"files": [
{
"filename": "document.pdf",
"size": 1234567,
"path": ".deer-flow/threads/abc123/user-data/uploads/document.pdf",
"virtual_path": "/mnt/user-data/uploads/document.pdf",
"artifact_url": "/api/threads/abc123/artifacts/mnt/user-data/uploads/document.pdf",
"extension": ".pdf",
"modified": 1705997600.0
}
],
"count": 1
}
```
#### Delete File
```http
DELETE /api/threads/{thread_id}/uploads/{filename}
```
**Response:**
```json
{
"success": true,
"message": "Deleted document.pdf"
}
```
### Thread Cleanup
Remove DeerFlow-managed local thread files under `.deer-flow/threads/{thread_id}` after the LangGraph thread itself has been deleted.
```http
DELETE /api/threads/{thread_id}
```
**Response:**
```json
{
"success": true,
"message": "Deleted local thread data for abc123"
}
```
**Error behavior:**
- `422` for invalid thread IDs
- `500` returns a generic `{"detail": "Failed to delete local thread data."}` response while full exception details stay in server logs
### Artifacts
#### Get Artifact
Download or view an artifact generated by the agent.
```http
GET /api/threads/{thread_id}/artifacts/{path}
```
**Path Examples:**
- `/api/threads/abc123/artifacts/mnt/user-data/outputs/result.txt`
- `/api/threads/abc123/artifacts/mnt/user-data/uploads/document.pdf`
**Query Parameters:**
- `download` (boolean): If `true`, force download with Content-Disposition header
**Response:** File content with appropriate Content-Type
---
## Error Responses
All APIs return errors in a consistent format:
```json
{
"detail": "Error message describing what went wrong"
}
```
**HTTP Status Codes:**
- `400` - Bad Request: Invalid input
- `404` - Not Found: Resource not found
- `422` - Validation Error: Request validation failed
- `500` - Internal Server Error: Server-side error
---
## Authentication
Currently, DeerFlow does not implement authentication. All APIs are accessible without credentials.
Note: This is about DeerFlow API authentication. MCP outbound connections can still use OAuth for configured HTTP/SSE MCP servers.
For production deployments, it is recommended to:
1. Use Nginx for basic auth or OAuth integration
2. Deploy behind a VPN or private network
3. Implement custom authentication middleware
---
## Rate Limiting
No rate limiting is implemented by default. For production deployments, configure rate limiting in Nginx:
```nginx
limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;
location /api/ {
limit_req zone=api burst=20 nodelay;
proxy_pass http://backend;
}
```
---
## WebSocket Support
The LangGraph server supports WebSocket connections for real-time streaming. Connect to:
```
ws://localhost:2026/api/langgraph/threads/{thread_id}/runs/stream
```
---
## SDK Usage
### Python (LangGraph SDK)
```python
from langgraph_sdk import get_client
client = get_client(url="http://localhost:2026/api/langgraph")
# Create thread
thread = await client.threads.create()
# Run agent
async for event in client.runs.stream(
thread["thread_id"],
"lead_agent",
input={"messages": [{"role": "user", "content": "Hello"}]},
config={"configurable": {"model_name": "gpt-4"}},
stream_mode=["values", "messages-tuple", "custom"],
):
print(event)
```
### JavaScript/TypeScript
```typescript
// Using fetch for Gateway API
const response = await fetch('/api/models');
const data = await response.json();
console.log(data.models);
// Using EventSource for streaming
const eventSource = new EventSource(
`/api/langgraph/threads/${threadId}/runs/stream`
);
eventSource.onmessage = (event) => {
console.log(JSON.parse(event.data));
};
```
### cURL Examples
```bash
# List models
curl http://localhost:2026/api/models
# Get MCP config
curl http://localhost:2026/api/mcp/config
# Upload file
curl -X POST http://localhost:2026/api/threads/abc123/uploads \
-F "files=@document.pdf"
# Enable skill
curl -X POST http://localhost:2026/api/skills/pdf-processing/enable
# Create thread and run agent
curl -X POST http://localhost:2026/api/langgraph/threads \
-H "Content-Type: application/json" \
-d '{}'
curl -X POST http://localhost:2026/api/langgraph/threads/abc123/runs \
-H "Content-Type: application/json" \
-d '{
"input": {"messages": [{"role": "user", "content": "Hello"}]},
"config": {"configurable": {"model_name": "gpt-4"}}
}'
```
+238
View File
@@ -0,0 +1,238 @@
# Apple Container Support
DeerFlow now supports Apple Container as the preferred container runtime on macOS, with automatic fallback to Docker.
## Overview
Starting with this version, DeerFlow automatically detects and uses Apple Container on macOS when available, falling back to Docker when:
- Apple Container is not installed
- Running on non-macOS platforms
This provides better performance on Apple Silicon Macs while maintaining compatibility across all platforms.
## Benefits
### On Apple Silicon Macs with Apple Container:
- **Better Performance**: Native ARM64 execution without Rosetta 2 translation
- **Lower Resource Usage**: Lighter weight than Docker Desktop
- **Native Integration**: Uses macOS Virtualization.framework
### Fallback to Docker:
- Full backward compatibility
- Works on all platforms (macOS, Linux, Windows)
- No configuration changes needed
## Requirements
### For Apple Container (macOS only):
- macOS 15.0 or later
- Apple Silicon (M1/M2/M3/M4)
- Apple Container CLI installed
### Installation:
```bash
# Download from GitHub releases
# https://github.com/apple/container/releases
# Verify installation
container --version
# Start the service
container system start
```
### For Docker (all platforms):
- Docker Desktop or Docker Engine
## How It Works
### Automatic Detection
The `AioSandboxProvider` automatically detects the available container runtime:
1. On macOS: Try `container --version`
- Success → Use Apple Container
- Failure → Fall back to Docker
2. On other platforms: Use Docker directly
### Runtime Differences
Both runtimes use nearly identical command syntax:
**Container Startup:**
```bash
# Apple Container
container run --rm -d -p 8080:8080 -v /host:/container -e KEY=value image
# Docker
docker run --rm -d -p 8080:8080 -v /host:/container -e KEY=value image
```
**Container Cleanup:**
```bash
# Apple Container (with --rm flag)
container stop <id> # Auto-removes due to --rm
# Docker (with --rm flag)
docker stop <id> # Auto-removes due to --rm
```
### Implementation Details
The implementation is in `backend/packages/harness/deerflow/community/aio_sandbox/aio_sandbox_provider.py`:
- `_detect_container_runtime()`: Detects available runtime at startup
- `_start_container()`: Uses detected runtime, skips Docker-specific options for Apple Container
- `_stop_container()`: Uses appropriate stop command for the runtime
## Configuration
No configuration changes are needed! The system works automatically.
However, you can verify the runtime in use by checking the logs:
```
INFO:deerflow.community.aio_sandbox.aio_sandbox_provider:Detected Apple Container: container version 0.1.0
INFO:deerflow.community.aio_sandbox.aio_sandbox_provider:Starting sandbox container using container: ...
```
Or for Docker:
```
INFO:deerflow.community.aio_sandbox.aio_sandbox_provider:Apple Container not available, falling back to Docker
INFO:deerflow.community.aio_sandbox.aio_sandbox_provider:Starting sandbox container using docker: ...
```
## Container Images
Both runtimes use OCI-compatible images. The default image works with both:
```yaml
sandbox:
use: deerflow.community.aio_sandbox:AioSandboxProvider
image: enterprise-public-cn-beijing.cr.volces.com/vefaas-public/all-in-one-sandbox:latest # Default image
```
Make sure your images are available for the appropriate architecture:
- ARM64 for Apple Container on Apple Silicon
- AMD64 for Docker on Intel Macs
- Multi-arch images work on both
### Pre-pulling Images (Recommended)
**Important**: Container images are typically large (500MB+) and are pulled on first use, which can cause a long wait time without clear feedback.
**Best Practice**: Pre-pull the image during setup:
```bash
# From project root
make setup-sandbox
```
This command will:
1. Read the configured image from `config.yaml` (or use default)
2. Detect available runtime (Apple Container or Docker)
3. Pull the image with progress indication
4. Verify the image is ready for use
**Manual pre-pull**:
```bash
# Using Apple Container
container image pull enterprise-public-cn-beijing.cr.volces.com/vefaas-public/all-in-one-sandbox:latest
# Using Docker
docker pull enterprise-public-cn-beijing.cr.volces.com/vefaas-public/all-in-one-sandbox:latest
```
If you skip pre-pulling, the image will be automatically pulled on first agent execution, which may take several minutes depending on your network speed.
## Cleanup Scripts
The project includes a unified cleanup script that handles both runtimes:
**Script:** `scripts/cleanup-containers.sh`
**Usage:**
```bash
# Clean up all DeerFlow sandbox containers
./scripts/cleanup-containers.sh deer-flow-sandbox
# Custom prefix
./scripts/cleanup-containers.sh my-prefix
```
**Makefile Integration:**
All cleanup commands in `Makefile` automatically handle both runtimes:
```bash
make stop # Stops all services and cleans up containers
make clean # Full cleanup including logs
```
## Testing
Test the container runtime detection:
```bash
cd backend
python test_container_runtime.py
```
This will:
1. Detect the available runtime
2. Optionally start a test container
3. Verify connectivity
4. Clean up
## Troubleshooting
### Apple Container not detected on macOS
1. Check if installed:
```bash
which container
container --version
```
2. Check if service is running:
```bash
container system start
```
3. Check logs for detection:
```bash
# Look for detection message in application logs
grep "container runtime" logs/*.log
```
### Containers not cleaning up
1. Manually check running containers:
```bash
# Apple Container
container list
# Docker
docker ps
```
2. Run cleanup script manually:
```bash
./scripts/cleanup-containers.sh deer-flow-sandbox
```
### Performance issues
- Apple Container should be faster on Apple Silicon
- If experiencing issues, you can force Docker by temporarily renaming the `container` command:
```bash
# Temporary workaround - not recommended for permanent use
sudo mv /opt/homebrew/bin/container /opt/homebrew/bin/container.bak
```
## References
- [Apple Container GitHub](https://github.com/apple/container)
- [Apple Container Documentation](https://github.com/apple/container/blob/main/docs/)
- [OCI Image Spec](https://github.com/opencontainers/image-spec)
+484
View File
@@ -0,0 +1,484 @@
# Architecture Overview
This document provides a comprehensive overview of the DeerFlow backend architecture.
## System Architecture
```
┌──────────────────────────────────────────────────────────────────────────┐
│ Client (Browser) │
└─────────────────────────────────┬────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────────────┐
│ Nginx (Port 2026) │
│ Unified Reverse Proxy Entry Point │
│ ┌────────────────────────────────────────────────────────────────────┐ │
│ │ /api/langgraph/* → LangGraph Server (2024) │ │
│ │ /api/* → Gateway API (8001) │ │
│ │ /* → Frontend (3000) │ │
│ └────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────┬────────────────────────────────────────┘
┌───────────────────────┼───────────────────────┐
│ │ │
▼ ▼ ▼
┌─────────────────────┐ ┌─────────────────────┐ ┌─────────────────────┐
│ LangGraph Server │ │ Gateway API │ │ Frontend │
│ (Port 2024) │ │ (Port 8001) │ │ (Port 3000) │
│ │ │ │ │ │
│ - Agent Runtime │ │ - Models API │ │ - Next.js App │
│ - Thread Mgmt │ │ - MCP Config │ │ - React UI │
│ - SSE Streaming │ │ - Skills Mgmt │ │ - Chat Interface │
│ - Checkpointing │ │ - File Uploads │ │ │
│ │ │ - Thread Cleanup │ │ │
│ │ │ - Artifacts │ │ │
└─────────────────────┘ └─────────────────────┘ └─────────────────────┘
│ │
│ ┌─────────────────┘
│ │
▼ ▼
┌──────────────────────────────────────────────────────────────────────────┐
│ Shared Configuration │
│ ┌─────────────────────────┐ ┌────────────────────────────────────────┐ │
│ │ config.yaml │ │ extensions_config.json │ │
│ │ - Models │ │ - MCP Servers │ │
│ │ - Tools │ │ - Skills State │ │
│ │ - Sandbox │ │ │ │
│ │ - Summarization │ │ │ │
│ └─────────────────────────┘ └────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
```
## Component Details
### LangGraph Server
The LangGraph server is the core agent runtime, built on LangGraph for robust multi-agent workflow orchestration.
**Entry Point**: `packages/harness/deerflow/agents/lead_agent/agent.py:make_lead_agent`
**Key Responsibilities**:
- Agent creation and configuration
- Thread state management
- Middleware chain execution
- Tool execution orchestration
- SSE streaming for real-time responses
**Configuration**: `langgraph.json`
```json
{
"agent": {
"type": "agent",
"path": "deerflow.agents:make_lead_agent"
}
}
```
### Gateway API
FastAPI application providing REST endpoints for non-agent operations.
**Entry Point**: `app/gateway/app.py`
**Routers**:
- `models.py` - `/api/models` - Model listing and details
- `mcp.py` - `/api/mcp` - MCP server configuration
- `skills.py` - `/api/skills` - Skills management
- `uploads.py` - `/api/threads/{id}/uploads` - File upload
- `threads.py` - `/api/threads/{id}` - Local DeerFlow thread data cleanup after LangGraph deletion
- `artifacts.py` - `/api/threads/{id}/artifacts` - Artifact serving
- `suggestions.py` - `/api/threads/{id}/suggestions` - Follow-up suggestion generation
The web conversation delete flow is now split across both backend surfaces: LangGraph handles `DELETE /api/langgraph/threads/{thread_id}` for thread state, then the Gateway `threads.py` router removes DeerFlow-managed filesystem data via `Paths.delete_thread_dir()`.
### Agent Architecture
```
┌─────────────────────────────────────────────────────────────────────────┐
│ make_lead_agent(config) │
└────────────────────────────────────┬────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────┐
│ Middleware Chain │
│ ┌──────────────────────────────────────────────────────────────────┐ │
│ │ 1. ThreadDataMiddleware - Initialize workspace/uploads/outputs │ │
│ │ 2. UploadsMiddleware - Process uploaded files │ │
│ │ 3. SandboxMiddleware - Acquire sandbox environment │ │
│ │ 4. SummarizationMiddleware - Context reduction (if enabled) │ │
│ │ 5. TitleMiddleware - Auto-generate titles │ │
│ │ 6. TodoListMiddleware - Task tracking (if plan_mode) │ │
│ │ 7. ViewImageMiddleware - Vision model support │ │
│ │ 8. ClarificationMiddleware - Handle clarifications │ │
│ └──────────────────────────────────────────────────────────────────┘ │
└────────────────────────────────────┬────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────┐
│ Agent Core │
│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────────┐ │
│ │ Model │ │ Tools │ │ System Prompt │ │
│ │ (from factory) │ │ (configured + │ │ (with skills) │ │
│ │ │ │ MCP + builtin) │ │ │ │
│ └──────────────────┘ └──────────────────┘ └──────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────┘
```
### Thread State
The `ThreadState` extends LangGraph's `AgentState` with additional fields:
```python
class ThreadState(AgentState):
# Core state from AgentState
messages: list[BaseMessage]
# DeerFlow extensions
sandbox: dict # Sandbox environment info
artifacts: list[str] # Generated file paths
thread_data: dict # {workspace, uploads, outputs} paths
title: str | None # Auto-generated conversation title
todos: list[dict] # Task tracking (plan mode)
viewed_images: dict # Vision model image data
```
### Sandbox System
```
┌─────────────────────────────────────────────────────────────────────────┐
│ Sandbox Architecture │
└─────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────┐
│ SandboxProvider │ (Abstract)
│ - acquire() │
│ - get() │
│ - release() │
└────────────┬────────────┘
┌────────────────────┼────────────────────┐
│ │
▼ ▼
┌─────────────────────────┐ ┌─────────────────────────┐
│ LocalSandboxProvider │ │ AioSandboxProvider │
│ (packages/harness/deerflow/sandbox/local.py) │ │ (packages/harness/deerflow/community/) │
│ │ │ │
│ - Singleton instance │ │ - Docker-based │
│ - Direct execution │ │ - Isolated containers │
│ - Development use │ │ - Production use │
└─────────────────────────┘ └─────────────────────────┘
┌─────────────────────────┐
│ Sandbox │ (Abstract)
│ - execute_command() │
│ - read_file() │
│ - write_file() │
│ - list_dir() │
└─────────────────────────┘
```
**Virtual Path Mapping**:
| Virtual Path | Physical Path |
|-------------|---------------|
| `/mnt/user-data/workspace` | `backend/.deer-flow/threads/{thread_id}/user-data/workspace` |
| `/mnt/user-data/uploads` | `backend/.deer-flow/threads/{thread_id}/user-data/uploads` |
| `/mnt/user-data/outputs` | `backend/.deer-flow/threads/{thread_id}/user-data/outputs` |
| `/mnt/skills` | `deer-flow/skills/` |
### Tool System
```
┌─────────────────────────────────────────────────────────────────────────┐
│ Tool Sources │
└─────────────────────────────────────────────────────────────────────────┘
┌─────────────────────┐ ┌─────────────────────┐ ┌─────────────────────┐
│ Built-in Tools │ │ Configured Tools │ │ MCP Tools │
│ (packages/harness/deerflow/tools/) │ │ (config.yaml) │ │ (extensions.json) │
├─────────────────────┤ ├─────────────────────┤ ├─────────────────────┤
│ - present_file │ │ - web_search │ │ - github │
│ - ask_clarification │ │ - web_fetch │ │ - filesystem │
│ - view_image │ │ - bash │ │ - postgres │
│ │ │ - read_file │ │ - brave-search │
│ │ │ - write_file │ │ - puppeteer │
│ │ │ - str_replace │ │ - ... │
│ │ │ - ls │ │ │
└─────────────────────┘ └─────────────────────┘ └─────────────────────┘
│ │ │
└───────────────────────┴───────────────────────┘
┌─────────────────────────┐
│ get_available_tools() │
│ (packages/harness/deerflow/tools/__init__) │
└─────────────────────────┘
```
### Model Factory
```
┌─────────────────────────────────────────────────────────────────────────┐
│ Model Factory │
│ (packages/harness/deerflow/models/factory.py) │
└─────────────────────────────────────────────────────────────────────────┘
config.yaml:
┌─────────────────────────────────────────────────────────────────────────┐
│ models: │
│ - name: gpt-4 │
│ display_name: GPT-4 │
│ use: langchain_openai:ChatOpenAI │
│ model: gpt-4 │
│ api_key: $OPENAI_API_KEY │
│ max_tokens: 4096 │
│ supports_thinking: false │
│ supports_vision: true │
└─────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────┐
│ create_chat_model() │
│ - name: str │
│ - thinking_enabled │
└────────────┬────────────┘
┌─────────────────────────┐
│ resolve_class() │
│ (reflection system) │
└────────────┬────────────┘
┌─────────────────────────┐
│ BaseChatModel │
│ (LangChain instance) │
└─────────────────────────┘
```
**Supported Providers**:
- OpenAI (`langchain_openai:ChatOpenAI`)
- Anthropic (`langchain_anthropic:ChatAnthropic`)
- DeepSeek (`langchain_deepseek:ChatDeepSeek`)
- Custom via LangChain integrations
### MCP Integration
```
┌─────────────────────────────────────────────────────────────────────────┐
│ MCP Integration │
│ (packages/harness/deerflow/mcp/manager.py) │
└─────────────────────────────────────────────────────────────────────────┘
extensions_config.json:
┌─────────────────────────────────────────────────────────────────────────┐
│ { │
│ "mcpServers": { │
│ "github": { │
│ "enabled": true, │
│ "type": "stdio", │
│ "command": "npx", │
│ "args": ["-y", "@modelcontextprotocol/server-github"], │
│ "env": {"GITHUB_TOKEN": "$GITHUB_TOKEN"} │
│ } │
│ } │
│ } │
└─────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────┐
│ MultiServerMCPClient │
│ (langchain-mcp-adapters)│
└────────────┬────────────┘
┌────────────────────┼────────────────────┐
│ │ │
▼ ▼ ▼
┌───────────┐ ┌───────────┐ ┌───────────┐
│ stdio │ │ SSE │ │ HTTP │
│ transport │ │ transport │ │ transport │
└───────────┘ └───────────┘ └───────────┘
```
### Skills System
```
┌─────────────────────────────────────────────────────────────────────────┐
│ Skills System │
│ (packages/harness/deerflow/skills/loader.py) │
└─────────────────────────────────────────────────────────────────────────┘
Directory Structure:
┌─────────────────────────────────────────────────────────────────────────┐
│ skills/ │
│ ├── public/ # Public skills (committed) │
│ │ ├── pdf-processing/ │
│ │ │ └── SKILL.md │
│ │ ├── frontend-design/ │
│ │ │ └── SKILL.md │
│ │ └── ... │
│ └── custom/ # Custom skills (gitignored) │
│ └── user-installed/ │
│ └── SKILL.md │
└─────────────────────────────────────────────────────────────────────────┘
SKILL.md Format:
┌─────────────────────────────────────────────────────────────────────────┐
│ --- │
│ name: PDF Processing │
│ description: Handle PDF documents efficiently │
│ license: MIT │
│ allowed-tools: │
│ - read_file │
│ - write_file │
│ - bash │
│ --- │
│ │
│ # Skill Instructions │
│ Content injected into system prompt... │
└─────────────────────────────────────────────────────────────────────────┘
```
### Request Flow
```
┌─────────────────────────────────────────────────────────────────────────┐
│ Request Flow Example │
│ User sends message to agent │
└─────────────────────────────────────────────────────────────────────────┘
1. Client → Nginx
POST /api/langgraph/threads/{thread_id}/runs
{"input": {"messages": [{"role": "user", "content": "Hello"}]}}
2. Nginx → LangGraph Server (2024)
Proxied to LangGraph server
3. LangGraph Server
a. Load/create thread state
b. Execute middleware chain:
- ThreadDataMiddleware: Set up paths
- UploadsMiddleware: Inject file list
- SandboxMiddleware: Acquire sandbox
- SummarizationMiddleware: Check token limits
- TitleMiddleware: Generate title if needed
- TodoListMiddleware: Load todos (if plan mode)
- ViewImageMiddleware: Process images
- ClarificationMiddleware: Check for clarifications
c. Execute agent:
- Model processes messages
- May call tools (bash, web_search, etc.)
- Tools execute via sandbox
- Results added to messages
d. Stream response via SSE
4. Client receives streaming response
```
## Data Flow
### File Upload Flow
```
1. Client uploads file
POST /api/threads/{thread_id}/uploads
Content-Type: multipart/form-data
2. Gateway receives file
- Validates file
- Stores in .deer-flow/threads/{thread_id}/user-data/uploads/
- If document: converts to Markdown via markitdown
3. Returns response
{
"files": [{
"filename": "doc.pdf",
"path": ".deer-flow/.../uploads/doc.pdf",
"virtual_path": "/mnt/user-data/uploads/doc.pdf",
"artifact_url": "/api/threads/.../artifacts/mnt/.../doc.pdf"
}]
}
4. Next agent run
- UploadsMiddleware lists files
- Injects file list into messages
- Agent can access via virtual_path
```
### Thread Cleanup Flow
```
1. Client deletes conversation via LangGraph
DELETE /api/langgraph/threads/{thread_id}
2. Web UI follows up with Gateway cleanup
DELETE /api/threads/{thread_id}
3. Gateway removes local DeerFlow-managed files
- Deletes .deer-flow/threads/{thread_id}/ recursively
- Missing directories are treated as a no-op
- Invalid thread IDs are rejected before filesystem access
```
### Configuration Reload
```
1. Client updates MCP config
PUT /api/mcp/config
2. Gateway writes extensions_config.json
- Updates mcpServers section
- File mtime changes
3. MCP Manager detects change
- get_cached_mcp_tools() checks mtime
- If changed: reinitializes MCP client
- Loads updated server configurations
4. Next agent run uses new tools
```
## Security Considerations
### Sandbox Isolation
- Agent code executes within sandbox boundaries
- Local sandbox: Direct execution (development only)
- Docker sandbox: Container isolation (production recommended)
- Path traversal prevention in file operations
### API Security
- Thread isolation: Each thread has separate data directories
- File validation: Uploads checked for path safety
- Environment variable resolution: Secrets not stored in config
### MCP Security
- Each MCP server runs in its own process
- Environment variables resolved at runtime
- Servers can be enabled/disabled independently
## Performance Considerations
### Caching
- MCP tools cached with file mtime invalidation
- Configuration loaded once, reloaded on file change
- Skills parsed once at startup, cached in memory
### Streaming
- SSE used for real-time response streaming
- Reduces time to first token
- Enables progress visibility for long operations
### Context Management
- Summarization middleware reduces context when limits approached
- Configurable triggers: tokens, messages, or fraction
- Preserves recent messages while summarizing older ones
File diff suppressed because it is too large Load Diff
+129
View File
@@ -0,0 +1,129 @@
# Authentication Upgrade Guide
DeerFlow 内置了认证模块。本文档面向从无认证版本升级的用户。
## 核心概念
认证模块采用**始终强制**策略:
- 首次启动时自动创建 admin 账号,随机密码打印到控制台日志
- 认证从一开始就是强制的,无竞争窗口
- 历史对话(升级前创建的 thread)自动迁移到 admin 名下
## 升级步骤
### 1. 更新代码
```bash
git pull origin main
cd backend && make install
```
### 2. 首次启动
```bash
make dev
```
控制台会输出:
```
============================================================
Admin account created on first boot
Email: admin@deerflow.dev
Password: aB3xK9mN_pQ7rT2w
Change it after login: Settings → Account
============================================================
```
如果未登录就重启了服务,不用担心——只要 setup 未完成,每次启动都会重置密码并重新打印到控制台。
### 3. 登录
访问 `http://localhost:2026/login`,使用控制台输出的邮箱和密码登录。
### 4. 修改密码
登录后进入 Settings → Account → Change Password。
### 5. 添加用户(可选)
其他用户通过 `/login` 页面注册,自动获得 **user** 角色。每个用户只能看到自己的对话。
## 安全机制
| 机制 | 说明 |
|------|------|
| JWT HttpOnly Cookie | Token 不暴露给 JavaScript,防止 XSS 窃取 |
| CSRF Double Submit Cookie | 所有 POST/PUT/DELETE 请求需携带 `X-CSRF-Token` |
| bcrypt 密码哈希 | 密码不以明文存储 |
| 多租户隔离 | 用户只能访问自己的 thread |
| HTTPS 自适应 | 检测 `x-forwarded-proto`,自动设置 `Secure` cookie 标志 |
## 常见操作
### 忘记密码
```bash
cd backend
# 重置 admin 密码
python -m app.gateway.auth.reset_admin
# 重置指定用户密码
python -m app.gateway.auth.reset_admin --email user@example.com
```
会输出新的随机密码。
### 完全重置
删除用户数据库,重启后自动创建新 admin:
```bash
rm -f backend/.deer-flow/users.db
# 重启服务,控制台输出新密码
```
## 数据存储
| 文件 | 内容 |
|------|------|
| `.deer-flow/users.db` | SQLite 用户数据库(密码哈希、角色) |
| `.env` 中的 `AUTH_JWT_SECRET` | JWT 签名密钥(未设置时自动生成临时密钥,重启后 session 失效) |
### 生产环境建议
```bash
# 生成持久化 JWT 密钥,避免重启后所有用户需重新登录
python -c "import secrets; print(secrets.token_urlsafe(32))"
# 将输出添加到 .env
# AUTH_JWT_SECRET=<生成的密钥>
```
## API 端点
| 端点 | 方法 | 说明 |
|------|------|------|
| `/api/v1/auth/login/local` | POST | 邮箱密码登录(OAuth2 form |
| `/api/v1/auth/register` | POST | 注册新用户(user 角色) |
| `/api/v1/auth/logout` | POST | 登出(清除 cookie |
| `/api/v1/auth/me` | GET | 获取当前用户信息 |
| `/api/v1/auth/change-password` | POST | 修改密码 |
| `/api/v1/auth/setup-status` | GET | 检查 admin 是否存在 |
## 兼容性
- **标准模式**`make dev`):完全兼容,admin 自动创建
- **Gateway 模式**`make dev-pro`):完全兼容
- **Docker 部署**:完全兼容,`.deer-flow/users.db` 需持久化卷挂载
- **IM 渠道**Feishu/Slack/Telegram):通过 LangGraph SDK 通信,不经过认证层
- **DeerFlowClient**(嵌入式):不经过 HTTP,不受认证影响
## 故障排查
| 症状 | 原因 | 解决 |
|------|------|------|
| 启动后没看到密码 | admin 已存在(非首次启动) | 用 `reset_admin` 重置,或删 `users.db` |
| 登录后 POST 返回 403 | CSRF token 缺失 | 确认前端已更新 |
| 重启后需要重新登录 | `AUTH_JWT_SECRET` 未持久化 | 在 `.env` 中设置固定密钥 |
+258
View File
@@ -0,0 +1,258 @@
# 自动 Thread Title 生成功能
## 功能说明
自动为对话线程生成标题,在用户首次提问并收到回复后自动触发。
## 实现方式
使用 `TitleMiddleware``after_model` 钩子中:
1. 检测是否是首次对话(1个用户消息 + 1个助手回复)
2. 检查 state 是否已有 title
3. 调用 LLM 生成简洁的标题(默认最多6个词)
4. 将 title 存储到 `ThreadState` 中(会被 checkpointer 持久化)
TitleMiddleware 会先把 LangChain message content 里的结构化 block/list 内容归一化为纯文本,再拼到 title prompt 里,避免把 Python/JSON 的原始 repr 泄漏到标题生成模型。
## ⚠️ 重要:存储机制
### Title 存储位置
Title 存储在 **`ThreadState.title`** 中,而非 thread metadata
```python
class ThreadState(AgentState):
sandbox: SandboxState | None = None
title: str | None = None # ✅ Title stored here
```
### 持久化说明
| 部署方式 | 持久化 | 说明 |
|---------|--------|------|
| **LangGraph Studio (本地)** | ❌ 否 | 仅内存存储,重启后丢失 |
| **LangGraph Platform** | ✅ 是 | 自动持久化到数据库 |
| **自定义 + Checkpointer** | ✅ 是 | 需配置 PostgreSQL/SQLite checkpointer |
### 如何启用持久化
如果需要在本地开发时也持久化 title,需要配置 checkpointer
```python
# 在 langgraph.json 同级目录创建 checkpointer.py
from langgraph.checkpoint.postgres import PostgresSaver
checkpointer = PostgresSaver.from_conn_string(
"postgresql://user:pass@localhost/dbname"
)
```
然后在 `langgraph.json` 中引用:
```json
{
"graphs": {
"lead_agent": "deerflow.agents:lead_agent"
},
"checkpointer": "checkpointer:checkpointer"
}
```
## 配置
`config.yaml` 中添加(可选):
```yaml
title:
enabled: true
max_words: 6
max_chars: 60
model_name: null # 使用默认模型
```
或在代码中配置:
```python
from deerflow.config.title_config import TitleConfig, set_title_config
set_title_config(TitleConfig(
enabled=True,
max_words=8,
max_chars=80,
))
```
## 客户端使用
### 获取 Thread Title
```typescript
// 方式1: 从 thread state 获取
const state = await client.threads.getState(threadId);
const title = state.values.title || "New Conversation";
// 方式2: 监听 stream 事件
for await (const chunk of client.runs.stream(threadId, assistantId, {
input: { messages: [{ role: "user", content: "Hello" }] }
})) {
if (chunk.event === "values" && chunk.data.title) {
console.log("Title:", chunk.data.title);
}
}
```
### 显示 Title
```typescript
// 在对话列表中显示
function ConversationList() {
const [threads, setThreads] = useState([]);
useEffect(() => {
async function loadThreads() {
const allThreads = await client.threads.list();
// 获取每个 thread 的 state 来读取 title
const threadsWithTitles = await Promise.all(
allThreads.map(async (t) => {
const state = await client.threads.getState(t.thread_id);
return {
id: t.thread_id,
title: state.values.title || "New Conversation",
updatedAt: t.updated_at,
};
})
);
setThreads(threadsWithTitles);
}
loadThreads();
}, []);
return (
<ul>
{threads.map(thread => (
<li key={thread.id}>
<a href={`/chat/${thread.id}`}>{thread.title}</a>
</li>
))}
</ul>
);
}
```
## 工作流程
```mermaid
sequenceDiagram
participant User
participant Client
participant LangGraph
participant TitleMiddleware
participant LLM
participant Checkpointer
User->>Client: 发送首条消息
Client->>LangGraph: POST /threads/{id}/runs
LangGraph->>Agent: 处理消息
Agent-->>LangGraph: 返回回复
LangGraph->>TitleMiddleware: after_agent()
TitleMiddleware->>TitleMiddleware: 检查是否需要生成 title
TitleMiddleware->>LLM: 生成 title
LLM-->>TitleMiddleware: 返回 title
TitleMiddleware->>LangGraph: return {"title": "..."}
LangGraph->>Checkpointer: 保存 state (含 title)
LangGraph-->>Client: 返回响应
Client->>Client: 从 state.values.title 读取
```
## 优势
**可靠持久化** - 使用 LangGraph 的 state 机制,自动持久化
**完全后端处理** - 客户端无需额外逻辑
**自动触发** - 首次对话后自动生成
**可配置** - 支持自定义长度、模型等
**容错性强** - 失败时使用 fallback 策略
**架构一致** - 与现有 SandboxMiddleware 保持一致
## 注意事项
1. **读取方式不同**Title 在 `state.values.title` 而非 `thread.metadata.title`
2. **性能考虑**title 生成会增加约 0.5-1 秒延迟,可通过使用更快的模型优化
3. **并发安全**middleware 在 agent 执行后运行,不会阻塞主流程
4. **Fallback 策略**:如果 LLM 调用失败,会使用用户消息的前几个词作为 title
## 测试
```python
# 测试 title 生成
import pytest
from deerflow.agents.title_middleware import TitleMiddleware
def test_title_generation():
# TODO: 添加单元测试
pass
```
## 故障排查
### Title 没有生成
1. 检查配置是否启用:`get_title_config().enabled == True`
2. 检查日志:查找 "Generated thread title" 或错误信息
3. 确认是首次对话:只有 1 个用户消息和 1 个助手回复时才会触发
### Title 生成但客户端看不到
1. 确认读取位置:应该从 `state.values.title` 读取,而非 `thread.metadata.title`
2. 检查 API 响应:确认 state 中包含 title 字段
3. 尝试重新获取 state`client.threads.getState(threadId)`
### Title 重启后丢失
1. 检查是否配置了 checkpointer(本地开发需要)
2. 确认部署方式:LangGraph Platform 会自动持久化
3. 查看数据库:确认 checkpointer 正常工作
## 架构设计
### 为什么使用 State 而非 Metadata
| 特性 | State | Metadata |
|------|-------|----------|
| **持久化** | ✅ 自动(通过 checkpointer | ⚠️ 取决于实现 |
| **版本控制** | ✅ 支持时间旅行 | ❌ 不支持 |
| **类型安全** | ✅ TypedDict 定义 | ❌ 任意字典 |
| **可追溯** | ✅ 每次更新都记录 | ⚠️ 只有最新值 |
| **标准化** | ✅ LangGraph 核心机制 | ⚠️ 扩展功能 |
### 实现细节
```python
# TitleMiddleware 核心逻辑
@override
def after_agent(self, state: TitleMiddlewareState, runtime: Runtime) -> dict | None:
"""Generate and set thread title after the first agent response."""
if self._should_generate_title(state, runtime):
title = self._generate_title(runtime)
print(f"Generated thread title: {title}")
# ✅ 返回 state 更新,会被 checkpointer 自动持久化
return {"title": title}
return None
```
## 相关文件
- [`packages/harness/deerflow/agents/thread_state.py`](../packages/harness/deerflow/agents/thread_state.py) - ThreadState 定义
- [`packages/harness/deerflow/agents/middlewares/title_middleware.py`](../packages/harness/deerflow/agents/middlewares/title_middleware.py) - TitleMiddleware 实现
- [`packages/harness/deerflow/config/title_config.py`](../packages/harness/deerflow/config/title_config.py) - 配置管理
- [`config.yaml`](../../config.example.yaml) - 配置文件
- [`packages/harness/deerflow/agents/lead_agent/agent.py`](../packages/harness/deerflow/agents/lead_agent/agent.py) - Middleware 注册
## 参考资料
- [LangGraph Checkpointer 文档](https://langchain-ai.github.io/langgraph/concepts/persistence/)
- [LangGraph State 管理](https://langchain-ai.github.io/langgraph/concepts/low_level/#state)
- [LangGraph Middleware](https://langchain-ai.github.io/langgraph/concepts/middleware/)
+369
View File
@@ -0,0 +1,369 @@
# Configuration Guide
This guide explains how to configure DeerFlow for your environment.
## Config Versioning
`config.example.yaml` contains a `config_version` field that tracks schema changes. When the example version is higher than your local `config.yaml`, the application emits a startup warning:
```
WARNING - Your config.yaml (version 0) is outdated — the latest version is 1.
Run `make config-upgrade` to merge new fields into your config.
```
- **Missing `config_version`** in your config is treated as version 0.
- Run `make config-upgrade` to auto-merge missing fields (your existing values are preserved, a `.bak` backup is created).
- When changing the config schema, bump `config_version` in `config.example.yaml`.
## Configuration Sections
### Models
Configure the LLM models available to the agent:
```yaml
models:
- name: gpt-4 # Internal identifier
display_name: GPT-4 # Human-readable name
use: langchain_openai:ChatOpenAI # LangChain class path
model: gpt-4 # Model identifier for API
api_key: $OPENAI_API_KEY # API key (use env var)
max_tokens: 4096 # Max tokens per request
temperature: 0.7 # Sampling temperature
```
**Supported Providers**:
- OpenAI (`langchain_openai:ChatOpenAI`)
- Anthropic (`langchain_anthropic:ChatAnthropic`)
- DeepSeek (`langchain_deepseek:ChatDeepSeek`)
- Claude Code OAuth (`deerflow.models.claude_provider:ClaudeChatModel`)
- Codex CLI (`deerflow.models.openai_codex_provider:CodexChatModel`)
- Any LangChain-compatible provider
CLI-backed provider examples:
```yaml
models:
- name: gpt-5.4
display_name: GPT-5.4 (Codex CLI)
use: deerflow.models.openai_codex_provider:CodexChatModel
model: gpt-5.4
supports_thinking: true
supports_reasoning_effort: true
- name: claude-sonnet-4.6
display_name: Claude Sonnet 4.6 (Claude Code OAuth)
use: deerflow.models.claude_provider:ClaudeChatModel
model: claude-sonnet-4-6
max_tokens: 4096
supports_thinking: true
```
**Auth behavior for CLI-backed providers**:
- `CodexChatModel` loads Codex CLI auth from `~/.codex/auth.json`
- The Codex Responses endpoint currently rejects `max_tokens` and `max_output_tokens`, so `CodexChatModel` does not expose a request-level token cap
- `ClaudeChatModel` accepts `CLAUDE_CODE_OAUTH_TOKEN`, `ANTHROPIC_AUTH_TOKEN`, `CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR`, `CLAUDE_CODE_CREDENTIALS_PATH`, or plaintext `~/.claude/.credentials.json`
- On macOS, DeerFlow does not probe Keychain automatically. Use `scripts/export_claude_code_oauth.py` to export Claude Code auth explicitly when needed
To use OpenAI's `/v1/responses` endpoint with LangChain, keep using `langchain_openai:ChatOpenAI` and set:
```yaml
models:
- name: gpt-5-responses
display_name: GPT-5 (Responses API)
use: langchain_openai:ChatOpenAI
model: gpt-5
api_key: $OPENAI_API_KEY
use_responses_api: true
output_version: responses/v1
```
For OpenAI-compatible gateways (for example Novita or OpenRouter), keep using `langchain_openai:ChatOpenAI` and set `base_url`:
```yaml
models:
- name: novita-deepseek-v3.2
display_name: Novita DeepSeek V3.2
use: langchain_openai:ChatOpenAI
model: deepseek/deepseek-v3.2
api_key: $NOVITA_API_KEY
base_url: https://api.novita.ai/openai
supports_thinking: true
when_thinking_enabled:
extra_body:
thinking:
type: enabled
- name: minimax-m2.5
display_name: MiniMax M2.5
use: langchain_openai:ChatOpenAI
model: MiniMax-M2.5
api_key: $MINIMAX_API_KEY
base_url: https://api.minimax.io/v1
max_tokens: 4096
temperature: 1.0 # MiniMax requires temperature in (0.0, 1.0]
supports_vision: true
- name: minimax-m2.5-highspeed
display_name: MiniMax M2.5 Highspeed
use: langchain_openai:ChatOpenAI
model: MiniMax-M2.5-highspeed
api_key: $MINIMAX_API_KEY
base_url: https://api.minimax.io/v1
max_tokens: 4096
temperature: 1.0 # MiniMax requires temperature in (0.0, 1.0]
supports_vision: true
- name: openrouter-gemini-2.5-flash
display_name: Gemini 2.5 Flash (OpenRouter)
use: langchain_openai:ChatOpenAI
model: google/gemini-2.5-flash-preview
api_key: $OPENAI_API_KEY
base_url: https://openrouter.ai/api/v1
```
If your OpenRouter key lives in a different environment variable name, point `api_key` at that variable explicitly (for example `api_key: $OPENROUTER_API_KEY`).
**Thinking Models**:
Some models support "thinking" mode for complex reasoning:
```yaml
models:
- name: deepseek-v3
supports_thinking: true
when_thinking_enabled:
extra_body:
thinking:
type: enabled
```
**Gemini with thinking via OpenAI-compatible gateway**:
When routing Gemini through an OpenAI-compatible proxy (Vertex AI OpenAI compat endpoint, AI Studio, or third-party gateways) with thinking enabled, the API attaches a `thought_signature` to each tool-call object returned in the response. Every subsequent request that replays those assistant messages **must** echo those signatures back on the tool-call entries or the API returns:
```
HTTP 400 INVALID_ARGUMENT: function call `<tool>` in the N. content block is
missing a `thought_signature`.
```
Standard `langchain_openai:ChatOpenAI` silently drops `thought_signature` when serialising messages. Use `deerflow.models.patched_openai:PatchedChatOpenAI` instead — it re-injects the tool-call signatures (sourced from `AIMessage.additional_kwargs["tool_calls"]`) into every outgoing payload:
```yaml
models:
- name: gemini-2.5-pro-thinking
display_name: Gemini 2.5 Pro (Thinking)
use: deerflow.models.patched_openai:PatchedChatOpenAI
model: google/gemini-2.5-pro-preview # model name as expected by your gateway
api_key: $GEMINI_API_KEY
base_url: https://<your-openai-compat-gateway>/v1
max_tokens: 16384
supports_thinking: true
supports_vision: true
when_thinking_enabled:
extra_body:
thinking:
type: enabled
```
For Gemini accessed **without** thinking (e.g. via OpenRouter where thinking is not activated), the plain `langchain_openai:ChatOpenAI` with `supports_thinking: false` is sufficient and no patch is needed.
### Tool Groups
Organize tools into logical groups:
```yaml
tool_groups:
- name: web # Web browsing and search
- name: file:read # Read-only file operations
- name: file:write # Write file operations
- name: bash # Shell command execution
```
### Tools
Configure specific tools available to the agent:
```yaml
tools:
- name: web_search
group: web
use: deerflow.community.tavily.tools:web_search_tool
max_results: 5
# api_key: $TAVILY_API_KEY # Optional
```
**Built-in Tools**:
- `web_search` - Search the web (Tavily)
- `web_fetch` - Fetch web pages (Jina AI)
- `ls` - List directory contents
- `read_file` - Read file contents
- `write_file` - Write file contents
- `str_replace` - String replacement in files
- `bash` - Execute bash commands
### Sandbox
DeerFlow supports multiple sandbox execution modes. Configure your preferred mode in `config.yaml`:
**Local Execution** (runs sandbox code directly on the host machine):
```yaml
sandbox:
use: deerflow.sandbox.local:LocalSandboxProvider # Local execution
allow_host_bash: false # default; host bash is disabled unless explicitly re-enabled
```
**Docker Execution** (runs sandbox code in isolated Docker containers):
```yaml
sandbox:
use: deerflow.community.aio_sandbox:AioSandboxProvider # Docker-based sandbox
```
**Docker Execution with Kubernetes** (runs sandbox code in Kubernetes pods via provisioner service):
This mode runs each sandbox in an isolated Kubernetes Pod on your **host machine's cluster**. Requires Docker Desktop K8s, OrbStack, or similar local K8s setup.
```yaml
sandbox:
use: deerflow.community.aio_sandbox:AioSandboxProvider
provisioner_url: http://provisioner:8002
```
When using Docker development (`make docker-start`), DeerFlow starts the `provisioner` service only if this provisioner mode is configured. In local or plain Docker sandbox modes, `provisioner` is skipped.
See [Provisioner Setup Guide](../../docker/provisioner/README.md) for detailed configuration, prerequisites, and troubleshooting.
Choose between local execution or Docker-based isolation:
**Option 1: Local Sandbox** (default, simpler setup):
```yaml
sandbox:
use: deerflow.sandbox.local:LocalSandboxProvider
allow_host_bash: false
```
`allow_host_bash` is intentionally `false` by default. DeerFlow's local sandbox is a host-side convenience mode, not a secure shell isolation boundary. If you need `bash`, prefer `AioSandboxProvider`. Only set `allow_host_bash: true` for fully trusted single-user local workflows.
**Option 2: Docker Sandbox** (isolated, more secure):
```yaml
sandbox:
use: deerflow.community.aio_sandbox:AioSandboxProvider
port: 8080
auto_start: true
container_prefix: deer-flow-sandbox
# Optional: Additional mounts
mounts:
- host_path: /path/on/host
container_path: /path/in/container
read_only: false
```
When you configure `sandbox.mounts`, DeerFlow exposes those `container_path` values in the agent prompt so the agent can discover and operate on mounted directories directly instead of assuming everything must live under `/mnt/user-data`.
### Skills
Configure the skills directory for specialized workflows:
```yaml
skills:
# Host path (optional, default: ../skills)
path: /custom/path/to/skills
# Container mount path (default: /mnt/skills)
container_path: /mnt/skills
```
**How Skills Work**:
- Skills are stored in `deer-flow/skills/{public,custom}/`
- Each skill has a `SKILL.md` file with metadata
- Skills are automatically discovered and loaded
- Available in both local and Docker sandbox via path mapping
**Per-Agent Skill Filtering**:
Custom agents can restrict which skills they load by defining a `skills` field in their `config.yaml` (located at `workspace/agents/<agent_name>/config.yaml`):
- **Omitted or `null`**: Loads all globally enabled skills (default fallback).
- **`[]` (empty list)**: Disables all skills for this specific agent.
- **`["skill-name"]`**: Loads only the explicitly specified skills.
### Title Generation
Automatic conversation title generation:
```yaml
title:
enabled: true
max_words: 6
max_chars: 60
model_name: null # Use first model in list
```
### GitHub API Token (Optional for GitHub Deep Research Skill)
The default GitHub API rate limits are quite restrictive. For frequent project research, we recommend configuring a personal access token (PAT) with read-only permissions.
**Configuration Steps**:
1. Uncomment the `GITHUB_TOKEN` line in the `.env` file and add your personal access token
2. Restart the DeerFlow service to apply changes
## Environment Variables
DeerFlow supports environment variable substitution using the `$` prefix:
```yaml
models:
- api_key: $OPENAI_API_KEY # Reads from environment
```
**Common Environment Variables**:
- `OPENAI_API_KEY` - OpenAI API key
- `ANTHROPIC_API_KEY` - Anthropic API key
- `DEEPSEEK_API_KEY` - DeepSeek API key
- `NOVITA_API_KEY` - Novita API key (OpenAI-compatible endpoint)
- `TAVILY_API_KEY` - Tavily search API key
- `DEER_FLOW_CONFIG_PATH` - Custom config file path
## Configuration Location
The configuration file should be placed in the **project root directory** (`deer-flow/config.yaml`), not in the backend directory.
## Configuration Priority
DeerFlow searches for configuration in this order:
1. Path specified in code via `config_path` argument
2. Path from `DEER_FLOW_CONFIG_PATH` environment variable
3. `config.yaml` in current working directory (typically `backend/` when running)
4. `config.yaml` in parent directory (project root: `deer-flow/`)
## Best Practices
1. **Place `config.yaml` in project root** - Not in `backend/` directory
2. **Never commit `config.yaml`** - It's already in `.gitignore`
3. **Use environment variables for secrets** - Don't hardcode API keys
4. **Keep `config.example.yaml` updated** - Document all new options
5. **Test configuration changes locally** - Before deploying
6. **Use Docker sandbox for production** - Better isolation and security
## Troubleshooting
### "Config file not found"
- Ensure `config.yaml` exists in the **project root** directory (`deer-flow/config.yaml`)
- The backend searches parent directory by default, so root location is preferred
- Alternatively, set `DEER_FLOW_CONFIG_PATH` environment variable to custom location
### "Invalid API key"
- Verify environment variables are set correctly
- Check that `$` prefix is used for env var references
### "Skills not loading"
- Check that `deer-flow/skills/` directory exists
- Verify skills have valid `SKILL.md` files
- Check `skills.path` configuration if using custom path
### "Docker sandbox fails to start"
- Ensure Docker is running
- Check port 8080 (or configured port) is available
- Verify Docker image is accessible
## Examples
See `config.example.yaml` for complete examples of all configuration options.
+293
View File
@@ -0,0 +1,293 @@
# 文件上传功能
## 概述
DeerFlow 后端提供了完整的文件上传功能,支持多文件上传,并自动将 Office 文档和 PDF 转换为 Markdown 格式。
## 功能特性
- ✅ 支持多文件同时上传
- ✅ 自动转换文档为 MarkdownPDF、PPT、Excel、Word
- ✅ 文件存储在线程隔离的目录中
- ✅ Agent 自动感知已上传的文件
- ✅ 支持文件列表查询和删除
## API 端点
### 1. 上传文件
```
POST /api/threads/{thread_id}/uploads
```
**请求体:** `multipart/form-data`
- `files`: 一个或多个文件
**响应:**
```json
{
"success": true,
"files": [
{
"filename": "document.pdf",
"size": 1234567,
"path": ".deer-flow/threads/{thread_id}/user-data/uploads/document.pdf",
"virtual_path": "/mnt/user-data/uploads/document.pdf",
"artifact_url": "/api/threads/{thread_id}/artifacts/mnt/user-data/uploads/document.pdf",
"markdown_file": "document.md",
"markdown_path": ".deer-flow/threads/{thread_id}/user-data/uploads/document.md",
"markdown_virtual_path": "/mnt/user-data/uploads/document.md",
"markdown_artifact_url": "/api/threads/{thread_id}/artifacts/mnt/user-data/uploads/document.md"
}
],
"message": "Successfully uploaded 1 file(s)"
}
```
**路径说明:**
- `path`: 实际文件系统路径(相对于 `backend/` 目录)
- `virtual_path`: Agent 在沙箱中使用的虚拟路径
- `artifact_url`: 前端通过 HTTP 访问文件的 URL
### 2. 列出已上传文件
```
GET /api/threads/{thread_id}/uploads/list
```
**响应:**
```json
{
"files": [
{
"filename": "document.pdf",
"size": 1234567,
"path": ".deer-flow/threads/{thread_id}/user-data/uploads/document.pdf",
"virtual_path": "/mnt/user-data/uploads/document.pdf",
"artifact_url": "/api/threads/{thread_id}/artifacts/mnt/user-data/uploads/document.pdf",
"extension": ".pdf",
"modified": 1705997600.0
}
],
"count": 1
}
```
### 3. 删除文件
```
DELETE /api/threads/{thread_id}/uploads/{filename}
```
**响应:**
```json
{
"success": true,
"message": "Deleted document.pdf"
}
```
## 支持的文档格式
以下格式会自动转换为 Markdown:
- PDF (`.pdf`)
- PowerPoint (`.ppt`, `.pptx`)
- Excel (`.xls`, `.xlsx`)
- Word (`.doc`, `.docx`)
转换后的 Markdown 文件会保存在同一目录下,文件名为原文件名 + `.md` 扩展名。
## Agent 集成
### 自动文件列举
Agent 在每次请求时会自动收到已上传文件的列表,格式如下:
```xml
<uploaded_files>
The following files have been uploaded and are available for use:
- document.pdf (1.2 MB)
Path: /mnt/user-data/uploads/document.pdf
- document.md (45.3 KB)
Path: /mnt/user-data/uploads/document.md
You can read these files using the `read_file` tool with the paths shown above.
</uploaded_files>
```
### 使用上传的文件
Agent 在沙箱中运行,使用虚拟路径访问文件。Agent 可以直接使用 `read_file` 工具读取上传的文件:
```python
# 读取原始 PDF(如果支持)
read_file(path="/mnt/user-data/uploads/document.pdf")
# 读取转换后的 Markdown(推荐)
read_file(path="/mnt/user-data/uploads/document.md")
```
**路径映射关系:**
- Agent 使用:`/mnt/user-data/uploads/document.pdf`(虚拟路径)
- 实际存储:`backend/.deer-flow/threads/{thread_id}/user-data/uploads/document.pdf`
- 前端访问:`/api/threads/{thread_id}/artifacts/mnt/user-data/uploads/document.pdf`HTTP URL
上传流程采用“线程目录优先”策略:
- 先写入 `backend/.deer-flow/threads/{thread_id}/user-data/uploads/` 作为权威存储
- 本地沙箱(`sandbox_id=local`)直接使用线程目录内容
- 非本地沙箱会额外同步到 `/mnt/user-data/uploads/*`,确保运行时可见
## 测试示例
### 使用 curl 测试
```bash
# 1. 上传单个文件
curl -X POST http://localhost:2026/api/threads/test-thread/uploads \
-F "files=@/path/to/document.pdf"
# 2. 上传多个文件
curl -X POST http://localhost:2026/api/threads/test-thread/uploads \
-F "files=@/path/to/document.pdf" \
-F "files=@/path/to/presentation.pptx" \
-F "files=@/path/to/spreadsheet.xlsx"
# 3. 列出已上传文件
curl http://localhost:2026/api/threads/test-thread/uploads/list
# 4. 删除文件
curl -X DELETE http://localhost:2026/api/threads/test-thread/uploads/document.pdf
```
### 使用 Python 测试
```python
import requests
thread_id = "test-thread"
base_url = "http://localhost:2026"
# 上传文件
files = [
("files", open("document.pdf", "rb")),
("files", open("presentation.pptx", "rb")),
]
response = requests.post(
f"{base_url}/api/threads/{thread_id}/uploads",
files=files
)
print(response.json())
# 列出文件
response = requests.get(f"{base_url}/api/threads/{thread_id}/uploads/list")
print(response.json())
# 删除文件
response = requests.delete(
f"{base_url}/api/threads/{thread_id}/uploads/document.pdf"
)
print(response.json())
```
## 文件存储结构
```
backend/.deer-flow/threads/
└── {thread_id}/
└── user-data/
└── uploads/
├── document.pdf # 原始文件
├── document.md # 转换后的 Markdown
├── presentation.pptx
├── presentation.md
└── ...
```
## 限制
- 最大文件大小:100MB(可在 nginx.conf 中配置 `client_max_body_size`
- 文件名安全性:系统会自动验证文件路径,防止目录遍历攻击
- 线程隔离:每个线程的上传文件相互隔离,无法跨线程访问
## 技术实现
### 组件
1. **Upload Router** (`app/gateway/routers/uploads.py`)
- 处理文件上传、列表、删除请求
- 使用 markitdown 转换文档
2. **Uploads Middleware** (`packages/harness/deerflow/agents/middlewares/uploads_middleware.py`)
- 在每次 Agent 请求前注入文件列表
- 自动生成格式化的文件列表消息
3. **Nginx 配置** (`nginx.conf`)
- 路由上传请求到 Gateway API
- 配置大文件上传支持
### 依赖
- `markitdown>=0.0.1a2` - 文档转换
- `python-multipart>=0.0.20` - 文件上传处理
## 故障排查
### 文件上传失败
1. 检查文件大小是否超过限制
2. 检查 Gateway API 是否正常运行
3. 检查磁盘空间是否充足
4. 查看 Gateway 日志:`make gateway`
### 文档转换失败
1. 检查 markitdown 是否正确安装:`uv run python -c "import markitdown"`
2. 查看日志中的具体错误信息
3. 某些损坏或加密的文档可能无法转换,但原文件仍会保存
### Agent 看不到上传的文件
1. 确认 UploadsMiddleware 已在 agent.py 中注册
2. 检查 thread_id 是否正确
3. 确认文件确实已上传到 `backend/.deer-flow/threads/{thread_id}/user-data/uploads/`
4. 非本地沙箱场景下,确认上传接口没有报错(需要成功完成 sandbox 同步)
## 开发建议
### 前端集成
```typescript
// 上传文件示例
async function uploadFiles(threadId: string, files: File[]) {
const formData = new FormData();
files.forEach(file => {
formData.append('files', file);
});
const response = await fetch(
`/api/threads/${threadId}/uploads`,
{
method: 'POST',
body: formData,
}
);
return response.json();
}
// 列出文件
async function listFiles(threadId: string) {
const response = await fetch(
`/api/threads/${threadId}/uploads/list`
);
return response.json();
}
```
### 扩展功能建议
1. **文件预览**:添加预览端点,支持在浏览器中直接查看文件
2. **批量删除**:支持一次删除多个文件
3. **文件搜索**:支持按文件名或类型搜索
4. **版本控制**:保留文件的多个版本
5. **压缩包支持**:自动解压 zip 文件
6. **图片 OCR**:对上传的图片进行 OCR 识别

Some files were not shown because too many files have changed in this diff Show More