mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-05-21 15:36:48 +00:00
00e0e9a49a
* feat(persistence): add SQLAlchemy 2.0 async ORM scaffold Introduce a unified database configuration (DatabaseConfig) that controls both the LangGraph checkpointer and the DeerFlow application persistence layer from a single `database:` config section. New modules: - deerflow.config.database_config — Pydantic config with memory/sqlite/postgres backends - deerflow.persistence — async engine lifecycle, DeclarativeBase with to_dict mixin, Alembic skeleton - deerflow.runtime.runs.store — RunStore ABC + MemoryRunStore implementation Gateway integration initializes/tears down the persistence engine in the existing langgraph_runtime() context manager. Legacy checkpointer config is preserved for backward compatibility. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(persistence): add RunEventStore ABC + MemoryRunEventStore Phase 2-A prerequisite for event storage: adds the unified run event stream interface (RunEventStore) with an in-memory implementation, RunEventsConfig, gateway integration, and comprehensive tests (27 cases). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(persistence): add ORM models, repositories, DB/JSONL event stores, RunJournal, and API endpoints Phase 2-B: run persistence + event storage + token tracking. - ORM models: RunRow (with token fields), ThreadMetaRow, RunEventRow - RunRepository implements RunStore ABC via SQLAlchemy ORM - ThreadMetaRepository with owner access control - DbRunEventStore with trace content truncation and cursor pagination - JsonlRunEventStore with per-run files and seq recovery from disk - RunJournal (BaseCallbackHandler) captures LLM/tool/lifecycle events, accumulates token usage by caller type, buffers and flushes to store - RunManager now accepts optional RunStore for persistent backing - Worker creates RunJournal, writes human_message, injects callbacks - Gateway deps use factory functions (RunRepository when DB available) - New endpoints: messages, run messages, run events, token-usage - ThreadCreateRequest gains assistant_id field - 92 tests pass (33 new), zero regressions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(persistence): add user feedback + follow-up run association Phase 2-C: feedback and follow-up tracking. - FeedbackRow ORM model (rating +1/-1, optional message_id, comment) - FeedbackRepository with CRUD, list_by_run/thread, aggregate stats - Feedback API endpoints: create, list, stats, delete - follow_up_to_run_id in RunCreateRequest (explicit or auto-detected from latest successful run on the thread) - Worker writes follow_up_to_run_id into human_message event metadata - Gateway deps: feedback_repo factory + getter - 17 new tests (14 FeedbackRepository + 3 follow-up association) - 109 total tests pass, zero regressions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test+config: comprehensive Phase 2 test coverage + deprecate checkpointer config - config.example.yaml: deprecate standalone checkpointer section, activate unified database:sqlite as default (drives both checkpointer + app data) - New: test_thread_meta_repo.py (14 tests) — full ThreadMetaRepository coverage including check_access owner logic, list_by_owner pagination - Extended test_run_repository.py (+4 tests) — completion preserves fields, list ordering desc, limit, owner_none returns all - Extended test_run_journal.py (+8 tests) — on_chain_error, track_tokens=false, middleware no ai_message, unknown caller tokens, convenience fields, tool_error, non-summarization custom event - Extended test_run_event_store.py (+7 tests) — DB batch seq continuity, make_run_event_store factory (memory/db/jsonl/fallback/unknown) - Extended test_phase2b_integration.py (+4 tests) — create_or_reject persists, follow-up metadata, summarization in history, full DB-backed lifecycle - Fixed DB integration test to use proper fake objects (not MagicMock) for JSON-serializable metadata - 157 total Phase 2 tests pass, zero regressions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * config: move default sqlite_dir to .deer-flow/data Keep SQLite databases alongside other DeerFlow-managed data (threads, memory) under the .deer-flow/ directory instead of a top-level ./data folder. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(persistence): remove UTFJSON, use engine-level json_serializer + datetime.now() - Replace custom UTFJSON type with standard sqlalchemy.JSON in all ORM models. Add json_serializer=json.dumps(ensure_ascii=False) to all create_async_engine calls so non-ASCII text (Chinese etc.) is stored as-is in both SQLite and Postgres. - Change ORM datetime defaults from datetime.now(UTC) to datetime.now(), remove UTC imports. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(gateway): simplify deps.py with getter factory + inline repos - Replace 6 identical getter functions with _require() factory. - Inline 3 _make_*_repo() factories into langgraph_runtime(), call get_session_factory() once instead of 3 times. - Add thread_meta upsert in start_run (services.py). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(docker): add UV_EXTRAS build arg for optional dependencies Support installing optional dependency groups (e.g. postgres) at Docker build time via UV_EXTRAS build arg: UV_EXTRAS=postgres docker compose build Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(journal): fix flush, token tracking, and consolidate tests RunJournal fixes: - _flush_sync: retain events in buffer when no event loop instead of dropping them; worker's finally block flushes via async flush(). - on_llm_end: add tool_calls filter and caller=="lead_agent" guard for ai_message events; mark message IDs for dedup with record_llm_usage. - worker.py: persist completion data (tokens, message count) to RunStore in finally block. Model factory: - Auto-inject stream_usage=True for BaseChatOpenAI subclasses with custom api_base, so usage_metadata is populated in streaming responses. Test consolidation: - Delete test_phase2b_integration.py (redundant with existing tests). - Move DB-backed lifecycle test into test_run_journal.py. - Add tests for stream_usage injection in test_model_factory.py. - Clean up executor/task_tool dead journal references. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(events): widen content type to str|dict in all store backends Allow event content to be a dict (for structured OpenAI-format messages) in addition to plain strings. Dict values are JSON-serialized for the DB backend and deserialized on read; memory and JSONL backends handle dicts natively. Trace truncation now serializes dicts to JSON before measuring. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(events): use metadata flag instead of heuristic for dict content detection Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(converters): add LangChain-to-OpenAI message format converters Pure functions langchain_to_openai_message, langchain_to_openai_completion, langchain_messages_to_openai, and _infer_finish_reason for converting LangChain BaseMessage objects to OpenAI Chat Completions format, used by RunJournal for event storage. 15 unit tests added. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(converters): handle empty list content as null, clean up test Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(events): human_message content uses OpenAI user message format Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(events): ai_message uses OpenAI format, add ai_tool_call message event - ai_message content now uses {"role": "assistant", "content": "..."} format - New ai_tool_call message event emitted when lead_agent LLM responds with tool_calls - ai_tool_call uses langchain_to_openai_message converter for consistent format - Both events include finish_reason in metadata ("stop" or "tool_calls") Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(events): add tool_result message event with OpenAI tool message format Cache tool_call_id from on_tool_start keyed by run_id as fallback for on_tool_end, then emit a tool_result message event (role=tool, tool_call_id, content) after each successful tool completion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(events): summary content uses OpenAI system message format Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(events): replace llm_start/llm_end with llm_request/llm_response in OpenAI format Add on_chat_model_start to capture structured prompt messages as llm_request events. Replace llm_end trace events with llm_response using OpenAI Chat Completions format. Track llm_call_index to pair request/response events. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(events): add record_middleware method for middleware trace events Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(events): add full run sequence integration test for OpenAI content format Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(events): align message events with checkpoint format and add middleware tag injection - Message events (ai_message, ai_tool_call, tool_result, human_message) now use BaseMessage.model_dump() format, matching LangGraph checkpoint values.messages - on_tool_end extracts tool_call_id/name/status from ToolMessage objects - on_tool_error now emits tool_result message events with error status - record_middleware uses middleware:{tag} event_type and middleware category - Summarization custom events use middleware:summarize category - TitleMiddleware injects middleware:title tag via get_config() inheritance - SummarizationMiddleware model bound with middleware:summarize tag - Worker writes human_message using HumanMessage.model_dump() Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(threads): switch search endpoint to threads_meta table and sync title - POST /api/threads/search now queries threads_meta table directly, removing the two-phase Store + Checkpointer scan approach - Add ThreadMetaRepository.search() with metadata/status filters - Add ThreadMetaRepository.update_display_name() for title sync - Worker syncs checkpoint title to threads_meta.display_name on run completion - Map display_name to values.title in search response for API compatibility Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(threads): history endpoint reads messages from event store - POST /api/threads/{thread_id}/history now combines two data sources: checkpointer for checkpoint_id, metadata, title, thread_data; event store for messages (complete history, not truncated by summarization) - Strip internal LangGraph metadata keys from response - Remove full channel_values serialization in favor of selective fields Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove duplicate optional-dependencies header in pyproject.toml Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(middleware): pass tagged config to TitleMiddleware ainvoke call Without the config, the middleware:title tag was not injected, causing the LLM response to be recorded as a lead_agent ai_message in run_events. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: resolve merge conflict in .env.example Keep both DATABASE_URL (from persistence-scaffold) and WECOM credentials (from main) after the merge. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(persistence): address review feedback on PR #1851 - Fix naive datetime.now() → datetime.now(UTC) in all ORM models - Fix seq race condition in DbRunEventStore.put() with FOR UPDATE and UNIQUE(thread_id, seq) constraint - Encapsulate _store access in RunManager.update_run_completion() - Deduplicate _store.put() logic in RunManager via _persist_to_store() - Add update_run_completion to RunStore ABC + MemoryRunStore - Wire follow_up_to_run_id through the full create path - Add error recovery to RunJournal._flush_sync() lost-event scenario - Add migration note for search_threads breaking change - Fix test_checkpointer_none_fix mock to set database=None Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update uv.lock Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(persistence): address 22 review comments from CodeQL, Copilot, and Code Quality Bug fixes: - Sanitize log params to prevent log injection (CodeQL) - Reset threads_meta.status to idle/error when run completes - Attach messages only to latest checkpoint in /history response - Write threads_meta on POST /threads so new threads appear in search Lint fixes: - Remove unused imports (journal.py, migrations/env.py, test_converters.py) - Convert lambda to named function (engine.py, Ruff E731) - Remove unused logger definitions in repos (Ruff F841) - Add logging to JSONL decode errors and empty except blocks - Separate assert side-effects in tests (CodeQL) - Remove unused local variables in tests (Ruff F841) - Fix max_trace_content truncation to use byte length, not char length Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: apply ruff format to persistence and runtime files Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Potential fix for pull request finding 'Statement has no effect' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> * refactor(runtime): introduce RunContext to reduce run_agent parameter bloat Extract checkpointer, store, event_store, run_events_config, thread_meta_repo, and follow_up_to_run_id into a frozen RunContext dataclass. Add get_run_context() in deps.py to build the base context from app.state singletons. start_run() uses dataclasses.replace() to enrich per-run fields before passing ctx to run_agent. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(gateway): move sanitize_log_param to app/gateway/utils.py Extract the log-injection sanitizer from routers/threads.py into a shared utils module and rename to sanitize_log_param (public API). Eliminates the reverse service → router import in services.py. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf: use SQL aggregation for feedback stats and thread token usage Replace Python-side counting in FeedbackRepository.aggregate_by_run with a single SELECT COUNT/SUM query. Add RunStore.aggregate_tokens_by_thread abstract method with SQL GROUP BY implementation in RunRepository and Python fallback in MemoryRunStore. Simplify the thread_token_usage endpoint to delegate to the new method, eliminating the limit=10000 truncation risk. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: annotate DbRunEventStore.put() as low-frequency path Add docstring clarifying that put() opens a per-call transaction with FOR UPDATE and should only be used for infrequent writes (currently just the initial human_message event). High-throughput callers should use put_batch() instead. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(threads): fall back to Store search when ThreadMetaRepository is unavailable When database.backend=memory (default) or no SQL session factory is configured, search_threads now queries the LangGraph Store instead of returning 503. Returns empty list if neither Store nor repo is available. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(persistence): introduce ThreadMetaStore ABC for backend-agnostic thread metadata Add ThreadMetaStore abstract base class with create/get/search/update/delete interface. ThreadMetaRepository (SQL) now inherits from it. New MemoryThreadMetaStore wraps LangGraph BaseStore for memory-mode deployments. deps.py now always provides a non-None thread_meta_repo, eliminating all `if thread_meta_repo is not None` guards in services.py, worker.py, and routers/threads.py. search_threads no longer needs a Store fallback branch. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(history): read messages from checkpointer instead of RunEventStore The /history endpoint now reads messages directly from the checkpointer's channel_values (the authoritative source) instead of querying RunEventStore.list_messages(). The RunEventStore API is preserved for other consumers. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(persistence): address new Copilot review comments - feedback.py: validate thread_id/run_id before deleting feedback - jsonl.py: add path traversal protection with ID validation - run_repo.py: parse `before` to datetime for PostgreSQL compat - thread_meta_repo.py: fix pagination when metadata filter is active - database_config.py: use resolve_path for sqlite_dir consistency Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Implement skill self-evolution and skill_manage flow (#1874) * chore: ignore .worktrees directory * Add skill_manage self-evolution flow * Fix CI regressions for skill_manage * Address PR review feedback for skill evolution * fix(skill-evolution): preserve history on delete * fix(skill-evolution): tighten scanner fallbacks * docs: add skill_manage e2e evidence screenshot * fix(skill-manage): avoid blocking fs ops in session runtime --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> * fix(config): resolve sqlite_dir relative to CWD, not Paths.base_dir resolve_path() resolves relative to Paths.base_dir (.deer-flow), which double-nested the path to .deer-flow/.deer-flow/data/app.db. Use Path.resolve() (CWD-relative) instead. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Feature/feishu receive file (#1608) * feat(feishu): add channel file materialization hook for inbound messages - Introduce Channel.receive_file(msg, thread_id) as a base method for file materialization; default is no-op. - Implement FeishuChannel.receive_file to download files/images from Feishu messages, save to sandbox, and inject virtual paths into msg.text. - Update ChannelManager to call receive_file for any channel if msg.files is present, enabling downstream model access to user-uploaded files. - No impact on Slack/Telegram or other channels (they inherit the default no-op). * style(backend): format code with ruff for lint compliance - Auto-formatted packages/harness/deerflow/agents/factory.py and tests/test_create_deerflow_agent.py using `ruff format` - Ensured both files conform to project linting standards - Fixes CI lint check failures caused by code style issues * fix(feishu): handle file write operation asynchronously to prevent blocking * fix(feishu): rename GetMessageResourceRequest to _GetMessageResourceRequest and remove redundant code * test(feishu): add tests for receive_file method and placeholder replacement * fix(manager): remove unnecessary type casting for channel retrieval * fix(feishu): update logging messages to reflect resource handling instead of image * fix(feishu): sanitize filename by replacing invalid characters in file uploads * fix(feishu): improve filename sanitization and reorder image key handling in message processing * fix(feishu): add thread lock to prevent filename conflicts during file downloads * fix(test): correct bad merge in test_feishu_parser.py * chore: run ruff and apply formatting cleanup fix(feishu): preserve rich-text attachment order and improve fallback filename handling * fix(docker): restore gateway env vars and fix langgraph empty arg issue (#1915) Two production docker-compose.yaml bugs prevent `make up` from working: 1. Gateway missing DEER_FLOW_CONFIG_PATH and DEER_FLOW_EXTENSIONS_CONFIG_PATH environment overrides. Added infb2d99f(#1836) but accidentally reverted byca2fb95(#1847). Without them, gateway reads host paths from .env via env_file, causing FileNotFoundError inside the container. 2. Langgraph command fails when LANGGRAPH_ALLOW_BLOCKING is unset (default). Empty $${allow_blocking} inserts a bare space between flags, causing ' --no-reload' to be parsed as unexpected extra argument. Fix by building args string first and conditionally appending --allow-blocking. Co-authored-by: cooper <cooperfu@tencent.com> * fix(frontend): resolve invalid HTML nesting and tabnabbing vulnerabilities (#1904) * fix(frontend): resolve invalid HTML nesting and tabnabbing vulnerabilities Fix `<button>` inside `<a>` invalid HTML in artifact components and add missing `noopener,noreferrer` to `window.open` calls to prevent reverse tabnabbing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(frontend): address Copilot review on tabnabbing and double-tab-open Remove redundant parent onClick on web_fetch ChainOfThoughtStep to prevent opening two tabs on link click, and explicitly null out window.opener after window.open() for defensive tabnabbing hardening. --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * refactor(persistence): organize entities into per-entity directories Restructure the persistence layer from horizontal "models/ + repositories/" split into vertical entity-aligned directories. Each entity (thread_meta, run, feedback) now owns its ORM model, abstract interface (where applicable), and concrete implementations under a single directory with an aggregating __init__.py for one-line imports. Layout: persistence/thread_meta/{base,model,sql,memory}.py persistence/run/{model,sql}.py persistence/feedback/{model,sql}.py models/__init__.py is kept as a facade so Alembic autogenerate continues to discover all ORM tables via Base.metadata. RunEventRow remains under models/run_event.py because its storage implementation lives in runtime/events/store/db.py and has no matching repository directory. The repositories/ directory is removed entirely. All call sites in gateway/deps.py and tests are updated to import from the new entity packages, e.g.: from deerflow.persistence.thread_meta import ThreadMetaRepository from deerflow.persistence.run import RunRepository from deerflow.persistence.feedback import FeedbackRepository Full test suite passes (1690 passed, 14 skipped). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(gateway): sync thread rename and delete through ThreadMetaStore The POST /threads/{id}/state endpoint previously synced title changes only to the LangGraph Store via _store_upsert. In sqlite mode the search endpoint reads from the ThreadMetaRepository SQL table, so renames never appeared in /threads/search until the next agent run completed (worker.py syncs title from checkpoint to thread_meta in its finally block). Likewise the DELETE /threads/{id} endpoint cleaned up the filesystem, Store, and checkpointer but left the threads_meta row orphaned in sqlite, so deleted threads kept appearing in /threads/search. Fix both endpoints by routing through the ThreadMetaStore abstraction which already has the correct sqlite/memory implementations wired up by deps.py. The rename path now calls update_display_name() and the delete path calls delete() — both work uniformly across backends. Verified end-to-end with curl in gateway mode against sqlite backend. Existing test suite (1690 passed) and focused router/repo tests pass. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(gateway): route all thread metadata access through ThreadMetaStore Following the rename/delete bug fix in PR1, migrate the remaining direct LangGraph Store reads/writes in the threads router and services to the ThreadMetaStore abstraction so that the sqlite and memory backends behave identically and the legacy dual-write paths can be removed. Migrated endpoints (threads.py): - create_thread: idempotency check + write now use thread_meta_repo.get/create instead of dual-writing the LangGraph Store and the SQL row. - get_thread: reads from thread_meta_repo.get; the checkpoint-only fallback for legacy threads is preserved. - patch_thread: replaced _store_get/_store_put with thread_meta_repo.update_metadata. - delete_thread_data: dropped the legacy store.adelete; thread_meta_repo.delete already covers it. Removed dead code (services.py): - _upsert_thread_in_store — redundant with the immediately following thread_meta_repo.create() call. - _sync_thread_title_after_run — worker.py's finally block already syncs the title via thread_meta_repo.update_display_name() after each run. Removed dead code (threads.py): - _store_get / _store_put / _store_upsert helpers (no remaining callers). - THREADS_NS constant. - get_store import (router no longer touches the LangGraph Store directly). New abstract method: - ThreadMetaStore.update_metadata(thread_id, metadata) merges metadata into the thread's metadata field. Implemented in both ThreadMetaRepository (SQL, read-modify-write inside one session) and MemoryThreadMetaStore. Three new unit tests cover merge / empty / nonexistent behaviour. Net change: -134 lines. Full test suite: 1693 passed, 14 skipped. Verified end-to-end with curl in gateway mode against sqlite backend (create / patch / get / rename / search / delete). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> Co-authored-by: DanielWalnut <45447813+hetaoBackend@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> Co-authored-by: JilongSun <965640067@qq.com> Co-authored-by: jie <49781832+stan-fu@users.noreply.github.com> Co-authored-by: cooper <cooperfu@tencent.com> Co-authored-by: yangzheli <43645580+yangzheli@users.noreply.github.com>
602 lines
27 KiB
Python
602 lines
27 KiB
Python
import logging
|
|
from datetime import datetime
|
|
from functools import lru_cache
|
|
|
|
from deerflow.config.agents_config import load_agent_soul
|
|
from deerflow.skills import load_skills
|
|
from deerflow.subagents import get_available_subagent_names
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
|
|
def _get_enabled_skills():
|
|
try:
|
|
return list(load_skills(enabled_only=True))
|
|
except Exception:
|
|
logger.exception("Failed to load enabled skills for prompt injection")
|
|
return []
|
|
|
|
|
|
def _skill_mutability_label(category: str) -> str:
|
|
return "[custom, editable]" if category == "custom" else "[built-in]"
|
|
|
|
|
|
def clear_skills_system_prompt_cache() -> None:
|
|
_get_cached_skills_prompt_section.cache_clear()
|
|
|
|
|
|
def _build_skill_evolution_section(skill_evolution_enabled: bool) -> str:
|
|
if not skill_evolution_enabled:
|
|
return ""
|
|
return """
|
|
## Skill Self-Evolution
|
|
After completing a task, consider creating or updating a skill when:
|
|
- The task required 5+ tool calls to resolve
|
|
- You overcame non-obvious errors or pitfalls
|
|
- The user corrected your approach and the corrected version worked
|
|
- You discovered a non-trivial, recurring workflow
|
|
If you used a skill and encountered issues not covered by it, patch it immediately.
|
|
Prefer patch over edit. Before creating a new skill, confirm with the user first.
|
|
Skip simple one-off tasks.
|
|
"""
|
|
|
|
|
|
def _build_subagent_section(max_concurrent: int) -> str:
|
|
"""Build the subagent system prompt section with dynamic concurrency limit.
|
|
|
|
Args:
|
|
max_concurrent: Maximum number of concurrent subagent calls allowed per response.
|
|
|
|
Returns:
|
|
Formatted subagent section string.
|
|
"""
|
|
n = max_concurrent
|
|
bash_available = "bash" in get_available_subagent_names()
|
|
available_subagents = (
|
|
"- **general-purpose**: For ANY non-trivial task - web research, code exploration, file operations, analysis, etc.\n- **bash**: For command execution (git, build, test, deploy operations)"
|
|
if bash_available
|
|
else "- **general-purpose**: For ANY non-trivial task - web research, code exploration, file operations, analysis, etc.\n"
|
|
"- **bash**: Not available in the current sandbox configuration. Use direct file/web tools or switch to AioSandboxProvider for isolated shell access."
|
|
)
|
|
direct_tool_examples = "bash, ls, read_file, web_search, etc." if bash_available else "ls, read_file, web_search, etc."
|
|
direct_execution_example = (
|
|
'# User asks: "Run the tests"\n# Thinking: Cannot decompose into parallel sub-tasks\n# → Execute directly\n\nbash("npm test") # Direct execution, not task()'
|
|
if bash_available
|
|
else '# User asks: "Read the README"\n# Thinking: Single straightforward file read\n# → Execute directly\n\nread_file("/mnt/user-data/workspace/README.md") # Direct execution, not task()'
|
|
)
|
|
return f"""<subagent_system>
|
|
**🚀 SUBAGENT MODE ACTIVE - DECOMPOSE, DELEGATE, SYNTHESIZE**
|
|
|
|
You are running with subagent capabilities enabled. Your role is to be a **task orchestrator**:
|
|
1. **DECOMPOSE**: Break complex tasks into parallel sub-tasks
|
|
2. **DELEGATE**: Launch multiple subagents simultaneously using parallel `task` calls
|
|
3. **SYNTHESIZE**: Collect and integrate results into a coherent answer
|
|
|
|
**CORE PRINCIPLE: Complex tasks should be decomposed and distributed across multiple subagents for parallel execution.**
|
|
|
|
**⛔ HARD CONCURRENCY LIMIT: MAXIMUM {n} `task` CALLS PER RESPONSE. THIS IS NOT OPTIONAL.**
|
|
- Each response, you may include **at most {n}** `task` tool calls. Any excess calls are **silently discarded** by the system — you will lose that work.
|
|
- **Before launching subagents, you MUST count your sub-tasks in your thinking:**
|
|
- If count ≤ {n}: Launch all in this response.
|
|
- If count > {n}: **Pick the {n} most important/foundational sub-tasks for this turn.** Save the rest for the next turn.
|
|
- **Multi-batch execution** (for >{n} sub-tasks):
|
|
- Turn 1: Launch sub-tasks 1-{n} in parallel → wait for results
|
|
- Turn 2: Launch next batch in parallel → wait for results
|
|
- ... continue until all sub-tasks are complete
|
|
- Final turn: Synthesize ALL results into a coherent answer
|
|
- **Example thinking pattern**: "I identified 6 sub-tasks. Since the limit is {n} per turn, I will launch the first {n} now, and the rest in the next turn."
|
|
|
|
**Available Subagents:**
|
|
{available_subagents}
|
|
|
|
**Your Orchestration Strategy:**
|
|
|
|
✅ **DECOMPOSE + PARALLEL EXECUTION (Preferred Approach):**
|
|
|
|
For complex queries, break them down into focused sub-tasks and execute in parallel batches (max {n} per turn):
|
|
|
|
**Example 1: "Why is Tencent's stock price declining?" (3 sub-tasks → 1 batch)**
|
|
→ Turn 1: Launch 3 subagents in parallel:
|
|
- Subagent 1: Recent financial reports, earnings data, and revenue trends
|
|
- Subagent 2: Negative news, controversies, and regulatory issues
|
|
- Subagent 3: Industry trends, competitor performance, and market sentiment
|
|
→ Turn 2: Synthesize results
|
|
|
|
**Example 2: "Compare 5 cloud providers" (5 sub-tasks → multi-batch)**
|
|
→ Turn 1: Launch {n} subagents in parallel (first batch)
|
|
→ Turn 2: Launch remaining subagents in parallel
|
|
→ Final turn: Synthesize ALL results into comprehensive comparison
|
|
|
|
**Example 3: "Refactor the authentication system"**
|
|
→ Turn 1: Launch 3 subagents in parallel:
|
|
- Subagent 1: Analyze current auth implementation and technical debt
|
|
- Subagent 2: Research best practices and security patterns
|
|
- Subagent 3: Review related tests, documentation, and vulnerabilities
|
|
→ Turn 2: Synthesize results
|
|
|
|
✅ **USE Parallel Subagents (max {n} per turn) when:**
|
|
- **Complex research questions**: Requires multiple information sources or perspectives
|
|
- **Multi-aspect analysis**: Task has several independent dimensions to explore
|
|
- **Large codebases**: Need to analyze different parts simultaneously
|
|
- **Comprehensive investigations**: Questions requiring thorough coverage from multiple angles
|
|
|
|
❌ **DO NOT use subagents (execute directly) when:**
|
|
- **Task cannot be decomposed**: If you can't break it into 2+ meaningful parallel sub-tasks, execute directly
|
|
- **Ultra-simple actions**: Read one file, quick edits, single commands
|
|
- **Need immediate clarification**: Must ask user before proceeding
|
|
- **Meta conversation**: Questions about conversation history
|
|
- **Sequential dependencies**: Each step depends on previous results (do steps yourself sequentially)
|
|
|
|
**CRITICAL WORKFLOW** (STRICTLY follow this before EVERY action):
|
|
1. **COUNT**: In your thinking, list all sub-tasks and count them explicitly: "I have N sub-tasks"
|
|
2. **PLAN BATCHES**: If N > {n}, explicitly plan which sub-tasks go in which batch:
|
|
- "Batch 1 (this turn): first {n} sub-tasks"
|
|
- "Batch 2 (next turn): next batch of sub-tasks"
|
|
3. **EXECUTE**: Launch ONLY the current batch (max {n} `task` calls). Do NOT launch sub-tasks from future batches.
|
|
4. **REPEAT**: After results return, launch the next batch. Continue until all batches complete.
|
|
5. **SYNTHESIZE**: After ALL batches are done, synthesize all results.
|
|
6. **Cannot decompose** → Execute directly using available tools ({direct_tool_examples})
|
|
|
|
**⛔ VIOLATION: Launching more than {n} `task` calls in a single response is a HARD ERROR. The system WILL discard excess calls and you WILL lose work. Always batch.**
|
|
|
|
**Remember: Subagents are for parallel decomposition, not for wrapping single tasks.**
|
|
|
|
**How It Works:**
|
|
- The task tool runs subagents asynchronously in the background
|
|
- The backend automatically polls for completion (you don't need to poll)
|
|
- The tool call will block until the subagent completes its work
|
|
- Once complete, the result is returned to you directly
|
|
|
|
**Usage Example 1 - Single Batch (≤{n} sub-tasks):**
|
|
|
|
```python
|
|
# User asks: "Why is Tencent's stock price declining?"
|
|
# Thinking: 3 sub-tasks → fits in 1 batch
|
|
|
|
# Turn 1: Launch 3 subagents in parallel
|
|
task(description="Tencent financial data", prompt="...", subagent_type="general-purpose")
|
|
task(description="Tencent news & regulation", prompt="...", subagent_type="general-purpose")
|
|
task(description="Industry & market trends", prompt="...", subagent_type="general-purpose")
|
|
# All 3 run in parallel → synthesize results
|
|
```
|
|
|
|
**Usage Example 2 - Multiple Batches (>{n} sub-tasks):**
|
|
|
|
```python
|
|
# User asks: "Compare AWS, Azure, GCP, Alibaba Cloud, and Oracle Cloud"
|
|
# Thinking: 5 sub-tasks → need multiple batches (max {n} per batch)
|
|
|
|
# Turn 1: Launch first batch of {n}
|
|
task(description="AWS analysis", prompt="...", subagent_type="general-purpose")
|
|
task(description="Azure analysis", prompt="...", subagent_type="general-purpose")
|
|
task(description="GCP analysis", prompt="...", subagent_type="general-purpose")
|
|
|
|
# Turn 2: Launch remaining batch (after first batch completes)
|
|
task(description="Alibaba Cloud analysis", prompt="...", subagent_type="general-purpose")
|
|
task(description="Oracle Cloud analysis", prompt="...", subagent_type="general-purpose")
|
|
|
|
# Turn 3: Synthesize ALL results from both batches
|
|
```
|
|
|
|
**Counter-Example - Direct Execution (NO subagents):**
|
|
|
|
```python
|
|
{direct_execution_example}
|
|
```
|
|
|
|
**CRITICAL**:
|
|
- **Max {n} `task` calls per turn** - the system enforces this, excess calls are discarded
|
|
- Only use `task` when you can launch 2+ subagents in parallel
|
|
- Single task = No value from subagents = Execute directly
|
|
- For >{n} sub-tasks, use sequential batches of {n} across multiple turns
|
|
</subagent_system>"""
|
|
|
|
|
|
SYSTEM_PROMPT_TEMPLATE = """
|
|
<role>
|
|
You are {agent_name}, an open-source super agent.
|
|
</role>
|
|
|
|
{soul}
|
|
{memory_context}
|
|
|
|
<thinking_style>
|
|
- Think concisely and strategically about the user's request BEFORE taking action
|
|
- Break down the task: What is clear? What is ambiguous? What is missing?
|
|
- **PRIORITY CHECK: If anything is unclear, missing, or has multiple interpretations, you MUST ask for clarification FIRST - do NOT proceed with work**
|
|
{subagent_thinking}- Never write down your full final answer or report in thinking process, but only outline
|
|
- CRITICAL: After thinking, you MUST provide your actual response to the user. Thinking is for planning, the response is for delivery.
|
|
- Your response must contain the actual answer, not just a reference to what you thought about
|
|
</thinking_style>
|
|
|
|
<clarification_system>
|
|
**WORKFLOW PRIORITY: CLARIFY → PLAN → ACT**
|
|
1. **FIRST**: Analyze the request in your thinking - identify what's unclear, missing, or ambiguous
|
|
2. **SECOND**: If clarification is needed, call `ask_clarification` tool IMMEDIATELY - do NOT start working
|
|
3. **THIRD**: Only after all clarifications are resolved, proceed with planning and execution
|
|
|
|
**CRITICAL RULE: Clarification ALWAYS comes BEFORE action. Never start working and clarify mid-execution.**
|
|
|
|
**MANDATORY Clarification Scenarios - You MUST call ask_clarification BEFORE starting work when:**
|
|
|
|
1. **Missing Information** (`missing_info`): Required details not provided
|
|
- Example: User says "create a web scraper" but doesn't specify the target website
|
|
- Example: "Deploy the app" without specifying environment
|
|
- **REQUIRED ACTION**: Call ask_clarification to get the missing information
|
|
|
|
2. **Ambiguous Requirements** (`ambiguous_requirement`): Multiple valid interpretations exist
|
|
- Example: "Optimize the code" could mean performance, readability, or memory usage
|
|
- Example: "Make it better" is unclear what aspect to improve
|
|
- **REQUIRED ACTION**: Call ask_clarification to clarify the exact requirement
|
|
|
|
3. **Approach Choices** (`approach_choice`): Several valid approaches exist
|
|
- Example: "Add authentication" could use JWT, OAuth, session-based, or API keys
|
|
- Example: "Store data" could use database, files, cache, etc.
|
|
- **REQUIRED ACTION**: Call ask_clarification to let user choose the approach
|
|
|
|
4. **Risky Operations** (`risk_confirmation`): Destructive actions need confirmation
|
|
- Example: Deleting files, modifying production configs, database operations
|
|
- Example: Overwriting existing code or data
|
|
- **REQUIRED ACTION**: Call ask_clarification to get explicit confirmation
|
|
|
|
5. **Suggestions** (`suggestion`): You have a recommendation but want approval
|
|
- Example: "I recommend refactoring this code. Should I proceed?"
|
|
- **REQUIRED ACTION**: Call ask_clarification to get approval
|
|
|
|
**STRICT ENFORCEMENT:**
|
|
- ❌ DO NOT start working and then ask for clarification mid-execution - clarify FIRST
|
|
- ❌ DO NOT skip clarification for "efficiency" - accuracy matters more than speed
|
|
- ❌ DO NOT make assumptions when information is missing - ALWAYS ask
|
|
- ❌ DO NOT proceed with guesses - STOP and call ask_clarification first
|
|
- ✅ Analyze the request in thinking → Identify unclear aspects → Ask BEFORE any action
|
|
- ✅ If you identify the need for clarification in your thinking, you MUST call the tool IMMEDIATELY
|
|
- ✅ After calling ask_clarification, execution will be interrupted automatically
|
|
- ✅ Wait for user response - do NOT continue with assumptions
|
|
|
|
**How to Use:**
|
|
```python
|
|
ask_clarification(
|
|
question="Your specific question here?",
|
|
clarification_type="missing_info", # or other type
|
|
context="Why you need this information", # optional but recommended
|
|
options=["option1", "option2"] # optional, for choices
|
|
)
|
|
```
|
|
|
|
**Example:**
|
|
User: "Deploy the application"
|
|
You (thinking): Missing environment info - I MUST ask for clarification
|
|
You (action): ask_clarification(
|
|
question="Which environment should I deploy to?",
|
|
clarification_type="approach_choice",
|
|
context="I need to know the target environment for proper configuration",
|
|
options=["development", "staging", "production"]
|
|
)
|
|
[Execution stops - wait for user response]
|
|
|
|
User: "staging"
|
|
You: "Deploying to staging..." [proceed]
|
|
</clarification_system>
|
|
|
|
{skills_section}
|
|
|
|
{deferred_tools_section}
|
|
|
|
{subagent_section}
|
|
|
|
<working_directory existed="true">
|
|
- User uploads: `/mnt/user-data/uploads` - Files uploaded by the user (automatically listed in context)
|
|
- User workspace: `/mnt/user-data/workspace` - Working directory for temporary files
|
|
- Output files: `/mnt/user-data/outputs` - Final deliverables must be saved here
|
|
|
|
**File Management:**
|
|
- Uploaded files are automatically listed in the <uploaded_files> section before each request
|
|
- Use `read_file` tool to read uploaded files using their paths from the list
|
|
- For PDF, PPT, Excel, and Word files, converted Markdown versions (*.md) are available alongside originals
|
|
- All temporary work happens in `/mnt/user-data/workspace`
|
|
- Final deliverables must be copied to `/mnt/user-data/outputs` and presented using `present_file` tool
|
|
{acp_section}
|
|
</working_directory>
|
|
|
|
<response_style>
|
|
- Clear and Concise: Avoid over-formatting unless requested
|
|
- Natural Tone: Use paragraphs and prose, not bullet points by default
|
|
- Action-Oriented: Focus on delivering results, not explaining processes
|
|
</response_style>
|
|
|
|
<citations>
|
|
**CRITICAL: Always include citations when using web search results**
|
|
|
|
- **When to Use**: MANDATORY after web_search, web_fetch, or any external information source
|
|
- **Format**: Use Markdown link format `[citation:TITLE](URL)` immediately after the claim
|
|
- **Placement**: Inline citations should appear right after the sentence or claim they support
|
|
- **Sources Section**: Also collect all citations in a "Sources" section at the end of reports
|
|
|
|
**Example - Inline Citations:**
|
|
```markdown
|
|
The key AI trends for 2026 include enhanced reasoning capabilities and multimodal integration
|
|
[citation:AI Trends 2026](https://techcrunch.com/ai-trends).
|
|
Recent breakthroughs in language models have also accelerated progress
|
|
[citation:OpenAI Research](https://openai.com/research).
|
|
```
|
|
|
|
**Example - Deep Research Report with Citations:**
|
|
```markdown
|
|
## Executive Summary
|
|
|
|
DeerFlow is an open-source AI agent framework that gained significant traction in early 2026
|
|
[citation:GitHub Repository](https://github.com/bytedance/deer-flow). The project focuses on
|
|
providing a production-ready agent system with sandbox execution and memory management
|
|
[citation:DeerFlow Documentation](https://deer-flow.dev/docs).
|
|
|
|
## Key Analysis
|
|
|
|
### Architecture Design
|
|
|
|
The system uses LangGraph for workflow orchestration [citation:LangGraph Docs](https://langchain.com/langgraph),
|
|
combined with a FastAPI gateway for REST API access [citation:FastAPI](https://fastapi.tiangolo.com).
|
|
|
|
## Sources
|
|
|
|
### Primary Sources
|
|
- [GitHub Repository](https://github.com/bytedance/deer-flow) - Official source code and documentation
|
|
- [DeerFlow Documentation](https://deer-flow.dev/docs) - Technical specifications
|
|
|
|
### Media Coverage
|
|
- [AI Trends 2026](https://techcrunch.com/ai-trends) - Industry analysis
|
|
```
|
|
|
|
**CRITICAL: Sources section format:**
|
|
- Every item in the Sources section MUST be a clickable markdown link with URL
|
|
- Use standard markdown link `[Title](URL) - Description` format (NOT `[citation:...]` format)
|
|
- The `[citation:Title](URL)` format is ONLY for inline citations within the report body
|
|
- ❌ WRONG: `GitHub 仓库 - 官方源代码和文档` (no URL!)
|
|
- ❌ WRONG in Sources: `[citation:GitHub Repository](url)` (citation prefix is for inline only!)
|
|
- ✅ RIGHT in Sources: `[GitHub Repository](https://github.com/bytedance/deer-flow) - 官方源代码和文档`
|
|
|
|
**WORKFLOW for Research Tasks:**
|
|
1. Use web_search to find sources → Extract {{title, url, snippet}} from results
|
|
2. Write content with inline citations: `claim [citation:Title](url)`
|
|
3. Collect all citations in a "Sources" section at the end
|
|
4. NEVER write claims without citations when sources are available
|
|
|
|
**CRITICAL RULES:**
|
|
- ❌ DO NOT write research content without citations
|
|
- ❌ DO NOT forget to extract URLs from search results
|
|
- ✅ ALWAYS add `[citation:Title](URL)` after claims from external sources
|
|
- ✅ ALWAYS include a "Sources" section listing all references
|
|
</citations>
|
|
|
|
<critical_reminders>
|
|
- **Clarification First**: ALWAYS clarify unclear/missing/ambiguous requirements BEFORE starting work - never assume or guess
|
|
{subagent_reminder}- Skill First: Always load the relevant skill before starting **complex** tasks.
|
|
- Progressive Loading: Load resources incrementally as referenced in skills
|
|
- Output Files: Final deliverables must be in `/mnt/user-data/outputs`
|
|
- Clarity: Be direct and helpful, avoid unnecessary meta-commentary
|
|
- Including Images and Mermaid: Images and Mermaid diagrams are always welcomed in the Markdown format, and you're encouraged to use `\n\n` or "```mermaid" to display images in response or Markdown files
|
|
- Multi-task: Better utilize parallel tool calling to call multiple tools at one time for better performance
|
|
- Language Consistency: Keep using the same language as user's
|
|
- Always Respond: Your thinking is internal. You MUST always provide a visible response to the user after thinking.
|
|
</critical_reminders>
|
|
"""
|
|
|
|
|
|
def _get_memory_context(agent_name: str | None = None) -> str:
|
|
"""Get memory context for injection into system prompt.
|
|
|
|
Args:
|
|
agent_name: If provided, loads per-agent memory. If None, loads global memory.
|
|
|
|
Returns:
|
|
Formatted memory context string wrapped in XML tags, or empty string if disabled.
|
|
"""
|
|
try:
|
|
from deerflow.agents.memory import format_memory_for_injection, get_memory_data
|
|
from deerflow.config.memory_config import get_memory_config
|
|
|
|
config = get_memory_config()
|
|
if not config.enabled or not config.injection_enabled:
|
|
return ""
|
|
|
|
memory_data = get_memory_data(agent_name)
|
|
memory_content = format_memory_for_injection(memory_data, max_tokens=config.max_injection_tokens)
|
|
|
|
if not memory_content.strip():
|
|
return ""
|
|
|
|
return f"""<memory>
|
|
{memory_content}
|
|
</memory>
|
|
"""
|
|
except Exception as e:
|
|
logger.error("Failed to load memory context: %s", e)
|
|
return ""
|
|
|
|
|
|
@lru_cache(maxsize=32)
|
|
def _get_cached_skills_prompt_section(
|
|
skill_signature: tuple[tuple[str, str, str, str], ...],
|
|
available_skills_key: tuple[str, ...] | None,
|
|
container_base_path: str,
|
|
skill_evolution_section: str,
|
|
) -> str:
|
|
filtered = [(name, description, category, location) for name, description, category, location in skill_signature if available_skills_key is None or name in available_skills_key]
|
|
skills_list = ""
|
|
if filtered:
|
|
skill_items = "\n".join(
|
|
f" <skill>\n <name>{name}</name>\n <description>{description} {_skill_mutability_label(category)}</description>\n <location>{location}</location>\n </skill>"
|
|
for name, description, category, location in filtered
|
|
)
|
|
skills_list = f"<available_skills>\n{skill_items}\n</available_skills>"
|
|
return f"""<skill_system>
|
|
You have access to skills that provide optimized workflows for specific tasks. Each skill contains best practices, frameworks, and references to additional resources.
|
|
|
|
**Progressive Loading Pattern:**
|
|
1. When a user query matches a skill's use case, immediately call `read_file` on the skill's main file using the path attribute provided in the skill tag below
|
|
2. Read and understand the skill's workflow and instructions
|
|
3. The skill file contains references to external resources under the same folder
|
|
4. Load referenced resources only when needed during execution
|
|
5. Follow the skill's instructions precisely
|
|
|
|
**Skills are located at:** {container_base_path}
|
|
{skill_evolution_section}
|
|
{skills_list}
|
|
|
|
</skill_system>"""
|
|
|
|
|
|
def get_skills_prompt_section(available_skills: set[str] | None = None) -> str:
|
|
"""Generate the skills prompt section with available skills list."""
|
|
skills = _get_enabled_skills()
|
|
|
|
try:
|
|
from deerflow.config import get_app_config
|
|
|
|
config = get_app_config()
|
|
container_base_path = config.skills.container_path
|
|
skill_evolution_enabled = config.skill_evolution.enabled
|
|
except Exception:
|
|
container_base_path = "/mnt/skills"
|
|
skill_evolution_enabled = False
|
|
|
|
if not skills and not skill_evolution_enabled:
|
|
return ""
|
|
|
|
if available_skills is not None and not any(skill.name in available_skills for skill in skills):
|
|
return ""
|
|
|
|
skill_signature = tuple((skill.name, skill.description, skill.category, skill.get_container_file_path(container_base_path)) for skill in skills)
|
|
available_key = tuple(sorted(available_skills)) if available_skills is not None else None
|
|
if not skill_signature and available_key is not None:
|
|
return ""
|
|
skill_evolution_section = _build_skill_evolution_section(skill_evolution_enabled)
|
|
return _get_cached_skills_prompt_section(skill_signature, available_key, container_base_path, skill_evolution_section)
|
|
|
|
|
|
def get_agent_soul(agent_name: str | None) -> str:
|
|
# Append SOUL.md (agent personality) if present
|
|
soul = load_agent_soul(agent_name)
|
|
if soul:
|
|
return f"<soul>\n{soul}\n</soul>\n" if soul else ""
|
|
return ""
|
|
|
|
|
|
def get_deferred_tools_prompt_section() -> str:
|
|
"""Generate <available-deferred-tools> block for the system prompt.
|
|
|
|
Lists only deferred tool names so the agent knows what exists
|
|
and can use tool_search to load them.
|
|
Returns empty string when tool_search is disabled or no tools are deferred.
|
|
"""
|
|
from deerflow.tools.builtins.tool_search import get_deferred_registry
|
|
|
|
try:
|
|
from deerflow.config import get_app_config
|
|
|
|
if not get_app_config().tool_search.enabled:
|
|
return ""
|
|
except Exception:
|
|
return ""
|
|
|
|
registry = get_deferred_registry()
|
|
if not registry:
|
|
return ""
|
|
|
|
names = "\n".join(e.name for e in registry.entries)
|
|
return f"<available-deferred-tools>\n{names}\n</available-deferred-tools>"
|
|
|
|
|
|
def _build_acp_section() -> str:
|
|
"""Build the ACP agent prompt section, only if ACP agents are configured."""
|
|
try:
|
|
from deerflow.config.acp_config import get_acp_agents
|
|
|
|
agents = get_acp_agents()
|
|
if not agents:
|
|
return ""
|
|
except Exception:
|
|
return ""
|
|
|
|
return (
|
|
"\n**ACP Agent Tasks (invoke_acp_agent):**\n"
|
|
"- ACP agents (e.g. codex, claude_code) run in their own independent workspace — NOT in `/mnt/user-data/`\n"
|
|
"- When writing prompts for ACP agents, describe the task only — do NOT reference `/mnt/user-data` paths\n"
|
|
"- ACP agent results are accessible at `/mnt/acp-workspace/` (read-only) — use `ls`, `read_file`, or `bash cp` to retrieve output files\n"
|
|
"- To deliver ACP output to the user: copy from `/mnt/acp-workspace/<file>` to `/mnt/user-data/outputs/<file>`, then use `present_file`"
|
|
)
|
|
|
|
|
|
def _build_custom_mounts_section() -> str:
|
|
"""Build a prompt section for explicitly configured sandbox mounts."""
|
|
try:
|
|
from deerflow.config import get_app_config
|
|
|
|
mounts = get_app_config().sandbox.mounts or []
|
|
except Exception:
|
|
logger.exception("Failed to load configured sandbox mounts for the lead-agent prompt")
|
|
return ""
|
|
|
|
if not mounts:
|
|
return ""
|
|
|
|
lines = []
|
|
for mount in mounts:
|
|
access = "read-only" if mount.read_only else "read-write"
|
|
lines.append(f"- Custom mount: `{mount.container_path}` - Host directory mapped into the sandbox ({access})")
|
|
|
|
mounts_list = "\n".join(lines)
|
|
return f"\n**Custom Mounted Directories:**\n{mounts_list}\n- If the user needs files outside `/mnt/user-data`, use these absolute container paths directly when they match the requested directory"
|
|
|
|
|
|
def apply_prompt_template(subagent_enabled: bool = False, max_concurrent_subagents: int = 3, *, agent_name: str | None = None, available_skills: set[str] | None = None) -> str:
|
|
# Get memory context
|
|
memory_context = _get_memory_context(agent_name)
|
|
|
|
# Include subagent section only if enabled (from runtime parameter)
|
|
n = max_concurrent_subagents
|
|
subagent_section = _build_subagent_section(n) if subagent_enabled else ""
|
|
|
|
# Add subagent reminder to critical_reminders if enabled
|
|
subagent_reminder = (
|
|
"- **Orchestrator Mode**: You are a task orchestrator - decompose complex tasks into parallel sub-tasks. "
|
|
f"**HARD LIMIT: max {n} `task` calls per response.** "
|
|
f"If >{n} sub-tasks, split into sequential batches of ≤{n}. Synthesize after ALL batches complete.\n"
|
|
if subagent_enabled
|
|
else ""
|
|
)
|
|
|
|
# Add subagent thinking guidance if enabled
|
|
subagent_thinking = (
|
|
"- **DECOMPOSITION CHECK: Can this task be broken into 2+ parallel sub-tasks? If YES, COUNT them. "
|
|
f"If count > {n}, you MUST plan batches of ≤{n} and only launch the FIRST batch now. "
|
|
f"NEVER launch more than {n} `task` calls in one response.**\n"
|
|
if subagent_enabled
|
|
else ""
|
|
)
|
|
|
|
# Get skills section
|
|
skills_section = get_skills_prompt_section(available_skills)
|
|
|
|
# Get deferred tools section (tool_search)
|
|
deferred_tools_section = get_deferred_tools_prompt_section()
|
|
|
|
# Build ACP agent section only if ACP agents are configured
|
|
acp_section = _build_acp_section()
|
|
custom_mounts_section = _build_custom_mounts_section()
|
|
acp_and_mounts_section = "\n".join(section for section in (acp_section, custom_mounts_section) if section)
|
|
|
|
# Format the prompt with dynamic skills and memory
|
|
prompt = SYSTEM_PROMPT_TEMPLATE.format(
|
|
agent_name=agent_name or "DeerFlow 2.0",
|
|
soul=get_agent_soul(agent_name),
|
|
skills_section=skills_section,
|
|
deferred_tools_section=deferred_tools_section,
|
|
memory_context=memory_context,
|
|
subagent_section=subagent_section,
|
|
subagent_reminder=subagent_reminder,
|
|
subagent_thinking=subagent_thinking,
|
|
acp_section=acp_and_mounts_section,
|
|
)
|
|
|
|
return prompt + f"\n<current_date>{datetime.now().strftime('%Y-%m-%d, %A')}</current_date>"
|