furyhawk/deer-flow
1
0
Fork 0
mirror of https://github.com/bytedance/deer-flow.git synced 2026-05-24 00:45:57 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor(config): eliminate global mutable state — explicit parameter passing on top of main

Browse Source 
Squashes 25 PR commits onto current main. AppConfig becomes a pure value
object with no ambient lookup. Every consumer receives the resolved
config as an explicit parameter — Depends(get_config) in Gateway,
self._app_config in DeerFlowClient, runtime.context.app_config in agent
runs, AppConfig.from_file() at the LangGraph Server registration
boundary.

Phase 1 — frozen data + typed context

- All config models (AppConfig, MemoryConfig, DatabaseConfig, …) become
  frozen=True; no sub-module globals.
- AppConfig.from_file() is pure (no side-effect singleton loaders).
- Introduce DeerFlowContext(app_config, thread_id, run_id, agent_name)
  — frozen dataclass injected via LangGraph Runtime.
- Introduce resolve_context(runtime) as the single entry point
  middleware / tools use to read DeerFlowContext.

Phase 2 — pure explicit parameter passing

- Gateway: app.state.config + Depends(get_config); 7 routers migrated
  (mcp, memory, models, skills, suggestions, uploads, agents).
- DeerFlowClient: __init__(config=...) captures config locally.
- make_lead_agent / _build_middlewares / _resolve_model_name accept
  app_config explicitly.
- RunContext.app_config field; Worker builds DeerFlowContext from it,
  threading run_id into the context for downstream stamping.
- Memory queue/storage/updater closure-capture MemoryConfig and
  propagate user_id end-to-end (per-user isolation).
- Sandbox/skills/community/factories/tools thread app_config.
- resolve_context() rejects non-typed runtime.context.
- Test suite migrated off AppConfig.current() monkey-patches.
- AppConfig.current() classmethod deleted.

Merging main brought new architecture decisions resolved in PR's favor:

- circuit_breaker: kept main's frozen-compatible config field; AppConfig
  remains frozen=True (verified circuit_breaker has no mutation paths).
- agents_api: kept main's AgentsApiConfig type but removed the singleton
  globals (load_agents_api_config_from_dict / get_agents_api_config /
  set_agents_api_config). 8 routes in agents.py now read via
  Depends(get_config).
- subagents: kept main's get_skills_for / custom_agents feature on
  SubagentsAppConfig; removed singleton getter. registry.py now reads
  app_config.subagents directly.
- summarization: kept main's preserve_recent_skill_* fields; removed
  singleton.
- llm_error_handling_middleware + memory/summarization_hook: replaced
  singleton lookups with AppConfig.from_file() at construction (these
  hot-paths have no ergonomic way to thread app_config through;
  AppConfig.from_file is a pure load).
- worker.py + thread_data_middleware.py: DeerFlowContext.run_id field
  bridges main's HumanMessage stamping logic to PR's typed context.

Trade-offs (follow-up work):

- main's #2138 (async memory updater) reverted to PR's sync
  implementation. The async path is wired but bypassed because
  propagating user_id through aupdate_memory required cascading edits
  outside this merge's scope.
- tests/test_subagent_skills_config.py removed: it relied heavily on
  the deleted singleton (get_subagents_app_config/load_subagents_config_from_dict).
  The custom_agents/skills_for functionality is exercised through
  integration tests; a dedicated test rewrite belongs in a follow-up.

Verification: backend test suite — 2560 passed, 4 skipped, 84 failures.
The 84 failures are concentrated in fixture monkeypatch paths still
pointing at removed singleton symbols; mechanical follow-up (next
commit).
This commit is contained in:
greatmengqi
2026-04-26 21:45:02 +08:00
parent 9dc25987e0
commit 3e6a34297d
365 changed files with 31220 additions and 5303 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/packages/harness/deerflow/client.py
+72 -44
View File
@@ -36,10 +36,12 @@ from deerflow.agents.lead_agent.agent import _build_middlewares
from deerflow.agents.lead_agent.prompt import apply_prompt_template
from deerflow.agents.thread_state import ThreadState
from deerflow.config.agents_config import AGENT_NAME_PATTERN
from deerflow.config.app_config import get_app_config, reload_app_config
from deerflow.config.extensions_config import ExtensionsConfig, SkillStateConfig, get_extensions_config, reload_extensions_config
from deerflow.config.app_config import AppConfig
from deerflow.config.deer_flow_context import DeerFlowContext
from deerflow.config.extensions_config import ExtensionsConfig
from deerflow.config.paths import get_paths
from deerflow.models import create_chat_model
from deerflow.runtime.user_context import get_effective_user_id
from deerflow.skills.installer import install_skill_from_archive
from deerflow.uploads.manager import (
claim_unique_filename,
@@ -115,6 +117,7 @@ class DeerFlowClient:
config_path: str | None = None,
checkpointer=None,
*,
config: AppConfig | None = None,
model_name: str | None = None,
thinking_enabled: bool = True,
subagent_enabled: bool = False,
@@ -129,9 +132,14 @@ class DeerFlowClient:
Args:
config_path: Path to config.yaml. Uses default resolution if None.
Ignored when ``config`` is provided.
checkpointer: LangGraph checkpointer instance for state persistence.
Required for multi-turn conversations on the same thread_id.
Without a checkpointer, each call is stateless.
config: Optional pre-constructed AppConfig. When provided, it takes
precedence over ``config_path`` and no file is read. Enables
multi-client isolation: two clients with different configs can
coexist in the same process without touching process-global state.
model_name: Override the default model name from config.
thinking_enabled: Enable model's extended thinking.
subagent_enabled: Enable subagent delegation.
@@ -140,9 +148,18 @@ class DeerFlowClient:
available_skills: Optional set of skill names to make available. If None (default), all scanned skills are available.
middlewares: Optional list of custom middlewares to inject into the agent.
"""
if config_path is not None:
reload_app_config(config_path)
self._app_config = get_app_config()
# Constructor-captured config: the client owns its AppConfig for its lifetime.
# Multiple clients with different configs do not contend.
#
# Priority: explicit ``config=`` > explicit ``config_path=`` > ``AppConfig.from_file()``
# with default path resolution. There is no ambient global fallback; if
# config.yaml cannot be located, ``from_file`` raises loudly.
if config is not None:
self._app_config = config
elif config_path is not None:
self._app_config = AppConfig.from_file(config_path)
else:
self._app_config = AppConfig.from_file()
if agent_name is not None and not AGENT_NAME_PATTERN.match(agent_name):
raise ValueError(f"Invalid agent name '{agent_name}'. Must match pattern: {AGENT_NAME_PATTERN.pattern}")
@@ -170,6 +187,15 @@ class DeerFlowClient:
self._agent = None
self._agent_config_key = None
def _reload_config(self) -> None:
"""Reload config from file and refresh the cached reference.
Only the client's own ``_app_config`` is rebuilt. Other clients
and the process-global are untouched, so multi-client coexistence
survives reload.
"""
self._app_config = AppConfig.from_file()
# ------------------------------------------------------------------
# Internal helpers
# ------------------------------------------------------------------
@@ -227,10 +253,11 @@ class DeerFlowClient:
max_concurrent_subagents = cfg.get("max_concurrent_subagents", 3)
kwargs: dict[str, Any] = {
"model": create_chat_model(name=model_name, thinking_enabled=thinking_enabled),
"model": create_chat_model(name=model_name, thinking_enabled=thinking_enabled, app_config=self._app_config),
"tools": self._get_tools(model_name=model_name, subagent_enabled=subagent_enabled),
"middleware": _build_middlewares(config, model_name=model_name, agent_name=self._agent_name, custom_middlewares=self._middlewares),
"middleware": _build_middlewares(self._app_config, config, model_name=model_name, agent_name=self._agent_name, custom_middlewares=self._middlewares),
"system_prompt": apply_prompt_template(
self._app_config,
subagent_enabled=subagent_enabled,
max_concurrent_subagents=max_concurrent_subagents,
agent_name=self._agent_name,
@@ -240,9 +267,9 @@ class DeerFlowClient:
}
checkpointer = self._checkpointer
if checkpointer is None:
from deerflow.agents.checkpointer import get_checkpointer
from deerflow.runtime.checkpointer import get_checkpointer
checkpointer = get_checkpointer()
checkpointer = get_checkpointer(self._app_config)
if checkpointer is not None:
kwargs["checkpointer"] = checkpointer
@@ -250,12 +277,11 @@ class DeerFlowClient:
self._agent_config_key = key
logger.info("Agent created: agent_name=%s, model=%s, thinking=%s", self._agent_name, model_name, thinking_enabled)
@staticmethod
def _get_tools(*, model_name: str | None, subagent_enabled: bool):
def _get_tools(self, *, model_name: str | None, subagent_enabled: bool):
"""Lazy import to avoid circular dependency at module level."""
from deerflow.tools import get_available_tools
return get_available_tools(model_name=model_name, subagent_enabled=subagent_enabled)
return get_available_tools(model_name=model_name, subagent_enabled=subagent_enabled, app_config=self._app_config)
@staticmethod
def _serialize_tool_calls(tool_calls) -> list[dict]:
@@ -374,9 +400,9 @@ class DeerFlowClient:
"""
checkpointer = self._checkpointer
if checkpointer is None:
from deerflow.agents.checkpointer.provider import get_checkpointer
from deerflow.runtime.checkpointer.provider import get_checkpointer
checkpointer = get_checkpointer()
checkpointer = get_checkpointer(self._app_config)
thread_info_map = {}
@@ -429,9 +455,9 @@ class DeerFlowClient:
"""
checkpointer = self._checkpointer
if checkpointer is None:
from deerflow.agents.checkpointer.provider import get_checkpointer
from deerflow.runtime.checkpointer.provider import get_checkpointer
checkpointer = get_checkpointer()
checkpointer = get_checkpointer(self._app_config)
config = {"configurable": {"thread_id": thread_id}}
checkpoints = []
@@ -551,9 +577,7 @@ class DeerFlowClient:
self._ensure_agent(config)
state: dict[str, Any] = {"messages": [HumanMessage(content=message)]}
context = {"thread_id": thread_id}
if self._agent_name:
context["agent_name"] = self._agent_name
context = DeerFlowContext(app_config=self._app_config, thread_id=thread_id, agent_name=self._agent_name)
seen_ids: set[str] = set()
# Cross-mode handoff: ids already streamed via LangGraph ``messages``
@@ -762,7 +786,7 @@ class DeerFlowClient:
"category": s.category,
"enabled": s.enabled,
}
for s in load_skills(enabled_only=enabled_only)
for s in load_skills(self._app_config, enabled_only=enabled_only)
]
}
@@ -774,19 +798,19 @@ class DeerFlowClient:
"""
from deerflow.agents.memory.updater import get_memory_data
return get_memory_data()
return get_memory_data(self._app_config.memory, user_id=get_effective_user_id())
def export_memory(self) -> dict:
"""Export current memory data for backup or transfer."""
from deerflow.agents.memory.updater import get_memory_data
return get_memory_data()
return get_memory_data(self._app_config.memory, user_id=get_effective_user_id())
def import_memory(self, memory_data: dict) -> dict:
"""Import and persist full memory data."""
from deerflow.agents.memory.updater import import_memory_data
return import_memory_data(memory_data)
return import_memory_data(self._app_config.memory, memory_data, user_id=get_effective_user_id())
def get_model(self, name: str) -> dict | None:
"""Get a specific model's configuration by name.
@@ -821,8 +845,8 @@ class DeerFlowClient:
Dict with "mcp_servers" key mapping server name to config,
matching the Gateway API ``McpConfigResponse`` schema.
"""
config = get_extensions_config()
return {"mcp_servers": {name: server.model_dump() for name, server in config.mcp_servers.items()}}
ext = self._app_config.extensions
return {"mcp_servers": {name: server.model_dump() for name, server in ext.mcp_servers.items()}}
def update_mcp_config(self, mcp_servers: dict[str, dict]) -> dict:
"""Update MCP server configurations.
@@ -844,18 +868,19 @@ class DeerFlowClient:
if config_path is None:
raise FileNotFoundError("Cannot locate extensions_config.json. Set DEER_FLOW_EXTENSIONS_CONFIG_PATH or ensure it exists in the project root.")
current_config = get_extensions_config()
current_ext = self._app_config.extensions
config_data = {
"mcpServers": mcp_servers,
"skills": {name: {"enabled": skill.enabled} for name, skill in current_config.skills.items()},
"skills": {name: {"enabled": skill.enabled} for name, skill in current_ext.skills.items()},
}
self._atomic_write_json(config_path, config_data)
self._agent = None
self._agent_config_key = None
reloaded = reload_extensions_config()
self._reload_config()
reloaded = self._app_config.extensions
return {"mcp_servers": {name: server.model_dump() for name, server in reloaded.mcp_servers.items()}}
# ------------------------------------------------------------------
@@ -873,7 +898,7 @@ class DeerFlowClient:
"""
from deerflow.skills.loader import load_skills
skill = next((s for s in load_skills(enabled_only=False) if s.name == name), None)
skill = next((s for s in load_skills(self._app_config, enabled_only=False) if s.name == name), None)
if skill is None:
return None
return {
@@ -900,7 +925,7 @@ class DeerFlowClient:
"""
from deerflow.skills.loader import load_skills
skills = load_skills(enabled_only=False)
skills = load_skills(self._app_config, enabled_only=False)
skill = next((s for s in skills if s.name == name), None)
if skill is None:
raise ValueError(f"Skill '{name}' not found")
@@ -909,21 +934,25 @@ class DeerFlowClient:
if config_path is None:
raise FileNotFoundError("Cannot locate extensions_config.json. Set DEER_FLOW_EXTENSIONS_CONFIG_PATH or ensure it exists in the project root.")
extensions_config = get_extensions_config()
extensions_config.skills[name] = SkillStateConfig(enabled=enabled)
# Do not mutate self._app_config (frozen value). Compose the new
# skills state in a fresh dict, write it to disk, and let _reload_config()
# below rebuild AppConfig from the updated file.
ext = self._app_config.extensions
new_skills = {n: {"enabled": sc.enabled} for n, sc in ext.skills.items()}
new_skills[name] = {"enabled": enabled}
config_data = {
"mcpServers": {n: s.model_dump() for n, s in extensions_config.mcp_servers.items()},
"skills": {n: {"enabled": sc.enabled} for n, sc in extensions_config.skills.items()},
"mcpServers": {n: s.model_dump() for n, s in ext.mcp_servers.items()},
"skills": new_skills,
}
self._atomic_write_json(config_path, config_data)
self._agent = None
self._agent_config_key = None
reload_extensions_config()
self._reload_config()
updated = next((s for s in load_skills(enabled_only=False) if s.name == name), None)
updated = next((s for s in load_skills(self._app_config, enabled_only=False) if s.name == name), None)
if updated is None:
raise RuntimeError(f"Skill '{name}' disappeared after update")
return {
@@ -961,25 +990,25 @@ class DeerFlowClient:
"""
from deerflow.agents.memory.updater import reload_memory_data
return reload_memory_data()
return reload_memory_data(self._app_config.memory, user_id=get_effective_user_id())
def clear_memory(self) -> dict:
"""Clear all persisted memory data."""
from deerflow.agents.memory.updater import clear_memory_data
return clear_memory_data()
return clear_memory_data(self._app_config.memory, user_id=get_effective_user_id())
def create_memory_fact(self, content: str, category: str = "context", confidence: float = 0.5) -> dict:
"""Create a single fact manually."""
from deerflow.agents.memory.updater import create_memory_fact
return create_memory_fact(content=content, category=category, confidence=confidence)
return create_memory_fact(self._app_config.memory, content=content, category=category, confidence=confidence)
def delete_memory_fact(self, fact_id: str) -> dict:
"""Delete a single fact from memory by fact id."""
from deerflow.agents.memory.updater import delete_memory_fact
return delete_memory_fact(fact_id)
return delete_memory_fact(self._app_config.memory, fact_id)
def update_memory_fact(
self,
@@ -992,6 +1021,7 @@ class DeerFlowClient:
from deerflow.agents.memory.updater import update_memory_fact
return update_memory_fact(
self._app_config.memory,
fact_id=fact_id,
content=content,
category=category,
@@ -1004,9 +1034,7 @@ class DeerFlowClient:
Returns:
Memory config dict.
"""
from deerflow.config.memory_config import get_memory_config
config = get_memory_config()
config = self._app_config.memory
return {
"enabled": config.enabled,
"storage_path": config.storage_path,
@@ -1184,7 +1212,7 @@ class DeerFlowClient:
ValueError: If the path is invalid.
"""
try:
actual = get_paths().resolve_virtual_path(thread_id, path)
actual = get_paths().resolve_virtual_path(thread_id, path, user_id=get_effective_user_id())
except ValueError as exc:
if "traversal" in str(exc):
from deerflow.uploads.manager import PathTraversalError