mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-06-13 10:55:59 +00:00
aa015462a7
* Add user-owned IM channel connections * Fix dev startup and channel connect popup * Use async channel connect flow * Harden dev service daemon startup * Support local IM channel connections * Align IM connections with local channels * Fix safe user id digest algorithm * Address Copilot IM channel feedback * Address IM channel review comments * Support all integrated IM channel connections * Format additional channel connection tests * Keep unavailable channel connect buttons clickable * Fix IM channel provider icons * Add runtime setup for enabled IM channels * Guard global shortcut key handling * Keep configured IM channels editable * Avoid password autofill for channel secrets * Make channel threads visible to connection owners * Persist IM runtime config locally * Allow disconnecting runtime IM channels * Route no-auth channel sessions to local user * Use default user for auth-disabled local mode * Show IM channel source on threads * Prefill IM channel runtime config * Reflect IM channel runtime health * Ignore Feishu message read events * Ignore Feishu non-content message events * Let setup wizard enable IM channels * Fix frontend formatting after merge * Stabilize backend tests without local config * Isolate channel runtime config tests * Address channel connection review comments * Use sha256 user buckets with legacy migration * Ensure runtime IM channels are ready after restart * Persist disconnected IM channel state * Address channel connection review comments * Address channel connection review findings Frontend connect flow: - Open the runtime-config dialog only when a provider still needs credentials; configured providers go straight to the connect flow, so the binding-code/deep-link path is reachable from the UI again. - After saving credentials, continue into the connect flow when a user binding is still required (multi-user mode) instead of stopping at a "Connected" toast. - Extract shared provider-state helpers to core/channels/provider-state and add unit + e2e coverage for the direct-connect and configure-then-connect paths. Provider status semantics: - Report connection_status from the user's newest connection row; with no binding it is not_connected, except in auth-disabled local mode where a configured running channel is effectively connected. Concurrency and event-loop correctness: - Offload ChannelRuntimeConfigStore construction and writes, channel service construction, and Slack connection replies to threads; add a tests/blocking_io/ anchor for the runtime-config handlers. - Consume binding codes with a conditional UPDATE so a code can only be used once under concurrent workers; retry upsert_connection as an update when a concurrent insert wins the unique constraint. - Serialize ensure_channel_ready per channel so concurrent provider polls cannot double-start a channel worker. Config and migration hardening: - Stop mutating the get_app_config()-cached Telegram provider config; the runtime store now owns the UI-entered bot username. - Register channel_connections in STARTUP_ONLY_FIELDS with the standardized startup-only Field description. - Match the legacy unsafe-id bucket by recomputing its exact SHA-1 name so another user's same-prefix bucket can never be migrated. - Remove the unused Telegram process_webhook_update path and document src/core/channels in the frontend docs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Address PR review comments on authz scoping and channel runtime Security (review feedback from ShenAC-SAC): - Scope internal-token callers to the connection owner carried in X-DeerFlow-Owner-User-Id instead of bypassing owner checks outright, in both require_permission(owner_check=True) and the stateless run endpoints. Internal callers keep access to their own and shared/legacy threads, and may claim a default-owned channel thread for its real owner, but a leaked internal token no longer grants cross-user thread access. - Require admin privileges for POST/DELETE /api/channels/{provider}/ runtime-config: runtime credentials and channel workers are instance-wide shared state (same model as the MCP config API). Read-only provider listing stays available to all users. Performance (review feedback from willem-bd): - Skip the redundant thread channel-metadata PATCH after the first successful backfill per thread. - Reuse the per-connection Slack WebClient until its token changes instead of constructing one per outbound message. - Reconcile channel readiness for all providers concurrently in GET /api/channels/providers. Also resolve the code-quality unused-import flag in the blocking-io anchor by pre-importing the channel service via importlib. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Fix prettier formatting in provider-state test Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Reconcile UI runtime channel config with config reload on restart Main now reloads a channel's config.yaml entry on restart_channel() (#3514, issue #3497). Adapt the user-owned connection flow to coexist: - configure_channel() restarts with reload_config=False — the caller just supplied the authoritative config (browser-entered credentials that are never written to config.yaml), so a file reload must not clobber it with the stale on-disk entry. - _load_channel_config() re-applies the UI runtime-store overlay used at startup, so an operator-triggered restart keeps browser-entered credentials for channels without a config.yaml entry and does not resurrect a channel disconnected from the UI. - Offload the reload's disk IO (config.yaml + runtime store) with asyncio.to_thread, matching the blocking-IO policy on this branch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
108 lines
5.5 KiB
Python
108 lines
5.5 KiB
Python
"""Single source of truth for the config hot-reload boundary.
|
|
|
|
Bytedance/deer-flow issue #3144: gateway request dependencies resolve
|
|
``AppConfig`` through ``get_app_config()`` on every request, so per-run
|
|
fields take effect on the next message without restarting the gateway.
|
|
The fields listed in this module are the **infrastructure** subset that
|
|
the gateway captures once at startup — engines, singletons, IM clients,
|
|
the logging handler — and that therefore require a process restart to
|
|
change at runtime.
|
|
|
|
The registry covers two kinds of entries:
|
|
|
|
- Top-level ``AppConfig`` fields (``database``, ``checkpointer``,
|
|
``run_events``, ``stream_bridge``, ``sandbox``, ``log_level``). For
|
|
these, :func:`format_field_description` produces the standardised
|
|
``"startup-only: ..."`` prefix that the matching Pydantic
|
|
``Field(description=...)`` carries, so the boundary surfaces in IDE
|
|
hover next to the field itself.
|
|
- Top-level ``config.yaml`` sections that are not part of the
|
|
``AppConfig`` schema (``channels``). These cannot be standardised at
|
|
the schema level, so the registry is their only canonical location.
|
|
|
|
Any future "needs restart" scanner — operator tooling, lint hooks, doc
|
|
generators — should drive off this registry rather than re-parsing
|
|
prose.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from collections.abc import Iterator
|
|
|
|
#: The standardised prefix every restart-required field description starts
|
|
#: with. ``test_reload_boundary`` enforces both directions: registered
|
|
#: fields must use this prefix in the schema, and any schema field using
|
|
#: this prefix must be in the registry.
|
|
STARTUP_ONLY_PREFIX = "startup-only:"
|
|
|
|
|
|
#: Restart-required field paths mapped to the human-readable reason.
|
|
#:
|
|
#: The reason text is what surfaces in ``Field(description=...)``, so it
|
|
#: must explain *what* code captures the snapshot — not just that the
|
|
#: field is restart-required — so an operator changing the value knows
|
|
#: which subsystem to restart.
|
|
STARTUP_ONLY_FIELDS: dict[str, str] = {
|
|
"database": ("init_engine_from_config() runs once during langgraph_runtime() startup; the SQLAlchemy engine holds the connection pool and is not rebuilt on config.yaml edits."),
|
|
"checkpointer": ("make_checkpointer() binds the persistent checkpointer once at startup, including SQLite WAL / busy_timeout settings."),
|
|
"run_events": ("make_run_event_store() picks the memory- vs SQL-backed implementation at startup and is frozen onto app.state.run_events_config to stay paired with the underlying event store."),
|
|
"stream_bridge": ("make_stream_bridge() constructs the stream-bridge singleton once during startup."),
|
|
"sandbox": ("get_sandbox_provider() caches the provider singleton (``_default_sandbox_provider``); a different ``sandbox.use`` class path only takes effect on next process start."),
|
|
"log_level": (
|
|
"apply_logging_level() runs only during app.py startup; it sets the deerflow/app logger levels and may lower root handler thresholds so configured messages can propagate. A freshly reloaded AppConfig does not retrigger it."
|
|
),
|
|
# Not part of the AppConfig Pydantic schema — channel credentials are
|
|
# consumed directly by ``start_channel_service()`` once at lifespan
|
|
# startup and the live channel clients are not rebuilt on
|
|
# config.yaml edits.
|
|
"channels": ("start_channel_service() is invoked once during startup; the live IM channel clients (Feishu, Slack, Telegram, DingTalk) are not rebuilt when channels.* changes."),
|
|
"channel_connections": (
|
|
"start_channel_service() wires the connection repository and channel workers once at startup, and the channel-connections router caches the merged provider config on app.state; channel_connections.* edits need a restart."
|
|
),
|
|
}
|
|
|
|
|
|
def iter_startup_only_field_paths() -> Iterator[str]:
|
|
"""Yield every registered restart-required field path."""
|
|
return iter(STARTUP_ONLY_FIELDS)
|
|
|
|
|
|
def is_startup_only_field(field_path: str) -> bool:
|
|
"""Return ``True`` when *field_path* is registered as restart-required.
|
|
|
|
Accepts only top-level paths (``"database"``, ``"sandbox"`` etc.);
|
|
nested keys like ``"database.url"`` are not modelled here because the
|
|
boundary is per-section, not per-leaf.
|
|
"""
|
|
return field_path in STARTUP_ONLY_FIELDS
|
|
|
|
|
|
def format_field_description(field_path: str, *, field_doc: str | None = None) -> str:
|
|
"""Build the standardised description for a registered field.
|
|
|
|
Used inside ``AppConfig`` ``Field(description=...)`` so the hover
|
|
text in IDEs matches the registry and the drift tests can pin one
|
|
side against the other.
|
|
|
|
Args:
|
|
field_path: A registered top-level field path (e.g. ``"log_level"``).
|
|
field_doc: Optional human-facing description for the field itself
|
|
(allowed values, semantics, etc.). When supplied, it is
|
|
appended after the ``startup-only:`` marker block separated by
|
|
a blank line so IDE hover shows both the restart-required
|
|
reason *and* the field's normal documentation. Composition
|
|
keeps the marker as the leading token machine-readable tooling
|
|
pivots on while restoring the prose that ``Field(description=)``
|
|
used to carry before the registry took over.
|
|
|
|
Raises:
|
|
KeyError: when *field_path* is not registered. This is deliberate
|
|
— silently returning a placeholder would let a typo bypass
|
|
the drift coverage.
|
|
"""
|
|
reason = STARTUP_ONLY_FIELDS[field_path]
|
|
header = f"{STARTUP_ONLY_PREFIX} {reason}"
|
|
if field_doc is None:
|
|
return header
|
|
return f"{header}\n\n{field_doc.strip()}"
|