mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-06-13 10:55:59 +00:00
feat(im): Add user-owned IM channel connections (#3487)
* Add user-owned IM channel connections * Fix dev startup and channel connect popup * Use async channel connect flow * Harden dev service daemon startup * Support local IM channel connections * Align IM connections with local channels * Fix safe user id digest algorithm * Address Copilot IM channel feedback * Address IM channel review comments * Support all integrated IM channel connections * Format additional channel connection tests * Keep unavailable channel connect buttons clickable * Fix IM channel provider icons * Add runtime setup for enabled IM channels * Guard global shortcut key handling * Keep configured IM channels editable * Avoid password autofill for channel secrets * Make channel threads visible to connection owners * Persist IM runtime config locally * Allow disconnecting runtime IM channels * Route no-auth channel sessions to local user * Use default user for auth-disabled local mode * Show IM channel source on threads * Prefill IM channel runtime config * Reflect IM channel runtime health * Ignore Feishu message read events * Ignore Feishu non-content message events * Let setup wizard enable IM channels * Fix frontend formatting after merge * Stabilize backend tests without local config * Isolate channel runtime config tests * Address channel connection review comments * Use sha256 user buckets with legacy migration * Ensure runtime IM channels are ready after restart * Persist disconnected IM channel state * Address channel connection review comments * Address channel connection review findings Frontend connect flow: - Open the runtime-config dialog only when a provider still needs credentials; configured providers go straight to the connect flow, so the binding-code/deep-link path is reachable from the UI again. - After saving credentials, continue into the connect flow when a user binding is still required (multi-user mode) instead of stopping at a "Connected" toast. - Extract shared provider-state helpers to core/channels/provider-state and add unit + e2e coverage for the direct-connect and configure-then-connect paths. Provider status semantics: - Report connection_status from the user's newest connection row; with no binding it is not_connected, except in auth-disabled local mode where a configured running channel is effectively connected. Concurrency and event-loop correctness: - Offload ChannelRuntimeConfigStore construction and writes, channel service construction, and Slack connection replies to threads; add a tests/blocking_io/ anchor for the runtime-config handlers. - Consume binding codes with a conditional UPDATE so a code can only be used once under concurrent workers; retry upsert_connection as an update when a concurrent insert wins the unique constraint. - Serialize ensure_channel_ready per channel so concurrent provider polls cannot double-start a channel worker. Config and migration hardening: - Stop mutating the get_app_config()-cached Telegram provider config; the runtime store now owns the UI-entered bot username. - Register channel_connections in STARTUP_ONLY_FIELDS with the standardized startup-only Field description. - Match the legacy unsafe-id bucket by recomputing its exact SHA-1 name so another user's same-prefix bucket can never be migrated. - Remove the unused Telegram process_webhook_update path and document src/core/channels in the frontend docs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Address PR review comments on authz scoping and channel runtime Security (review feedback from ShenAC-SAC): - Scope internal-token callers to the connection owner carried in X-DeerFlow-Owner-User-Id instead of bypassing owner checks outright, in both require_permission(owner_check=True) and the stateless run endpoints. Internal callers keep access to their own and shared/legacy threads, and may claim a default-owned channel thread for its real owner, but a leaked internal token no longer grants cross-user thread access. - Require admin privileges for POST/DELETE /api/channels/{provider}/ runtime-config: runtime credentials and channel workers are instance-wide shared state (same model as the MCP config API). Read-only provider listing stays available to all users. Performance (review feedback from willem-bd): - Skip the redundant thread channel-metadata PATCH after the first successful backfill per thread. - Reuse the per-connection Slack WebClient until its token changes instead of constructing one per outbound message. - Reconcile channel readiness for all providers concurrently in GET /api/channels/providers. Also resolve the code-quality unused-import flag in the blocking-io anchor by pre-importing the channel service via importlib. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Fix prettier formatting in provider-state test Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Reconcile UI runtime channel config with config reload on restart Main now reloads a channel's config.yaml entry on restart_channel() (#3514, issue #3497). Adapt the user-owned connection flow to coexist: - configure_channel() restarts with reload_config=False — the caller just supplied the authoritative config (browser-entered credentials that are never written to config.yaml), so a file reload must not clobber it with the stale on-disk entry. - _load_channel_config() re-applies the UI runtime-store overlay used at startup, so an operator-triggered restart keeps browser-entered credentials for channels without a config.yaml entry and does not resurrect a channel disconnected from the UI. - Offload the reload's disk IO (config.yaml + runtime store) with asyncio.to_thread, matching the blocking-IO policy on this branch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
DanielWalnut
@@ -4,6 +4,18 @@ from __future__ import annotations
|
||||
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from deerflow.config.app_config import AppConfig, reset_app_config, set_app_config
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def _stub_app_config():
|
||||
"""Keep run-context tests independent from a developer-local config.yaml."""
|
||||
set_app_config(AppConfig.model_validate({"sandbox": {"use": "deerflow.sandbox.local:LocalSandboxProvider"}}))
|
||||
yield
|
||||
reset_app_config()
|
||||
|
||||
|
||||
def test_format_sse_basic():
|
||||
from app.gateway.services import format_sse
|
||||
@@ -36,6 +48,12 @@ def test_format_sse_no_event_id():
|
||||
assert "id:" not in frame
|
||||
|
||||
|
||||
def test_sanitize_log_param_strips_control_characters():
|
||||
from app.gateway.utils import sanitize_log_param
|
||||
|
||||
assert sanitize_log_param("thread\nid\rwith\x00controls") == "threadidwithcontrols"
|
||||
|
||||
|
||||
def test_normalize_stream_modes_none():
|
||||
from app.gateway.services import normalize_stream_modes
|
||||
|
||||
@@ -474,6 +492,83 @@ def test_inject_authenticated_user_context_skips_internal_role():
|
||||
assert config["context"]["user_id"] == "channel-user-7"
|
||||
|
||||
|
||||
def test_start_run_uses_internal_owner_header_for_persistence(_stub_app_config):
|
||||
import asyncio
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
from langgraph.checkpoint.memory import InMemorySaver
|
||||
from langgraph.store.memory import InMemoryStore
|
||||
|
||||
from app.gateway.internal_auth import INTERNAL_OWNER_USER_ID_HEADER_NAME, INTERNAL_SYSTEM_ROLE
|
||||
from app.gateway.services import start_run
|
||||
from deerflow.persistence.thread_meta.memory import MemoryThreadMetaStore
|
||||
from deerflow.runtime import RunManager
|
||||
from deerflow.runtime.runs.store.memory import MemoryRunStore
|
||||
from deerflow.runtime.user_context import get_effective_user_id
|
||||
|
||||
async def _scenario():
|
||||
run_store = MemoryRunStore()
|
||||
thread_store = MemoryThreadMetaStore(InMemoryStore())
|
||||
await thread_store.create("channel-thread", user_id="default", metadata={"legacy": True})
|
||||
run_manager = RunManager(store=run_store)
|
||||
state = SimpleNamespace(
|
||||
stream_bridge=SimpleNamespace(),
|
||||
run_manager=run_manager,
|
||||
checkpointer=InMemorySaver(),
|
||||
store=InMemoryStore(),
|
||||
run_event_store=SimpleNamespace(),
|
||||
run_events_config=None,
|
||||
thread_store=thread_store,
|
||||
)
|
||||
request = SimpleNamespace(
|
||||
headers={INTERNAL_OWNER_USER_ID_HEADER_NAME: "owner-1"},
|
||||
state=SimpleNamespace(user=SimpleNamespace(id="default", system_role=INTERNAL_SYSTEM_ROLE)),
|
||||
app=SimpleNamespace(state=state),
|
||||
)
|
||||
body = SimpleNamespace(
|
||||
assistant_id="lead_agent",
|
||||
input={"messages": [{"role": "human", "content": "hi"}]},
|
||||
metadata={},
|
||||
config=None,
|
||||
context=None,
|
||||
on_disconnect="cancel",
|
||||
multitask_strategy="reject",
|
||||
stream_mode=None,
|
||||
stream_subgraphs=False,
|
||||
interrupt_before=None,
|
||||
interrupt_after=None,
|
||||
)
|
||||
task_context: dict[str, str] = {}
|
||||
|
||||
async def fake_run_agent(*args, **kwargs):
|
||||
task_context["user_id"] = get_effective_user_id()
|
||||
|
||||
with (
|
||||
patch("app.gateway.services.resolve_agent_factory", return_value=object()),
|
||||
patch("app.gateway.services.run_agent", side_effect=fake_run_agent),
|
||||
):
|
||||
record = await start_run(body, "channel-thread", request)
|
||||
await record.task
|
||||
|
||||
owner_run = await run_store.get(record.run_id, user_id="owner-1")
|
||||
default_run = await run_store.get(record.run_id, user_id="default")
|
||||
owner_thread = await thread_store.get("channel-thread", user_id="owner-1")
|
||||
default_thread = await thread_store.get("channel-thread", user_id="default")
|
||||
return owner_run, default_run, owner_thread, default_thread, task_context
|
||||
|
||||
owner_run, default_run, owner_thread, default_thread, task_context = asyncio.run(_scenario())
|
||||
|
||||
assert owner_run is not None
|
||||
assert owner_run["user_id"] == "owner-1"
|
||||
assert default_run is None
|
||||
assert owner_thread is not None
|
||||
assert owner_thread["user_id"] == "owner-1"
|
||||
assert owner_thread["metadata"] == {"legacy": True}
|
||||
assert default_thread is None
|
||||
assert task_context["user_id"] == "owner-1"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# build_run_config — context / configurable precedence (LangGraph >= 0.6.0)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
Reference in New Issue
Block a user