mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-06-13 19:06:01 +00:00
feat(im): Add user-owned IM channel connections (#3487)
* Add user-owned IM channel connections * Fix dev startup and channel connect popup * Use async channel connect flow * Harden dev service daemon startup * Support local IM channel connections * Align IM connections with local channels * Fix safe user id digest algorithm * Address Copilot IM channel feedback * Address IM channel review comments * Support all integrated IM channel connections * Format additional channel connection tests * Keep unavailable channel connect buttons clickable * Fix IM channel provider icons * Add runtime setup for enabled IM channels * Guard global shortcut key handling * Keep configured IM channels editable * Avoid password autofill for channel secrets * Make channel threads visible to connection owners * Persist IM runtime config locally * Allow disconnecting runtime IM channels * Route no-auth channel sessions to local user * Use default user for auth-disabled local mode * Show IM channel source on threads * Prefill IM channel runtime config * Reflect IM channel runtime health * Ignore Feishu message read events * Ignore Feishu non-content message events * Let setup wizard enable IM channels * Fix frontend formatting after merge * Stabilize backend tests without local config * Isolate channel runtime config tests * Address channel connection review comments * Use sha256 user buckets with legacy migration * Ensure runtime IM channels are ready after restart * Persist disconnected IM channel state * Address channel connection review comments * Address channel connection review findings Frontend connect flow: - Open the runtime-config dialog only when a provider still needs credentials; configured providers go straight to the connect flow, so the binding-code/deep-link path is reachable from the UI again. - After saving credentials, continue into the connect flow when a user binding is still required (multi-user mode) instead of stopping at a "Connected" toast. - Extract shared provider-state helpers to core/channels/provider-state and add unit + e2e coverage for the direct-connect and configure-then-connect paths. Provider status semantics: - Report connection_status from the user's newest connection row; with no binding it is not_connected, except in auth-disabled local mode where a configured running channel is effectively connected. Concurrency and event-loop correctness: - Offload ChannelRuntimeConfigStore construction and writes, channel service construction, and Slack connection replies to threads; add a tests/blocking_io/ anchor for the runtime-config handlers. - Consume binding codes with a conditional UPDATE so a code can only be used once under concurrent workers; retry upsert_connection as an update when a concurrent insert wins the unique constraint. - Serialize ensure_channel_ready per channel so concurrent provider polls cannot double-start a channel worker. Config and migration hardening: - Stop mutating the get_app_config()-cached Telegram provider config; the runtime store now owns the UI-entered bot username. - Register channel_connections in STARTUP_ONLY_FIELDS with the standardized startup-only Field description. - Match the legacy unsafe-id bucket by recomputing its exact SHA-1 name so another user's same-prefix bucket can never be migrated. - Remove the unused Telegram process_webhook_update path and document src/core/channels in the frontend docs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Address PR review comments on authz scoping and channel runtime Security (review feedback from ShenAC-SAC): - Scope internal-token callers to the connection owner carried in X-DeerFlow-Owner-User-Id instead of bypassing owner checks outright, in both require_permission(owner_check=True) and the stateless run endpoints. Internal callers keep access to their own and shared/legacy threads, and may claim a default-owned channel thread for its real owner, but a leaked internal token no longer grants cross-user thread access. - Require admin privileges for POST/DELETE /api/channels/{provider}/ runtime-config: runtime credentials and channel workers are instance-wide shared state (same model as the MCP config API). Read-only provider listing stays available to all users. Performance (review feedback from willem-bd): - Skip the redundant thread channel-metadata PATCH after the first successful backfill per thread. - Reuse the per-connection Slack WebClient until its token changes instead of constructing one per outbound message. - Reconcile channel readiness for all providers concurrently in GET /api/channels/providers. Also resolve the code-quality unused-import flag in the blocking-io anchor by pre-importing the channel service via importlib. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Fix prettier formatting in provider-state test Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Reconcile UI runtime channel config with config reload on restart Main now reloads a channel's config.yaml entry on restart_channel() (#3514, issue #3497). Adapt the user-owned connection flow to coexist: - configure_channel() restarts with reload_config=False — the caller just supplied the authoritative config (browser-entered credentials that are never written to config.yaml), so a file reload must not clobber it with the stale on-disk entry. - _load_channel_config() re-applies the UI runtime-store overlay used at startup, so an operator-triggered restart keeps browser-entered credentials for channels without a config.yaml entry and does not resurrect a channel disconnected from the UI. - Offload the reload's disk IO (config.yaml + runtime store) with asyncio.to_thread, matching the blocking-IO policy on this branch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
DanielWalnut
@@ -0,0 +1,251 @@
|
||||
"""Connection binding tests for browser-connectable IM channels beyond Telegram/Slack/Discord."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from unittest.mock import AsyncMock, MagicMock
|
||||
|
||||
from app.channels.message_bus import InboundMessage, MessageBus
|
||||
|
||||
|
||||
async def _make_repo(tmp_path, name: str):
|
||||
from deerflow.persistence.channel_connections import ChannelConnectionRepository
|
||||
from deerflow.persistence.engine import get_session_factory, init_engine
|
||||
|
||||
await init_engine("sqlite", url=f"sqlite+aiosqlite:///{tmp_path / f'{name}.db'}", sqlite_dir=str(tmp_path))
|
||||
return ChannelConnectionRepository(get_session_factory())
|
||||
|
||||
|
||||
async def _seed_state(repo, provider: str, state: str, owner_user_id: str = "deerflow-user-1") -> None:
|
||||
await repo.create_oauth_state(
|
||||
owner_user_id=owner_user_id,
|
||||
provider=provider,
|
||||
state=state,
|
||||
expires_at=datetime.now(UTC) + timedelta(minutes=5),
|
||||
)
|
||||
|
||||
|
||||
def test_feishu_connect_command_binds_identity(tmp_path):
|
||||
import anyio
|
||||
|
||||
from app.channels.feishu import FeishuChannel
|
||||
|
||||
async def go():
|
||||
repo = await _make_repo(tmp_path, "feishu")
|
||||
state = "feishu-bind-code"
|
||||
await _seed_state(repo, "feishu", state)
|
||||
channel = FeishuChannel(
|
||||
bus=MessageBus(),
|
||||
config={"app_id": "app", "app_secret": "secret", "connection_repo": repo},
|
||||
)
|
||||
channel._reply_card = AsyncMock()
|
||||
|
||||
handled = await channel._bind_connection_from_connect_code(
|
||||
message_id="om-message-1",
|
||||
chat_id="oc-chat-1",
|
||||
user_id="ou-user-1",
|
||||
code=state,
|
||||
)
|
||||
|
||||
connections = await repo.list_connections("deerflow-user-1")
|
||||
assert handled is True
|
||||
assert len(connections) == 1
|
||||
assert connections[0]["provider"] == "feishu"
|
||||
assert connections[0]["external_account_id"] == "ou-user-1"
|
||||
assert connections[0]["workspace_id"] == "oc-chat-1"
|
||||
channel._reply_card.assert_awaited_once_with("om-message-1", "Feishu connected to DeerFlow.")
|
||||
await repo.close()
|
||||
|
||||
anyio.run(go)
|
||||
|
||||
|
||||
def test_dingtalk_connect_command_binds_identity(tmp_path):
|
||||
import anyio
|
||||
|
||||
from app.channels.dingtalk import _CONVERSATION_TYPE_GROUP, DingTalkChannel
|
||||
|
||||
async def go():
|
||||
repo = await _make_repo(tmp_path, "dingtalk")
|
||||
state = "dingtalk-bind-code"
|
||||
await _seed_state(repo, "dingtalk", state)
|
||||
channel = DingTalkChannel(
|
||||
bus=MessageBus(),
|
||||
config={"client_id": "client", "client_secret": "secret", "connection_repo": repo},
|
||||
)
|
||||
channel._send_connection_reply = AsyncMock()
|
||||
|
||||
handled = await channel._bind_connection_from_connect_code(
|
||||
conversation_type=_CONVERSATION_TYPE_GROUP,
|
||||
sender_staff_id="staff-user-1",
|
||||
sender_nick="Alice",
|
||||
conversation_id="cid-group-1",
|
||||
code=state,
|
||||
)
|
||||
|
||||
connections = await repo.list_connections("deerflow-user-1")
|
||||
assert handled is True
|
||||
assert len(connections) == 1
|
||||
assert connections[0]["provider"] == "dingtalk"
|
||||
assert connections[0]["external_account_id"] == "staff-user-1"
|
||||
assert connections[0]["external_account_name"] == "Alice"
|
||||
assert connections[0]["workspace_id"] == "cid-group-1"
|
||||
channel._send_connection_reply.assert_awaited_once()
|
||||
await repo.close()
|
||||
|
||||
anyio.run(go)
|
||||
|
||||
|
||||
def test_wechat_connect_command_binds_identity(tmp_path):
|
||||
import anyio
|
||||
|
||||
from app.channels.wechat import WechatChannel
|
||||
|
||||
async def go():
|
||||
repo = await _make_repo(tmp_path, "wechat")
|
||||
state = "wechat-bind-code"
|
||||
await _seed_state(repo, "wechat", state)
|
||||
channel = WechatChannel(
|
||||
bus=MessageBus(),
|
||||
config={"bot_token": "token", "connection_repo": repo},
|
||||
)
|
||||
channel._send_connection_reply = AsyncMock()
|
||||
|
||||
handled = await channel._bind_connection_from_connect_code(
|
||||
chat_id="wx-user-1",
|
||||
context_token="ctx-1",
|
||||
code=state,
|
||||
)
|
||||
|
||||
connections = await repo.list_connections("deerflow-user-1")
|
||||
assert handled is True
|
||||
assert len(connections) == 1
|
||||
assert connections[0]["provider"] == "wechat"
|
||||
assert connections[0]["external_account_id"] == "wx-user-1"
|
||||
assert connections[0]["workspace_id"] == "wx-user-1"
|
||||
channel._send_connection_reply.assert_awaited_once_with("wx-user-1", "ctx-1", "WeChat connected to DeerFlow.")
|
||||
await repo.close()
|
||||
|
||||
anyio.run(go)
|
||||
|
||||
|
||||
def test_wecom_connect_command_binds_identity(tmp_path):
|
||||
import anyio
|
||||
|
||||
from app.channels.wecom import WeComChannel
|
||||
|
||||
async def go():
|
||||
repo = await _make_repo(tmp_path, "wecom")
|
||||
state = "wecom-bind-code"
|
||||
await _seed_state(repo, "wecom", state)
|
||||
channel = WeComChannel(
|
||||
bus=MessageBus(),
|
||||
config={"bot_id": "bot", "bot_secret": "secret", "connection_repo": repo},
|
||||
)
|
||||
channel._ws_client = MagicMock()
|
||||
channel._ws_client.reply = AsyncMock()
|
||||
frame = {"body": {"aibotid": "bot-1", "chattype": "single"}}
|
||||
|
||||
handled = await channel._bind_connection_from_connect_code(
|
||||
frame=frame,
|
||||
user_id="wecom-user-1",
|
||||
code=state,
|
||||
)
|
||||
|
||||
connections = await repo.list_connections("deerflow-user-1")
|
||||
assert handled is True
|
||||
assert len(connections) == 1
|
||||
assert connections[0]["provider"] == "wecom"
|
||||
assert connections[0]["external_account_id"] == "wecom-user-1"
|
||||
assert connections[0]["workspace_id"] == "bot-1"
|
||||
channel._ws_client.reply.assert_awaited_once_with(frame, {"msgtype": "text", "text": {"content": "WeCom connected to DeerFlow."}})
|
||||
await repo.close()
|
||||
|
||||
anyio.run(go)
|
||||
|
||||
|
||||
def test_additional_channels_attach_owner_identity(tmp_path):
|
||||
import anyio
|
||||
|
||||
from app.channels.dingtalk import _CONVERSATION_TYPE_GROUP, DingTalkChannel
|
||||
from app.channels.feishu import FeishuChannel
|
||||
from app.channels.wechat import WechatChannel
|
||||
from app.channels.wecom import WeComChannel
|
||||
|
||||
async def go():
|
||||
repo = await _make_repo(tmp_path, "additional-identity")
|
||||
await repo.upsert_connection(
|
||||
owner_user_id="deerflow-user-1",
|
||||
provider="feishu",
|
||||
external_account_id="ou-user-1",
|
||||
workspace_id="oc-chat-1",
|
||||
)
|
||||
await repo.upsert_connection(
|
||||
owner_user_id="deerflow-user-1",
|
||||
provider="dingtalk",
|
||||
external_account_id="staff-user-1",
|
||||
workspace_id="cid-group-1",
|
||||
)
|
||||
await repo.upsert_connection(
|
||||
owner_user_id="deerflow-user-1",
|
||||
provider="wechat",
|
||||
external_account_id="wx-user-1",
|
||||
workspace_id="wx-user-1",
|
||||
)
|
||||
await repo.upsert_connection(
|
||||
owner_user_id="deerflow-user-1",
|
||||
provider="wecom",
|
||||
external_account_id="wecom-user-1",
|
||||
workspace_id="bot-1",
|
||||
)
|
||||
|
||||
cases = [
|
||||
(
|
||||
FeishuChannel(bus=MessageBus(), config={"connection_repo": repo}),
|
||||
InboundMessage(channel_name="feishu", chat_id="oc-chat-1", user_id="ou-user-1", text="hello"),
|
||||
),
|
||||
(
|
||||
DingTalkChannel(bus=MessageBus(), config={"connection_repo": repo}),
|
||||
InboundMessage(
|
||||
channel_name="dingtalk",
|
||||
chat_id="cid-group-1",
|
||||
user_id="staff-user-1",
|
||||
text="hello",
|
||||
metadata={
|
||||
"conversation_type": _CONVERSATION_TYPE_GROUP,
|
||||
"conversation_id": "cid-group-1",
|
||||
},
|
||||
),
|
||||
),
|
||||
(
|
||||
WechatChannel(bus=MessageBus(), config={"connection_repo": repo}),
|
||||
InboundMessage(channel_name="wechat", chat_id="wx-user-1", user_id="wx-user-1", text="hello"),
|
||||
),
|
||||
(
|
||||
WeComChannel(bus=MessageBus(), config={"connection_repo": repo}),
|
||||
InboundMessage(
|
||||
channel_name="wecom",
|
||||
chat_id="wecom-user-1",
|
||||
user_id="wecom-user-1",
|
||||
text="hello",
|
||||
metadata={"aibotid": "bot-1"},
|
||||
),
|
||||
),
|
||||
]
|
||||
|
||||
for channel, inbound in cases:
|
||||
attached = await channel._attach_connection_identity(inbound)
|
||||
assert attached.owner_user_id == "deerflow-user-1"
|
||||
assert attached.connection_id
|
||||
assert (
|
||||
attached.workspace_id
|
||||
== {
|
||||
"feishu": "oc-chat-1",
|
||||
"dingtalk": "cid-group-1",
|
||||
"wechat": "wx-user-1",
|
||||
"wecom": "bot-1",
|
||||
}[channel.name]
|
||||
)
|
||||
|
||||
await repo.close()
|
||||
|
||||
anyio.run(go)
|
||||
Reference in New Issue
Block a user