feat(im): Add user-owned IM channel connections (#3487)

* Add user-owned IM channel connections

* Fix dev startup and channel connect popup

* Use async channel connect flow

* Harden dev service daemon startup

* Support local IM channel connections

* Align IM connections with local channels

* Fix safe user id digest algorithm

* Address Copilot IM channel feedback

* Address IM channel review comments

* Support all integrated IM channel connections

* Format additional channel connection tests

* Keep unavailable channel connect buttons clickable

* Fix IM channel provider icons

* Add runtime setup for enabled IM channels

* Guard global shortcut key handling

* Keep configured IM channels editable

* Avoid password autofill for channel secrets

* Make channel threads visible to connection owners

* Persist IM runtime config locally

* Allow disconnecting runtime IM channels

* Route no-auth channel sessions to local user

* Use default user for auth-disabled local mode

* Show IM channel source on threads

* Prefill IM channel runtime config

* Reflect IM channel runtime health

* Ignore Feishu message read events

* Ignore Feishu non-content message events

* Let setup wizard enable IM channels

* Fix frontend formatting after merge

* Stabilize backend tests without local config

* Isolate channel runtime config tests

* Address channel connection review comments

* Use sha256 user buckets with legacy migration

* Ensure runtime IM channels are ready after restart

* Persist disconnected IM channel state

* Address channel connection review comments

* Address channel connection review findings

Frontend connect flow:
- Open the runtime-config dialog only when a provider still needs
  credentials; configured providers go straight to the connect flow, so
  the binding-code/deep-link path is reachable from the UI again.
- After saving credentials, continue into the connect flow when a user
  binding is still required (multi-user mode) instead of stopping at a
  "Connected" toast.
- Extract shared provider-state helpers to core/channels/provider-state
  and add unit + e2e coverage for the direct-connect and
  configure-then-connect paths.

Provider status semantics:
- Report connection_status from the user's newest connection row;
  with no binding it is not_connected, except in auth-disabled local
  mode where a configured running channel is effectively connected.

Concurrency and event-loop correctness:
- Offload ChannelRuntimeConfigStore construction and writes, channel
  service construction, and Slack connection replies to threads; add a
  tests/blocking_io/ anchor for the runtime-config handlers.
- Consume binding codes with a conditional UPDATE so a code can only be
  used once under concurrent workers; retry upsert_connection as an
  update when a concurrent insert wins the unique constraint.
- Serialize ensure_channel_ready per channel so concurrent provider
  polls cannot double-start a channel worker.

Config and migration hardening:
- Stop mutating the get_app_config()-cached Telegram provider config;
  the runtime store now owns the UI-entered bot username.
- Register channel_connections in STARTUP_ONLY_FIELDS with the
  standardized startup-only Field description.
- Match the legacy unsafe-id bucket by recomputing its exact SHA-1 name
  so another user's same-prefix bucket can never be migrated.
- Remove the unused Telegram process_webhook_update path and document
  src/core/channels in the frontend docs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Address PR review comments on authz scoping and channel runtime

Security (review feedback from ShenAC-SAC):
- Scope internal-token callers to the connection owner carried in
  X-DeerFlow-Owner-User-Id instead of bypassing owner checks outright,
  in both require_permission(owner_check=True) and the stateless run
  endpoints. Internal callers keep access to their own and
  shared/legacy threads, and may claim a default-owned channel thread
  for its real owner, but a leaked internal token no longer grants
  cross-user thread access.
- Require admin privileges for POST/DELETE /api/channels/{provider}/
  runtime-config: runtime credentials and channel workers are
  instance-wide shared state (same model as the MCP config API).
  Read-only provider listing stays available to all users.

Performance (review feedback from willem-bd):
- Skip the redundant thread channel-metadata PATCH after the first
  successful backfill per thread.
- Reuse the per-connection Slack WebClient until its token changes
  instead of constructing one per outbound message.
- Reconcile channel readiness for all providers concurrently in
  GET /api/channels/providers.

Also resolve the code-quality unused-import flag in the blocking-io
anchor by pre-importing the channel service via importlib.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Fix prettier formatting in provider-state test

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Reconcile UI runtime channel config with config reload on restart

Main now reloads a channel's config.yaml entry on restart_channel()
(#3514, issue #3497). Adapt the user-owned connection flow to coexist:

- configure_channel() restarts with reload_config=False — the caller
  just supplied the authoritative config (browser-entered credentials
  that are never written to config.yaml), so a file reload must not
  clobber it with the stale on-disk entry.
- _load_channel_config() re-applies the UI runtime-store overlay used
  at startup, so an operator-triggered restart keeps browser-entered
  credentials for channels without a config.yaml entry and does not
  resurrect a channel disconnected from the UI.
- Offload the reload's disk IO (config.yaml + runtime store) with
  asyncio.to_thread, matching the blocking-IO policy on this branch.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
DanielWalnut
2026-06-12 15:24:58 +08:00
committed by GitHub
parent b8f5ed360f
commit aa015462a7
96 changed files with 8585 additions and 277 deletions
+46
View File
@@ -0,0 +1,46 @@
"""Step: browser-connectable IM channel enablement."""
from __future__ import annotations
from dataclasses import dataclass
from wizard.ui import ask_multi_choice, print_header, print_info, print_success
CHANNEL_CONNECTION_OPTIONS: tuple[tuple[str, str, str], ...] = (
("telegram", "Telegram", "direct messages through your DeerFlow bot"),
("slack", "Slack", "workspace messages and mentions"),
("discord", "Discord", "server messages through your DeerFlow bot"),
("feishu", "Feishu / Lark", "messages through your DeerFlow app"),
("dingtalk", "DingTalk", "Stream Push messages through your DeerFlow bot"),
("wechat", "WeChat", "iLink messages through your DeerFlow bot"),
("wecom", "WeCom", "messages through your DeerFlow AI bot"),
)
@dataclass
class ChannelConnectionsStepResult:
enabled_providers: list[str]
def run_channels_step(step_label: str = "Step 4/5") -> ChannelConnectionsStepResult:
print_header(f"{step_label} · IM Channels (optional)")
print_info("Choose which IM channels should appear in the DeerFlow sidebar and Settings.")
print_info("Credentials can be entered later from the browser with Connect or Modify.")
print()
options = [f"{display_name}{description}" for _, display_name, description in CHANNEL_CONNECTION_OPTIONS]
selected = ask_multi_choice(
"Enable channels (comma-separated numbers, 'all', or Enter for none)",
options,
default=[],
)
enabled_providers = [CHANNEL_CONNECTION_OPTIONS[idx][0] for idx in selected]
if enabled_providers:
display_names = [CHANNEL_CONNECTION_OPTIONS[idx][1] for idx in selected]
print_success(f"Enabled channels: {', '.join(display_names)}")
else:
print_info("No IM channels selected; channel connections will stay disabled.")
return ChannelConnectionsStepResult(enabled_providers=enabled_providers)
+43
View File
@@ -224,6 +224,49 @@ def ask_choice(prompt: str, options: list[str], default: int | None = None) -> i
return _ask_choice_with_numbers(prompt, options, default=default)
def ask_multi_choice(prompt: str, options: list[str], default: list[int] | None = None) -> list[int]:
"""Present a numbered multi-select menu and return 0-based indexes."""
has_default = default is not None
default_indexes = list(default or [])
for i, opt in enumerate(options, 1):
marker = f" {green('*')}" if has_default and i - 1 in default_indexes else " "
print(f"{marker} {i}. {opt}")
print()
suffix = ""
if default_indexes:
suffix = f" [{','.join(str(idx + 1) for idx in default_indexes)}]"
elif has_default:
suffix = " [none]"
while True:
raw = input(f"{prompt}{suffix}: ").strip().lower()
if raw == "" and has_default:
return default_indexes
if raw in {"none", "no", "n", "skip"}:
return []
if raw == "all":
return list(range(len(options)))
parts = [part.strip() for part in raw.replace(" ", ",").split(",") if part.strip()]
selected: list[int] = []
valid = bool(parts)
for part in parts:
if not part.isdigit():
valid = False
break
idx = int(part) - 1
if not 0 <= idx < len(options):
valid = False
break
if idx not in selected:
selected.append(idx)
if valid:
return selected
print(f" Enter comma-separated numbers between 1 and {len(options)}, 'all', or 'none'.")
def ask_text(prompt: str, default: str = "", required: bool = False) -> str:
"""Ask for a text value, returning default if the user presses Enter."""
suffix = f" [{default}]" if default else ""
+27
View File
@@ -12,6 +12,16 @@ from typing import Any
import yaml
CHANNEL_CONNECTION_PROVIDERS: tuple[str, ...] = (
"telegram",
"slack",
"discord",
"feishu",
"dingtalk",
"wechat",
"wecom",
)
def _project_root() -> Path:
return Path(__file__).resolve().parents[2]
@@ -151,6 +161,18 @@ def _make_model_config_name(model_name: str) -> str:
return base.replace(".", "-")
def _build_channel_connections_config(enabled_providers: list[str]) -> dict[str, Any]:
selected = set(enabled_providers)
unknown = selected.difference(CHANNEL_CONNECTION_PROVIDERS)
if unknown:
raise ValueError(f"Unknown channel connection provider(s): {', '.join(sorted(unknown))}")
return {
"enabled": bool(selected),
**{provider: {"enabled": provider in selected} for provider in CHANNEL_CONNECTION_PROVIDERS},
}
def build_minimal_config(
*,
provider_use: str,
@@ -170,6 +192,7 @@ def build_minimal_config(
allow_host_bash: bool = False,
include_bash_tool: bool = False,
include_write_tools: bool = True,
channel_connection_providers: list[str] | None = None,
config_version: int = 5,
base_config: dict[str, Any] | None = None,
) -> str:
@@ -219,6 +242,8 @@ def build_minimal_config(
else:
sandbox_config.pop("allow_host_bash", None)
data["sandbox"] = sandbox_config
if channel_connection_providers is not None:
data["channel_connections"] = _build_channel_connections_config(channel_connection_providers)
header = (
f"# DeerFlow Configuration\n"
@@ -250,6 +275,7 @@ def write_config_yaml(
allow_host_bash: bool = False,
include_bash_tool: bool = False,
include_write_tools: bool = True,
channel_connection_providers: list[str] | None = None,
) -> None:
"""Write (or overwrite) config.yaml with a minimal working configuration."""
# Read config_version from config.example.yaml if present
@@ -284,6 +310,7 @@ def write_config_yaml(
allow_host_bash=allow_host_bash,
include_bash_tool=include_bash_tool,
include_write_tools=include_write_tools,
channel_connection_providers=channel_connection_providers,
config_version=config_version,
base_config=example_defaults,
)