mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-06-13 19:06:01 +00:00
feat(im): Add user-owned IM channel connections (#3487)
* Add user-owned IM channel connections * Fix dev startup and channel connect popup * Use async channel connect flow * Harden dev service daemon startup * Support local IM channel connections * Align IM connections with local channels * Fix safe user id digest algorithm * Address Copilot IM channel feedback * Address IM channel review comments * Support all integrated IM channel connections * Format additional channel connection tests * Keep unavailable channel connect buttons clickable * Fix IM channel provider icons * Add runtime setup for enabled IM channels * Guard global shortcut key handling * Keep configured IM channels editable * Avoid password autofill for channel secrets * Make channel threads visible to connection owners * Persist IM runtime config locally * Allow disconnecting runtime IM channels * Route no-auth channel sessions to local user * Use default user for auth-disabled local mode * Show IM channel source on threads * Prefill IM channel runtime config * Reflect IM channel runtime health * Ignore Feishu message read events * Ignore Feishu non-content message events * Let setup wizard enable IM channels * Fix frontend formatting after merge * Stabilize backend tests without local config * Isolate channel runtime config tests * Address channel connection review comments * Use sha256 user buckets with legacy migration * Ensure runtime IM channels are ready after restart * Persist disconnected IM channel state * Address channel connection review comments * Address channel connection review findings Frontend connect flow: - Open the runtime-config dialog only when a provider still needs credentials; configured providers go straight to the connect flow, so the binding-code/deep-link path is reachable from the UI again. - After saving credentials, continue into the connect flow when a user binding is still required (multi-user mode) instead of stopping at a "Connected" toast. - Extract shared provider-state helpers to core/channels/provider-state and add unit + e2e coverage for the direct-connect and configure-then-connect paths. Provider status semantics: - Report connection_status from the user's newest connection row; with no binding it is not_connected, except in auth-disabled local mode where a configured running channel is effectively connected. Concurrency and event-loop correctness: - Offload ChannelRuntimeConfigStore construction and writes, channel service construction, and Slack connection replies to threads; add a tests/blocking_io/ anchor for the runtime-config handlers. - Consume binding codes with a conditional UPDATE so a code can only be used once under concurrent workers; retry upsert_connection as an update when a concurrent insert wins the unique constraint. - Serialize ensure_channel_ready per channel so concurrent provider polls cannot double-start a channel worker. Config and migration hardening: - Stop mutating the get_app_config()-cached Telegram provider config; the runtime store now owns the UI-entered bot username. - Register channel_connections in STARTUP_ONLY_FIELDS with the standardized startup-only Field description. - Match the legacy unsafe-id bucket by recomputing its exact SHA-1 name so another user's same-prefix bucket can never be migrated. - Remove the unused Telegram process_webhook_update path and document src/core/channels in the frontend docs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Address PR review comments on authz scoping and channel runtime Security (review feedback from ShenAC-SAC): - Scope internal-token callers to the connection owner carried in X-DeerFlow-Owner-User-Id instead of bypassing owner checks outright, in both require_permission(owner_check=True) and the stateless run endpoints. Internal callers keep access to their own and shared/legacy threads, and may claim a default-owned channel thread for its real owner, but a leaked internal token no longer grants cross-user thread access. - Require admin privileges for POST/DELETE /api/channels/{provider}/ runtime-config: runtime credentials and channel workers are instance-wide shared state (same model as the MCP config API). Read-only provider listing stays available to all users. Performance (review feedback from willem-bd): - Skip the redundant thread channel-metadata PATCH after the first successful backfill per thread. - Reuse the per-connection Slack WebClient until its token changes instead of constructing one per outbound message. - Reconcile channel readiness for all providers concurrently in GET /api/channels/providers. Also resolve the code-quality unused-import flag in the blocking-io anchor by pre-importing the channel service via importlib. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Fix prettier formatting in provider-state test Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Reconcile UI runtime channel config with config reload on restart Main now reloads a channel's config.yaml entry on restart_channel() (#3514, issue #3497). Adapt the user-owned connection flow to coexist: - configure_channel() restarts with reload_config=False — the caller just supplied the authoritative config (browser-entered credentials that are never written to config.yaml), so a file reload must not clobber it with the stale on-disk entry. - _load_channel_config() re-applies the UI runtime-store overlay used at startup, so an operator-triggered restart keeps browser-entered credentials for channels without a config.yaml entry and does not resurrect a channel disconnected from the UI. - Offload the reload's disk IO (config.yaml + runtime store) with asyncio.to_thread, matching the blocking-IO policy on this branch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
DanielWalnut
@@ -0,0 +1,452 @@
|
||||
import { expect, test, type Page } from "@playwright/test";
|
||||
|
||||
import { mockLangGraphAPI } from "./utils/mock-api";
|
||||
|
||||
const channelProviders = [
|
||||
["telegram", "Telegram", "deep_link"],
|
||||
["slack", "Slack", "binding_code"],
|
||||
["discord", "Discord", "binding_code"],
|
||||
["feishu", "Feishu", "binding_code"],
|
||||
["dingtalk", "DingTalk", "binding_code"],
|
||||
["wechat", "WeChat", "binding_code"],
|
||||
["wecom", "WeCom", "binding_code"],
|
||||
] as const;
|
||||
|
||||
type MockChannelProvider = {
|
||||
provider: string;
|
||||
display_name: string;
|
||||
enabled: boolean;
|
||||
configured: boolean;
|
||||
connectable: boolean;
|
||||
auth_mode: string;
|
||||
connection_status: string;
|
||||
unavailable_reason?: string | null;
|
||||
credential_fields?: Array<{
|
||||
name: string;
|
||||
label: string;
|
||||
type: string;
|
||||
required: boolean;
|
||||
}>;
|
||||
credential_values?: Record<string, string>;
|
||||
};
|
||||
|
||||
function defaultProviders(): MockChannelProvider[] {
|
||||
return channelProviders.map(([provider, displayName, authMode]) => ({
|
||||
provider,
|
||||
display_name: displayName,
|
||||
enabled: true,
|
||||
configured: true,
|
||||
connectable: true,
|
||||
auth_mode: authMode,
|
||||
connection_status: "connected",
|
||||
credential_fields: [
|
||||
{
|
||||
name: "token",
|
||||
label: "Token",
|
||||
type: "password",
|
||||
required: true,
|
||||
},
|
||||
],
|
||||
}));
|
||||
}
|
||||
|
||||
function mockChannelsAPI(
|
||||
page: Page,
|
||||
providers: MockChannelProvider[] = defaultProviders(),
|
||||
onSlackConnect?: () => void,
|
||||
) {
|
||||
void page.route("**/api/channels/providers", (route) => {
|
||||
return route.fulfill({
|
||||
status: 200,
|
||||
contentType: "application/json",
|
||||
body: JSON.stringify({
|
||||
enabled: true,
|
||||
providers,
|
||||
}),
|
||||
});
|
||||
});
|
||||
|
||||
void page.route("**/api/channels/connections", (route) => {
|
||||
return route.fulfill({
|
||||
status: 200,
|
||||
contentType: "application/json",
|
||||
body: JSON.stringify({ connections: [] }),
|
||||
});
|
||||
});
|
||||
|
||||
void page.route("**/api/channels/slack/connect", (route) => {
|
||||
onSlackConnect?.();
|
||||
return route.fulfill({
|
||||
status: 200,
|
||||
contentType: "application/json",
|
||||
body: JSON.stringify({
|
||||
provider: "slack",
|
||||
mode: "binding_code",
|
||||
url: null,
|
||||
code: "abc123",
|
||||
instruction: "Send /connect abc123 to the DeerFlow Slack bot.",
|
||||
expires_in: 600,
|
||||
}),
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
test.describe("IM channels", () => {
|
||||
test("sidebar and settings expose channel connections", async ({ page }) => {
|
||||
mockLangGraphAPI(page);
|
||||
mockChannelsAPI(page);
|
||||
|
||||
await page.goto("/workspace/chats/new");
|
||||
|
||||
const sidebar = page.locator("[data-sidebar='sidebar']");
|
||||
await expect(sidebar.getByText("Channels")).toBeVisible({
|
||||
timeout: 15_000,
|
||||
});
|
||||
await expect(sidebar.getByText("Telegram")).toBeVisible();
|
||||
await expect(sidebar.getByText("Slack")).toBeVisible();
|
||||
await expect(sidebar.getByText("Discord")).toBeVisible();
|
||||
await expect(sidebar.getByText("Feishu")).toBeVisible();
|
||||
await expect(sidebar.getByText("DingTalk")).toBeVisible();
|
||||
await expect(sidebar.getByText("WeChat")).toBeVisible();
|
||||
await expect(sidebar.getByText("WeCom")).toBeVisible();
|
||||
await expect(
|
||||
sidebar.getByRole("button", { name: "Connected" }),
|
||||
).toHaveCount(7);
|
||||
|
||||
await sidebar.getByRole("button", { name: /Settings and more/ }).click();
|
||||
await page.getByRole("menuitem", { name: "Settings" }).click();
|
||||
await page.getByRole("button", { name: "Channels" }).click();
|
||||
|
||||
await expect(page.getByText("Telegram direct messages")).toBeVisible();
|
||||
await expect(page.getByText("Slack workspace messages")).toBeVisible();
|
||||
await expect(page.getByText("Discord server messages")).toBeVisible();
|
||||
await expect(page.getByText("Feishu and Lark messages")).toBeVisible();
|
||||
await expect(page.getByText("DingTalk Stream Push messages")).toBeVisible();
|
||||
await expect(page.getByText("WeChat iLink messages")).toBeVisible();
|
||||
await expect(page.getByText("WeCom messages")).toBeVisible();
|
||||
|
||||
const dialog = page.getByRole("dialog", { name: "Settings" });
|
||||
await expect(dialog.getByRole("button", { name: "Modify" })).toHaveCount(7);
|
||||
});
|
||||
|
||||
test("only enabled providers are shown and runtime setup stays editable", async ({
|
||||
page,
|
||||
}) => {
|
||||
mockLangGraphAPI(page);
|
||||
let slackConfigured = false;
|
||||
let submittedValues: Record<string, string> | undefined;
|
||||
|
||||
void page.route("**/api/channels/providers", (route) => {
|
||||
return route.fulfill({
|
||||
status: 200,
|
||||
contentType: "application/json",
|
||||
body: JSON.stringify({
|
||||
enabled: true,
|
||||
providers: [
|
||||
{
|
||||
provider: "slack",
|
||||
display_name: "Slack",
|
||||
enabled: true,
|
||||
configured: slackConfigured,
|
||||
connectable: slackConfigured,
|
||||
auth_mode: "binding_code",
|
||||
connection_status: slackConfigured
|
||||
? "connected"
|
||||
: "not_connected",
|
||||
credential_fields: [
|
||||
{
|
||||
name: "bot_token",
|
||||
label: "Bot token",
|
||||
type: "password",
|
||||
required: true,
|
||||
},
|
||||
{
|
||||
name: "app_token",
|
||||
label: "App token",
|
||||
type: "password",
|
||||
required: true,
|
||||
},
|
||||
],
|
||||
credential_values: slackConfigured
|
||||
? {
|
||||
bot_token: "********",
|
||||
app_token: "********",
|
||||
}
|
||||
: {},
|
||||
},
|
||||
{
|
||||
provider: "discord",
|
||||
display_name: "Discord",
|
||||
enabled: false,
|
||||
configured: false,
|
||||
connectable: false,
|
||||
auth_mode: "binding_code",
|
||||
connection_status: "not_connected",
|
||||
credential_fields: [],
|
||||
},
|
||||
],
|
||||
}),
|
||||
});
|
||||
});
|
||||
|
||||
void page.route("**/api/channels/connections", (route) => {
|
||||
return route.fulfill({
|
||||
status: 200,
|
||||
contentType: "application/json",
|
||||
body: JSON.stringify({ connections: [] }),
|
||||
});
|
||||
});
|
||||
|
||||
void page.route("**/api/channels/slack/runtime-config", async (route) => {
|
||||
const body = route.request().postDataJSON() as {
|
||||
values: Record<string, string>;
|
||||
};
|
||||
submittedValues = body.values;
|
||||
slackConfigured = true;
|
||||
return route.fulfill({
|
||||
status: 200,
|
||||
contentType: "application/json",
|
||||
body: JSON.stringify({
|
||||
provider: "slack",
|
||||
display_name: "Slack",
|
||||
enabled: true,
|
||||
configured: true,
|
||||
connectable: true,
|
||||
auth_mode: "binding_code",
|
||||
connection_status: "connected",
|
||||
credential_fields: [],
|
||||
credential_values: {},
|
||||
}),
|
||||
});
|
||||
});
|
||||
|
||||
void page.route("**/api/channels/slack/connect", (route) => route.abort());
|
||||
|
||||
await page.goto("/workspace/chats/new");
|
||||
|
||||
const sidebar = page.locator("[data-sidebar='sidebar']");
|
||||
await expect(sidebar.getByText("Slack")).toBeVisible({ timeout: 15_000 });
|
||||
await expect(sidebar.getByText("Discord")).toBeHidden();
|
||||
const connectButton = sidebar.getByRole("button", { name: "Connect" });
|
||||
await expect(connectButton).toBeEnabled();
|
||||
|
||||
await connectButton.click();
|
||||
|
||||
const setupDialog = page.getByRole("dialog", { name: "Connect Slack" });
|
||||
await expect(setupDialog).toBeVisible();
|
||||
const botTokenInput = setupDialog.getByLabel("Bot token");
|
||||
await expect(botTokenInput).toHaveAttribute("type", "text");
|
||||
await expect(botTokenInput).toHaveAttribute("autocomplete", "off");
|
||||
await expect(botTokenInput).toHaveAttribute("data-lpignore", "true");
|
||||
await expect(botTokenInput).toHaveAttribute("data-1p-ignore", "true");
|
||||
await expect(botTokenInput).toHaveCSS("-webkit-text-security", "disc");
|
||||
await setupDialog.getByLabel("Bot token").fill("xoxb-ui");
|
||||
await setupDialog.getByLabel("App token").fill("xapp-ui");
|
||||
await setupDialog.getByRole("button", { name: "Save and connect" }).click();
|
||||
|
||||
await expect(setupDialog).toBeHidden();
|
||||
await expect(
|
||||
sidebar.getByRole("button", { name: "Connected" }),
|
||||
).toBeVisible();
|
||||
await sidebar.getByRole("button", { name: "Connected" }).click();
|
||||
await expect(
|
||||
page.getByRole("dialog", { name: "Modify Slack" }),
|
||||
).toBeVisible();
|
||||
await expect(page.getByLabel("Bot token")).toHaveValue("********");
|
||||
await expect(page.getByLabel("App token")).toHaveValue("********");
|
||||
expect(submittedValues).toEqual({
|
||||
bot_token: "xoxb-ui",
|
||||
app_token: "xapp-ui",
|
||||
});
|
||||
});
|
||||
|
||||
test("configured provider connects directly with a binding-code instruction", async ({
|
||||
page,
|
||||
}) => {
|
||||
mockLangGraphAPI(page);
|
||||
let slackConnectCalls = 0;
|
||||
mockChannelsAPI(
|
||||
page,
|
||||
[
|
||||
{
|
||||
provider: "slack",
|
||||
display_name: "Slack",
|
||||
enabled: true,
|
||||
configured: true,
|
||||
connectable: true,
|
||||
auth_mode: "binding_code",
|
||||
connection_status: "not_connected",
|
||||
credential_fields: [
|
||||
{
|
||||
name: "bot_token",
|
||||
label: "Bot token",
|
||||
type: "password",
|
||||
required: true,
|
||||
},
|
||||
],
|
||||
credential_values: { bot_token: "********" },
|
||||
},
|
||||
],
|
||||
() => {
|
||||
slackConnectCalls += 1;
|
||||
},
|
||||
);
|
||||
|
||||
await page.goto("/workspace/chats/new");
|
||||
|
||||
const sidebar = page.locator("[data-sidebar='sidebar']");
|
||||
await expect(sidebar.getByText("Slack")).toBeVisible({ timeout: 15_000 });
|
||||
await sidebar.getByRole("button", { name: "Connect" }).click();
|
||||
|
||||
await expect(
|
||||
page.getByText("Send /connect abc123 to the DeerFlow Slack bot."),
|
||||
).toBeVisible();
|
||||
expect(slackConnectCalls).toBe(1);
|
||||
});
|
||||
|
||||
test("runtime setup continues into the connect flow when a binding is still required", async ({
|
||||
page,
|
||||
}) => {
|
||||
mockLangGraphAPI(page);
|
||||
let slackConfigured = false;
|
||||
let slackConnectCalls = 0;
|
||||
|
||||
void page.route("**/api/channels/providers", (route) => {
|
||||
return route.fulfill({
|
||||
status: 200,
|
||||
contentType: "application/json",
|
||||
body: JSON.stringify({
|
||||
enabled: true,
|
||||
providers: [
|
||||
{
|
||||
provider: "slack",
|
||||
display_name: "Slack",
|
||||
enabled: true,
|
||||
configured: slackConfigured,
|
||||
connectable: slackConfigured,
|
||||
auth_mode: "binding_code",
|
||||
connection_status: "not_connected",
|
||||
credential_fields: [
|
||||
{
|
||||
name: "bot_token",
|
||||
label: "Bot token",
|
||||
type: "password",
|
||||
required: true,
|
||||
},
|
||||
],
|
||||
credential_values: {},
|
||||
},
|
||||
],
|
||||
}),
|
||||
});
|
||||
});
|
||||
|
||||
void page.route("**/api/channels/connections", (route) => {
|
||||
return route.fulfill({
|
||||
status: 200,
|
||||
contentType: "application/json",
|
||||
body: JSON.stringify({ connections: [] }),
|
||||
});
|
||||
});
|
||||
|
||||
void page.route("**/api/channels/slack/runtime-config", (route) => {
|
||||
slackConfigured = true;
|
||||
return route.fulfill({
|
||||
status: 200,
|
||||
contentType: "application/json",
|
||||
body: JSON.stringify({
|
||||
provider: "slack",
|
||||
display_name: "Slack",
|
||||
enabled: true,
|
||||
configured: true,
|
||||
connectable: true,
|
||||
auth_mode: "binding_code",
|
||||
connection_status: "not_connected",
|
||||
credential_fields: [],
|
||||
credential_values: {},
|
||||
}),
|
||||
});
|
||||
});
|
||||
|
||||
void page.route("**/api/channels/slack/connect", (route) => {
|
||||
slackConnectCalls += 1;
|
||||
return route.fulfill({
|
||||
status: 200,
|
||||
contentType: "application/json",
|
||||
body: JSON.stringify({
|
||||
provider: "slack",
|
||||
mode: "binding_code",
|
||||
url: null,
|
||||
code: "abc123",
|
||||
instruction: "Send /connect abc123 to the DeerFlow Slack bot.",
|
||||
expires_in: 600,
|
||||
}),
|
||||
});
|
||||
});
|
||||
|
||||
await page.goto("/workspace/chats/new");
|
||||
|
||||
const sidebar = page.locator("[data-sidebar='sidebar']");
|
||||
await expect(sidebar.getByText("Slack")).toBeVisible({ timeout: 15_000 });
|
||||
await sidebar.getByRole("button", { name: "Connect" }).click();
|
||||
|
||||
const setupDialog = page.getByRole("dialog", { name: "Connect Slack" });
|
||||
await expect(setupDialog).toBeVisible();
|
||||
await setupDialog.getByLabel("Bot token").fill("xoxb-ui");
|
||||
await setupDialog.getByRole("button", { name: "Save and connect" }).click();
|
||||
|
||||
await expect(setupDialog).toBeHidden();
|
||||
await expect(
|
||||
page.getByText("Send /connect abc123 to the DeerFlow Slack bot."),
|
||||
).toBeVisible();
|
||||
expect(slackConnectCalls).toBe(1);
|
||||
});
|
||||
|
||||
test("runtime setup dialog prefills editable credential values", async ({
|
||||
page,
|
||||
}) => {
|
||||
mockLangGraphAPI(page);
|
||||
mockChannelsAPI(page, [
|
||||
{
|
||||
provider: "feishu",
|
||||
display_name: "Feishu",
|
||||
enabled: true,
|
||||
configured: true,
|
||||
connectable: true,
|
||||
auth_mode: "binding_code",
|
||||
connection_status: "connected",
|
||||
credential_fields: [
|
||||
{
|
||||
name: "app_id",
|
||||
label: "App ID",
|
||||
type: "text",
|
||||
required: true,
|
||||
},
|
||||
{
|
||||
name: "app_secret",
|
||||
label: "App secret",
|
||||
type: "password",
|
||||
required: true,
|
||||
},
|
||||
],
|
||||
credential_values: {
|
||||
app_id: "cli_feishu_app",
|
||||
app_secret: "********",
|
||||
},
|
||||
},
|
||||
]);
|
||||
|
||||
await page.goto("/workspace/chats/new");
|
||||
|
||||
const sidebar = page.locator("[data-sidebar='sidebar']");
|
||||
await expect(sidebar.getByText("Feishu")).toBeVisible({ timeout: 15_000 });
|
||||
await sidebar.getByRole("button", { name: "Connected" }).click();
|
||||
|
||||
const setupDialog = page.getByRole("dialog", { name: "Modify Feishu" });
|
||||
await expect(setupDialog).toBeVisible();
|
||||
await expect(setupDialog.getByLabel("App ID")).toHaveValue(
|
||||
"cli_feishu_app",
|
||||
);
|
||||
await expect(setupDialog.getByLabel("App secret")).toHaveValue("********");
|
||||
});
|
||||
});
|
||||
@@ -258,4 +258,43 @@ test.describe("Thread history", () => {
|
||||
});
|
||||
await expect(main.getByText("Second conversation")).toBeVisible();
|
||||
});
|
||||
|
||||
test("IM channel threads show their source in thread lists", async ({
|
||||
page,
|
||||
}) => {
|
||||
mockLangGraphAPI(page, {
|
||||
threads: [
|
||||
{
|
||||
thread_id: MOCK_THREAD_ID,
|
||||
title: "Feishu conversation",
|
||||
updated_at: "2025-06-03T12:00:00Z",
|
||||
metadata: {
|
||||
channel_source: {
|
||||
type: "im_channel",
|
||||
provider: "feishu",
|
||||
chat_id: "oc_mock",
|
||||
},
|
||||
},
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
await page.goto("/workspace/chats/new");
|
||||
|
||||
const sidebarThread = page.locator(
|
||||
`a[href='/workspace/chats/${MOCK_THREAD_ID}']`,
|
||||
);
|
||||
await expect(sidebarThread).toBeVisible({ timeout: 15_000 });
|
||||
await expect(sidebarThread.getByLabel("Feishu channel")).toBeVisible();
|
||||
|
||||
await page.goto("/workspace/chats");
|
||||
|
||||
const mainThread = page
|
||||
.locator("main")
|
||||
.locator(`a[href='/workspace/chats/${MOCK_THREAD_ID}']`);
|
||||
await expect(mainThread.getByText("Feishu conversation")).toBeVisible({
|
||||
timeout: 15_000,
|
||||
});
|
||||
await expect(mainThread.getByText("Feishu", { exact: true })).toBeVisible();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -25,6 +25,7 @@ export type MockThread = {
|
||||
title?: string;
|
||||
updated_at?: string;
|
||||
agent_name?: string;
|
||||
metadata?: Record<string, unknown>;
|
||||
messages?: unknown[];
|
||||
artifacts?: string[];
|
||||
};
|
||||
@@ -90,7 +91,10 @@ export function mockLangGraphAPI(page: Page, options?: MockAPIOptions) {
|
||||
thread_id: t.thread_id,
|
||||
created_at: "2025-01-01T00:00:00Z",
|
||||
updated_at: t.updated_at ?? "2025-01-01T00:00:00Z",
|
||||
metadata: t.agent_name ? { agent_name: t.agent_name } : {},
|
||||
metadata: {
|
||||
...(t.metadata ?? {}),
|
||||
...(t.agent_name ? { agent_name: t.agent_name } : {}),
|
||||
},
|
||||
status: "idle",
|
||||
values: { title: t.title ?? "Untitled" },
|
||||
}));
|
||||
|
||||
Reference in New Issue
Block a user