feat(channels): enhance Discord with mention-only mode, thread routing, and typing indicators (#2842)

* feat(channels): enhance Discord with mention-only mode, thread routing, and typing indicators

Add mention_only config to only respond when bot is mentioned, with
allowed_channels override. Add thread_mode for Hermes-style auto-thread
creation. Add periodic typing indicators while bot is processing.

* fix(discord): include allowed_channels in mention_only skip condition (line 274)

* docs: fix Discord config example to match boolean thread_mode implementation

* style: format with ruff

* fix(discord): apply Copilot review fixes and resolve lint errors

- Remove unused Optional import
- Fix thread_ts type hints to str | None
- Fix has_mention logic for None values
- Implement thread_mode fallback to channel replies on thread creation failure
- Fix thread_mode docstring alignment
- Fix allowed_channels comment formatting in config.example.yaml

* fix(discord): reset context for orphaned threads in mention_only mode

When a message arrives in a thread not tracked by _active_threads,
clear thread_id and typing_target so the message falls through to
the standard channel handling pipeline, which creates a fresh thread
instead of incorrectly routing to the stale thread.

* fix(discord): create new thread on @ when channel has existing tracked thread

When mention_only is enabled and a user @-s the bot in a channel
that already has a tracked thread, create a new thread instead of
incorrectly routing to the old one.

* fix(discord): allow no-@ thread replies while skipping no-@ channel messages

The skip block for no-@ messages was too aggressive — it blocked
continuation replies within tracked threads AND incorrectly routed
no-@ channel messages to the existing thread.

Now:
- Thread message, no @ → routed to existing tracked thread
- Channel message, no @ → skipped
- Channel message, with @ → creates new thread

* feat(discord): add checkmark reaction to acknowledge received messages

* Move discord.py to optional dependency and auto-detect from config.yaml

- Add discord extra to [project.optional-dependencies] in pyproject.toml
- Update detect_uv_extras.py to map channels.discord.enabled: true -> --extra discord
- Set UV_EXTRAS=discord in docker-compose-dev.yaml gateway env

* fix(discord): persist thread-channel mappings to store for recovery after restart

Discord's _active_threads dict was purely in-memory, so all channel-to-thread
mappings were lost on server restart. This fix bridges ChannelStore into
DiscordChannel:

- Save thread mappings to store.json after every thread creation
- Restore active threads from store on DiscordChannel startup
- Pass channel_store to all channels via service.py config injection

Store keys follow the pattern: discord:<channel_id>:<thread_id>

* fix(discord): address Copilot review — fix types, typing targets, cross-thread safety, and config comments

* fix(tests): add multitask_strategy param to mock for clarification follow-up test

* fix(tests): explicitly set model_name=None for title middleware test isolation

* fix(discord): use trigger_typing() instead of typing() for typing indicators

discord.py 2.x TextChannel.typing() and Thread.typing() are async context
managers, not one-shot coroutines. Use trigger_typing() for periodic
typing indicator pings.

* fix(discord): cancel typing tasks on channel shutdown

Prevents 'Task was destroyed but it is pending' warnings when the
Discord client stops while typing indicator loops are still running.

* fix(scripts): detect nested YAML config for discord extra

section_value() only matched top-level YAML sections. Added
nested_section_value() that handles two-level nesting (e.g.,
channels.discord.enabled), so auto-detection of the discord
extra works when config uses the standard nested format.

* fix(docker): remove hard-coded UV_EXTRAS=discord from dev compose

Relies on auto-detection via detect_uv_extras.py instead of forcing
discord.py install even when channels.discord.enabled is false.
Matches production docker-compose.yaml behavior (UV_EXTRAS:-).

* refactor(nginx): move proxy_buffering/proxy_cache to server level

DRY cleanup — these directives were repeated in 14 location blocks.
Set at server level once, reducing duplication and risk of drift.

* fix(discord): use dedicated JSON file for thread persistence

Replace ChannelStore usage for Discord thread-ID persistence with a
dedicated discord_threads.json file. ChannelStore is designed to map
IM conversations to DeerFlow thread IDs — using it to persist Discord
thread IDs was semantically wrong and confusing.

Changes:
- _save_thread() now reads/writes a simple {channel_id: thread_id} JSON dict
- _load_active_threads() reads directly from the JSON file
- File path derived from ChannelStore directory (when available) or
  defaults to ~/.deer-flow/channels/discord_threads.json
- Removed unused ChannelStore import

* fix(discord): address WillemJiang's code review comments on PR #2842

1. Remove semantically incorrect message_in_thread variable. At this code
   point (after the Thread case is handled above), we're guaranteed to be in
   a channel, not a thread. Always apply mention_only check here.

2. Add _active_thread_ids reverse-lookup set for O(1) thread ID membership
   checks instead of O(n) scan of _active_threads.values(). Keep the set
   in sync with _active_threads in _load_active_threads() and _save_thread().

3. Add _thread_store_lock (threading.Lock) to protect _active_threads and
   the JSON file from concurrent access between the Discord loop thread
   (_run_client) and the main thread (_load_active_threads, _save_thread).
This commit is contained in:
Yi Tang
2026-05-15 15:30:05 +01:00
committed by GitHub
parent 7c42ab3e16
commit 48e038f752
12 changed files with 482 additions and 26 deletions
+291 -11
View File
@@ -3,8 +3,10 @@
from __future__ import annotations from __future__ import annotations
import asyncio import asyncio
import json
import logging import logging
import threading import threading
from pathlib import Path
from typing import Any from typing import Any
from app.channels.base import Channel from app.channels.base import Channel
@@ -21,6 +23,12 @@ class DiscordChannel(Channel):
Configuration keys (in ``config.yaml`` under ``channels.discord``): Configuration keys (in ``config.yaml`` under ``channels.discord``):
- ``bot_token``: Discord Bot token. - ``bot_token``: Discord Bot token.
- ``allowed_guilds``: (optional) List of allowed Discord guild IDs. Empty = allow all. - ``allowed_guilds``: (optional) List of allowed Discord guild IDs. Empty = allow all.
- ``mention_only``: (optional) If true, only respond when the bot is mentioned.
- ``allowed_channels``: (optional) List of channel IDs where messages are always accepted
(even when mention_only is true). Use for channels where you want the bot to respond
without mentions. Empty = mention_only applies everywhere.
- ``thread_mode``: (optional) If true, group a channel conversation into a thread.
Default: same as ``mention_only``.
""" """
def __init__(self, bus: MessageBus, config: dict[str, Any]) -> None: def __init__(self, bus: MessageBus, config: dict[str, Any]) -> None:
@@ -32,6 +40,29 @@ class DiscordChannel(Channel):
self._allowed_guilds.add(int(guild_id)) self._allowed_guilds.add(int(guild_id))
except (TypeError, ValueError): except (TypeError, ValueError):
continue continue
self._mention_only: bool = bool(config.get("mention_only", False))
self._thread_mode: bool = config.get("thread_mode", self._mention_only)
self._allowed_channels: set[str] = set()
for channel_id in config.get("allowed_channels", []):
self._allowed_channels.add(str(channel_id))
# Session tracking: channel_id -> Discord thread_id (in-memory, persisted to JSON).
# Uses a dedicated JSON file separate from ChannelStore, which maps IM
# conversations to DeerFlow thread IDs — a different concern.
self._active_threads: dict[str, str] = {}
# Reverse-lookup set for O(1) thread ID checks (avoids O(n) scan of _active_threads.values()).
self._active_thread_ids: set[str] = set()
# Lock protecting _active_threads and the JSON file from concurrent access.
# _run_client (Discord loop thread) and the main thread both read/write.
self._thread_store_lock = threading.Lock()
store = config.get("channel_store")
if store is not None:
self._thread_store_path = store._path.parent / "discord_threads.json"
else:
self._thread_store_path = Path.home() / ".deer-flow" / "channels" / "discord_threads.json"
# Typing indicator management
self._typing_tasks: dict[str, asyncio.Task] = {}
self._client = None self._client = None
self._thread: threading.Thread | None = None self._thread: threading.Thread | None = None
@@ -75,12 +106,56 @@ class DiscordChannel(Channel):
self._thread = threading.Thread(target=self._run_client, daemon=True) self._thread = threading.Thread(target=self._run_client, daemon=True)
self._thread.start() self._thread.start()
self._load_active_threads()
logger.info("Discord channel started") logger.info("Discord channel started")
def _load_active_threads(self) -> None:
"""Restore Discord thread mappings from the dedicated JSON file on startup."""
with self._thread_store_lock:
try:
if not self._thread_store_path.exists():
logger.debug("[Discord] no thread mappings file at %s", self._thread_store_path)
return
data = json.loads(self._thread_store_path.read_text())
self._active_threads.clear()
self._active_thread_ids.clear()
for channel_id, thread_id in data.items():
self._active_threads[channel_id] = thread_id
self._active_thread_ids.add(thread_id)
if self._active_threads:
logger.info("[Discord] restored %d thread mappings from %s", len(self._active_threads), self._thread_store_path)
except Exception:
logger.exception("[Discord] failed to load thread mappings")
def _save_thread(self, channel_id: str, thread_id: str) -> None:
"""Persist a Discord thread mapping to the dedicated JSON file."""
with self._thread_store_lock:
try:
data: dict[str, str] = {}
if self._thread_store_path.exists():
data = json.loads(self._thread_store_path.read_text())
old_id = data.get(channel_id)
data[channel_id] = thread_id
# Update reverse-lookup set
if old_id:
self._active_thread_ids.discard(old_id)
self._active_thread_ids.add(thread_id)
self._thread_store_path.parent.mkdir(parents=True, exist_ok=True)
self._thread_store_path.write_text(json.dumps(data, indent=2))
except Exception:
logger.exception("[Discord] failed to save thread mapping for channel %s", channel_id)
async def stop(self) -> None: async def stop(self) -> None:
self._running = False self._running = False
self.bus.unsubscribe_outbound(self._on_outbound) self.bus.unsubscribe_outbound(self._on_outbound)
# Cancel all active typing indicator tasks
for target_id, task in list(self._typing_tasks.items()):
if not task.done():
task.cancel()
logger.debug("[Discord] cancelled typing task for target %s", target_id)
self._typing_tasks.clear()
if self._client and self._discord_loop and self._discord_loop.is_running(): if self._client and self._discord_loop and self._discord_loop.is_running():
close_future = asyncio.run_coroutine_threadsafe(self._client.close(), self._discord_loop) close_future = asyncio.run_coroutine_threadsafe(self._client.close(), self._discord_loop)
try: try:
@@ -100,6 +175,10 @@ class DiscordChannel(Channel):
logger.info("Discord channel stopped") logger.info("Discord channel stopped")
async def send(self, msg: OutboundMessage) -> None: async def send(self, msg: OutboundMessage) -> None:
# Stop typing indicator once we're sending the response
stop_future = asyncio.run_coroutine_threadsafe(self._stop_typing(msg.chat_id, msg.thread_ts), self._discord_loop)
await asyncio.wrap_future(stop_future)
target = await self._resolve_target(msg) target = await self._resolve_target(msg)
if target is None: if target is None:
logger.error("[Discord] target not found for chat_id=%s thread_ts=%s", msg.chat_id, msg.thread_ts) logger.error("[Discord] target not found for chat_id=%s thread_ts=%s", msg.chat_id, msg.thread_ts)
@@ -111,6 +190,9 @@ class DiscordChannel(Channel):
await asyncio.wrap_future(send_future) await asyncio.wrap_future(send_future)
async def send_file(self, msg: OutboundMessage, attachment: ResolvedAttachment) -> bool: async def send_file(self, msg: OutboundMessage, attachment: ResolvedAttachment) -> bool:
stop_future = asyncio.run_coroutine_threadsafe(self._stop_typing(msg.chat_id, msg.thread_ts), self._discord_loop)
await asyncio.wrap_future(stop_future)
target = await self._resolve_target(msg) target = await self._resolve_target(msg)
if target is None: if target is None:
logger.error("[Discord] target not found for file upload chat_id=%s thread_ts=%s", msg.chat_id, msg.thread_ts) logger.error("[Discord] target not found for file upload chat_id=%s thread_ts=%s", msg.chat_id, msg.thread_ts)
@@ -130,6 +212,41 @@ class DiscordChannel(Channel):
logger.exception("[Discord] failed to upload file: %s", attachment.filename) logger.exception("[Discord] failed to upload file: %s", attachment.filename)
return False return False
async def _start_typing(self, channel, chat_id: str, thread_ts: str | None = None) -> None:
"""Starts a loop to send periodic typing indicators."""
target_id = thread_ts or chat_id
if target_id in self._typing_tasks:
return # Already typing for this target
async def _typing_loop():
try:
while True:
try:
await channel.trigger_typing()
except Exception:
pass
await asyncio.sleep(10)
except asyncio.CancelledError:
pass
task = asyncio.create_task(_typing_loop())
self._typing_tasks[target_id] = task
async def _stop_typing(self, chat_id: str, thread_ts: str | None = None) -> None:
"""Stops the typing loop for a specific target."""
target_id = thread_ts or chat_id
task = self._typing_tasks.pop(target_id, None)
if task and not task.done():
task.cancel()
logger.debug("[Discord] stopped typing indicator for target %s", target_id)
async def _add_reaction(self, message) -> None:
"""Add a checkmark reaction to acknowledge the message was received."""
try:
await message.add_reaction("")
except Exception:
logger.debug("[Discord] failed to add reaction to message %s", message.id, exc_info=True)
async def _on_message(self, message) -> None: async def _on_message(self, message) -> None:
if not self._running or not self._client: if not self._running or not self._client:
return return
@@ -152,15 +269,143 @@ class DiscordChannel(Channel):
if self._discord_module is None: if self._discord_module is None:
return return
if isinstance(message.channel, self._discord_module.Thread): # Determine whether the bot is mentioned in this message
chat_id = str(message.channel.parent_id or message.channel.id) user = self._client.user if self._client else None
thread_id = str(message.channel.id) if user:
bot_mention = user.mention # <@ID>
alt_mention = f"<@!{user.id}>" # <@!ID> (ping variant)
standard_mention = f"<@{user.id}>"
else: else:
thread = await self._create_thread(message) bot_mention = None
if thread is None: alt_mention = None
standard_mention = ""
has_mention = (bot_mention and bot_mention in message.content) or (alt_mention and alt_mention in message.content) or (standard_mention and standard_mention in message.content)
# Strip mention from text for processing
if has_mention:
text = text.replace(bot_mention or "", "").replace(alt_mention or "", "").replace(standard_mention or "", "").strip()
# Don't return early if text is empty — still process the mention (e.g., create thread)
# --- Determine thread/channel routing and typing target ---
thread_id = None
chat_id = None
typing_target = None # The Discord object to type into
if isinstance(message.channel, self._discord_module.Thread):
# --- Message already inside a thread ---
thread_obj = message.channel
thread_id = str(thread_obj.id)
chat_id = str(thread_obj.parent_id or thread_obj.id)
typing_target = thread_obj
# If this is a known active thread, process normally
if thread_id in self._active_thread_ids:
msg_type = InboundMessageType.COMMAND if text.startswith("/") else InboundMessageType.CHAT
inbound = self._make_inbound(
chat_id=chat_id,
user_id=str(message.author.id),
text=text,
msg_type=msg_type,
thread_ts=thread_id,
metadata={
"guild_id": str(guild.id) if guild else None,
"channel_id": str(message.channel.id),
"message_id": str(message.id),
},
)
inbound.topic_id = thread_id
self._publish(inbound)
# Start typing indicator in the thread
if typing_target:
asyncio.create_task(self._start_typing(typing_target, chat_id, thread_id))
asyncio.create_task(self._add_reaction(message))
return return
chat_id = str(message.channel.id)
thread_id = str(thread.id) # Thread not tracked (orphaned) — create new thread and handle below
logger.debug("[Discord] message in orphaned thread %s, will create new thread", thread_id)
thread_id = None
typing_target = None
# At this point we're guaranteed to be in a channel, not a thread
# (the Thread case is handled above). Apply mention_only for all
# non-thread messages — no special case needed.
channel_id = str(message.channel.id)
# Check if there's an active thread for this channel
if channel_id in self._active_threads:
# respect mention_only: if enabled, only process messages that mention the bot
# (unless the channel is in allowed_channels)
# Messages within a thread are always allowed through (continuation).
# At this code point we know the message is in a channel, not a thread
# (Thread case handled above), so always apply the check.
if self._mention_only and not has_mention and channel_id not in self._allowed_channels:
logger.debug("[Discord] skipping no-@ message in channel %s (not in thread)", channel_id)
return
# mention_only + fresh @ → create new thread instead of routing to existing one
if self._mention_only and has_mention:
thread_obj = await self._create_thread(message)
if thread_obj is not None:
target_thread_id = str(thread_obj.id)
self._active_threads[channel_id] = target_thread_id
self._save_thread(channel_id, target_thread_id)
thread_id = target_thread_id
chat_id = channel_id
typing_target = thread_obj
logger.info("[Discord] created new thread %s in channel %s on mention (replacing existing thread)", target_thread_id, channel_id)
else:
logger.info("[Discord] thread creation failed in channel %s, falling back to channel replies", channel_id)
thread_id = channel_id
chat_id = channel_id
typing_target = message.channel
else:
# Existing session → route to the existing thread
target_thread_id = self._active_threads[channel_id]
logger.debug("[Discord] routing message in channel %s to existing thread %s", channel_id, target_thread_id)
thread_id = target_thread_id
chat_id = channel_id
typing_target = await self._get_channel_or_thread(target_thread_id)
elif self._mention_only and not has_mention and channel_id not in self._allowed_channels:
# Not mentioned and not in an allowed channel → skip
logger.debug("[Discord] skipping message without mention in channel %s", channel_id)
return
elif self._mention_only and has_mention:
# First mention in this channel → create thread
thread_obj = await self._create_thread(message)
if thread_obj is not None:
target_thread_id = str(thread_obj.id)
self._active_threads[channel_id] = target_thread_id
self._save_thread(channel_id, target_thread_id)
thread_id = target_thread_id
chat_id = channel_id
typing_target = thread_obj # Type into the new thread
logger.info("[Discord] created thread %s in channel %s for user %s", target_thread_id, channel_id, message.author.display_name)
else:
# Fallback: thread creation failed (disabled/permissions), reply in channel
logger.info("[Discord] thread creation failed in channel %s, falling back to channel replies", channel_id)
thread_id = channel_id
chat_id = channel_id
typing_target = message.channel # Type into the channel
elif self._thread_mode:
# thread_mode but mention_only is False → create thread anyway for conversation grouping
thread_obj = await self._create_thread(message)
if thread_obj is None:
# Thread creation failed (disabled/permissions), fall back to channel replies
logger.info("[Discord] thread creation failed in channel %s, falling back to channel replies", channel_id)
thread_id = channel_id
chat_id = channel_id
typing_target = message.channel # Type into the channel
else:
target_thread_id = str(thread_obj.id)
self._active_threads[channel_id] = target_thread_id
self._save_thread(channel_id, target_thread_id)
thread_id = target_thread_id
chat_id = channel_id
typing_target = thread_obj # Type into the new thread
else:
# No threading — reply directly in channel
thread_id = channel_id
chat_id = channel_id
typing_target = message.channel # Type into the channel
msg_type = InboundMessageType.COMMAND if text.startswith("/") else InboundMessageType.CHAT msg_type = InboundMessageType.COMMAND if text.startswith("/") else InboundMessageType.CHAT
inbound = self._make_inbound( inbound = self._make_inbound(
@@ -177,6 +422,15 @@ class DiscordChannel(Channel):
) )
inbound.topic_id = thread_id inbound.topic_id = thread_id
# Start typing indicator in the correct target (thread or channel)
if typing_target:
asyncio.create_task(self._start_typing(typing_target, chat_id, thread_id))
self._publish(inbound)
asyncio.create_task(self._add_reaction(message))
def _publish(self, inbound) -> None:
"""Publish an inbound message to the main event loop."""
if self._main_loop and self._main_loop.is_running(): if self._main_loop and self._main_loop.is_running():
future = asyncio.run_coroutine_threadsafe(self.bus.publish_inbound(inbound), self._main_loop) future = asyncio.run_coroutine_threadsafe(self.bus.publish_inbound(inbound), self._main_loop)
future.add_done_callback(lambda f: logger.exception("[Discord] publish_inbound failed", exc_info=f.exception()) if f.exception() else None) future.add_done_callback(lambda f: logger.exception("[Discord] publish_inbound failed", exc_info=f.exception()) if f.exception() else None)
@@ -198,14 +452,40 @@ class DiscordChannel(Channel):
async def _create_thread(self, message): async def _create_thread(self, message):
try: try:
if self._discord_module is None:
return None
# Only TextChannel (type 0) and NewsChannel (type 10) support threads
channel_type = message.channel.type
if channel_type not in (
self._discord_module.ChannelType.text,
self._discord_module.ChannelType.news,
):
logger.info(
"[Discord] channel type %s (%s) does not support threads",
channel_type.value,
channel_type.name,
)
return None
thread_name = f"deerflow-{message.author.display_name}-{message.id}"[:100] thread_name = f"deerflow-{message.author.display_name}-{message.id}"[:100]
return await message.create_thread(name=thread_name) return await message.create_thread(name=thread_name)
except self._discord_module.errors.HTTPException as exc:
if exc.code == 50024:
logger.info(
"[Discord] cannot create thread in channel %s (error code 50024): %s",
message.channel.id,
channel_type.name if (channel_type := message.channel.type) else "unknown",
)
else:
logger.exception(
"[Discord] failed to create thread for message=%s (HTTPException %s)",
message.id,
exc.code,
)
return None
except Exception: except Exception:
logger.exception("[Discord] failed to create thread for message=%s (threads may be disabled or missing permissions)", message.id) logger.exception("[Discord] failed to create thread for message=%s (threads may be disabled or missing permissions)", message.id)
try:
await message.channel.send("Could not create a thread for your message. Please check that threads are enabled in this channel.")
except Exception:
pass
return None return None
async def _resolve_target(self, msg: OutboundMessage): async def _resolve_target(self, msg: OutboundMessage):
+9
View File
@@ -787,13 +787,22 @@ class ChannelManager:
return return
logger.info("[Manager] invoking runs.wait(thread_id=%s, text=%r)", thread_id, msg.text[:100]) logger.info("[Manager] invoking runs.wait(thread_id=%s, text=%r)", thread_id, msg.text[:100])
try:
result = await client.runs.wait( result = await client.runs.wait(
thread_id, thread_id,
assistant_id, assistant_id,
input={"messages": [{"role": "human", "content": msg.text}]}, input={"messages": [{"role": "human", "content": msg.text}]},
config=run_config, config=run_config,
context=run_context, context=run_context,
multitask_strategy="reject",
) )
except Exception as exc:
if _is_thread_busy_error(exc):
logger.warning("[Manager] thread busy (concurrent run rejected): thread_id=%s", thread_id)
await self._send_error(msg, THREAD_BUSY_MESSAGE)
return
else:
raise
response_text = _extract_response_text(result) response_text = _extract_response_text(result)
artifacts = _extract_artifacts(result) artifacts = _extract_artifacts(result)
+2
View File
@@ -167,6 +167,8 @@ class ChannelService:
return False return False
try: try:
config = dict(config)
config["channel_store"] = self.store
channel = channel_cls(bus=self.bus, config=config) channel = channel_cls(bus=self.bus, config=config)
self._channels[name] = channel self._channels[name] = channel
await channel.start() await channel.start()
+1
View File
@@ -25,6 +25,7 @@ dependencies = [
[project.optional-dependencies] [project.optional-dependencies]
postgres = ["deerflow-harness[postgres]"] postgres = ["deerflow-harness[postgres]"]
discord = ["discord.py>=2.7.0"]
[dependency-groups] [dependency-groups]
dev = [ dev = [
+1 -1
View File
@@ -761,7 +761,7 @@ class TestChannelManager:
history_by_checkpoint: dict[tuple[str, str], list[str]] = {} history_by_checkpoint: dict[tuple[str, str], list[str]] = {}
async def _runs_wait(thread_id, assistant_id, *, input, config, context): async def _runs_wait(thread_id, assistant_id, *, input, config, context, multitask_strategy=None):
del assistant_id, context # unused in this test, kept for signature parity del assistant_id, context # unused in this test, kept for signature parity
checkpoint_ns = config.get("configurable", {}).get("checkpoint_ns") checkpoint_ns = config.get("configurable", {}).get("checkpoint_ns")
-1
View File
@@ -454,7 +454,6 @@ class TestAStream:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_with_tools_emits_tool_call_chunk(self): async def test_with_tools_emits_tool_call_chunk(self):
tool_calls = [{"name": "fn", "args": {}, "id": "c1"}] tool_calls = [{"name": "fn", "args": {}, "id": "c1"}]
with patch.object(MindIEChatModel, "_agenerate", new_callable=AsyncMock) as mock_ag, patch.object(MindIEChatModel, "__init__", return_value=None): with patch.object(MindIEChatModel, "_agenerate", new_callable=AsyncMock) as mock_ag, patch.object(MindIEChatModel, "__init__", return_value=None):
mock_ag.return_value = _make_chat_result("ok", tool_calls=tool_calls) mock_ag.return_value = _make_chat_result("ok", tool_calls=tool_calls)
@@ -93,7 +93,7 @@ class TestTitleMiddlewareCoreLogic:
assert middleware._should_generate_title(state) is False assert middleware._should_generate_title(state) is False
def test_generate_title_uses_async_model_and_respects_max_chars(self, monkeypatch): def test_generate_title_uses_async_model_and_respects_max_chars(self, monkeypatch):
_set_test_title_config(max_chars=12) _set_test_title_config(max_chars=12, model_name=None)
middleware = TitleMiddleware() middleware = TitleMiddleware()
model = MagicMock() model = MagicMock()
model.ainvoke = AsyncMock(return_value=AIMessage(content="短标题")) model.ainvoke = AsyncMock(return_value=AIMessage(content="短标题"))
+19 -2
View File
@@ -1,5 +1,5 @@
version = 1 version = 1
revision = 3 revision = 2
requires-python = ">=3.12" requires-python = ">=3.12"
resolution-markers = [ resolution-markers = [
"python_full_version >= '3.14' and sys_platform == 'win32'", "python_full_version >= '3.14' and sys_platform == 'win32'",
@@ -763,6 +763,9 @@ dependencies = [
] ]
[package.optional-dependencies] [package.optional-dependencies]
discord = [
{ name = "discord-py" },
]
postgres = [ postgres = [
{ name = "deerflow-harness", extra = ["postgres"] }, { name = "deerflow-harness", extra = ["postgres"] },
] ]
@@ -781,6 +784,7 @@ requires-dist = [
{ name = "deerflow-harness", editable = "packages/harness" }, { name = "deerflow-harness", editable = "packages/harness" },
{ name = "deerflow-harness", extras = ["postgres"], marker = "extra == 'postgres'", editable = "packages/harness" }, { name = "deerflow-harness", extras = ["postgres"], marker = "extra == 'postgres'", editable = "packages/harness" },
{ name = "dingtalk-stream", specifier = ">=0.24.3" }, { name = "dingtalk-stream", specifier = ">=0.24.3" },
{ name = "discord-py", marker = "extra == 'discord'", specifier = ">=2.7.0" },
{ name = "email-validator", specifier = ">=2.0.0" }, { name = "email-validator", specifier = ">=2.0.0" },
{ name = "fastapi", specifier = ">=0.115.0" }, { name = "fastapi", specifier = ">=0.115.0" },
{ name = "httpx", specifier = ">=0.28.0" }, { name = "httpx", specifier = ">=0.28.0" },
@@ -795,7 +799,7 @@ requires-dist = [
{ name = "uvicorn", extras = ["standard"], specifier = ">=0.34.0" }, { name = "uvicorn", extras = ["standard"], specifier = ">=0.34.0" },
{ name = "wecom-aibot-python-sdk", specifier = ">=0.1.6" }, { name = "wecom-aibot-python-sdk", specifier = ">=0.1.6" },
] ]
provides-extras = ["postgres"] provides-extras = ["postgres", "discord"]
[package.metadata.requires-dev] [package.metadata.requires-dev]
dev = [ dev = [
@@ -923,6 +927,19 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/4c/44/102dede3f371277598df6aa9725b82e3add068c729333c7a5dbc12764579/dingtalk_stream-0.24.3-py3-none-any.whl", hash = "sha256:2160403656985962878bf60cdf5adf41619f21067348e06f07a7c7eebf5943ad", size = 27813, upload-time = "2025-10-24T09:36:57.497Z" }, { url = "https://files.pythonhosted.org/packages/4c/44/102dede3f371277598df6aa9725b82e3add068c729333c7a5dbc12764579/dingtalk_stream-0.24.3-py3-none-any.whl", hash = "sha256:2160403656985962878bf60cdf5adf41619f21067348e06f07a7c7eebf5943ad", size = 27813, upload-time = "2025-10-24T09:36:57.497Z" },
] ]
[[package]]
name = "discord-py"
version = "2.7.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "aiohttp" },
{ name = "audioop-lts", marker = "python_full_version >= '3.13'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/ef/57/9a2d9abdabdc9db8ef28ce0cf4129669e1c8717ba28d607b5ba357c4de3b/discord_py-2.7.1.tar.gz", hash = "sha256:24d5e6a45535152e4b98148a9dd6b550d25dc2c9fb41b6d670319411641249da", size = 1106326, upload-time = "2026-03-03T18:40:46.24Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/f7/a7/17208c3b3f92319e7fad259f1c6d5a5baf8fd0654c54846ced329f83c3eb/discord_py-2.7.1-py3-none-any.whl", hash = "sha256:849dca2c63b171146f3a7f3f8acc04248098e9e6203412ce3cf2745f284f7439", size = 1227550, upload-time = "2026-03-03T18:40:44.492Z" },
]
[[package]] [[package]]
name = "distro" name = "distro"
version = "1.9.0" version = "1.9.0"
+8
View File
@@ -1029,6 +1029,14 @@ run_events:
# client_secret: $DINGTALK_CLIENT_SECRET # client_secret: $DINGTALK_CLIENT_SECRET
# allowed_users: [] # empty = allow all # allowed_users: [] # empty = allow all
# card_template_id: "" # Optional: AI Card template ID for streaming updates # card_template_id: "" # Optional: AI Card template ID for streaming updates
#
# discord:
# enabled: false
# bot_token: $DISCORD_BOT_TOKEN
# allowed_guilds: [] # empty = allow all guilds; can also be a single guild ID
# mention_only: false # If true, only respond when the bot is mentioned
# allowed_channels: [] # Optional: channel IDs exempt from mention_only (bot responds without mention)
# thread_mode: false # If true, group a channel conversation into a thread
# ============================================================================ # ============================================================================
# Guardrails Configuration # Guardrails Configuration
+20 -2
View File
@@ -28,6 +28,10 @@ http {
set $gateway_upstream gateway:8001; set $gateway_upstream gateway:8001;
set $frontend_upstream frontend:3000; set $frontend_upstream frontend:3000;
# Default proxy settings for all locations (streaming/SSE support)
proxy_buffering off;
proxy_cache off;
# Keep the unified nginx endpoint same-origin by default. When split # Keep the unified nginx endpoint same-origin by default. When split
# frontend/backend or port-forwarded deployments need browser CORS, # frontend/backend or port-forwarded deployments need browser CORS,
# configure the Gateway allowlist with GATEWAY_CORS_ORIGINS so CORS and # configure the Gateway allowlist with GATEWAY_CORS_ORIGINS so CORS and
@@ -49,8 +53,6 @@ http {
proxy_set_header Connection ''; proxy_set_header Connection '';
# SSE/Streaming support # SSE/Streaming support
proxy_buffering off;
proxy_cache off;
proxy_set_header X-Accel-Buffering no; proxy_set_header X-Accel-Buffering no;
# Timeouts for long-running requests # Timeouts for long-running requests
@@ -70,6 +72,7 @@ http {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
} }
# Custom API: Memory endpoint # Custom API: Memory endpoint
@@ -80,6 +83,7 @@ http {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
} }
# Custom API: MCP configuration endpoint # Custom API: MCP configuration endpoint
@@ -90,6 +94,7 @@ http {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
} }
# Custom API: Skills configuration endpoint # Custom API: Skills configuration endpoint
@@ -100,6 +105,7 @@ http {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
} }
# Custom API: Agents endpoint # Custom API: Agents endpoint
@@ -110,6 +116,7 @@ http {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
} }
# Custom API: Uploads endpoint # Custom API: Uploads endpoint
@@ -124,6 +131,8 @@ http {
# Large file upload support # Large file upload support
client_max_body_size 100M; client_max_body_size 100M;
proxy_request_buffering off; proxy_request_buffering off;
# Disable response buffering to avoid permission errors
} }
# Custom API: Other endpoints under /api/threads # Custom API: Other endpoints under /api/threads
@@ -134,6 +143,7 @@ http {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
} }
# API Documentation: Swagger UI # API Documentation: Swagger UI
@@ -144,6 +154,7 @@ http {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
} }
# API Documentation: ReDoc # API Documentation: ReDoc
@@ -154,6 +165,7 @@ http {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
} }
# API Documentation: OpenAPI Schema # API Documentation: OpenAPI Schema
@@ -164,6 +176,7 @@ http {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
} }
# Health check endpoint (gateway) # Health check endpoint (gateway)
@@ -174,6 +187,7 @@ http {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
} }
# ── Provisioner API (sandbox management) ──────────────────────── # ── Provisioner API (sandbox management) ────────────────────────
@@ -187,6 +201,7 @@ http {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
} }
# Catch-all for /api/ routes not covered above (e.g. /api/v1/auth/*). # Catch-all for /api/ routes not covered above (e.g. /api/v1/auth/*).
@@ -198,6 +213,9 @@ http {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
# Disable buffering to avoid permission errors when nginx
# runs as a non-root user (e.g. local development).
} }
# All other requests go to frontend # All other requests go to frontend
+41
View File
@@ -70,6 +70,11 @@ http {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
# Disable buffering to avoid permission errors when nginx
# runs as a non-root user (e.g. local development).
proxy_buffering off;
proxy_cache off;
} }
# Custom API: Memory endpoint # Custom API: Memory endpoint
@@ -80,6 +85,9 @@ http {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_cache off;
} }
# Custom API: MCP configuration endpoint # Custom API: MCP configuration endpoint
@@ -90,6 +98,9 @@ http {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_cache off;
} }
# Custom API: Skills configuration endpoint # Custom API: Skills configuration endpoint
@@ -100,6 +111,9 @@ http {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_cache off;
} }
# Custom API: Agents endpoint # Custom API: Agents endpoint
@@ -110,6 +124,9 @@ http {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_cache off;
} }
# Custom API: Uploads endpoint # Custom API: Uploads endpoint
@@ -124,6 +141,10 @@ http {
# Large file upload support # Large file upload support
client_max_body_size 100M; client_max_body_size 100M;
proxy_request_buffering off; proxy_request_buffering off;
# Disable response buffering to avoid permission errors
proxy_buffering off;
proxy_cache off;
} }
# Custom API: Other endpoints under /api/threads # Custom API: Other endpoints under /api/threads
@@ -134,6 +155,9 @@ http {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_cache off;
} }
# API Documentation: Swagger UI # API Documentation: Swagger UI
@@ -144,6 +168,9 @@ http {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_cache off;
} }
# API Documentation: ReDoc # API Documentation: ReDoc
@@ -154,6 +181,9 @@ http {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_cache off;
} }
# API Documentation: OpenAPI Schema # API Documentation: OpenAPI Schema
@@ -164,6 +194,9 @@ http {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_cache off;
} }
# Health check endpoint (gateway) # Health check endpoint (gateway)
@@ -174,6 +207,9 @@ http {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_cache off;
} }
# Catch-all for any /api/* prefix not matched by a more specific block above. # Catch-all for any /api/* prefix not matched by a more specific block above.
@@ -193,6 +229,11 @@ http {
# Auth endpoints set HttpOnly cookies — make sure nginx doesn't # Auth endpoints set HttpOnly cookies — make sure nginx doesn't
# strip the Set-Cookie header from upstream responses. # strip the Set-Cookie header from upstream responses.
proxy_pass_header Set-Cookie; proxy_pass_header Set-Cookie;
# Disable buffering to avoid permission errors when nginx
# runs as a non-root user (e.g. local development).
proxy_buffering off;
proxy_cache off;
} }
# All other requests go to frontend # All other requests go to frontend
+81
View File
@@ -72,6 +72,7 @@ def find_config_file() -> Path | None:
_SECTION_RE = re.compile(r"^([A-Za-z_][\w-]*)\s*:\s*$") _SECTION_RE = re.compile(r"^([A-Za-z_][\w-]*)\s*:\s*$")
_INDENTED_SECTION_RE = re.compile(r"^\s+([A-Za-z_][\w-]*)\s*:\s*$")
_KEY_RE = re.compile(r"^\s+([A-Za-z_][\w-]*)\s*:\s*(\S.*?)\s*$") _KEY_RE = re.compile(r"^\s+([A-Za-z_][\w-]*)\s*:\s*(\S.*?)\s*$")
@@ -141,6 +142,84 @@ def section_value(lines: list[str], section: str, key: str) -> str | None:
return None return None
def nested_section_value(lines: list[str], section_path: str, key: str) -> str | None:
"""Return the value of a nested YAML key like ``channels.discord.enabled``.
Handles two levels of nesting:
channels:
discord:
enabled: true
"""
parts = section_path.split(".")
if len(parts) != 2:
return None
parent_section, child_section = parts
inside_parent = False
inside_child = False
parent_indent: int | None = None
child_indent: int | None = None
for raw in lines:
line = _strip_comment(raw)
if not line.strip():
continue
stripped = line.lstrip()
indent = len(line) - len(stripped)
# Top-level section match
sect_match = _SECTION_RE.match(line)
if sect_match:
if indent == 0:
inside_parent = sect_match.group(1) == parent_section
inside_child = False
parent_indent = None
child_indent = None
continue
if not inside_parent:
continue
# Track parent indent from first child
if parent_indent is None and indent > 0:
parent_indent = indent
# If indent goes back to 0, we left the parent section
if indent == 0:
inside_parent = False
inside_child = False
continue
# Check if we're at the parent's child level (subsection)
if parent_indent is not None and indent == parent_indent:
# This could be a subsection or a direct key of parent
sub_match = _INDENTED_SECTION_RE.match(line)
if sub_match and sub_match.group(1) == child_section:
inside_child = True
child_indent = None
continue
else:
inside_child = False
continue
if not inside_child:
continue
# We're inside the subsection — track child indent
if child_indent is None and indent > (parent_indent or 0):
child_indent = indent
if child_indent is not None and indent != child_indent:
continue
key_match = _KEY_RE.match(line)
if key_match and key_match.group(1) == key:
return _unquote(key_match.group(2).strip())
return None
def detect_from_config(path: Path) -> list[str]: def detect_from_config(path: Path) -> list[str]:
try: try:
text = path.read_text(encoding="utf-8", errors="replace") text = path.read_text(encoding="utf-8", errors="replace")
@@ -152,6 +231,8 @@ def detect_from_config(path: Path) -> list[str]:
extras.add("postgres") extras.add("postgres")
if (section_value(lines, "checkpointer", "type") or "").lower() == "postgres": if (section_value(lines, "checkpointer", "type") or "").lower() == "postgres":
extras.add("postgres") extras.add("postgres")
if (nested_section_value(lines, "channels.discord", "enabled") or "").lower() == "true":
extras.add("discord")
return sorted(extras) return sorted(extras)