00e0e9a49a
* feat(persistence): add SQLAlchemy 2.0 async ORM scaffold Introduce a unified database configuration (DatabaseConfig) that controls both the LangGraph checkpointer and the DeerFlow application persistence layer from a single `database:` config section. New modules: - deerflow.config.database_config — Pydantic config with memory/sqlite/postgres backends - deerflow.persistence — async engine lifecycle, DeclarativeBase with to_dict mixin, Alembic skeleton - deerflow.runtime.runs.store — RunStore ABC + MemoryRunStore implementation Gateway integration initializes/tears down the persistence engine in the existing langgraph_runtime() context manager. Legacy checkpointer config is preserved for backward compatibility. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(persistence): add RunEventStore ABC + MemoryRunEventStore Phase 2-A prerequisite for event storage: adds the unified run event stream interface (RunEventStore) with an in-memory implementation, RunEventsConfig, gateway integration, and comprehensive tests (27 cases). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(persistence): add ORM models, repositories, DB/JSONL event stores, RunJournal, and API endpoints Phase 2-B: run persistence + event storage + token tracking. - ORM models: RunRow (with token fields), ThreadMetaRow, RunEventRow - RunRepository implements RunStore ABC via SQLAlchemy ORM - ThreadMetaRepository with owner access control - DbRunEventStore with trace content truncation and cursor pagination - JsonlRunEventStore with per-run files and seq recovery from disk - RunJournal (BaseCallbackHandler) captures LLM/tool/lifecycle events, accumulates token usage by caller type, buffers and flushes to store - RunManager now accepts optional RunStore for persistent backing - Worker creates RunJournal, writes human_message, injects callbacks - Gateway deps use factory functions (RunRepository when DB available) - New endpoints: messages, run messages, run events, token-usage - ThreadCreateRequest gains assistant_id field - 92 tests pass (33 new), zero regressions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(persistence): add user feedback + follow-up run association Phase 2-C: feedback and follow-up tracking. - FeedbackRow ORM model (rating +1/-1, optional message_id, comment) - FeedbackRepository with CRUD, list_by_run/thread, aggregate stats - Feedback API endpoints: create, list, stats, delete - follow_up_to_run_id in RunCreateRequest (explicit or auto-detected from latest successful run on the thread) - Worker writes follow_up_to_run_id into human_message event metadata - Gateway deps: feedback_repo factory + getter - 17 new tests (14 FeedbackRepository + 3 follow-up association) - 109 total tests pass, zero regressions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test+config: comprehensive Phase 2 test coverage + deprecate checkpointer config - config.example.yaml: deprecate standalone checkpointer section, activate unified database:sqlite as default (drives both checkpointer + app data) - New: test_thread_meta_repo.py (14 tests) — full ThreadMetaRepository coverage including check_access owner logic, list_by_owner pagination - Extended test_run_repository.py (+4 tests) — completion preserves fields, list ordering desc, limit, owner_none returns all - Extended test_run_journal.py (+8 tests) — on_chain_error, track_tokens=false, middleware no ai_message, unknown caller tokens, convenience fields, tool_error, non-summarization custom event - Extended test_run_event_store.py (+7 tests) — DB batch seq continuity, make_run_event_store factory (memory/db/jsonl/fallback/unknown) - Extended test_phase2b_integration.py (+4 tests) — create_or_reject persists, follow-up metadata, summarization in history, full DB-backed lifecycle - Fixed DB integration test to use proper fake objects (not MagicMock) for JSON-serializable metadata - 157 total Phase 2 tests pass, zero regressions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * config: move default sqlite_dir to .deer-flow/data Keep SQLite databases alongside other DeerFlow-managed data (threads, memory) under the .deer-flow/ directory instead of a top-level ./data folder. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(persistence): remove UTFJSON, use engine-level json_serializer + datetime.now() - Replace custom UTFJSON type with standard sqlalchemy.JSON in all ORM models. Add json_serializer=json.dumps(ensure_ascii=False) to all create_async_engine calls so non-ASCII text (Chinese etc.) is stored as-is in both SQLite and Postgres. - Change ORM datetime defaults from datetime.now(UTC) to datetime.now(), remove UTC imports. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(gateway): simplify deps.py with getter factory + inline repos - Replace 6 identical getter functions with _require() factory. - Inline 3 _make_*_repo() factories into langgraph_runtime(), call get_session_factory() once instead of 3 times. - Add thread_meta upsert in start_run (services.py). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(docker): add UV_EXTRAS build arg for optional dependencies Support installing optional dependency groups (e.g. postgres) at Docker build time via UV_EXTRAS build arg: UV_EXTRAS=postgres docker compose build Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(journal): fix flush, token tracking, and consolidate tests RunJournal fixes: - _flush_sync: retain events in buffer when no event loop instead of dropping them; worker's finally block flushes via async flush(). - on_llm_end: add tool_calls filter and caller=="lead_agent" guard for ai_message events; mark message IDs for dedup with record_llm_usage. - worker.py: persist completion data (tokens, message count) to RunStore in finally block. Model factory: - Auto-inject stream_usage=True for BaseChatOpenAI subclasses with custom api_base, so usage_metadata is populated in streaming responses. Test consolidation: - Delete test_phase2b_integration.py (redundant with existing tests). - Move DB-backed lifecycle test into test_run_journal.py. - Add tests for stream_usage injection in test_model_factory.py. - Clean up executor/task_tool dead journal references. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(events): widen content type to str|dict in all store backends Allow event content to be a dict (for structured OpenAI-format messages) in addition to plain strings. Dict values are JSON-serialized for the DB backend and deserialized on read; memory and JSONL backends handle dicts natively. Trace truncation now serializes dicts to JSON before measuring. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(events): use metadata flag instead of heuristic for dict content detection Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(converters): add LangChain-to-OpenAI message format converters Pure functions langchain_to_openai_message, langchain_to_openai_completion, langchain_messages_to_openai, and _infer_finish_reason for converting LangChain BaseMessage objects to OpenAI Chat Completions format, used by RunJournal for event storage. 15 unit tests added. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(converters): handle empty list content as null, clean up test Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(events): human_message content uses OpenAI user message format Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(events): ai_message uses OpenAI format, add ai_tool_call message event - ai_message content now uses {"role": "assistant", "content": "..."} format - New ai_tool_call message event emitted when lead_agent LLM responds with tool_calls - ai_tool_call uses langchain_to_openai_message converter for consistent format - Both events include finish_reason in metadata ("stop" or "tool_calls") Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(events): add tool_result message event with OpenAI tool message format Cache tool_call_id from on_tool_start keyed by run_id as fallback for on_tool_end, then emit a tool_result message event (role=tool, tool_call_id, content) after each successful tool completion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(events): summary content uses OpenAI system message format Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(events): replace llm_start/llm_end with llm_request/llm_response in OpenAI format Add on_chat_model_start to capture structured prompt messages as llm_request events. Replace llm_end trace events with llm_response using OpenAI Chat Completions format. Track llm_call_index to pair request/response events. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(events): add record_middleware method for middleware trace events Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(events): add full run sequence integration test for OpenAI content format Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(events): align message events with checkpoint format and add middleware tag injection - Message events (ai_message, ai_tool_call, tool_result, human_message) now use BaseMessage.model_dump() format, matching LangGraph checkpoint values.messages - on_tool_end extracts tool_call_id/name/status from ToolMessage objects - on_tool_error now emits tool_result message events with error status - record_middleware uses middleware:{tag} event_type and middleware category - Summarization custom events use middleware:summarize category - TitleMiddleware injects middleware:title tag via get_config() inheritance - SummarizationMiddleware model bound with middleware:summarize tag - Worker writes human_message using HumanMessage.model_dump() Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(threads): switch search endpoint to threads_meta table and sync title - POST /api/threads/search now queries threads_meta table directly, removing the two-phase Store + Checkpointer scan approach - Add ThreadMetaRepository.search() with metadata/status filters - Add ThreadMetaRepository.update_display_name() for title sync - Worker syncs checkpoint title to threads_meta.display_name on run completion - Map display_name to values.title in search response for API compatibility Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(threads): history endpoint reads messages from event store - POST /api/threads/{thread_id}/history now combines two data sources: checkpointer for checkpoint_id, metadata, title, thread_data; event store for messages (complete history, not truncated by summarization) - Strip internal LangGraph metadata keys from response - Remove full channel_values serialization in favor of selective fields Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove duplicate optional-dependencies header in pyproject.toml Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(middleware): pass tagged config to TitleMiddleware ainvoke call Without the config, the middleware:title tag was not injected, causing the LLM response to be recorded as a lead_agent ai_message in run_events. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: resolve merge conflict in .env.example Keep both DATABASE_URL (from persistence-scaffold) and WECOM credentials (from main) after the merge. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(persistence): address review feedback on PR #1851 - Fix naive datetime.now() → datetime.now(UTC) in all ORM models - Fix seq race condition in DbRunEventStore.put() with FOR UPDATE and UNIQUE(thread_id, seq) constraint - Encapsulate _store access in RunManager.update_run_completion() - Deduplicate _store.put() logic in RunManager via _persist_to_store() - Add update_run_completion to RunStore ABC + MemoryRunStore - Wire follow_up_to_run_id through the full create path - Add error recovery to RunJournal._flush_sync() lost-event scenario - Add migration note for search_threads breaking change - Fix test_checkpointer_none_fix mock to set database=None Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update uv.lock Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(persistence): address 22 review comments from CodeQL, Copilot, and Code Quality Bug fixes: - Sanitize log params to prevent log injection (CodeQL) - Reset threads_meta.status to idle/error when run completes - Attach messages only to latest checkpoint in /history response - Write threads_meta on POST /threads so new threads appear in search Lint fixes: - Remove unused imports (journal.py, migrations/env.py, test_converters.py) - Convert lambda to named function (engine.py, Ruff E731) - Remove unused logger definitions in repos (Ruff F841) - Add logging to JSONL decode errors and empty except blocks - Separate assert side-effects in tests (CodeQL) - Remove unused local variables in tests (Ruff F841) - Fix max_trace_content truncation to use byte length, not char length Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: apply ruff format to persistence and runtime files Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Potential fix for pull request finding 'Statement has no effect' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> * refactor(runtime): introduce RunContext to reduce run_agent parameter bloat Extract checkpointer, store, event_store, run_events_config, thread_meta_repo, and follow_up_to_run_id into a frozen RunContext dataclass. Add get_run_context() in deps.py to build the base context from app.state singletons. start_run() uses dataclasses.replace() to enrich per-run fields before passing ctx to run_agent. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(gateway): move sanitize_log_param to app/gateway/utils.py Extract the log-injection sanitizer from routers/threads.py into a shared utils module and rename to sanitize_log_param (public API). Eliminates the reverse service → router import in services.py. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf: use SQL aggregation for feedback stats and thread token usage Replace Python-side counting in FeedbackRepository.aggregate_by_run with a single SELECT COUNT/SUM query. Add RunStore.aggregate_tokens_by_thread abstract method with SQL GROUP BY implementation in RunRepository and Python fallback in MemoryRunStore. Simplify the thread_token_usage endpoint to delegate to the new method, eliminating the limit=10000 truncation risk. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: annotate DbRunEventStore.put() as low-frequency path Add docstring clarifying that put() opens a per-call transaction with FOR UPDATE and should only be used for infrequent writes (currently just the initial human_message event). High-throughput callers should use put_batch() instead. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(threads): fall back to Store search when ThreadMetaRepository is unavailable When database.backend=memory (default) or no SQL session factory is configured, search_threads now queries the LangGraph Store instead of returning 503. Returns empty list if neither Store nor repo is available. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(persistence): introduce ThreadMetaStore ABC for backend-agnostic thread metadata Add ThreadMetaStore abstract base class with create/get/search/update/delete interface. ThreadMetaRepository (SQL) now inherits from it. New MemoryThreadMetaStore wraps LangGraph BaseStore for memory-mode deployments. deps.py now always provides a non-None thread_meta_repo, eliminating all `if thread_meta_repo is not None` guards in services.py, worker.py, and routers/threads.py. search_threads no longer needs a Store fallback branch. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(history): read messages from checkpointer instead of RunEventStore The /history endpoint now reads messages directly from the checkpointer's channel_values (the authoritative source) instead of querying RunEventStore.list_messages(). The RunEventStore API is preserved for other consumers. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(persistence): address new Copilot review comments - feedback.py: validate thread_id/run_id before deleting feedback - jsonl.py: add path traversal protection with ID validation - run_repo.py: parse `before` to datetime for PostgreSQL compat - thread_meta_repo.py: fix pagination when metadata filter is active - database_config.py: use resolve_path for sqlite_dir consistency Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Implement skill self-evolution and skill_manage flow (#1874) * chore: ignore .worktrees directory * Add skill_manage self-evolution flow * Fix CI regressions for skill_manage * Address PR review feedback for skill evolution * fix(skill-evolution): preserve history on delete * fix(skill-evolution): tighten scanner fallbacks * docs: add skill_manage e2e evidence screenshot * fix(skill-manage): avoid blocking fs ops in session runtime --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> * fix(config): resolve sqlite_dir relative to CWD, not Paths.base_dir resolve_path() resolves relative to Paths.base_dir (.deer-flow), which double-nested the path to .deer-flow/.deer-flow/data/app.db. Use Path.resolve() (CWD-relative) instead. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Feature/feishu receive file (#1608) * feat(feishu): add channel file materialization hook for inbound messages - Introduce Channel.receive_file(msg, thread_id) as a base method for file materialization; default is no-op. - Implement FeishuChannel.receive_file to download files/images from Feishu messages, save to sandbox, and inject virtual paths into msg.text. - Update ChannelManager to call receive_file for any channel if msg.files is present, enabling downstream model access to user-uploaded files. - No impact on Slack/Telegram or other channels (they inherit the default no-op). * style(backend): format code with ruff for lint compliance - Auto-formatted packages/harness/deerflow/agents/factory.py and tests/test_create_deerflow_agent.py using `ruff format` - Ensured both files conform to project linting standards - Fixes CI lint check failures caused by code style issues * fix(feishu): handle file write operation asynchronously to prevent blocking * fix(feishu): rename GetMessageResourceRequest to _GetMessageResourceRequest and remove redundant code * test(feishu): add tests for receive_file method and placeholder replacement * fix(manager): remove unnecessary type casting for channel retrieval * fix(feishu): update logging messages to reflect resource handling instead of image * fix(feishu): sanitize filename by replacing invalid characters in file uploads * fix(feishu): improve filename sanitization and reorder image key handling in message processing * fix(feishu): add thread lock to prevent filename conflicts during file downloads * fix(test): correct bad merge in test_feishu_parser.py * chore: run ruff and apply formatting cleanup fix(feishu): preserve rich-text attachment order and improve fallback filename handling * fix(docker): restore gateway env vars and fix langgraph empty arg issue (#1915) Two production docker-compose.yaml bugs prevent `make up` from working: 1. Gateway missing DEER_FLOW_CONFIG_PATH and DEER_FLOW_EXTENSIONS_CONFIG_PATH environment overrides. Added infb2d99f(#1836) but accidentally reverted byca2fb95(#1847). Without them, gateway reads host paths from .env via env_file, causing FileNotFoundError inside the container. 2. Langgraph command fails when LANGGRAPH_ALLOW_BLOCKING is unset (default). Empty $${allow_blocking} inserts a bare space between flags, causing ' --no-reload' to be parsed as unexpected extra argument. Fix by building args string first and conditionally appending --allow-blocking. Co-authored-by: cooper <cooperfu@tencent.com> * fix(frontend): resolve invalid HTML nesting and tabnabbing vulnerabilities (#1904) * fix(frontend): resolve invalid HTML nesting and tabnabbing vulnerabilities Fix `<button>` inside `<a>` invalid HTML in artifact components and add missing `noopener,noreferrer` to `window.open` calls to prevent reverse tabnabbing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(frontend): address Copilot review on tabnabbing and double-tab-open Remove redundant parent onClick on web_fetch ChainOfThoughtStep to prevent opening two tabs on link click, and explicitly null out window.opener after window.open() for defensive tabnabbing hardening. --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * refactor(persistence): organize entities into per-entity directories Restructure the persistence layer from horizontal "models/ + repositories/" split into vertical entity-aligned directories. Each entity (thread_meta, run, feedback) now owns its ORM model, abstract interface (where applicable), and concrete implementations under a single directory with an aggregating __init__.py for one-line imports. Layout: persistence/thread_meta/{base,model,sql,memory}.py persistence/run/{model,sql}.py persistence/feedback/{model,sql}.py models/__init__.py is kept as a facade so Alembic autogenerate continues to discover all ORM tables via Base.metadata. RunEventRow remains under models/run_event.py because its storage implementation lives in runtime/events/store/db.py and has no matching repository directory. The repositories/ directory is removed entirely. All call sites in gateway/deps.py and tests are updated to import from the new entity packages, e.g.: from deerflow.persistence.thread_meta import ThreadMetaRepository from deerflow.persistence.run import RunRepository from deerflow.persistence.feedback import FeedbackRepository Full test suite passes (1690 passed, 14 skipped). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(gateway): sync thread rename and delete through ThreadMetaStore The POST /threads/{id}/state endpoint previously synced title changes only to the LangGraph Store via _store_upsert. In sqlite mode the search endpoint reads from the ThreadMetaRepository SQL table, so renames never appeared in /threads/search until the next agent run completed (worker.py syncs title from checkpoint to thread_meta in its finally block). Likewise the DELETE /threads/{id} endpoint cleaned up the filesystem, Store, and checkpointer but left the threads_meta row orphaned in sqlite, so deleted threads kept appearing in /threads/search. Fix both endpoints by routing through the ThreadMetaStore abstraction which already has the correct sqlite/memory implementations wired up by deps.py. The rename path now calls update_display_name() and the delete path calls delete() — both work uniformly across backends. Verified end-to-end with curl in gateway mode against sqlite backend. Existing test suite (1690 passed) and focused router/repo tests pass. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(gateway): route all thread metadata access through ThreadMetaStore Following the rename/delete bug fix in PR1, migrate the remaining direct LangGraph Store reads/writes in the threads router and services to the ThreadMetaStore abstraction so that the sqlite and memory backends behave identically and the legacy dual-write paths can be removed. Migrated endpoints (threads.py): - create_thread: idempotency check + write now use thread_meta_repo.get/create instead of dual-writing the LangGraph Store and the SQL row. - get_thread: reads from thread_meta_repo.get; the checkpoint-only fallback for legacy threads is preserved. - patch_thread: replaced _store_get/_store_put with thread_meta_repo.update_metadata. - delete_thread_data: dropped the legacy store.adelete; thread_meta_repo.delete already covers it. Removed dead code (services.py): - _upsert_thread_in_store — redundant with the immediately following thread_meta_repo.create() call. - _sync_thread_title_after_run — worker.py's finally block already syncs the title via thread_meta_repo.update_display_name() after each run. Removed dead code (threads.py): - _store_get / _store_put / _store_upsert helpers (no remaining callers). - THREADS_NS constant. - get_store import (router no longer touches the LangGraph Store directly). New abstract method: - ThreadMetaStore.update_metadata(thread_id, metadata) merges metadata into the thread's metadata field. Implemented in both ThreadMetaRepository (SQL, read-modify-write inside one session) and MemoryThreadMetaStore. Three new unit tests cover merge / empty / nonexistent behaviour. Net change: -134 lines. Full test suite: 1693 passed, 14 skipped. Verified end-to-end with curl in gateway mode against sqlite backend (create / patch / get / rename / search / delete). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> Co-authored-by: DanielWalnut <45447813+hetaoBackend@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> Co-authored-by: JilongSun <965640067@qq.com> Co-authored-by: jie <49781832+stan-fu@users.noreply.github.com> Co-authored-by: cooper <cooperfu@tencent.com> Co-authored-by: yangzheli <43645580+yangzheli@users.noreply.github.com>
693 lines
32 KiB
Python
693 lines
32 KiB
Python
"""Feishu/Lark channel — connects to Feishu via WebSocket (no public IP needed)."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
import json
|
|
import logging
|
|
import re
|
|
import threading
|
|
from typing import Any, Literal
|
|
|
|
from app.channels.base import Channel
|
|
from app.channels.commands import KNOWN_CHANNEL_COMMANDS
|
|
from app.channels.message_bus import InboundMessage, InboundMessageType, MessageBus, OutboundMessage, ResolvedAttachment
|
|
from deerflow.config.paths import VIRTUAL_PATH_PREFIX, get_paths
|
|
from deerflow.sandbox.sandbox_provider import get_sandbox_provider
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
|
|
def _is_feishu_command(text: str) -> bool:
|
|
if not text.startswith("/"):
|
|
return False
|
|
return text.split(maxsplit=1)[0].lower() in KNOWN_CHANNEL_COMMANDS
|
|
|
|
|
|
class FeishuChannel(Channel):
|
|
"""Feishu/Lark IM channel using the ``lark-oapi`` WebSocket client.
|
|
|
|
Configuration keys (in ``config.yaml`` under ``channels.feishu``):
|
|
- ``app_id``: Feishu app ID.
|
|
- ``app_secret``: Feishu app secret.
|
|
- ``verification_token``: (optional) Event verification token.
|
|
|
|
The channel uses WebSocket long-connection mode so no public IP is required.
|
|
|
|
Message flow:
|
|
1. User sends a message → bot adds "OK" emoji reaction
|
|
2. Bot replies in thread: "Working on it......"
|
|
3. Agent processes the message and returns a result
|
|
4. Bot replies in thread with the result
|
|
5. Bot adds "DONE" emoji reaction to the original message
|
|
"""
|
|
|
|
def __init__(self, bus: MessageBus, config: dict[str, Any]) -> None:
|
|
super().__init__(name="feishu", bus=bus, config=config)
|
|
self._thread: threading.Thread | None = None
|
|
self._main_loop: asyncio.AbstractEventLoop | None = None
|
|
self._api_client = None
|
|
self._CreateMessageReactionRequest = None
|
|
self._CreateMessageReactionRequestBody = None
|
|
self._Emoji = None
|
|
self._PatchMessageRequest = None
|
|
self._PatchMessageRequestBody = None
|
|
self._background_tasks: set[asyncio.Task] = set()
|
|
self._running_card_ids: dict[str, str] = {}
|
|
self._running_card_tasks: dict[str, asyncio.Task] = {}
|
|
self._CreateFileRequest = None
|
|
self._CreateFileRequestBody = None
|
|
self._CreateImageRequest = None
|
|
self._CreateImageRequestBody = None
|
|
self._GetMessageResourceRequest = None
|
|
self._thread_lock = threading.Lock()
|
|
|
|
async def start(self) -> None:
|
|
if self._running:
|
|
return
|
|
|
|
try:
|
|
import lark_oapi as lark
|
|
from lark_oapi.api.im.v1 import (
|
|
CreateFileRequest,
|
|
CreateFileRequestBody,
|
|
CreateImageRequest,
|
|
CreateImageRequestBody,
|
|
CreateMessageReactionRequest,
|
|
CreateMessageReactionRequestBody,
|
|
CreateMessageRequest,
|
|
CreateMessageRequestBody,
|
|
Emoji,
|
|
GetMessageResourceRequest,
|
|
PatchMessageRequest,
|
|
PatchMessageRequestBody,
|
|
ReplyMessageRequest,
|
|
ReplyMessageRequestBody,
|
|
)
|
|
except ImportError:
|
|
logger.error("lark-oapi is not installed. Install it with: uv add lark-oapi")
|
|
return
|
|
|
|
self._lark = lark
|
|
self._CreateMessageRequest = CreateMessageRequest
|
|
self._CreateMessageRequestBody = CreateMessageRequestBody
|
|
self._ReplyMessageRequest = ReplyMessageRequest
|
|
self._ReplyMessageRequestBody = ReplyMessageRequestBody
|
|
self._CreateMessageReactionRequest = CreateMessageReactionRequest
|
|
self._CreateMessageReactionRequestBody = CreateMessageReactionRequestBody
|
|
self._Emoji = Emoji
|
|
self._PatchMessageRequest = PatchMessageRequest
|
|
self._PatchMessageRequestBody = PatchMessageRequestBody
|
|
self._CreateFileRequest = CreateFileRequest
|
|
self._CreateFileRequestBody = CreateFileRequestBody
|
|
self._CreateImageRequest = CreateImageRequest
|
|
self._CreateImageRequestBody = CreateImageRequestBody
|
|
self._GetMessageResourceRequest = GetMessageResourceRequest
|
|
|
|
app_id = self.config.get("app_id", "")
|
|
app_secret = self.config.get("app_secret", "")
|
|
domain = self.config.get("domain", "https://open.feishu.cn")
|
|
|
|
if not app_id or not app_secret:
|
|
logger.error("Feishu channel requires app_id and app_secret")
|
|
return
|
|
|
|
self._api_client = lark.Client.builder().app_id(app_id).app_secret(app_secret).domain(domain).build()
|
|
logger.info("[Feishu] using domain: %s", domain)
|
|
self._main_loop = asyncio.get_event_loop()
|
|
|
|
self._running = True
|
|
self.bus.subscribe_outbound(self._on_outbound)
|
|
|
|
# Both ws.Client construction and start() must happen in a dedicated
|
|
# thread with its own event loop. lark-oapi caches the running loop
|
|
# at construction time and later calls loop.run_until_complete(),
|
|
# which conflicts with an already-running uvloop.
|
|
self._thread = threading.Thread(
|
|
target=self._run_ws,
|
|
args=(app_id, app_secret, domain),
|
|
daemon=True,
|
|
)
|
|
self._thread.start()
|
|
logger.info("Feishu channel started")
|
|
|
|
def _run_ws(self, app_id: str, app_secret: str, domain: str) -> None:
|
|
"""Construct and run the lark WS client in a thread with a fresh event loop.
|
|
|
|
The lark-oapi SDK captures a module-level event loop at import time
|
|
(``lark_oapi.ws.client.loop``). When uvicorn uses uvloop, that
|
|
captured loop is the *main* thread's uvloop — which is already
|
|
running, so ``loop.run_until_complete()`` inside ``Client.start()``
|
|
raises ``RuntimeError``.
|
|
|
|
We work around this by creating a plain asyncio event loop for this
|
|
thread and patching the SDK's module-level reference before calling
|
|
``start()``.
|
|
"""
|
|
loop = asyncio.new_event_loop()
|
|
asyncio.set_event_loop(loop)
|
|
try:
|
|
import lark_oapi as lark
|
|
import lark_oapi.ws.client as _ws_client_mod
|
|
|
|
# Replace the SDK's module-level loop so Client.start() uses
|
|
# this thread's (non-running) event loop instead of the main
|
|
# thread's uvloop.
|
|
_ws_client_mod.loop = loop
|
|
|
|
event_handler = lark.EventDispatcherHandler.builder("", "").register_p2_im_message_receive_v1(self._on_message).build()
|
|
ws_client = lark.ws.Client(
|
|
app_id=app_id,
|
|
app_secret=app_secret,
|
|
event_handler=event_handler,
|
|
log_level=lark.LogLevel.INFO,
|
|
domain=domain,
|
|
)
|
|
ws_client.start()
|
|
except Exception:
|
|
if self._running:
|
|
logger.exception("Feishu WebSocket error")
|
|
|
|
async def stop(self) -> None:
|
|
self._running = False
|
|
self.bus.unsubscribe_outbound(self._on_outbound)
|
|
for task in list(self._background_tasks):
|
|
task.cancel()
|
|
self._background_tasks.clear()
|
|
for task in list(self._running_card_tasks.values()):
|
|
task.cancel()
|
|
self._running_card_tasks.clear()
|
|
if self._thread:
|
|
self._thread.join(timeout=5)
|
|
self._thread = None
|
|
logger.info("Feishu channel stopped")
|
|
|
|
async def send(self, msg: OutboundMessage, *, _max_retries: int = 3) -> None:
|
|
if not self._api_client:
|
|
logger.warning("[Feishu] send called but no api_client available")
|
|
return
|
|
|
|
logger.info(
|
|
"[Feishu] sending reply: chat_id=%s, thread_ts=%s, text_len=%d",
|
|
msg.chat_id,
|
|
msg.thread_ts,
|
|
len(msg.text),
|
|
)
|
|
|
|
last_exc: Exception | None = None
|
|
for attempt in range(_max_retries):
|
|
try:
|
|
await self._send_card_message(msg)
|
|
return # success
|
|
except Exception as exc:
|
|
last_exc = exc
|
|
if attempt < _max_retries - 1:
|
|
delay = 2**attempt # 1s, 2s
|
|
logger.warning(
|
|
"[Feishu] send failed (attempt %d/%d), retrying in %ds: %s",
|
|
attempt + 1,
|
|
_max_retries,
|
|
delay,
|
|
exc,
|
|
)
|
|
await asyncio.sleep(delay)
|
|
|
|
logger.error("[Feishu] send failed after %d attempts: %s", _max_retries, last_exc)
|
|
if last_exc is None:
|
|
raise RuntimeError("Feishu send failed without an exception from any attempt")
|
|
raise last_exc
|
|
|
|
async def send_file(self, msg: OutboundMessage, attachment: ResolvedAttachment) -> bool:
|
|
if not self._api_client:
|
|
return False
|
|
|
|
# Check size limits (image: 10MB, file: 30MB)
|
|
if attachment.is_image and attachment.size > 10 * 1024 * 1024:
|
|
logger.warning("[Feishu] image too large (%d bytes), skipping: %s", attachment.size, attachment.filename)
|
|
return False
|
|
if not attachment.is_image and attachment.size > 30 * 1024 * 1024:
|
|
logger.warning("[Feishu] file too large (%d bytes), skipping: %s", attachment.size, attachment.filename)
|
|
return False
|
|
|
|
try:
|
|
if attachment.is_image:
|
|
file_key = await self._upload_image(attachment.actual_path)
|
|
msg_type = "image"
|
|
content = json.dumps({"image_key": file_key})
|
|
else:
|
|
file_key = await self._upload_file(attachment.actual_path, attachment.filename)
|
|
msg_type = "file"
|
|
content = json.dumps({"file_key": file_key})
|
|
|
|
if msg.thread_ts:
|
|
request = self._ReplyMessageRequest.builder().message_id(msg.thread_ts).request_body(self._ReplyMessageRequestBody.builder().msg_type(msg_type).content(content).reply_in_thread(True).build()).build()
|
|
await asyncio.to_thread(self._api_client.im.v1.message.reply, request)
|
|
else:
|
|
request = self._CreateMessageRequest.builder().receive_id_type("chat_id").request_body(self._CreateMessageRequestBody.builder().receive_id(msg.chat_id).msg_type(msg_type).content(content).build()).build()
|
|
await asyncio.to_thread(self._api_client.im.v1.message.create, request)
|
|
|
|
logger.info("[Feishu] file sent: %s (type=%s)", attachment.filename, msg_type)
|
|
return True
|
|
except Exception:
|
|
logger.exception("[Feishu] failed to upload/send file: %s", attachment.filename)
|
|
return False
|
|
|
|
async def _upload_image(self, path) -> str:
|
|
"""Upload an image to Feishu and return the image_key."""
|
|
with open(str(path), "rb") as f:
|
|
request = self._CreateImageRequest.builder().request_body(self._CreateImageRequestBody.builder().image_type("message").image(f).build()).build()
|
|
response = await asyncio.to_thread(self._api_client.im.v1.image.create, request)
|
|
if not response.success():
|
|
raise RuntimeError(f"Feishu image upload failed: code={response.code}, msg={response.msg}")
|
|
return response.data.image_key
|
|
|
|
async def _upload_file(self, path, filename: str) -> str:
|
|
"""Upload a file to Feishu and return the file_key."""
|
|
suffix = path.suffix.lower() if hasattr(path, "suffix") else ""
|
|
if suffix in (".xls", ".xlsx", ".csv"):
|
|
file_type = "xls"
|
|
elif suffix in (".ppt", ".pptx"):
|
|
file_type = "ppt"
|
|
elif suffix == ".pdf":
|
|
file_type = "pdf"
|
|
elif suffix in (".doc", ".docx"):
|
|
file_type = "doc"
|
|
else:
|
|
file_type = "stream"
|
|
|
|
with open(str(path), "rb") as f:
|
|
request = self._CreateFileRequest.builder().request_body(self._CreateFileRequestBody.builder().file_type(file_type).file_name(filename).file(f).build()).build()
|
|
response = await asyncio.to_thread(self._api_client.im.v1.file.create, request)
|
|
if not response.success():
|
|
raise RuntimeError(f"Feishu file upload failed: code={response.code}, msg={response.msg}")
|
|
return response.data.file_key
|
|
|
|
async def receive_file(self, msg: InboundMessage, thread_id: str) -> InboundMessage:
|
|
"""Download a Feishu file into the thread uploads directory.
|
|
|
|
Returns the sandbox virtual path when the image is persisted successfully.
|
|
"""
|
|
if not msg.thread_ts:
|
|
logger.warning("[Feishu] received file message without thread_ts, cannot associate with conversation: %s", msg)
|
|
return msg
|
|
files = msg.files
|
|
if not files:
|
|
logger.warning("[Feishu] received message with no files: %s", msg)
|
|
return msg
|
|
text = msg.text
|
|
for file in files:
|
|
if file.get("image_key"):
|
|
virtual_path = await self._receive_single_file(msg.thread_ts, file["image_key"], "image", thread_id)
|
|
text = text.replace("[image]", virtual_path, 1)
|
|
elif file.get("file_key"):
|
|
virtual_path = await self._receive_single_file(msg.thread_ts, file["file_key"], "file", thread_id)
|
|
text = text.replace("[file]", virtual_path, 1)
|
|
msg.text = text
|
|
return msg
|
|
|
|
async def _receive_single_file(self, message_id: str, file_key: str, type: Literal["image", "file"], thread_id: str) -> str:
|
|
request = self._GetMessageResourceRequest.builder().message_id(message_id).file_key(file_key).type(type).build()
|
|
|
|
def inner():
|
|
return self._api_client.im.v1.message_resource.get(request)
|
|
|
|
try:
|
|
response = await asyncio.to_thread(inner)
|
|
except Exception:
|
|
logger.exception("[Feishu] resource get request failed for resource_key=%s type=%s", file_key, type)
|
|
return f"Failed to obtain the [{type}]"
|
|
|
|
if not response.success():
|
|
logger.warning(
|
|
"[Feishu] resource get failed: resource_key=%s, type=%s, code=%s, msg=%s, log_id=%s ",
|
|
file_key,
|
|
type,
|
|
response.code,
|
|
response.msg,
|
|
response.get_log_id(),
|
|
)
|
|
return f"Failed to obtain the [{type}]"
|
|
|
|
image_stream = getattr(response, "file", None)
|
|
if image_stream is None:
|
|
logger.warning("[Feishu] resource get returned no file stream: resource_key=%s, type=%s", file_key, type)
|
|
return f"Failed to obtain the [{type}]"
|
|
|
|
try:
|
|
content: bytes = await asyncio.to_thread(image_stream.read)
|
|
except Exception:
|
|
logger.exception("[Feishu] failed to read resource stream: resource_key=%s, type=%s", file_key, type)
|
|
return f"Failed to obtain the [{type}]"
|
|
|
|
if not content:
|
|
logger.warning("[Feishu] empty resource content: resource_key=%s, type=%s", file_key, type)
|
|
return f"Failed to obtain the [{type}]"
|
|
|
|
paths = get_paths()
|
|
paths.ensure_thread_dirs(thread_id)
|
|
uploads_dir = paths.sandbox_uploads_dir(thread_id).resolve()
|
|
|
|
ext = "png" if type == "image" else "bin"
|
|
raw_filename = getattr(response, "file_name", "") or f"feishu_{file_key[-12:]}.{ext}"
|
|
|
|
# Sanitize filename: preserve extension, replace path chars in name part
|
|
if "." in raw_filename:
|
|
name_part, ext = raw_filename.rsplit(".", 1)
|
|
name_part = re.sub(r"[./\\]", "_", name_part)
|
|
filename = f"{name_part}.{ext}"
|
|
else:
|
|
filename = re.sub(r"[./\\]", "_", raw_filename)
|
|
resolved_target = uploads_dir / filename
|
|
|
|
def down_load():
|
|
# use thread_lock to avoid filename conflicts when writing
|
|
with self._thread_lock:
|
|
resolved_target.write_bytes(content)
|
|
|
|
try:
|
|
await asyncio.to_thread(down_load)
|
|
except Exception:
|
|
logger.exception("[Feishu] failed to persist downloaded resource: %s, type=%s", resolved_target, type)
|
|
return f"Failed to obtain the [{type}]"
|
|
|
|
virtual_path = f"{VIRTUAL_PATH_PREFIX}/uploads/{resolved_target.name}"
|
|
|
|
try:
|
|
sandbox_provider = get_sandbox_provider()
|
|
sandbox_id = sandbox_provider.acquire(thread_id)
|
|
if sandbox_id != "local":
|
|
sandbox = sandbox_provider.get(sandbox_id)
|
|
if sandbox is None:
|
|
logger.warning("[Feishu] sandbox not found for thread_id=%s", thread_id)
|
|
return f"Failed to obtain the [{type}]"
|
|
sandbox.update_file(virtual_path, content)
|
|
except Exception:
|
|
logger.exception("[Feishu] failed to sync resource into non-local sandbox: %s", virtual_path)
|
|
return f"Failed to obtain the [{type}]"
|
|
|
|
logger.info("[Feishu] downloaded resource mapped: file_key=%s -> %s", file_key, virtual_path)
|
|
return virtual_path
|
|
|
|
# -- message formatting ------------------------------------------------
|
|
|
|
@staticmethod
|
|
def _build_card_content(text: str) -> str:
|
|
"""Build a Feishu interactive card with markdown content.
|
|
|
|
Feishu's interactive card format natively renders markdown, including
|
|
headers, bold/italic, code blocks, lists, and links.
|
|
"""
|
|
card = {
|
|
"config": {"wide_screen_mode": True, "update_multi": True},
|
|
"elements": [{"tag": "markdown", "content": text}],
|
|
}
|
|
return json.dumps(card)
|
|
|
|
# -- reaction helpers --------------------------------------------------
|
|
|
|
async def _add_reaction(self, message_id: str, emoji_type: str = "THUMBSUP") -> None:
|
|
"""Add an emoji reaction to a message."""
|
|
if not self._api_client or not self._CreateMessageReactionRequest:
|
|
return
|
|
try:
|
|
request = self._CreateMessageReactionRequest.builder().message_id(message_id).request_body(self._CreateMessageReactionRequestBody.builder().reaction_type(self._Emoji.builder().emoji_type(emoji_type).build()).build()).build()
|
|
await asyncio.to_thread(self._api_client.im.v1.message_reaction.create, request)
|
|
logger.info("[Feishu] reaction '%s' added to message %s", emoji_type, message_id)
|
|
except Exception:
|
|
logger.exception("[Feishu] failed to add reaction '%s' to message %s", emoji_type, message_id)
|
|
|
|
async def _reply_card(self, message_id: str, text: str) -> str | None:
|
|
"""Reply with an interactive card and return the created card message ID."""
|
|
if not self._api_client:
|
|
return None
|
|
|
|
content = self._build_card_content(text)
|
|
request = self._ReplyMessageRequest.builder().message_id(message_id).request_body(self._ReplyMessageRequestBody.builder().msg_type("interactive").content(content).reply_in_thread(True).build()).build()
|
|
response = await asyncio.to_thread(self._api_client.im.v1.message.reply, request)
|
|
response_data = getattr(response, "data", None)
|
|
return getattr(response_data, "message_id", None)
|
|
|
|
async def _create_card(self, chat_id: str, text: str) -> None:
|
|
"""Create a new card message in the target chat."""
|
|
if not self._api_client:
|
|
return
|
|
|
|
content = self._build_card_content(text)
|
|
request = self._CreateMessageRequest.builder().receive_id_type("chat_id").request_body(self._CreateMessageRequestBody.builder().receive_id(chat_id).msg_type("interactive").content(content).build()).build()
|
|
await asyncio.to_thread(self._api_client.im.v1.message.create, request)
|
|
|
|
async def _update_card(self, message_id: str, text: str) -> None:
|
|
"""Patch an existing card message in place."""
|
|
if not self._api_client or not self._PatchMessageRequest:
|
|
return
|
|
|
|
content = self._build_card_content(text)
|
|
request = self._PatchMessageRequest.builder().message_id(message_id).request_body(self._PatchMessageRequestBody.builder().content(content).build()).build()
|
|
await asyncio.to_thread(self._api_client.im.v1.message.patch, request)
|
|
|
|
def _track_background_task(self, task: asyncio.Task, *, name: str, msg_id: str) -> None:
|
|
"""Keep a strong reference to fire-and-forget tasks and surface errors."""
|
|
self._background_tasks.add(task)
|
|
task.add_done_callback(lambda done_task, task_name=name, mid=msg_id: self._finalize_background_task(done_task, task_name, mid))
|
|
|
|
def _finalize_background_task(self, task: asyncio.Task, name: str, msg_id: str) -> None:
|
|
self._background_tasks.discard(task)
|
|
self._log_task_error(task, name, msg_id)
|
|
|
|
async def _create_running_card(self, source_message_id: str, text: str) -> str | None:
|
|
"""Create the running card and cache its message ID when available."""
|
|
running_card_id = await self._reply_card(source_message_id, text)
|
|
if running_card_id:
|
|
self._running_card_ids[source_message_id] = running_card_id
|
|
logger.info("[Feishu] running card created: source=%s card=%s", source_message_id, running_card_id)
|
|
else:
|
|
logger.warning("[Feishu] running card creation returned no message_id for source=%s, subsequent updates will fall back to new replies", source_message_id)
|
|
return running_card_id
|
|
|
|
def _ensure_running_card_started(self, source_message_id: str, text: str = "Working on it...") -> asyncio.Task | None:
|
|
"""Start running-card creation once per source message."""
|
|
running_card_id = self._running_card_ids.get(source_message_id)
|
|
if running_card_id:
|
|
return None
|
|
|
|
running_card_task = self._running_card_tasks.get(source_message_id)
|
|
if running_card_task:
|
|
return running_card_task
|
|
|
|
running_card_task = asyncio.create_task(self._create_running_card(source_message_id, text))
|
|
self._running_card_tasks[source_message_id] = running_card_task
|
|
running_card_task.add_done_callback(lambda done_task, mid=source_message_id: self._finalize_running_card_task(mid, done_task))
|
|
return running_card_task
|
|
|
|
def _finalize_running_card_task(self, source_message_id: str, task: asyncio.Task) -> None:
|
|
if self._running_card_tasks.get(source_message_id) is task:
|
|
self._running_card_tasks.pop(source_message_id, None)
|
|
self._log_task_error(task, "create_running_card", source_message_id)
|
|
|
|
async def _ensure_running_card(self, source_message_id: str, text: str = "Working on it...") -> str | None:
|
|
"""Ensure the in-thread running card exists and track its message ID."""
|
|
running_card_id = self._running_card_ids.get(source_message_id)
|
|
if running_card_id:
|
|
return running_card_id
|
|
|
|
running_card_task = self._ensure_running_card_started(source_message_id, text)
|
|
if running_card_task is None:
|
|
return self._running_card_ids.get(source_message_id)
|
|
return await running_card_task
|
|
|
|
async def _send_running_reply(self, message_id: str) -> None:
|
|
"""Reply to a message in-thread with a running card."""
|
|
try:
|
|
await self._ensure_running_card(message_id)
|
|
except Exception:
|
|
logger.exception("[Feishu] failed to send running reply for message %s", message_id)
|
|
|
|
async def _send_card_message(self, msg: OutboundMessage) -> None:
|
|
"""Send or update the Feishu card tied to the current request."""
|
|
source_message_id = msg.thread_ts
|
|
if source_message_id:
|
|
running_card_id = self._running_card_ids.get(source_message_id)
|
|
awaited_running_card_task = False
|
|
|
|
if not running_card_id:
|
|
running_card_task = self._running_card_tasks.get(source_message_id)
|
|
if running_card_task:
|
|
awaited_running_card_task = True
|
|
running_card_id = await running_card_task
|
|
|
|
if running_card_id:
|
|
try:
|
|
await self._update_card(running_card_id, msg.text)
|
|
except Exception:
|
|
if not msg.is_final:
|
|
raise
|
|
logger.exception(
|
|
"[Feishu] failed to patch running card %s, falling back to final reply",
|
|
running_card_id,
|
|
)
|
|
await self._reply_card(source_message_id, msg.text)
|
|
else:
|
|
logger.info("[Feishu] running card updated: source=%s card=%s", source_message_id, running_card_id)
|
|
elif msg.is_final:
|
|
await self._reply_card(source_message_id, msg.text)
|
|
elif awaited_running_card_task:
|
|
logger.warning(
|
|
"[Feishu] running card task finished without message_id for source=%s, skipping duplicate non-final creation",
|
|
source_message_id,
|
|
)
|
|
else:
|
|
await self._ensure_running_card(source_message_id, msg.text)
|
|
|
|
if msg.is_final:
|
|
self._running_card_ids.pop(source_message_id, None)
|
|
await self._add_reaction(source_message_id, "DONE")
|
|
return
|
|
|
|
await self._create_card(msg.chat_id, msg.text)
|
|
|
|
# -- internal ----------------------------------------------------------
|
|
|
|
@staticmethod
|
|
def _log_future_error(fut, name: str, msg_id: str) -> None:
|
|
"""Callback for run_coroutine_threadsafe futures to surface errors."""
|
|
try:
|
|
exc = fut.exception()
|
|
if exc:
|
|
logger.error("[Feishu] %s failed for msg_id=%s: %s", name, msg_id, exc)
|
|
except Exception:
|
|
pass
|
|
|
|
@staticmethod
|
|
def _log_task_error(task: asyncio.Task, name: str, msg_id: str) -> None:
|
|
"""Callback for background asyncio tasks to surface errors."""
|
|
try:
|
|
exc = task.exception()
|
|
if exc:
|
|
logger.error("[Feishu] %s failed for msg_id=%s: %s", name, msg_id, exc)
|
|
except asyncio.CancelledError:
|
|
logger.info("[Feishu] %s cancelled for msg_id=%s", name, msg_id)
|
|
except Exception:
|
|
pass
|
|
|
|
async def _prepare_inbound(self, msg_id: str, inbound) -> None:
|
|
"""Kick off Feishu side effects without delaying inbound dispatch."""
|
|
reaction_task = asyncio.create_task(self._add_reaction(msg_id, "OK"))
|
|
self._track_background_task(reaction_task, name="add_reaction", msg_id=msg_id)
|
|
self._ensure_running_card_started(msg_id)
|
|
await self.bus.publish_inbound(inbound)
|
|
|
|
def _on_message(self, event) -> None:
|
|
"""Called by lark-oapi when a message is received (runs in lark thread)."""
|
|
try:
|
|
logger.info("[Feishu] raw event received: type=%s", type(event).__name__)
|
|
message = event.event.message
|
|
chat_id = message.chat_id
|
|
msg_id = message.message_id
|
|
sender_id = event.event.sender.sender_id.open_id
|
|
|
|
# root_id is set when the message is a reply within a Feishu thread.
|
|
# Use it as topic_id so all replies share the same DeerFlow thread.
|
|
root_id = getattr(message, "root_id", None) or None
|
|
|
|
# Parse message content
|
|
content = json.loads(message.content)
|
|
|
|
# files_list store the any-file-key in feishu messages, which can be used to download the file content later
|
|
# In Feishu channel, image_keys are independent of file_keys.
|
|
# The file_key includes files, videos, and audio, but does not include stickers.
|
|
files_list = []
|
|
|
|
if "text" in content:
|
|
# Handle plain text messages
|
|
text = content["text"]
|
|
elif "file_key" in content:
|
|
file_key = content.get("file_key")
|
|
if isinstance(file_key, str) and file_key:
|
|
files_list.append({"file_key": file_key})
|
|
text = "[file]"
|
|
else:
|
|
text = ""
|
|
elif "image_key" in content:
|
|
image_key = content.get("image_key")
|
|
if isinstance(image_key, str) and image_key:
|
|
files_list.append({"image_key": image_key})
|
|
text = "[image]"
|
|
else:
|
|
text = ""
|
|
elif "content" in content and isinstance(content["content"], list):
|
|
# Handle rich-text messages with a top-level "content" list (e.g., topic groups/posts)
|
|
text_paragraphs: list[str] = []
|
|
for paragraph in content["content"]:
|
|
if isinstance(paragraph, list):
|
|
paragraph_text_parts: list[str] = []
|
|
for element in paragraph:
|
|
if isinstance(element, dict):
|
|
# Include both normal text and @ mentions
|
|
if element.get("tag") in ("text", "at"):
|
|
text_value = element.get("text", "")
|
|
if text_value:
|
|
paragraph_text_parts.append(text_value)
|
|
elif element.get("tag") == "img":
|
|
image_key = element.get("image_key")
|
|
if isinstance(image_key, str) and image_key:
|
|
files_list.append({"image_key": image_key})
|
|
paragraph_text_parts.append("[image]")
|
|
elif element.get("tag") in ("file", "media"):
|
|
file_key = element.get("file_key")
|
|
if isinstance(file_key, str) and file_key:
|
|
files_list.append({"file_key": file_key})
|
|
paragraph_text_parts.append("[file]")
|
|
if paragraph_text_parts:
|
|
# Join text segments within a paragraph with spaces to avoid "helloworld"
|
|
text_paragraphs.append(" ".join(paragraph_text_parts))
|
|
|
|
# Join paragraphs with blank lines to preserve paragraph boundaries
|
|
text = "\n\n".join(text_paragraphs)
|
|
else:
|
|
text = ""
|
|
text = text.strip()
|
|
|
|
logger.info(
|
|
"[Feishu] parsed message: chat_id=%s, msg_id=%s, root_id=%s, sender=%s, text=%r",
|
|
chat_id,
|
|
msg_id,
|
|
root_id,
|
|
sender_id,
|
|
text[:100] if text else "",
|
|
)
|
|
|
|
if not (text or files_list):
|
|
logger.info("[Feishu] empty text, ignoring message")
|
|
return
|
|
|
|
# Only treat known slash commands as commands; absolute paths and
|
|
# other slash-prefixed text should be handled as normal chat.
|
|
if _is_feishu_command(text):
|
|
msg_type = InboundMessageType.COMMAND
|
|
else:
|
|
msg_type = InboundMessageType.CHAT
|
|
|
|
# topic_id: use root_id for replies (same topic), msg_id for new messages (new topic)
|
|
topic_id = root_id or msg_id
|
|
|
|
inbound = self._make_inbound(
|
|
chat_id=chat_id,
|
|
user_id=sender_id,
|
|
text=text,
|
|
msg_type=msg_type,
|
|
thread_ts=msg_id,
|
|
files=files_list,
|
|
metadata={"message_id": msg_id, "root_id": root_id},
|
|
)
|
|
inbound.topic_id = topic_id
|
|
|
|
# Schedule on the async event loop
|
|
if self._main_loop and self._main_loop.is_running():
|
|
logger.info("[Feishu] publishing inbound message to bus (type=%s, msg_id=%s)", msg_type.value, msg_id)
|
|
fut = asyncio.run_coroutine_threadsafe(self._prepare_inbound(msg_id, inbound), self._main_loop)
|
|
fut.add_done_callback(lambda f, mid=msg_id: self._log_future_error(f, "prepare_inbound", mid))
|
|
else:
|
|
logger.warning("[Feishu] main loop not running, cannot publish inbound message")
|
|
except Exception:
|
|
logger.exception("[Feishu] error processing message")
|