Files
5b65d543b1 fix(channels): key inbound dedupe on chat-scoped workspaces (#4287)
* fix(channels): key inbound dedupe on chat-scoped workspaces

* fix(channels): release the inbound dedupe key on swallowed transient failures

_release_inbound_dedupe_key runs from _handle_message's generic exception
handler, but three sites handle their error in place and never re-raise, so
the key recorded on receipt survives the full dedupe TTL and the provider's
redelivery -- the retry that would have recovered the failure -- is dropped:

- _handle_streaming_chat swallows every stream error into its finally block
- the fire-and-forget runs.create busy branch
- the non-streaming runs.wait busy branch

Keying chat-scoped workspaces gives telegram/feishu/wechat/dingtalk a dedupe
key for the first time, which newly exposes them to this. The mechanism is
pre-existing, not introduced here: WeCom already had a key (streaming plus
aibotid) and shows the same black hole on main.

Also parametrize the dispatch dedupe test over all three chat-scoped
providers so the streaming path is covered end-to-end, and restate the
workspace-fallback comment in terms of what the chain actually guarantees --
both fallbacks are appended last and gated on every earlier source being
absent, so they can only turn "no key" into a key, never change one.

* fix(channels): publish final reply before dedupe release

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-19 20:01:52 +08:00

60 KiB

Changelog

All notable changes to DeerFlow are documented in this file.

The format is based on Keep a Changelog and this project adheres to Semantic Versioning.

[Unreleased]

This section accumulates work toward the 2.1.0 milestone (milestone 2).

⚠ Breaking changes

  • skills: A directory containing SKILL.md is now a runtime package boundary. Nested SKILL.md files inside that package are supporting data and are no longer registered as independent skills; unusual custom layouts must move independently loadable skills under a namespace directory without its own SKILL.md. (#4098)
  • memory: The memory system is now pluggable (memory.manager_class selects a backend; default deermem is self-contained). DeerMem-private settings moved from the top level of memory: into memory.backend_config, and the /memory/config response (and client.get_memory_config()) changed shape. (#4122)
  • memory: /memory/config and client.get_memory_config() no longer return flat DeerMem fields (storage_path, max_facts, debounce_seconds, token_counting, guaranteed_*, staleness_*, ...). They return {enabled, mode, injection_enabled, manager_class, backend_config} where backend_config is an opaque dict the active backend self-interprets. Memory data responses (/memory, /memory/status data) are unchanged. External API/SDK clients reading the old flat fields must read backend_config instead. (#4122)
  • memory: Custom memory.storage_class moved: the old default path deerflow.agents.memory.storage.FileMemoryStorage no longer exists (now deerflow.agents.memory.backends.deermem.deermem.core.storage.FileMemoryStorage). Custom MemoryStorage subclasses must accept config in __init__ (was no-arg). A broken/old storage_class logs an error and falls back to FileMemoryStorage (won't crash) -- update the path + signature to restore it. (#4122)
  • memory: storage_path semantics changed from a FILE path to a root DIRECTORY. Pre-abstraction, an absolute storage_path was the shared memory file (opting out of per-user isolation) and a relative value was the global file under the data base_dir. Now storage_path (absolute or relative) is the root directory; per-user memory lives at {storage_path}/users/{uid}/memory.json. An upgrade keeping the old default storage_path: memory.json (a relative file name) would orphan per-user memory or hit NotADirectoryError on save, so the legacy migration drops file-style storage_path values (ending in .json) with a warning and the factory raises if storage_path resolves to an existing file. Set memory.backend_config.storage_path to a directory for a custom root. (#4122)

Added

Agents & runtime

  • middleware: New TokenBudgetMiddleware enforces a per-run token budget, shared additively across the lead agent and subagents. (#3412)
  • middleware: Structured tool-result metadata and a tool-progress state machine give the runtime first-class visibility into multi-step tool flows. (#3601)
  • context: Record the effective memory identity per run and persist durable context (system messages, memory, and tool state) across summarization, emitting it as structured runtime metadata so compaction no longer drops it. (#3556, #3887, #3906)
  • runtime: Goal continuations let a run resume toward a goal across multiple agent turns, with continuation_count tracked and capped. (#3858)
  • subagents: A system-maintained delegation ledger prevents redundant re-delegation of an in-flight task, and a total delegation cap bounds fan-out per run. (#3877, #4115)
  • subagents: Persist and display subagent step history in the thread. (#3845)
  • tools: Structured synopses replace raw oversized tool output in previews. (#3377)
  • files: Deterministic read-before-write version gate for file tools prevents clobbering concurrent edits. (#3912)
  • gateway: Cache-aware cost accounting attributes token costs to cached vs. uncached paths; a Redis stream bridge enables distributed event streaming; and manual context compaction is exposed to the user. (#3920, #3191, #3969)

Memory

  • memory: Memory consolidation synthesizes fragmented facts, and a staleness review prunes silently-outdated facts using LLM-assigned per-fact expected_valid_days / staleFactsToExtend. (#3996, #3860, #4143)
  • memory: Guaranteed injection of correction facts (with graceful fallback) so user corrections always reach the model. (#3592)

Skills

  • skills: Native SkillScan (phase 1) statically analyzes skill packages at load, and describe_skill enables deferred discovery so the model fetches a skill's schema on demand instead of loading all skills up front. (#3033, #3775)
  • skills: Per-user custom skill isolation with sandbox mounting. (#3889)

Models & integrations

  • community: New web search/fetch engines - GroundRoute, Crawl4AI (web_fetch), and a fastCRW provider - plus a Browserless web_capture screenshot tool and Brave image_search. (#3675, #3821, #3585, #3881, #3866)
  • mcp: Per-server tool_call_timeout for MCP tool calls, and routing hints that guide the model to the right server. (#3843, #4004)

Channels

  • channels: Expose the IM channel_user_id to sandbox commands as DEERFLOW_CHANNEL_USER_ID. (#3926)
  • channels: Queue rapid same-thread messages and preserve topic-card previews across batches. (#3988)

Auth & guardrails

  • auth: Generic OIDC/SSO authentication with Keycloak support. (#3506)
  • guardrails: Authenticated runtime context is exposed in GuardrailRequest, and security interventions are persisted as run events. (#3665, #3837)

Sandbox & provisioner

  • sandbox: New E2B and BoxLite (micro-VM) sandbox providers; BoxLite ships with a warm pool. (#3883, #3940, #3951)
  • provisioner: ClusterIP Services and scoped per-skill PVC mounts, plus a configurable sandbox container port. (#4016, #3928)

Frontend

  • frontend: Branching support for assistant turns and side conversations for quoted follow-ups. (#3950, #3934)
  • frontend: Regenerate the latest answer. (#3637)
  • frontend: Citation-sources evidence panel, workspace change review for agent runs, and a visualized ask_clarification card. (#3907, #3945, #3956)
  • frontend: Voice dictation, prompt-history recall with arrow keys, composer input polishing, and a "(thought for N seconds)" thinking-duration chip. (#4036, #3718, #3986, #3627)
  • frontend: Feature-gate the agents UI behind the agents_api flag, and persist AI turn duration in backend and UI. (#3769, #3663)

Observability & tooling

  • observability: Trace-id correlation with enhanced logging and agent observability via Monocle. (#3902, #4024)
  • tooling: A Hermes-like terminal workbench (deerflow CLI) backed by DeerFlowClient, plus a redacted community support-bundle generator. (#3760, #3886)
  • setup: The setup wizard now asks whether OpenAI-compatible gateway models support thinking, and a Volcengine Coding Plan quick-setup path was added. (#3428, #4141)

Changed

  • sandbox: The Helm chart now defaults per-sandbox Services to ClusterIP instead of NodePort, so the code-execution sandbox is reachable only inside the cluster via Service DNS (http://sandbox-<id>-svc.<ns>.svc.cluster.local) and is no longer bound on every node's interfaces - including the externally-reachable ones on GKE/EKS/AKS. Existing chart installs flip NodePort -> ClusterIP on upgrade. To preserve the old reachability (an external probe hitting the 30xxx port, or the Docker-Compose/hybrid path where the gateway is not in K8s), set provisioner.sandboxServiceType: NodePort (with provisioner.nodeHost if needed). The provisioner itself is unchanged (mode-aware since #4016). (#4190)
  • skills: An active restrictive skill must explicitly list task in allowed-tools to delegate to a subagent. Read-only discovery infrastructure (tool_search and describe_skill) remains available, but cannot grant schema visibility or execution for a denied business tool. (#4098)
  • memory: Pre-abstraction top-level memory.* DeerMem fields (storage_path, max_facts, debounce_seconds, model_name, token_counting, staleness_*, consolidation_*, ...) are auto-migrated into backend_config on load with a warning, so an upgrade does NOT silently revert customized settings to defaults (model_name -> backend_config.model.model). Move them under memory.backend_config in config.yaml to silence the warning. (#4122)
  • memory: Added memory.mode (middleware | tool); tool mode registers memory tools (memory_search/add/update/delete) the model calls directly instead of passive per-turn summarization. manager_class resolution is now fail-fast (raises ValueError on an unknown backend instead of silently falling back). (#4023)
  • middleware: Declarative layered middleware builder; ThreadData now runs before Uploads. (#3809)
  • sandbox: The host->virtual output-masking regex now has a single owner, eliminating duplicated pattern compilation. (#4108)
  • docs: AGENTS.md is now the source of truth for agent guidance, imported by CLAUDE.md via @AGENTS.md; module guides refreshed. (#3770)

Fixed

  • skills: Apply allowed-tools only to slash-activated or actually loaded lead-agent skills, preventing passive enabled skills and evaluation fixtures from removing MCP, web, file, and delegation tools from every run. (#4095, #4098, #4192)
  • models: Honor api_base on every BaseChatOpenAI subclass (VllmChatModel, MindIEChatModel, PatchedChatMiMo, PatchedChatStepFun, PatchedChatMiniMax), not just ChatOpenAI / PatchedChatOpenAI. Those five previously dropped the configured endpoint silently and then failed every request with an opaque unexpected keyword argument 'api_base'; the unknown-config-key warning was disabled for them as well. Both now gate on issubclass(BaseChatOpenAI). (#4146)
  • agents: Coalesce SystemMessages before the LLM request; ensure a visible response after tool runs; avoid a default LLM title call before stream end; reserve ellipsis room so the local title respects max_chars; and snap the tool-output tail forward so fallback truncation respects max_chars. (#3711, #4033, #3885, #4052, #4017)
  • agents: Skip dateless reminders in the dynamic-context date scan; load SOUL.md from agent dirs without config.yaml; require config.yaml in update_agent's legacy-agent guard; and refuse empty SOUL.md updates. (#3685, #4136, #4166, #4219)
  • middleware: Window the loop-detection tool-frequency counter so long runs no longer false-trip; prevent the title middleware from streaming tokens; fix positional fallback consuming an unrelated todo when the same-content list is exhausted; acquire the token-budget lock across _apply, before_agent, _clear_run_state, and _drain_pending_warnings; drop orphan ToolMessages so strict providers don't 400; sanitize invalid tool-call arguments; and recover from empty tool-call names and malformed tool-call ids in dangling repair. (#4072, #3566, #3709, #3714, #4080, #4193, #4008, #4246)
  • subagents: Inherit LoopDetectionMiddleware and summarization middleware so tool loops break and steps are captured; surface the turn-budget cap as MAX_TURNS_REACHED with a partial result; unify guardrail caps on the additive stop_reason + token_budget; inject durable context before compaction; preserve the parent checkpoint namespace; prohibit the task tool in the general-purpose system prompt; re-buffer subagent events on flush failure to avoid losing steps; and fix the lost loop_capped stop reason when a subagent's run_id is None. (#3931, #4009, #3949, #3980, #4040, #4215, #4161, #4082, #4059)
  • memory: Harden against null/empty edge cases - skip whitespace-only facts; coerce null confidence / source.confidence in updates, searches, and the three remaining raw reads; treat explicit null backend_config values as omitted; fix KeyError / UnboundLocalError when a fact has no id or the facts list is empty; stop the busy-spin in the debounced update queue; and flush the memory queue on graceful shutdown to prevent loss. (#3719, #4074, #4076, #4034, #4217, #3993, #3992, #4073, #4181)
  • runs: Close multi-worker ownership gaps in run atomicity; degrade cancel to lease takeover for multi-worker; keep create_thread idempotent when the insert loses a race; read stop_reason from runtime context; and persist run duration in checkpoints for history reads. (#4003, #4064, #3800, #4188, #4118)
  • runtime: Serialize SQLite event-store writes to prevent per-thread sequence collisions; skip hidden human messages in the journal; and drop the silent delta-discard in _merge_stream_text. (#4077, #3698, #4085)
  • gateway: Attach thread-message feedback by real event_type; offload blocking filesystem IO in artifact serving, gateway uploads, and the Discord channel; limit the uploaded-file context manifest; and live-tail malformed Redis reconnect ids. (#3651, #3551, #3935, #3927, #3917, #4012)
  • uploads: Claim the converted-Markdown companion filename before writing it, so two convertible uploads sharing a stem (or a convertible plus a same-stem .md upload) no longer silently clobber each other within one request. When uploads.auto_convert_documents is on, the companion .md now gets a unique name (e.g. a_1.md); POST /threads/{id}/uploads and DeerFlowClient.upload_files both report the actual name in markdown_file. (#4288)
  • config: Coerce null object config sections to their defaults; honor the unified database configuration in the store and sync checkpointer; and have legacy DB backfill create missing Index objects on existing tables. (#3573, #3904, #3994, #4090)
  • models: Apply the stream_chunk_timeout default to all BaseChatOpenAI subclasses; and normalize api_base -> base_url for ChatOpenAI with a warning on unknown config keys. (#4102, #3790)
  • mcp: Isolate tool-discovery failures per server; synchronize the session-pool singleton lifecycle; invalidate the tools cache on config content
    • path (not just newer mtime); validate MCP tool names at load so deferred prompts stay inert; and route tools by source server, not name prefix. (#3772, #3797, #4124, #4154, #3812)
  • skills: Activate a slash skill once per run, not per model call; close the skill-install security-scan coverage gap; recognize fully deleted skill packages in review CI and remaining requests / httpx methods as network sinks in SkillScan; reuse the resolved app config in the no-arg skills prompt section; and reload mounted skills without restarting the Gateway. (#4103, #3924, #4169, #4130, #4160, #4264)
  • sandbox: Guard the reverse path-translation and output-masking regexes with segment boundaries; handle one-sided line ranges and empty files in read_file / str_replace; align the AIO bash working directory; use os.sep in the reverse-resolve containment check on Windows; normalize Windows backslash paths in bash commands; stop glob / grep / ls from surfacing disabled skills' files; and allow valid heredoc commands in the sandbox audit. (#4035, #4053, #4078, #4079, #4051, #4058, #3869, #4096, #3786)
  • sandbox: Synchronize the sandbox provider singleton lifecycle (with concurrency regression tests) and keep k8s calls off the event loop in the provisioner. (#3730, #3941)
  • sandbox: Align sandbox artifact mounts with the channel user; fix local-dev (make dev) on non-root / NFS hosts; reap macOS nginx processes on stop; and fix production Postgres UV-extras detection in Docker. (#3729, #3590, #3828, #3897)
  • channels: Validate the channel provider before resolving its config; dedupe GitHub webhook redeliveries and drop redundant GitHub review-comment webhook fan-out; scope the slash-skill whitelist check to the run's owner; batch Feishu file messages into one thread and dispatch Feishu group commands prefixed with a bot @mention; accept leading @mentions before /connect bind codes and don't treat a bare "connect" as a bind command; stop Feishu from creating thread topics and throttle card updates; let the UI runtime channel config win over config.yaml; fix require_mention gating on whitespace-only bot_login / mention_login; guard null quote fields in WeCom; and key inbound dedupe on chat-scoped workspaces so Telegram, Feishu, WeChat and DingTalk redeliveries stop re-running the agent on a default (unbound) configuration, releasing the dedupe key on transient failures so a redelivery can still recover. (#4100, #4104, #4131, #4129, #3753, #4229, #4222, #4251, #3810, #3674, #4055, #4069, #4287)
  • frontend: Preserve messages and durable context across summarization; preserve artifacts and stabilize artifact paths during streaming; resolve relative artifact image paths; retain presented artifacts in the header dropdown; keep orphan tool messages visible; show assistant text during tool steps; reset new chat on client-side navigation; prevent stream cancellation on concurrent submit; fix stale-run reconnect and cancel handling; fix chat math rendering, single-tilde markdown, double reasoning rendering, UTF-16 markdown binary classification, and <memory> tags in Streamdown; make recent-chat rows fully clickable; validate attachment limits before upload and fix uploaded-file metadata in message copy; fix mobile workspace and accessibility blockers, the card tool-message bug, and side-chat toolbar / panel-button behavior; block unresolved suggestion-template placeholders; refresh notification permissions; show the branch action only for completed turns; enable regenerate in custom agent chats; and generate a fallback title for interrupted first-turn runs. (#3826, #3791, #4094, #4038, #3854, #3880, #4114, #3673, #3878, #3908, #3557, #4245, #3870, #3966, #4209, #3733, #3900, #3944, #3740, #3976, #3959, #3961, #3764, #3768, #4147, #3967, #3874, #3644)
  • tui: Interrupt an active run before /quit exits. (#4235)
  • harness: Don't flag the outline as truncated at exactly MAX_OUTLINE_ENTRIES headings. (#3856)
  • tracing: Attach Langfuse trace metadata to the goal evaluator. (#4202)
  • context: Resolve the context-compress bug. (#4065)
  • threaddata: Fix AttributeError when runtime.context is None. (#3989)
  • goal: Stop continuation_count double-bump during stand-down. (#4199)
  • circuit-breaker: Stop wedging after a non-retriable half-open probe. (#3991)
  • github: Match allow_authors logins case-insensitively. (#4218)
  • community: image_search now returns the full-resolution image URL. (#3990)

Performance

  • runtime: Index MemoryRunStore by thread_id and MemoryRunEventStore events by run_id to avoid O(n) scans. (#3562, #3686)
  • subagents: Deduplicate streamed AI messages via a seen-id set (O(n²) -> O(n)). (#3687)
  • sandbox: Cache LocalSandbox path-rewrite regexes and local-path masking patterns per instance instead of recompiling per search match. (#3648, #3713)

Security

  • prompt-injection: New input-sanitization middleware defends against prompt-injection, forged framework tags in the input guardrail are blocked, and system context is injected as a SystemMessage for role isolation. (#3662, #4155, #3661)
  • prompt-injection: HTML-escape untrusted content rendered into model prompts
    • memory facts and summaries, SOUL.md, subagent descriptions, and the conversation block in the memory-update prompt - and neutralize prompt-injection tags in web_capture tool results. (#4028, #4119, #4137, #4157, #4162, #4099, #4060)
  • secrets: Scrub inherited secret environment variables (MYSQL_PWD, REDISCLI_AUTH, abbreviated *_PASS, and Postgres PGPASSFILE) from the skill environment; request-scoped secrets are bound for both slash-activated and autonomously-invoked skills. (#4018, #4026, #3871, #3938)
  • web_fetch: SSRF guard for self-hosted providers. (#3942)
  • guardrails: An empty allowlist now denies all tools instead of failing open. (#4067)
  • authz: Global skills-management endpoints now require admin; the legacy skills mount is gated by user visibility; artifacts honor a trusted owner-user-id header; and the trusted authorization principal is propagated through the runtime. (#3855, #3985, #3982, #4203)
  • auth: Persist the csrf_token cookie for the access-token lifetime. (#3872)
  • storage: Stop persisting base64 image data in checkpoint state. (#4140)

2.0.0 — 2026-06-15

DeerFlow 2.0 is a ground-up rewrite around a "super agent" harness with sub-agents, persistent memory, sandbox execution, and an extensible skills/tools system. It shares no code with the 1.x line, which now lives on the main-1.x branch.

This release closes milestone 2.0.0 with 180 merged pull requests since the first 2.0 milestone tag.

⚠ Breaking changes

  • harness: Hydrate runs from RunStore and persist interrupted status. Run cancellation/multitask semantics now require a working RunStore on the worker that owns the run; cross-worker cancels return 409 instead of silently appearing successful. (#2932)

Added

Agents & runtime

  • agent: Custom-agent self-updates with user isolation — agents can persist edits to their own SOUL.md / config.yaml from inside a normal chat. (#2713)
  • loop-detection: Make loop detection configurable with per-tool frequency overrides; keep configurable on/off switch. (#2586, #2711)
  • loop-detection: Defer warning injection so detector pairs cleanly with tool-call lifecycle. (#2752)
  • run: Propagate model_name from the gateway request through the runtime and persistence stack into the SQLite-backed store. (#2775)
  • subagents: Stream subagent token usage to the header via terminal task events. (#2882)
  • memory: Add memory.token_counting config to opt out of tiktoken for network-restricted deployments. (#3465)
  • suggest: Make AI follow-up question suggestions optional. (#3591)

Models & integrations

  • models: Add StepFun reasoning model adapter. (#3461)
  • community: Add Brave Search web search tool. (#3528)
  • channels: Enhance Discord with mention-only mode, thread routing, and typing indicators. (#2842)
  • im: Add user-owned IM channel connections — users can bind their own Slack/Telegram/Discord/Feishu/DingTalk/WeChat/WeCom accounts on top of the operator-configured bots. (#3487)
  • models: Add patched MiMo reasoning content support. (#3298)
  • models: Add MiniMax provider for image/video/podcast skills plus a new music-generation skill. (#3437)
  • community: Add SearXNG and Browserless web search/fetch tools. (#3451)
  • community: Add Serper Google Images provider for image_search. (#3575)
  • channels: Stream Telegram agent replies by editing the placeholder message in place. (#3534)

Observability

  • trace: Set the LangGraph trace name to lead_agent (or the custom agent's agent_name) for cleaner Langfuse/LangSmith traces. (#3101)
  • frontend: Refine token usage display modes. (#2329)
  • defaults: Enable token usage tracking by default. (#2841)
  • defaults: Raise default summarization trigger threshold. (#3174)
  • trace: Attribute subagent spans to the parent thread's Langfuse trace. (#3611)

Skills

  • skill: Add blocking-io-guard skill for blocking-IO triage and runtime anchors. (#3503)
  • skill: Add maintainer issue and PR workflow skill. (#3554)
  • skill: Strengthen the maintainer orchestrator review workflow. (#3606)

Performance

  • harness: Push thread metadata filters into SQL instead of post-filtering in Python. (#2865)
  • runtime: Index runs by thread_id to avoid O(n) scans in RunManager. (#3499)
  • runtime: Index messages in MemoryRunEventStore to avoid O(n) scans. (#3531)
  • persistence: Cache Base.to_dict column reflection per class. (#3654)
  • sandbox: Speed up should_ignore_name in glob/grep walks. (#3657)

Security

  • upload: Reject symlinked upload destinations. (#2623)
  • uploads: Add Windows support for safe symlink-protected uploads. (#2794)
  • mcp: Mask sensitive values in MCP config API responses. (#2667)
  • mcp: Harden the MCP config endpoint against malformed input. (#3425)
  • auth: Reject cross-site auth POSTs. (#2740)
  • gateway: Cap skill artifact preview decompression to prevent zip-bomb-style abuse. (#2963)
  • sandbox: Mount the host Docker socket only in aio (DooD) sandbox mode. (#3517)
  • sandbox: Do not bind-mount host CLI auth dirs by default. (#3521)

Fixed

Runtime, gateway & persistence

  • runtime: Rollback restore checkpoint now supersedes newer checkpoints. (#2582)
  • runtime: Persist run message summaries. (#2850)
  • runtime: Bound write_file execution-failure observations to keep failure traces from blowing out the context. (#3133)
  • runtime: Protect the sync singleton's init and reset paths. (#3413)
  • runtime: Avoid PostgreSQL aggregate FOR UPDATE on run events. (#2962)
  • runs: Restore historical runs from persistent store after a gateway restart. (#2989)
  • gateway: Return ISO 8601 timestamps from threads endpoints. (#2599)
  • gateway: Make cancel idempotent for already-interrupted runs. (#3058)
  • gateway: Split stream_existing_run into per-method routes for unique OpenAPI operationIds. (#3228)
  • events: Serialize structured DB event content. (#2762)
  • persistence: Emit timezone-aware timestamps from SQLite-backed stores. (#3130)
  • persistence: Reuse token usage model grouping expression. (#2910)
  • runs: Ignore stale run reconnect conflicts. (#3284)
  • nginx: Defer CORS to the gateway allowlist instead of double-applying it. (#2861)
  • persistence: Fix runtime journal run lifecycle events. (#3470)
  • gateway: Enforce thread ownership on stateless run endpoints. (#3473)
  • runtime: Propagate interrupt through SSE values events for the LangGraph SDK. (#3605)
  • serialization: Strip base64 image data from streamed values events. (#3631)
  • history: Strip base64 image data from REST endpoint responses. (#3535)
  • gateway: Attribute token usage to the actual models. (#3658)

Agents, subagents & middleware

  • subagents: Make subagent timeout terminal state atomic. (#2583)
  • subagents: Use model override for tools and middleware. (#2641)
  • subagents: Consolidate system_prompt and skills into a single SystemMessage. (#2701)
  • subagent: Isolate subagents from the parent run's checkpointer. (#3559)
  • agents: Make update_agent honor runtime.context user_id like setup_agent does. (#2867)
  • agents: Resolve duplicate todos channel type conflict in TodoMiddleware. (#3200)
  • agents: Offload blocking filesystem IO in the custom-agent router off the event loop. (#3457)
  • agents: Keep new agent bootstrap in user scope. (#2784)
  • loop-detection: Keep tool-call pairing on warn injection. (#2725)
  • middleware: Sync raw tool-call metadata. (#2757)
  • middleware: Handle invalid tool calls in dangling pairing middleware. (#2891)
  • middleware: Prevent todo completion reminder IM-message leak. (#2907)
  • middleware: Normalize tool result adjacency before model calls. (#2939)
  • agents: Require config.yaml in resolve_agent_dir to skip memory-only directories. (#3481)
  • agents: Sync agent_name across context/configurable and reject empty soul. (#3553)
  • middleware: Offload the uploads scan in UploadsMiddleware off the event loop. (#3311)
  • middleware: Offload memory injection off the event loop to prevent tiktoken blocking. (#3411)
  • middleware: Externalize oversized tool output into the sandbox for non-mounted sandboxes. (#3417)
  • middleware: Preserve the sandbox reducer in middleware state. (#3629)
  • subagents: Raise general-purpose max_turns to 150 and default timeout to 30 min. (#3610)

Memory & tracing

  • memory: Replace short-lived asyncio.run() with a persistent event loop. (#2627)
  • memory: Isolate queued memory updates by agent. (#2941)
  • memory: Parse wrapped memory-update JSON responses. (#3252)
  • tracing: Propagate session_id and user_id into Langfuse traces. (#2944)
  • trace: Decode unicode escape sequences in non-ASCII memory trace info. (#3104)

Tools, sandbox & MCP

  • mcp: Fix env resolution in MCP config lists. (#2556)
  • models: Record Codex token usage in usage_metadata. (#2585)
  • sandbox: Supplement list_running in RemoteSandboxBackend. (#2716)
  • sandbox: Disable MSYS path conversion for Git Bash on Windows. (#2766)
  • sandbox: Avoid blocking sandbox readiness polling. (#2822)
  • sandbox: Uphold the /mnt/user-data contract at the Sandbox API boundary. (#2881)
  • sandbox: Scope provisioner PVC data by user. (#2973)
  • sandbox: Merge idempotent sandbox state updates. (#3518)
  • tools: Introduce Runtime type alias to eliminate Pydantic serialization warnings. (#2774)
  • tools: Preserve tool_search promotions across re-entrant get_available_tools. (#2885)
  • harness: Wrap async-only config tools for sync client execution. (#2878)
  • harness: Wrap all async-only tools for sync clients. (#2935)
  • tool-search: Reliably hide deferred MCP schemas by removing the ContextVar. (#3342)
  • search: Fix DDGS Wikipedia region handling. (#3423)
  • web_fetch: Support a proxy for the Jina reader in restricted networks. (#3430)
  • sandbox: Persist lazily-acquired sandbox state via Command. (#3464)
  • sandbox: Fix stale AIO sandbox cache reuse. (#3494)
  • sandbox: Create a shell session before retrying on a fresh id. (#3577)
  • sandbox: Stop flagging string-literal path fragments as unsafe absolute paths. (#3623)
  • sandbox: Return an actionable hint when read_file hits a binary file. (#3624)
  • mcp: Make stdio MCP-produced files resolvable via virtual sandbox paths. (#3600)
  • mcp: Surface admin-required state on the settings tools page. (#3533)
  • mcp: Add a tools cache reset endpoint. (#3602)
  • uploads: Fix the upload file size contract. (#3408)

Skills & channels

  • skills: Enforce allowed-tools metadata. (#2626)
  • skills: Harden slash skill activation across chat channels. (#3466)
  • skills: Fix custom skill install permissions. (#3241)
  • channels: Authenticate gateway command requests. (#2742)
  • skills: Surface the offending line and a quoting hint on SKILL.md YAML errors. (#3335)
  • skills: Keep skill archive installation off the event loop. (#3505)
  • channels: Ignore hidden control messages when extracting replies. (#3270)
  • channels: Reload config on channel restart. (#3514)
  • channels: Surface WeCom WebSocket connection failures. (#3526)
  • channels: Close the Discord file handle after upload. (#3561)
  • channels: Require a bound identity for user-owned IM messages. (#3578)
  • channels: Scope IM files and helper commands to the owner. (#3579)
  • channels: Make runtime provider state authoritative. (#3580)
  • channels: Harden runtime credential management APIs. (#3581)
  • channels: Make the channel connect flow deterministic. (#3582)
  • channels: Centralize shared channel retry helpers. (#3583)
  • channels: Add operational guardrails. (#3584)
  • channels: Unsubscribe channel listeners by equality. (#3608)

Auth

  • auth: Replace setup-status 429 rate limit with a cached response. (#2915)
  • auth: Persist auto-generated JWT secret so it survives restarts. (#2933)
  • auth: Align auth-disabled mode with mock history loading. (#3471)

Frontend

  • frontend: Restore localhost fallback for getGatewayConfig in prod mode. (#2718)
  • chat: Prevent the first user message from being swallowed in new conversations. (#2731)
  • frontend: Use backend thread token usage for the header total. (#2800)
  • frontend: Wait for async chat submit before clearing the input. (#2940)
  • frontend: Resolve login page flickering and the resize-observer loop. (#2954)
  • frontend: Deduplicate restored thread messages. (#2958)
  • frontend: Avoid duplicate optimistic user message. (#3002)
  • frontend: Hide the copy button for streaming assistant messages. (#3176)
  • frontend: Show a new thread in the sidebar immediately on creation. (#3283)
  • frontend: Isolate new chat thread messages. (#3508)
  • frontend: Cap deeply nested list indentation to prevent render crashes. (#3393, #3570)
  • token-usage: Dedupe token usage aggregation by message id. (#2770)
  • frontend: Fall back to Streamdown clipboard copy. (#3397)
  • frontend: Remove the Backspace shortcut for deleting prompt attachments. (#3410)
  • frontend: Restructure the Memory settings toolbar into two rows. (#3433)
  • suggestions: Strip inline <think> reasoning before parsing follow-up questions. (#3435)
  • frontend: Stop fetching follow-up suggestions when they are disabled. (#3599)
  • frontend: Paginate the workspace chat list beyond 50 threads. (#3485)
  • frontend: Prevent user message bubble overflow with long unbreakable strings. (#3488)
  • frontend: Keep the workspace interactive when the SSR auth probe cannot reach the gateway. (#3495)
  • frontend: Render user messages as plain text and cap blockquote nesting. (#3502)
  • frontend: Reset the active chat after deletion. (#3519)
  • frontend: Improve the mobile workspace layout. (#3646)
  • frontend: Render full content for multi-part AI messages. (#3649)

Build, deploy, scripts & config

  • packaging: Add postgres extra for store/checkpointer support; clarify install guidance. (#2584)
  • harness: Resolve runtime paths from the project root. (#2642)
  • docker: Force nginx to resolve upstream names at request time. (#2717)
  • docker: Default Gateway to a single worker to prevent multi-worker breakage. (#3475)
  • scripts: Preserve uv extras across make dev restarts. (#2767, #2754)
  • scripts: Clean up local nginx on stop. (#3005)
  • deploy: Fall back to python / openssl when python3 is absent for secret generation. (#3074)
  • config: Make the reload boundary discoverable from code. (#3144, #3153)
  • replay-e2e: Key replay fixtures by caller and conversation. (#3453)
  • setup: Refresh LLM provider wizard defaults. (#3421)
  • config: Coerce null config.yaml list sections to an empty list. (#3434)
  • scripts: Exclude runtime state from gateway reload. (#3426)
  • scripts: Create the backend/sandbox dir before the uvicorn reload-exclude. (#3460)
  • scripts: Stop next-server correctly after make start-daemon. (#3498)
  • makefile: Fix per-commit hooks installation. (#3569)
  • replay-e2e: Match replay by conversation, not the living system prompt. (#3436)

Changed

  • provider (refactor): Share assistant payload replay matching across providers. (#3307)
  • lead-agent (refactor): Make build_middlewares public to drop the last cross-module private import. (#3458)
  • todo (refactor): Remove the unused completion reminder counter. (#3530)

Documentation

  • Document blocking-IO detection usage and maintenance. (#3233)
  • Clean standalone LangGraph server remnants from docs. (#3301)
  • Add AI assistance disclosure to the PR template and CONTRIBUTING. (#3398)
  • Document custom AIO sandbox images. (#3548)

Internal

  • dev: Add async/thread boundary detector. (#2936)
  • runtime: Add lifecycle end-to-end coverage. (#2946)
  • windows: Add PYTHONIOENCODING and PYTHONUTF8 to backend Makefile targets. (#3069)
  • blocking-io: Fail-loud repo-root resolution and shared detector CLI shim. (#3512)
  • runtime: Add a Blockbuster runtime anchor for JsonlRunEventStore async IO. (#3313)
  • ci: Consolidate PR/issue labeling and fix the reviewing-job crash and label thrash. (#3455)