mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-05-21 23:46:50 +00:00
feat(loop-detection): defer warning injection (#2752)
* fix(loop-detection): defer warn injection to wrap_model_call The warn branch in LoopDetectionMiddleware injected a HumanMessage into state from after_model. The tools node had not yet produced ToolMessage responses to the previous AIMessage(tool_calls=...), so the new HumanMessage landed *between* the assistant's tool_calls and their responses. OpenAI/Moonshot reject the next request with "tool_call_ids did not have response messages" because their validators require tool_calls to be followed immediately by tool messages. Detection now runs in after_model as before, but only enqueues the warning into a per-thread list. Injection happens in wrap_model_call, where every prior ToolMessage is already present in request.messages. The warning is appended at the end as HumanMessage(name="loop_warning") — pairing intact, AIMessage semantics untouched, no SystemMessage issues for Anthropic. Closes #2029, addresses #2255 #2293 #2304 #2511. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(channels): remove loop warning display filter * feat(loop-detection): scope pending warnings by run * docs(loop-detection): update docs * test(loop-detection): assert deferred warnings are queued * fix(loop-detection): cap transient warning state * docs: update docs * add async awrap_model_call test coverage * docs(loop-detection): document transient warnings --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Nan Gao
@@ -4,22 +4,22 @@
|
||||
|
||||
`create_deerflow_agent` 通过 `RuntimeFeatures` 组装的完整 middleware 链（默认全开时）：
|
||||
|
||||
| # | Middleware | `before_agent` | `before_model` | `after_model` | `after_agent` | `wrap_tool_call` | 主 Agent | Subagent | 来源 |
|
||||
|---|-----------|:-:|:-:|:-:|:-:|:-:|:-:|:-:|------|
|
||||
| 0 | ThreadDataMiddleware | ✓ | | | | | ✓ | ✓ | `sandbox` |
|
||||
| 1 | UploadsMiddleware | ✓ | | | | | ✓ | ✗ | `sandbox` |
|
||||
| 2 | SandboxMiddleware | ✓ | | | ✓ | | ✓ | ✓ | `sandbox` |
|
||||
| 3 | DanglingToolCallMiddleware | | | ✓ | | | ✓ | ✗ | 始终开启 |
|
||||
| 4 | GuardrailMiddleware | | | | | ✓ | ✓ | ✓ | *Phase 2 纳入* |
|
||||
| 5 | ToolErrorHandlingMiddleware | | | | | ✓ | ✓ | ✓ | 始终开启 |
|
||||
| 6 | SummarizationMiddleware | | | ✓ | | | ✓ | ✗ | `summarization` |
|
||||
| 7 | TodoMiddleware | | | ✓ | | | ✓ | ✗ | `plan_mode` 参数 |
|
||||
| 8 | TitleMiddleware | | | ✓ | | | ✓ | ✗ | `auto_title` |
|
||||
| 9 | MemoryMiddleware | | | | ✓ | | ✓ | ✗ | `memory` |
|
||||
| 10 | ViewImageMiddleware | | ✓ | | | | ✓ | ✗ | `vision` |
|
||||
| 11 | SubagentLimitMiddleware | | | ✓ | | | ✓ | ✗ | `subagent` |
|
||||
| 12 | LoopDetectionMiddleware | | | ✓ | | | ✓ | ✗ | 始终开启 |
|
||||
| 13 | ClarificationMiddleware | | | ✓ | | | ✓ | ✗ | 始终最后 |
|
||||
| # | Middleware | `before_agent` | `before_model` | `after_model` | `after_agent` | `wrap_model_call` | `wrap_tool_call` | 主 Agent | Subagent | 来源 |
|
||||
|---|-----------|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|------|
|
||||
| 0 | ThreadDataMiddleware | ✓ | | | | | | ✓ | ✓ | `sandbox` |
|
||||
| 1 | UploadsMiddleware | ✓ | | | | | | ✓ | ✗ | `sandbox` |
|
||||
| 2 | SandboxMiddleware | ✓ | | | ✓ | | | ✓ | ✓ | `sandbox` |
|
||||
| 3 | DanglingToolCallMiddleware | | | | | ✓ | | ✓ | ✗ | 始终开启 |
|
||||
| 4 | GuardrailMiddleware | | | | | | ✓ | ✓ | ✓ | *Phase 2 纳入* |
|
||||
| 5 | ToolErrorHandlingMiddleware | | | | | | ✓ | ✓ | ✓ | 始终开启 |
|
||||
| 6 | SummarizationMiddleware | | ✓ | | | | | ✓ | ✗ | `summarization` |
|
||||
| 7 | TodoMiddleware | | ✓ | ✓ | | ✓ | | ✓ | ✗ | `plan_mode` 参数 |
|
||||
| 8 | TitleMiddleware | | | ✓ | | | | ✓ | ✗ | `auto_title` |
|
||||
| 9 | MemoryMiddleware | | | | ✓ | | | ✓ | ✗ | `memory` |
|
||||
| 10 | ViewImageMiddleware | | ✓ | | | | | ✓ | ✗ | `vision` |
|
||||
| 11 | SubagentLimitMiddleware | | | ✓ | | | | ✓ | ✗ | `subagent` |
|
||||
| 12 | LoopDetectionMiddleware | ✓ | | ✓ | ✓ | ✓ | | ✓ | ✗ | 始终开启 |
|
||||
| 13 | ClarificationMiddleware | | | | | | ✓ | ✓ | ✗ | 始终最后 |
|
||||
|
||||
主 agent **14 个** middleware（`make_lead_agent`），subagent **4 个**（ThreadData、Sandbox、Guardrail、ToolErrorHandling）。`create_deerflow_agent` Phase 1 实现 **13 个**（Guardrail 仅支持自定义实例，无内置默认）。
|
||||
|
||||
@@ -35,7 +35,7 @@ graph TB
|
||||
|
||||
subgraph BA ["<b>before_agent</b> 正序 0→N"]
|
||||
direction TB
|
||||
TD["[0] ThreadData<br/>创建线程目录"] --> UL["[1] Uploads<br/>扫描上传文件"] --> SB["[2] Sandbox<br/>获取沙箱"]
|
||||
TD["[0] ThreadData<br/>创建线程目录"] --> UL["[1] Uploads<br/>扫描上传文件"] --> SB["[2] Sandbox<br/>获取沙箱"] --> LD_BA["[12] LoopDetection<br/>清理 stale warning"]
|
||||
end
|
||||
|
||||
subgraph BM ["<b>before_model</b> 正序 0→N"]
|
||||
@@ -43,34 +43,42 @@ graph TB
|
||||
VI["[10] ViewImage<br/>注入图片 base64"]
|
||||
end
|
||||
|
||||
SB --> VI
|
||||
VI --> M["<b>MODEL</b>"]
|
||||
subgraph WM ["<b>wrap_model_call</b>"]
|
||||
direction TB
|
||||
DTC_WM["[3] DanglingToolCall<br/>补悬空 ToolMessage"] --> LD_WM["[12] LoopDetection<br/>注入当前 run warning"]
|
||||
end
|
||||
|
||||
LD_BA --> VI
|
||||
VI --> DTC_WM
|
||||
LD_WM --> M["<b>MODEL</b>"]
|
||||
|
||||
subgraph AM ["<b>after_model</b> 反序 N→0"]
|
||||
direction TB
|
||||
CL["[13] Clarification<br/>拦截 ask_clarification"] --> LD["[12] LoopDetection<br/>检测循环"] --> SL["[11] SubagentLimit<br/>截断多余 task"] --> TI["[8] Title<br/>生成标题"] --> SM["[6] Summarization<br/>上下文压缩"] --> DTC["[3] DanglingToolCall<br/>补缺失 ToolMessage"]
|
||||
LD["[12] LoopDetection<br/>检测循环/排队 warning"] --> SL["[11] SubagentLimit<br/>截断多余 task"] --> TI["[8] Title<br/>生成标题"]
|
||||
end
|
||||
|
||||
M --> CL
|
||||
M --> LD
|
||||
|
||||
subgraph AA ["<b>after_agent</b> 反序 N→0"]
|
||||
direction TB
|
||||
SBR["[2] Sandbox<br/>释放沙箱"] --> MEM["[9] Memory<br/>入队记忆"]
|
||||
LD_CLEAN["[12] LoopDetection<br/>清理 pending warning"] --> MEM["[9] Memory<br/>入队记忆"] --> SBR["[2] Sandbox<br/>释放沙箱"]
|
||||
end
|
||||
|
||||
DTC --> SBR
|
||||
MEM --> END(["response"])
|
||||
TI --> LD_CLEAN
|
||||
SBR --> END(["response"])
|
||||
|
||||
classDef beforeNode fill:#a0a8b5,stroke:#636b7a,color:#2d3239
|
||||
classDef modelNode fill:#b5a8a0,stroke:#7a6b63,color:#2d3239
|
||||
classDef wrapModelNode fill:#a8a0b5,stroke:#6b637a,color:#2d3239
|
||||
classDef afterModelNode fill:#b5a0a8,stroke:#7a636b,color:#2d3239
|
||||
classDef afterAgentNode fill:#a0b5a8,stroke:#637a6b,color:#2d3239
|
||||
classDef terminalNode fill:#a8b5a0,stroke:#6b7a63,color:#2d3239
|
||||
|
||||
class TD,UL,SB,VI beforeNode
|
||||
class TD,UL,SB,LD_BA,VI beforeNode
|
||||
class DTC_WM,LD_WM wrapModelNode
|
||||
class M modelNode
|
||||
class CL,LD,SL,TI,SM,DTC afterModelNode
|
||||
class SBR,MEM afterAgentNode
|
||||
class LD,SL,TI afterModelNode
|
||||
class LD_CLEAN,SBR,MEM afterAgentNode
|
||||
class START,END terminalNode
|
||||
```
|
||||
|
||||
@@ -82,13 +90,12 @@ sequenceDiagram
|
||||
participant TD as ThreadDataMiddleware
|
||||
participant UL as UploadsMiddleware
|
||||
participant SB as SandboxMiddleware
|
||||
participant LD as LoopDetectionMiddleware
|
||||
participant VI as ViewImageMiddleware
|
||||
participant DTC as DanglingToolCallMiddleware
|
||||
participant M as MODEL
|
||||
participant CL as ClarificationMiddleware
|
||||
participant SL as SubagentLimitMiddleware
|
||||
participant TI as TitleMiddleware
|
||||
participant SM as SummarizationMiddleware
|
||||
participant DTC as DanglingToolCallMiddleware
|
||||
participant MEM as MemoryMiddleware
|
||||
|
||||
U ->> TD: invoke
|
||||
@@ -103,19 +110,26 @@ sequenceDiagram
|
||||
activate SB
|
||||
Note right of SB: before_agent 获取沙箱
|
||||
|
||||
SB ->> VI: before_model
|
||||
SB ->> LD: before_agent
|
||||
activate LD
|
||||
Note right of LD: before_agent 清理同 thread 旧 run 的 pending warning
|
||||
LD ->> VI: before_model
|
||||
activate VI
|
||||
Note right of VI: before_model 注入图片 base64
|
||||
|
||||
VI ->> M: messages + tools
|
||||
VI ->> DTC: wrap_model_call
|
||||
activate DTC
|
||||
Note right of DTC: wrap_model_call 补悬空 ToolMessage
|
||||
DTC ->> LD: wrap_model_call
|
||||
Note right of LD: wrap_model_call drain 当前 run warning 并追加到末尾
|
||||
LD ->> M: messages + tools
|
||||
activate M
|
||||
M -->> CL: AI response
|
||||
M -->> LD: AI response
|
||||
deactivate M
|
||||
|
||||
activate CL
|
||||
Note right of CL: after_model 拦截 ask_clarification
|
||||
CL -->> SL: after_model
|
||||
deactivate CL
|
||||
Note right of LD: after_model 检测循环；warning 入队，hard-stop 清 tool_calls
|
||||
LD -->> SL: after_model
|
||||
deactivate LD
|
||||
|
||||
activate SL
|
||||
Note right of SL: after_model 截断多余 task
|
||||
@@ -124,22 +138,18 @@ sequenceDiagram
|
||||
|
||||
activate TI
|
||||
Note right of TI: after_model 生成标题
|
||||
TI -->> SM: after_model
|
||||
TI -->> DTC: done
|
||||
deactivate TI
|
||||
|
||||
activate SM
|
||||
Note right of SM: after_model 上下文压缩
|
||||
SM -->> DTC: after_model
|
||||
deactivate SM
|
||||
|
||||
activate DTC
|
||||
Note right of DTC: after_model 补缺失 ToolMessage
|
||||
DTC -->> VI: done
|
||||
deactivate DTC
|
||||
|
||||
VI -->> SB: done
|
||||
deactivate VI
|
||||
|
||||
Note right of LD: after_agent 清理当前 run 未消费 warning
|
||||
|
||||
Note right of MEM: after_agent 入队记忆
|
||||
|
||||
Note right of SB: after_agent 释放沙箱
|
||||
SB -->> UL: done
|
||||
deactivate SB
|
||||
@@ -147,8 +157,6 @@ sequenceDiagram
|
||||
UL -->> TD: done
|
||||
deactivate UL
|
||||
|
||||
Note right of MEM: after_agent 入队记忆
|
||||
|
||||
TD -->> U: response
|
||||
deactivate TD
|
||||
```
|
||||
@@ -224,12 +232,12 @@ sequenceDiagram
|
||||
participant TD as ThreadData
|
||||
participant UL as Uploads
|
||||
participant SB as Sandbox
|
||||
participant LD as LoopDetection
|
||||
participant VI as ViewImage
|
||||
participant DTC as DanglingToolCall
|
||||
participant M as MODEL
|
||||
participant CL as Clarification
|
||||
participant SL as SubagentLimit
|
||||
participant TI as Title
|
||||
participant SM as Summarization
|
||||
participant MEM as Memory
|
||||
|
||||
U ->> TD: invoke
|
||||
@@ -238,34 +246,40 @@ sequenceDiagram
|
||||
Note right of UL: before_agent 扫描文件
|
||||
UL ->> SB: .
|
||||
Note right of SB: before_agent 获取沙箱
|
||||
SB ->> LD: .
|
||||
Note right of LD: before_agent 清理 stale pending warning
|
||||
|
||||
loop 每轮对话（tool call 循环）
|
||||
SB ->> VI: .
|
||||
Note right of VI: before_model 注入图片
|
||||
VI ->> M: messages + tools
|
||||
M -->> CL: AI response
|
||||
Note right of CL: after_model 拦截 ask_clarification
|
||||
CL -->> SL: .
|
||||
VI ->> DTC: .
|
||||
Note right of DTC: wrap_model_call 补悬空工具结果
|
||||
DTC ->> LD: .
|
||||
Note right of LD: wrap_model_call 注入当前 run warning
|
||||
LD ->> M: messages + tools
|
||||
M -->> LD: AI response
|
||||
Note right of LD: after_model 检测循环/排队 warning
|
||||
LD -->> SL: .
|
||||
Note right of SL: after_model 截断多余 task
|
||||
SL -->> TI: .
|
||||
Note right of TI: after_model 生成标题
|
||||
TI -->> SM: .
|
||||
Note right of SM: after_model 上下文压缩
|
||||
end
|
||||
|
||||
Note right of SB: after_agent 释放沙箱
|
||||
SB -->> MEM: .
|
||||
Note right of LD: after_agent 清理当前 run pending warning
|
||||
LD -->> MEM: .
|
||||
Note right of MEM: after_agent 入队记忆
|
||||
MEM -->> U: response
|
||||
MEM -->> SB: .
|
||||
Note right of SB: after_agent 释放沙箱
|
||||
SB -->> U: response
|
||||
```
|
||||
|
||||
> [!warning] 不是洋葱
|
||||
> 14 个 middleware 中只有 SandboxMiddleware 有 before/after 对称（获取/释放）。其余都是单向的：要么只在 `before_*` 做事，要么只在 `after_*` 做事。`before_agent` / `after_agent` 只跑一次，`before_model` / `after_model` 每轮循环都跑。
|
||||
> 大部分 middleware 只用一个阶段。SandboxMiddleware 使用 `before_agent`/`after_agent` 做资源获取/释放；LoopDetectionMiddleware 也使用这两个钩子，但用途是清理 run-scoped pending warnings，不是资源生命周期对称。`before_agent` / `after_agent` 只跑一次，`before_model` / `after_model` / `wrap_model_call` 每轮循环都跑。
|
||||
|
||||
硬依赖只有 2 处：
|
||||
|
||||
1. **ThreadData 在 Sandbox 之前** — sandbox 需要线程目录
|
||||
2. **Clarification 在列表最后** — `after_model` 反序时最先执行，第一个拦截 `ask_clarification`
|
||||
2. **Clarification 在列表最后** — `wrap_tool_call` 处理 `ask_clarification` 时优先拦截，并通过 `Command(goto=END)` 中断执行
|
||||
|
||||
### 结论
|
||||
|
||||
@@ -273,19 +287,19 @@ sequenceDiagram
|
||||
|---|---|---|
|
||||
| 每个 middleware | before + after 对称 | 大多只用一个钩子 |
|
||||
| 激活条 | 嵌套（外长内短） | 不嵌套（串行） |
|
||||
| 反序的意义 | 清理与初始化配对 | 仅影响 after_model 的执行优先级 |
|
||||
| 反序的意义 | 清理与初始化配对 | 影响 `after_model` / `after_agent` 的执行优先级 |
|
||||
| 典型例子 | Auth: 校验 token / 清理上下文 | ThreadData: 只创建目录，没有清理 |
|
||||
|
||||
## 关键设计点
|
||||
|
||||
### ClarificationMiddleware 为什么在列表最后？
|
||||
|
||||
位置最后 = `after_model` 最先执行。它需要**第一个**看到 model 输出，检查是否有 `ask_clarification` tool call。如果有，立即中断（`Command(goto=END)`），后续 middleware 的 `after_model` 不再执行。
|
||||
位置最后使它在工具调用包装链中优先拦截 `ask_clarification`。如果命中，它返回 `Command(goto=END)`，把格式化后的澄清问题写成 `ToolMessage` 并中断执行。
|
||||
|
||||
### SandboxMiddleware 的对称性
|
||||
|
||||
`before_agent`（正序第 3 个）获取沙箱，`after_agent`（反序第 1 个）释放沙箱。外层进入 → 外层退出，天然的洋葱对称。
|
||||
|
||||
### 大部分 middleware 只用一个钩子
|
||||
### LoopDetectionMiddleware 为什么同时用多个钩子？
|
||||
|
||||
14 个 middleware 中，只有 SandboxMiddleware 同时用了 `before_agent` + `after_agent`（获取/释放）。其余都只在一个阶段执行。洋葱模型的反序特性主要影响 `after_model` 阶段的执行顺序。
|
||||
`after_model` 只做检测：重复工具调用达到 warning 阈值时，把 warning 放入 `(thread_id, run_id)` 作用域的 pending 队列。真正注入发生在下一次 `wrap_model_call`：此时上一轮 `AIMessage(tool_calls)` 对应的 `ToolMessage` 已经在请求里，warning 追加在末尾，不会破坏 OpenAI/Moonshot 的 tool-call pairing。`before_agent` 清理同一 thread 下旧 run 的残留 warning，`after_agent` 清理当前 run 没被消费的 warning。
|
||||
|
||||
Reference in New Issue
Block a user