mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-05-23 00:16:48 +00:00
fix(runtime): suppress tool execution when provider safety-terminates with tool_calls (#3035)
* fix(runtime): suppress tool execution when provider safety-terminates with tool_calls When a provider stops generation for safety reasons (OpenAI/Moonshot finish_reason=content_filter, Anthropic stop_reason=refusal, Gemini finish_reason=SAFETY/BLOCKLIST/PROHIBITED_CONTENT/SPII/RECITATION/ IMAGE_SAFETY/...), the response may still carry truncated tool_calls. LangChain's tool router treats any non-empty tool_calls as executable, so partial arguments (e.g. write_file with a half-finished markdown) get dispatched and the agent loops on retry. Add SafetyFinishReasonMiddleware at after_model: detect safety termination via a pluggable detector registry, clear both structured tool_calls and raw additional_kwargs.tool_calls / function_call, preserve response_metadata.finish_reason for downstream observers, stamp additional_kwargs.safety_termination for traces, append a user-facing explanation to message content (list-aware for thinking blocks), and emit a safety_termination custom stream event so SSE consumers can reconcile any "tool starting..." UI. Default detectors cover OpenAI-compatible content_filter, Anthropic refusal, and Gemini safety enums (text + image). Custom providers are added via reflection (same pattern as guardrails). Wired into both lead-agent and subagent runtimes. Closes #3028 * fix(runtime): persist safety_termination as a middleware audit event Address review on #3035: the SSE custom event is great for live consumers but invisible to post-run audit. RunEventStore should carry its own row so operators can answer "which runs were safety-suppressed today?" from a single SQL query without joining the message body. Worker now exposes the run-scoped RunJournal via runtime.context["__run_journal"] (sentinel key, internal channel). SafetyFinishReasonMiddleware calls the previously-unused RunJournal.record_middleware, which emits event_type = "middleware:safety_termination" category = "middleware" content = {name, hook, action, changes={ detector, reason_field, reason_value, suppressed_tool_call_count, suppressed_tool_call_names, suppressed_tool_call_ids, message_id, extras}} Tool *arguments* are deliberately excluded — those are the very content the provider filtered and persisting them would defeat the purpose of the safety filter (per review note in #3035). Graceful skips when journal is absent (subagent runtime, unit tests, no-event-store local dev). Journal exceptions never propagate into the agent loop. Refs #3028 * fix(runtime): satisfy ruff format + address Copilot review - ruff format on safety_finish_reason_config.py and e2e demo (CI lint failed on ruff format --check; backend Makefile lint target runs ruff check AND ruff format --check). - Docstring on SafetyFinishReasonConfig now says resolve_variable to match the actual loader used in from_config (the wording was resolve_class previously; behavior is unchanged — resolve_variable mirrors how guardrails.provider is loaded). - Switch the AIMessage type check in SafetyFinishReasonMiddleware._apply from getattr(last, "type") == "ai" to isinstance(last, AIMessage), matching TokenUsageMiddleware / TodoMiddleware / ViewImageMiddleware / SummarizationMiddleware which are the dominant pattern. Refs #3028
This commit is contained in:
+36
-1
@@ -15,7 +15,7 @@
|
||||
# ============================================================================
|
||||
# Bump this number when the config schema changes.
|
||||
# Run `make config-upgrade` to merge new fields into your local config.yaml.
|
||||
config_version: 9
|
||||
config_version: 10
|
||||
|
||||
# ============================================================================
|
||||
# Logging
|
||||
@@ -535,6 +535,41 @@ loop_detection:
|
||||
# warn: 150
|
||||
# hard_limit: 300
|
||||
|
||||
# ============================================================================
|
||||
# Provider Safety Termination Configuration
|
||||
# ============================================================================
|
||||
# Intercept AIMessages where the provider stopped generation for safety reasons
|
||||
# (e.g. OpenAI finish_reason='content_filter', Anthropic stop_reason='refusal',
|
||||
# Gemini finish_reason='SAFETY') while still returning tool_calls. The
|
||||
# tool_calls in such responses are typically truncated/unreliable and must
|
||||
# not be executed. See issue #3028 for the full failure mode.
|
||||
#
|
||||
# Detectors are loaded by class path via reflection (same pattern as
|
||||
# guardrails / models / tools). The built-in set covers OpenAI-compatible
|
||||
# content_filter, Anthropic refusal, and Gemini SAFETY/BLOCKLIST/
|
||||
# PROHIBITED_CONTENT/SPII/RECITATION.
|
||||
|
||||
safety_finish_reason:
|
||||
enabled: true
|
||||
# Leave `detectors` unset to use the built-in detector set. Set to a
|
||||
# non-empty list to fully override (use `enabled: false` to disable instead
|
||||
# of providing an empty list).
|
||||
#
|
||||
# Example — extend the OpenAI-compatible detector for a Chinese provider
|
||||
# whose gateway uses a non-standard finish_reason token:
|
||||
# detectors:
|
||||
# - use: deerflow.agents.middlewares.safety_termination_detectors:OpenAICompatibleContentFilterDetector
|
||||
# config:
|
||||
# finish_reasons: ["content_filter", "sensitive", "risk_control"]
|
||||
# - use: deerflow.agents.middlewares.safety_termination_detectors:AnthropicRefusalDetector
|
||||
# - use: deerflow.agents.middlewares.safety_termination_detectors:GeminiSafetyDetector
|
||||
#
|
||||
# Example — add a custom detector for an in-house provider:
|
||||
# detectors:
|
||||
# - use: my_company.deerflow_ext:WenxinSafetyDetector
|
||||
# config:
|
||||
# error_codes: [336003, 17, 18]
|
||||
|
||||
# ============================================================================
|
||||
# Sandbox Configuration
|
||||
# ============================================================================
|
||||
|
||||
Reference in New Issue
Block a user