mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-05-23 16:35:59 +00:00
Xinmin Zeng
be0eae9825
* fix(runtime): suppress tool execution when provider safety-terminates with tool_calls When a provider stops generation for safety reasons (OpenAI/Moonshot finish_reason=content_filter, Anthropic stop_reason=refusal, Gemini finish_reason=SAFETY/BLOCKLIST/PROHIBITED_CONTENT/SPII/RECITATION/ IMAGE_SAFETY/...), the response may still carry truncated tool_calls. LangChain's tool router treats any non-empty tool_calls as executable, so partial arguments (e.g. write_file with a half-finished markdown) get dispatched and the agent loops on retry. Add SafetyFinishReasonMiddleware at after_model: detect safety termination via a pluggable detector registry, clear both structured tool_calls and raw additional_kwargs.tool_calls / function_call, preserve response_metadata.finish_reason for downstream observers, stamp additional_kwargs.safety_termination for traces, append a user-facing explanation to message content (list-aware for thinking blocks), and emit a safety_termination custom stream event so SSE consumers can reconcile any "tool starting..." UI. Default detectors cover OpenAI-compatible content_filter, Anthropic refusal, and Gemini safety enums (text + image). Custom providers are added via reflection (same pattern as guardrails). Wired into both lead-agent and subagent runtimes. Closes #3028 * fix(runtime): persist safety_termination as a middleware audit event Address review on #3035: the SSE custom event is great for live consumers but invisible to post-run audit. RunEventStore should carry its own row so operators can answer "which runs were safety-suppressed today?" from a single SQL query without joining the message body. Worker now exposes the run-scoped RunJournal via runtime.context["__run_journal"] (sentinel key, internal channel). SafetyFinishReasonMiddleware calls the previously-unused RunJournal.record_middleware, which emits event_type = "middleware:safety_termination" category = "middleware" content = {name, hook, action, changes={ detector, reason_field, reason_value, suppressed_tool_call_count, suppressed_tool_call_names, suppressed_tool_call_ids, message_id, extras}} Tool *arguments* are deliberately excluded — those are the very content the provider filtered and persisting them would defeat the purpose of the safety filter (per review note in #3035). Graceful skips when journal is absent (subagent runtime, unit tests, no-event-store local dev). Journal exceptions never propagate into the agent loop. Refs #3028 * fix(runtime): satisfy ruff format + address Copilot review - ruff format on safety_finish_reason_config.py and e2e demo (CI lint failed on ruff format --check; backend Makefile lint target runs ruff check AND ruff format --check). - Docstring on SafetyFinishReasonConfig now says resolve_variable to match the actual loader used in from_config (the wording was resolve_class previously; behavior is unchanged — resolve_variable mirrors how guardrails.provider is loaded). - Switch the AIMessage type check in SafetyFinishReasonMiddleware._apply from getattr(last, "type") == "ai" to isinstance(last, AIMessage), matching TokenUsageMiddleware / TodoMiddleware / ViewImageMiddleware / SummarizationMiddleware which are the dominant pattern. Refs #3028
48 lines
1.7 KiB
Python
48 lines
1.7 KiB
Python
"""Configuration for SafetyFinishReasonMiddleware.
|
|
|
|
Mirrors the shape of GuardrailsConfig: detectors are loaded by class path
|
|
through ``deerflow.reflection.resolve_variable`` (same loader the
|
|
``guardrails.provider`` config uses) so users can drop in custom provider
|
|
detectors without modifying core code.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from pydantic import BaseModel, Field
|
|
|
|
|
|
class SafetyDetectorConfig(BaseModel):
|
|
"""One detector entry under ``safety_finish_reason.detectors``."""
|
|
|
|
use: str = Field(
|
|
description=("Class path of a SafetyTerminationDetector implementation (e.g. 'deerflow.agents.middlewares.safety_termination_detectors:OpenAICompatibleContentFilterDetector')."),
|
|
)
|
|
config: dict = Field(
|
|
default_factory=dict,
|
|
description="Constructor kwargs passed to the detector class.",
|
|
)
|
|
|
|
|
|
class SafetyFinishReasonConfig(BaseModel):
|
|
"""Configuration for the SafetyFinishReasonMiddleware.
|
|
|
|
The middleware intercepts AIMessages where the provider signaled a
|
|
safety-related termination (e.g. OpenAI ``finish_reason='content_filter'``)
|
|
while still returning tool calls, and suppresses those tool calls so the
|
|
half-truncated arguments never execute.
|
|
"""
|
|
|
|
enabled: bool = Field(
|
|
default=True,
|
|
description="Master switch for the SafetyFinishReasonMiddleware.",
|
|
)
|
|
detectors: list[SafetyDetectorConfig] | None = Field(
|
|
default=None,
|
|
description=(
|
|
"Custom detector list. Leave unset (None) to use the built-in "
|
|
"set covering OpenAI-compatible content_filter, Anthropic "
|
|
"refusal, and Gemini SAFETY/BLOCKLIST/PROHIBITED_CONTENT/SPII/"
|
|
"RECITATION. Provide a non-null list to fully override."
|
|
),
|
|
)
|