fix(runtime): suppress tool execution when provider safety-terminates with tool_calls (#3035)

* fix(runtime): suppress tool execution when provider safety-terminates with tool_calls When a provider stops generation for safety reasons (OpenAI/Moonshot finish_reason=content_filter, Anthropic stop_reason=refusal, Gemini finish_reason=SAFETY/BLOCKLIST/PROHIBITED_CONTENT/SPII/RECITATION/ IMAGE_SAFETY/...), the response may still carry truncated tool_calls. LangChain's tool router treats any non-empty tool_calls as executable, so partial arguments (e.g. write_file with a half-finished markdown) get dispatched and the agent loops on retry. Add SafetyFinishReasonMiddleware at after_model: detect safety termination via a pluggable detector registry, clear both structured tool_calls and raw additional_kwargs.tool_calls / function_call, preserve response_metadata.finish_reason for downstream observers, stamp additional_kwargs.safety_termination for traces, append a user-facing explanation to message content (list-aware for thinking blocks), and emit a safety_termination custom stream event so SSE consumers can reconcile any "tool starting..." UI. Default detectors cover OpenAI-compatible content_filter, Anthropic refusal, and Gemini safety enums (text + image). Custom providers are added via reflection (same pattern as guardrails). Wired into both lead-agent and subagent runtimes. Closes #3028 * fix(runtime): persist safety_termination as a middleware audit event Address review on #3035: the SSE custom event is great for live consumers but invisible to post-run audit. RunEventStore should carry its own row so operators can answer "which runs were safety-suppressed today?" from a single SQL query without joining the message body. Worker now exposes the run-scoped RunJournal via runtime.context["__run_journal"] (sentinel key, internal channel). SafetyFinishReasonMiddleware calls the previously-unused RunJournal.record_middleware, which emits event_type = "middleware:safety_termination" category = "middleware" content = {name, hook, action, changes={ detector, reason_field, reason_value, suppressed_tool_call_count, suppressed_tool_call_names, suppressed_tool_call_ids, message_id, extras}} Tool *arguments* are deliberately excluded — those are the very content the provider filtered and persisting them would defeat the purpose of the safety filter (per review note in #3035). Graceful skips when journal is absent (subagent runtime, unit tests, no-event-store local dev). Journal exceptions never propagate into the agent loop. Refs #3028 * fix(runtime): satisfy ruff format + address Copilot review - ruff format on safety_finish_reason_config.py and e2e demo (CI lint failed on ruff format --check; backend Makefile lint target runs ruff check AND ruff format --check). - Docstring on SafetyFinishReasonConfig now says resolve_variable to match the actual loader used in from_config (the wording was resolve_class previously; behavior is unchanged — resolve_variable mirrors how guardrails.provider is loaded). - Switch the AIMessage type check in SafetyFinishReasonMiddleware._apply from getattr(last, "type") == "ai" to isinstance(last, AIMessage), matching TokenUsageMiddleware / TodoMiddleware / ViewImageMiddleware / SummarizationMiddleware which are the dominant pattern. Refs #3028
2026-05-23 00:16:48 +00:00 · 2026-05-22 21:20:28 +08:00
parent 253542ea0d
commit be0eae9825
14 changed files with 1936 additions and 5 deletions
@@ -29,6 +29,7 @@ from deerflow.agents.memory.summarization_hook import memory_flush_hook
 from deerflow.agents.middlewares.clarification_middleware import ClarificationMiddleware
 from deerflow.agents.middlewares.loop_detection_middleware import LoopDetectionMiddleware
 from deerflow.agents.middlewares.memory_middleware import MemoryMiddleware
+from deerflow.agents.middlewares.safety_finish_reason_middleware import SafetyFinishReasonMiddleware
 from deerflow.agents.middlewares.subagent_limit_middleware import SubagentLimitMiddleware
 from deerflow.agents.middlewares.summarization_middleware import BeforeSummarizationHook, DeerFlowSummarizationMiddleware
 from deerflow.agents.middlewares.title_middleware import TitleMiddleware
@@ -338,6 +339,15 @@ def _build_middlewares(
    if custom_middlewares:
        middlewares.extend(custom_middlewares)

+    # SafetyFinishReasonMiddleware — suppress tool execution when the provider
+    # safety-terminated the response. Registered after custom middlewares so
+    # that LangChain's reverse-order after_model dispatch runs Safety first;
+    # cleared tool_calls then flow through Loop/Subagent accounting without
+    # firing extra alarms. See safety_finish_reason_middleware.py docstring.
+    safety_config = resolved_app_config.safety_finish_reason
+    if safety_config.enabled:
+        middlewares.append(SafetyFinishReasonMiddleware.from_config(safety_config))
+
    # ClarificationMiddleware should always be last
    middlewares.append(ClarificationMiddleware())
    return middlewares
@@ -0,0 +1,317 @@
+"""Suppress tool execution when the provider safety-terminated the response.
+
+Background — see issue bytedance/deer-flow#3028.
+
+Some providers (OpenAI ``finish_reason='content_filter'``, Anthropic
+``stop_reason='refusal'``, Gemini ``finish_reason='SAFETY'`` ...) can stop
+generation mid-stream while still returning partially-formed ``tool_calls``.
+LangChain's tool router treats any AIMessage with a non-empty ``tool_calls``
+field as "go execute these", so half-truncated arguments — e.g. a markdown
+``write_file`` that stops in the middle of a sentence — get dispatched as if
+they were complete. The agent then sees the truncated file, tries to fix it,
+gets filtered again, and loops.
+
+This middleware sits at ``after_model`` and gates that behaviour: when a
+configured ``SafetyTerminationDetector`` fires *and* the AIMessage carries
+tool calls, we strip the tool calls (both structured and raw provider
+payloads), append a user-facing explanation, and stash observability fields
+in ``additional_kwargs.safety_termination`` so logs, traces, and SSE
+consumers can see what happened.
+
+Hook choice: ``after_model`` (not ``wrap_model_call``) because the response
+is a *normal* return — not an exception — and we want to participate in the
+same after-model chain as ``LoopDetectionMiddleware``, with which we share
+the same tool-call-suppression mechanic but a different trigger.
+
+Placement: register *after* ``LoopDetectionMiddleware`` in the middleware
+list. LangChain factory wires ``after_model`` edges in reverse list order
+(``langchain/agents/factory.py:add_edge("model", middleware_w_after_model[-1])``,
+then walks ``range(len-1, 0, -1)``), so the *last* registered middleware is
+the *first* to observe the model output. Registering Safety after Loop
+means Safety sees the raw response first, clears tool calls if it fires,
+and Loop then accounts against the cleaned message.
+"""
+
+from __future__ import annotations
+
+import logging
+from typing import TYPE_CHECKING, override
+
+from langchain.agents import AgentState
+from langchain.agents.middleware import AgentMiddleware
+from langchain_core.messages import AIMessage
+from langgraph.runtime import Runtime
+
+from deerflow.agents.middlewares.safety_termination_detectors import (
+    SafetyTermination,
+    SafetyTerminationDetector,
+    default_detectors,
+)
+from deerflow.agents.middlewares.tool_call_metadata import clone_ai_message_with_tool_calls
+
+if TYPE_CHECKING:
+    from deerflow.config.safety_finish_reason_config import SafetyFinishReasonConfig
+
+logger = logging.getLogger(__name__)
+
+
+_USER_FACING_MESSAGE = (
+    "The model provider stopped this response with a safety-related signal "
+    "({reason_field}={reason_value!r}, detector={detector!r}). Any tool "
+    "calls produced in this turn were suppressed because their arguments "
+    "may be truncated and unsafe to execute. Please rephrase the request "
+    "or ask for a narrower output."
+)
+
+
+class SafetyFinishReasonMiddleware(AgentMiddleware[AgentState]):
+    """Strip tool_calls from AIMessages flagged by a SafetyTerminationDetector."""
+
+    def __init__(self, detectors: list[SafetyTerminationDetector] | None = None) -> None:
+        super().__init__()
+        # Copy so caller mutations after construction don't leak into us.
+        self._detectors: list[SafetyTerminationDetector] = list(detectors) if detectors else default_detectors()
+
+    @classmethod
+    def from_config(cls, config: SafetyFinishReasonConfig) -> SafetyFinishReasonMiddleware:
+        """Construct from validated Pydantic config, honouring the
+        reflection-loaded detector list when provided.
+
+        An explicit empty list is intentionally rejected — it would silently
+        disable detection while leaving the middleware in the chain, which
+        is the worst of both worlds. Use ``enabled: false`` instead.
+        """
+        if config.detectors is None:
+            return cls()
+
+        if not config.detectors:
+            raise ValueError("safety_finish_reason.detectors must be omitted (use built-ins) or contain at least one entry; use enabled=false to disable the middleware entirely.")
+
+        from deerflow.reflection import resolve_variable
+
+        detectors: list[SafetyTerminationDetector] = []
+        for entry in config.detectors:
+            detector_cls = resolve_variable(entry.use)
+            kwargs = dict(entry.config) if entry.config else {}
+            detector = detector_cls(**kwargs)
+            if not isinstance(detector, SafetyTerminationDetector):
+                raise TypeError(f"{entry.use} did not produce a SafetyTerminationDetector (got {type(detector).__name__}); ensure it has a `name` attribute and a `detect(message)` method")
+            detectors.append(detector)
+        return cls(detectors=detectors)
+
+    # ----- detection -------------------------------------------------------
+
+    def _detect(self, message: AIMessage) -> SafetyTermination | None:
+        for detector in self._detectors:
+            try:
+                hit = detector.detect(message)
+            except Exception:  # noqa: BLE001 - never let a buggy detector break the agent run
+                logger.exception("SafetyTerminationDetector %r raised; treating as no-match", getattr(detector, "name", type(detector).__name__))
+                continue
+            if hit is not None:
+                return hit
+        return None
+
+    # ----- message rewriting ----------------------------------------------
+
+    @staticmethod
+    def _append_user_message(content: object, text: str) -> str | list:
+        """Append a plain-text explanation to AIMessage content.
+
+        Mirrors ``LoopDetectionMiddleware._append_text`` so list-content
+        responses (Anthropic thinking blocks, vLLM reasoning splits) keep
+        their structure instead of being string-coerced into a TypeError.
+        """
+        if content is None or content == "":
+            return text
+        if isinstance(content, list):
+            return [*content, {"type": "text", "text": f"\n\n{text}"}]
+        if isinstance(content, str):
+            return content + f"\n\n{text}"
+        return str(content) + f"\n\n{text}"
+
+    def _build_suppressed_message(
+        self,
+        message: AIMessage,
+        termination: SafetyTermination,
+    ) -> AIMessage:
+        suppressed_names = [tc.get("name") or "unknown" for tc in (message.tool_calls or [])]
+        explanation = _USER_FACING_MESSAGE.format(
+            reason_field=termination.reason_field,
+            reason_value=termination.reason_value,
+            detector=termination.detector,
+        )
+        new_content = self._append_user_message(message.content, explanation)
+
+        # clone_ai_message_with_tool_calls handles structured tool_calls,
+        # raw additional_kwargs.tool_calls, and function_call in one shot.
+        # It only rewrites finish_reason when the old value was "tool_calls",
+        # which is not our case — content_filter / refusal / SAFETY stay put
+        # so downstream SSE / converters keep seeing the real provider reason.
+        cleared = clone_ai_message_with_tool_calls(message, [], content=new_content)
+
+        # Re-clone additional_kwargs so we don't accidentally mutate the
+        # dict returned by clone_ai_message_with_tool_calls (which already
+        # made a shallow copy, but downstream model_copy still references
+        # it). Then stamp the observability record.
+        kwargs = dict(getattr(cleared, "additional_kwargs", None) or {})
+        kwargs["safety_termination"] = {
+            "detector": termination.detector,
+            "reason_field": termination.reason_field,
+            "reason_value": termination.reason_value,
+            "suppressed_tool_call_count": len(suppressed_names),
+            "suppressed_tool_call_names": suppressed_names,
+            "extras": dict(termination.extras) if termination.extras else {},
+        }
+        return cleared.model_copy(update={"additional_kwargs": kwargs})
+
+    # ----- observability ---------------------------------------------------
+
+    def _emit_event(
+        self,
+        termination: SafetyTermination,
+        suppressed_names: list[str],
+        runtime: Runtime,
+    ) -> None:
+        """Notify SSE consumers (e.g. the web UI) that a tool turn was
+        suppressed so they can reconcile any "tool starting..." placeholders
+        already streamed to the user. Failures are logged at debug and
+        ignored — this is a best-effort signal."""
+        try:
+            from langgraph.config import get_stream_writer
+
+            writer = get_stream_writer()
+        except Exception:  # noqa: BLE001
+            logger.debug("get_stream_writer unavailable; skipping safety_termination event", exc_info=True)
+            return
+
+        thread_id = None
+        if runtime is not None and getattr(runtime, "context", None):
+            thread_id = runtime.context.get("thread_id") if isinstance(runtime.context, dict) else None
+
+        try:
+            writer(
+                {
+                    "type": "safety_termination",
+                    "detector": termination.detector,
+                    "reason_field": termination.reason_field,
+                    "reason_value": termination.reason_value,
+                    "suppressed_tool_call_count": len(suppressed_names),
+                    "suppressed_tool_call_names": suppressed_names,
+                    "thread_id": thread_id,
+                }
+            )
+        except Exception:  # noqa: BLE001
+            logger.debug("Failed to emit safety_termination stream event", exc_info=True)
+
+    def _record_audit_event(
+        self,
+        termination: SafetyTermination,
+        message,
+        tool_calls: list[dict],
+        runtime: Runtime,
+    ) -> None:
+        """Write a ``middleware:safety_termination`` record to RunEventStore
+        for post-run auditability.
+
+        The custom stream event in ``_emit_event`` is consumed by live SSE
+        clients and disappears after the run; this event is persisted so an
+        operator can answer "which runs were safety-suppressed today?" from
+        a single SQL query without joining the message body. Worker exposes
+        the run-scoped ``RunJournal`` via ``runtime.context["__run_journal"]``;
+        absent in unit-test / subagent / no-event-store paths, in which case
+        we silently skip.
+
+        Tool **arguments** are deliberately **not** recorded — those are the
+        very content the provider filtered; persisting them would defeat the
+        purpose of the safety filter. Names / count / ids are sufficient for
+        audit and debugging (issue #3028 review).
+        """
+        journal = None
+        if runtime is not None and getattr(runtime, "context", None):
+            context = runtime.context
+            if isinstance(context, dict):
+                journal = context.get("__run_journal")
+        if journal is None:
+            return
+
+        suppressed_names = [tc.get("name") or "unknown" for tc in tool_calls]
+        suppressed_ids = [tc.get("id") for tc in tool_calls if tc.get("id")]
+
+        changes = {
+            "detector": termination.detector,
+            "reason_field": termination.reason_field,
+            "reason_value": termination.reason_value,
+            "suppressed_tool_call_count": len(tool_calls),
+            "suppressed_tool_call_names": suppressed_names,
+            "suppressed_tool_call_ids": suppressed_ids,
+            "message_id": getattr(message, "id", None),
+            "extras": dict(termination.extras) if termination.extras else {},
+        }
+
+        try:
+            journal.record_middleware(
+                tag="safety_termination",
+                name=type(self).__name__,
+                hook="after_model",
+                action="suppress_tool_calls",
+                changes=changes,
+            )
+        except Exception:  # noqa: BLE001
+            # Audit-event persistence must never break agent execution.
+            logger.debug("Failed to record middleware:safety_termination event", exc_info=True)
+
+    # ----- main apply ------------------------------------------------------
+
+    def _apply(self, state: AgentState, runtime: Runtime) -> dict | None:
+        messages = state.get("messages", [])
+        if not messages:
+            return None
+
+        last = messages[-1]
+        if not isinstance(last, AIMessage):
+            return None
+
+        # Issue scope: only intervene when there's something to suppress.
+        # ``content_filter`` without tool_calls is allowed through unchanged
+        # so the partial text response (if any) reaches the user naturally.
+        tool_calls = last.tool_calls
+        if not tool_calls:
+            return None
+
+        termination = self._detect(last)
+        if termination is None:
+            return None
+
+        patched = self._build_suppressed_message(last, termination)
+
+        thread_id = None
+        if runtime is not None and getattr(runtime, "context", None):
+            thread_id = runtime.context.get("thread_id") if isinstance(runtime.context, dict) else None
+
+        logger.warning(
+            "Provider safety termination detected — suppressed %d tool call(s)",
+            len(tool_calls),
+            extra={
+                "thread_id": thread_id,
+                "detector": termination.detector,
+                "reason_field": termination.reason_field,
+                "reason_value": termination.reason_value,
+                "suppressed_tool_call_names": [tc.get("name") for tc in tool_calls],
+            },
+        )
+
+        self._emit_event(termination, [tc.get("name") or "unknown" for tc in tool_calls], runtime)
+        self._record_audit_event(termination, last, list(tool_calls), runtime)
+
+        return {"messages": [patched]}
+
+    # ----- hooks -----------------------------------------------------------
+
+    @override
+    def after_model(self, state: AgentState, runtime: Runtime) -> dict | None:
+        return self._apply(state, runtime)
+
+    @override
+    async def aafter_model(self, state: AgentState, runtime: Runtime) -> dict | None:
+        return self._apply(state, runtime)
@@ -0,0 +1,237 @@
+"""Detectors for provider-side safety termination signals.
+
+Different LLM providers signal "I stopped this response for safety reasons"
+through different fields with different values. This module defines a small
+strategy interface and three built-in detectors that cover the major
+providers DeerFlow supports today. New providers (Wenxin, Hunyuan, Bedrock
+adapters, in-house gateways, ...) can be added by implementing
+``SafetyTerminationDetector`` and wiring it through
+``config.yaml: safety_finish_reason.detectors``.
+
+The middleware that consumes these detectors lives in
+``safety_finish_reason_middleware.py``.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Any, Protocol, runtime_checkable
+
+from langchain_core.messages import AIMessage
+
+
+@dataclass(frozen=True)
+class SafetyTermination:
+    """A detected safety-related termination signal.
+
+    Attributes:
+        detector: Name of the detector that produced this result. Used for
+            observability so operators can see which provider rule fired.
+        reason_field: The message metadata field that carried the signal
+            (e.g. ``finish_reason``, ``stop_reason``).
+        reason_value: The actual value of that field
+            (e.g. ``content_filter``, ``refusal``, ``SAFETY``).
+        extras: Provider-specific metadata that may help downstream
+            consumers (e.g. Azure OpenAI content_filter_results, Gemini
+            safety_ratings). Detectors are free to populate or skip this.
+    """
+
+    detector: str
+    reason_field: str
+    reason_value: str
+    extras: dict[str, Any] = field(default_factory=dict)
+
+
+@runtime_checkable
+class SafetyTerminationDetector(Protocol):
+    """Strategy interface for provider safety termination detection."""
+
+    name: str
+
+    def detect(self, message: AIMessage) -> SafetyTermination | None:
+        """Return a SafetyTermination if *message* indicates provider safety
+        termination, otherwise return ``None``.
+
+        Implementations must be side-effect free and tolerant of missing or
+        oddly-typed metadata — detectors run on every model response.
+        """
+        ...
+
+
+def _get_metadata_value(message: AIMessage, field_name: str) -> str | None:
+    """Read a string-typed value from either ``response_metadata`` or
+    ``additional_kwargs``.
+
+    LangChain provider adapters are inconsistent about where they stash
+    provider stop signals. Most modern adapters use ``response_metadata``,
+    but some legacy / passthrough paths still surface them via
+    ``additional_kwargs``. We check both, in that order, and only accept
+    string values — Pydantic enums or dicts are ignored so we never raise
+    on malformed inputs.
+    """
+    for container_name in ("response_metadata", "additional_kwargs"):
+        container = getattr(message, container_name, None) or {}
+        if not isinstance(container, dict):
+            continue
+        value = container.get(field_name)
+        if isinstance(value, str) and value:
+            return value
+    return None
+
+
+class OpenAICompatibleContentFilterDetector:
+    """OpenAI-compatible content_filter signal.
+
+    Covers OpenAI, Azure OpenAI, Moonshot/Kimi, DeepSeek, Mistral, vLLM,
+    Qwen (OpenAI-compatible mode), and any other adapter that follows the
+    OpenAI ``finish_reason`` convention.
+
+    Some Chinese providers ship custom OpenAI-compatible gateways that use
+    alternative tokens like ``sensitive`` or ``violation``. Extend the set
+    via the ``finish_reasons`` kwarg in config.
+    """
+
+    name = "openai_compatible_content_filter"
+
+    def __init__(self, finish_reasons: list[str] | tuple[str, ...] | None = None) -> None:
+        configured = finish_reasons if finish_reasons is not None else ("content_filter",)
+        self._finish_reasons: frozenset[str] = frozenset(r.lower() for r in configured)
+
+    def detect(self, message: AIMessage) -> SafetyTermination | None:
+        value = _get_metadata_value(message, "finish_reason")
+        if value is None or value.lower() not in self._finish_reasons:
+            return None
+
+        extras: dict[str, Any] = {}
+        # Azure OpenAI ships a structured content_filter_results block; carry it
+        # through so operators can see *what* was filtered without re-tracing.
+        response_metadata = getattr(message, "response_metadata", None) or {}
+        if isinstance(response_metadata, dict):
+            filter_results = response_metadata.get("content_filter_results")
+            if filter_results:
+                extras["content_filter_results"] = filter_results
+
+        return SafetyTermination(
+            detector=self.name,
+            reason_field="finish_reason",
+            reason_value=value,
+            extras=extras,
+        )
+
+
+class AnthropicRefusalDetector:
+    """Anthropic ``stop_reason == "refusal"`` signal.
+
+    Anthropic models surface safety refusals via a dedicated ``stop_reason``
+    rather than ``finish_reason``. See:
+    https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/handle-streaming-refusals
+    """
+
+    name = "anthropic_refusal"
+
+    def __init__(self, stop_reasons: list[str] | tuple[str, ...] | None = None) -> None:
+        configured = stop_reasons if stop_reasons is not None else ("refusal",)
+        self._stop_reasons: frozenset[str] = frozenset(r.lower() for r in configured)
+
+    def detect(self, message: AIMessage) -> SafetyTermination | None:
+        value = _get_metadata_value(message, "stop_reason")
+        if value is None or value.lower() not in self._stop_reasons:
+            return None
+        return SafetyTermination(
+            detector=self.name,
+            reason_field="stop_reason",
+            reason_value=value,
+        )
+
+
+class GeminiSafetyDetector:
+    """Gemini / Vertex AI safety-related finish reasons.
+
+    Gemini uses the same ``finish_reason`` field as OpenAI but with an
+    enumerated upper-case taxonomy. The default set covers every Gemini
+    finish_reason that means "the model stopped because the content/image
+    tripped a safety, blocklist, recitation, or PII filter" — i.e. cases
+    where any tool_calls returned alongside are likely truncated/
+    unreliable. Full enum:
+    https://docs.cloud.google.com/python/docs/reference/aiplatform/latest/google.cloud.aiplatform_v1.types.Candidate.FinishReason
+
+    Intentionally **excluded** from the default set:
+    - ``STOP``                       — normal termination.
+    - ``MAX_TOKENS``                 — output length truncation, not safety
+                                       (same root failure mode as
+                                       content_filter, but issue #3028
+                                       scopes it out; expose separately if
+                                       desired).
+    - ``LANGUAGE`` / ``NO_IMAGE``    — capability mismatches, unrelated to
+                                       safety; tool_calls would be absent
+                                       anyway.
+    - ``MALFORMED_FUNCTION_CALL`` /
+      ``UNEXPECTED_TOOL_CALL``       — tool-call protocol errors. The
+                                       tool_calls are *also* unreliable
+                                       here, but the failure category is
+                                       distinct from safety filtering;
+                                       handle in a dedicated detector to
+                                       keep observability records honest.
+    - ``OTHER`` / ``IMAGE_OTHER`` /
+      ``FINISH_REASON_UNSPECIFIED``  — too broad to enable by default;
+                                       opt in via ``finish_reasons=`` if
+                                       your provider abuses these.
+    """
+
+    name = "gemini_safety"
+
+    _DEFAULT_FINISH_REASONS = (
+        # Text safety
+        "SAFETY",
+        "BLOCKLIST",
+        "PROHIBITED_CONTENT",
+        "SPII",
+        "RECITATION",
+        # Image safety (multimodal generation)
+        "IMAGE_SAFETY",
+        "IMAGE_PROHIBITED_CONTENT",
+        "IMAGE_RECITATION",
+    )
+
+    def __init__(self, finish_reasons: list[str] | tuple[str, ...] | None = None) -> None:
+        configured = finish_reasons if finish_reasons is not None else self._DEFAULT_FINISH_REASONS
+        self._finish_reasons: frozenset[str] = frozenset(r.upper() for r in configured)
+
+    def detect(self, message: AIMessage) -> SafetyTermination | None:
+        value = _get_metadata_value(message, "finish_reason")
+        if value is None or value.upper() not in self._finish_reasons:
+            return None
+
+        extras: dict[str, Any] = {}
+        response_metadata = getattr(message, "response_metadata", None) or {}
+        if isinstance(response_metadata, dict):
+            # Gemini surfaces per-category scoring under safety_ratings.
+            ratings = response_metadata.get("safety_ratings")
+            if ratings:
+                extras["safety_ratings"] = ratings
+
+        return SafetyTermination(
+            detector=self.name,
+            reason_field="finish_reason",
+            reason_value=value,
+            extras=extras,
+        )
+
+
+def default_detectors() -> list[SafetyTerminationDetector]:
+    """Built-in detector set used when no custom detectors are configured."""
+    return [
+        OpenAICompatibleContentFilterDetector(),
+        AnthropicRefusalDetector(),
+        GeminiSafetyDetector(),
+    ]
+
+
+__all__ = [
+    "AnthropicRefusalDetector",
+    "GeminiSafetyDetector",
+    "OpenAICompatibleContentFilterDetector",
+    "SafetyTermination",
+    "SafetyTerminationDetector",
+    "default_detectors",
+]
@@ -164,4 +164,14 @@ def build_subagent_runtime_middlewares(

        middlewares.append(ViewImageMiddleware())

+    # Same provider safety-termination guard the lead agent uses — subagents
+    # are equally exposed to truncated tool_calls returned with
+    # finish_reason=content_filter (and friends), and the bad call would then
+    # propagate back to the lead agent via the task tool result.
+    safety_config = app_config.safety_finish_reason
+    if safety_config.enabled:
+        from deerflow.agents.middlewares.safety_finish_reason_middleware import SafetyFinishReasonMiddleware
+
+        middlewares.append(SafetyFinishReasonMiddleware.from_config(safety_config))
+
    return middlewares
@@ -20,6 +20,7 @@ from deerflow.config.memory_config import MemoryConfig, load_memory_config_from_
 from deerflow.config.model_config import ModelConfig
 from deerflow.config.run_events_config import RunEventsConfig
 from deerflow.config.runtime_paths import existing_project_file
+from deerflow.config.safety_finish_reason_config import SafetyFinishReasonConfig
 from deerflow.config.sandbox_config import SandboxConfig
 from deerflow.config.skill_evolution_config import SkillEvolutionConfig
 from deerflow.config.skills_config import SkillsConfig
@@ -102,6 +103,7 @@ class AppConfig(BaseModel):
    guardrails: GuardrailsConfig = Field(default_factory=GuardrailsConfig, description="Guardrail middleware configuration")
    circuit_breaker: CircuitBreakerConfig = Field(default_factory=CircuitBreakerConfig, description="LLM circuit breaker configuration")
    loop_detection: LoopDetectionConfig = Field(default_factory=LoopDetectionConfig, description="Loop detection middleware configuration")
+    safety_finish_reason: SafetyFinishReasonConfig = Field(default_factory=SafetyFinishReasonConfig, description="Provider safety-filter finish_reason interception middleware configuration")
    model_config = ConfigDict(extra="allow")
    database: DatabaseConfig = Field(default_factory=DatabaseConfig, description="Unified database backend configuration")
    run_events: RunEventsConfig = Field(default_factory=RunEventsConfig, description="Run event storage configuration")
@@ -0,0 +1,47 @@
+"""Configuration for SafetyFinishReasonMiddleware.
+
+Mirrors the shape of GuardrailsConfig: detectors are loaded by class path
+through ``deerflow.reflection.resolve_variable`` (same loader the
+``guardrails.provider`` config uses) so users can drop in custom provider
+detectors without modifying core code.
+"""
+
+from __future__ import annotations
+
+from pydantic import BaseModel, Field
+
+
+class SafetyDetectorConfig(BaseModel):
+    """One detector entry under ``safety_finish_reason.detectors``."""
+
+    use: str = Field(
+        description=("Class path of a SafetyTerminationDetector implementation (e.g. 'deerflow.agents.middlewares.safety_termination_detectors:OpenAICompatibleContentFilterDetector')."),
+    )
+    config: dict = Field(
+        default_factory=dict,
+        description="Constructor kwargs passed to the detector class.",
+    )
+
+
+class SafetyFinishReasonConfig(BaseModel):
+    """Configuration for the SafetyFinishReasonMiddleware.
+
+    The middleware intercepts AIMessages where the provider signaled a
+    safety-related termination (e.g. OpenAI ``finish_reason='content_filter'``)
+    while still returning tool calls, and suppresses those tool calls so the
+    half-truncated arguments never execute.
+    """
+
+    enabled: bool = Field(
+        default=True,
+        description="Master switch for the SafetyFinishReasonMiddleware.",
+    )
+    detectors: list[SafetyDetectorConfig] | None = Field(
+        default=None,
+        description=(
+            "Custom detector list. Leave unset (None) to use the built-in "
+            "set covering OpenAI-compatible content_filter, Anthropic "
+            "refusal, and Gemini SAFETY/BLOCKLIST/PROHIBITED_CONTENT/SPII/"
+            "RECITATION. Provide a non-null list to fully override."
+        ),
+    )
@@ -219,6 +219,12 @@ async def run_agent(
        # manually here because we drive the graph through ``agent.astream(config=...)``
        # without passing the official ``context=`` parameter.
        runtime_ctx = _build_runtime_context(thread_id, run_id, config.get("context"), ctx.app_config)
+        # Expose the run-scoped journal under a sentinel key so middleware can
+        # write audit events (e.g. SafetyFinishReasonMiddleware recording
+        # suppressed tool calls). Double-underscore prefix marks it as a
+        # runtime-internal channel; user code must not depend on the key name.
+        if journal is not None:
+            runtime_ctx["__run_journal"] = journal
        _install_runtime_context(config, runtime_ctx)
        runtime = Runtime(context=cast(Any, runtime_ctx), store=store)
        config.setdefault("configurable", {})["__pregel_runtime"] = runtime