fix(skills): apply allowed-tools only to active skills (#4098)

* fix(skills): scope allowed-tools to active skills

* fix(skills): tolerate stale active skill paths

* chore: retrigger CI

* fix(skills): document policy activation limits

* perf(skills): reuse per-step tool policy decisions

* fix(skills): harden runtime tool policy contracts

* fix(skills): redact cached policy decisions

* fix(skills): make slash tool policy authoritative

* fix(skills): preserve policy-safe discovery tools

* test(skills): cover explicit task delegation policy
This commit is contained in:
Huixin615
2026-07-16 14:12:02 +08:00
committed by GitHub
parent e7d7da9e7c
commit 65afc9b1d2
25 changed files with 2006 additions and 124 deletions
+14
View File
@@ -9,6 +9,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### ⚠ Breaking changes
- **skills:** A directory containing `SKILL.md` is now a runtime package
boundary. Nested `SKILL.md` files inside that package are supporting data and
are no longer registered as independent skills; unusual custom layouts must
move independently loadable skills under a namespace directory without its
own `SKILL.md`. ([#4098])
- **memory:** The memory system is now pluggable (`memory.manager_class` selects
a backend; default `deermem` is self-contained). DeerMem-private settings moved
from the top level of `memory:` into `memory.backend_config`, and the
@@ -40,6 +45,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Changed
- **skills:** An active restrictive skill must explicitly list `task` in
`allowed-tools` to delegate to a subagent. Read-only discovery infrastructure
(`tool_search` and `describe_skill`) remains available, but cannot grant schema
visibility or execution for a denied business tool. ([#4098])
- **memory:** Pre-abstraction top-level `memory.*` DeerMem fields
(`storage_path`, `max_facts`, `debounce_seconds`, `model_name`,
`token_counting`, `staleness_*`, `consolidation_*`, ...) are **auto-migrated
@@ -55,6 +64,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
- **skills:** Apply `allowed-tools` only to slash-activated or actually loaded
lead-agent skills, preventing passive enabled skills and evaluation fixtures
from removing `task`, web, and file tools from every run. ([#4095], [#4098])
- **models:** Honor `api_base` on every `BaseChatOpenAI` subclass (`VllmChatModel`,
`MindIEChatModel`, `PatchedChatMiMo`, `PatchedChatStepFun`, `PatchedChatMiniMax`),
not just `ChatOpenAI` / `PatchedChatOpenAI`. Those five previously dropped the
@@ -575,4 +587,6 @@ with **180 merged pull requests** since the first 2.0 milestone tag.
[#3654]: https://github.com/bytedance/deer-flow/pull/3654
[#3657]: https://github.com/bytedance/deer-flow/pull/3657
[#3658]: https://github.com/bytedance/deer-flow/pull/3658
[#4095]: https://github.com/bytedance/deer-flow/issues/4095
[#4098]: https://github.com/bytedance/deer-flow/pull/4098
[#4146]: https://github.com/bytedance/deer-flow/pull/4146
+4
View File
@@ -644,8 +644,12 @@ A standard Agent Skill is a structured capability module — a Markdown file tha
Skills are loaded progressively — only when the task needs them, not all at once. This keeps the context window lean and makes DeerFlow work well even with token-sensitive models.
A skill directory is a package boundary: once DeerFlow finds its `SKILL.md`, nested `SKILL.md` files under that package (for example evaluation fixtures) remain supporting data and are not registered as runtime skills. Namespace directories without their own `SKILL.md` can still group nested skills.
Users can explicitly activate an enabled skill for a single turn by starting the request with `/skill-name`, for example `/data-analysis analyze uploads/foo.csv`. DeerFlow loads that skill's `SKILL.md` as hidden current-turn context while leaving the base prompt limited to skill metadata. Slash activation respects disabled skills, custom-agent skill whitelists, and existing channel commands such as `/new` and `/help`.
An enabled skill's `allowed-tools` policy applies only after that skill is explicitly slash-activated or captured in the thread's active skill context after a `read_file` load. Merely enabling, advertising, or listing a skill in a custom agent's `skills` allowlist does not reduce the lead agent's normal toolset. During a slash-activated run, that explicit skill's policy is authoritative: reading another `SKILL.md` may provide instructions but cannot widen the slash skill's tools. Without slash activation, policies from skills actually loaded into active context retain their union semantics. Once active, the policy filters both model-visible tool schemas and tool execution. Framework discovery tools (`tool_search` and `describe_skill`) remain available so an allowed deferred tool or installed skill can still be discovered, but discovery and promotion never grant permission to execute a business tool omitted from `allowed-tools`. `task` is not framework-exempt; a restrictive skill must list it explicitly to delegate to a subagent. Per-step policy decisions are internal runtime context and are removed from observable or persisted context copies. Registry failures and an active set with no remaining valid skill fail closed to framework-safe tools; individual stale paths are ignored only when another valid active skill remains. This is best-effort behavioral scoping, not a hard security boundary: loading skill instructions through another tool is not captured, and active-skill entries can be evicted from bounded context.
When you install `.skill` archives through the Gateway, DeerFlow accepts standard optional frontmatter metadata such as `version`, `author`, and `compatibility` instead of rejecting otherwise valid external skills.
Skill installs and agent-managed skill edits run through **SkillScan**, a native deterministic safety scanner before the LLM-based skill scanner. Phase 1 runs offline with no Semgrep/OpenGrep dependency, blocks high-confidence `CRITICAL` findings such as private keys or shell execution, and passes warning findings to the LLM scanner for contextual review. Set `skill_scan.enabled: false` in `config.yaml` to disable only the deterministic analyzers; safe archive extraction and the LLM scanner still run.
+20 -18
View File
@@ -241,23 +241,24 @@ Lead-agent middlewares are assembled in strict order across three functions: the
14. **DynamicContextMiddleware** - Injects the current date (and optionally memory) as a `<system-reminder>` into the first HumanMessage, keeping the base system prompt fully static for prefix-cache reuse
15. **SkillActivationMiddleware** - Detects strict `/skill-name task` syntax on the latest real user message, resolves only enabled and runtime-allowed skills, injects the `SKILL.md` body as hidden current-turn context, and records a `middleware:skill_activation` audit event
16. **DurableContextMiddleware** - Captures `task` delegations into `ThreadState.delegations` (including in-progress dispatches and terminal result summaries) and loaded skill-file references (name/path/description, parsed in-memory - not the body) into `ThreadState.skill_context` before summarization can compact the paired tool-call/result messages, then projects durable context into each model request. Static authority rules are injected as a `SystemMessage`; untrusted field values (`summary_text`, delegation results, skill descriptions) are injected separately as a hidden `HumanMessage` data block so compressed history, delegated work, and which skills are active stay visible without being stored as `messages` or promoted to system-role instructions. `build_subagent_runtime_middlewares` also attaches this middleware immediately before subagent summarization so a compacted `summary_text` is projected ahead of a preserved assistant/tool tail instead of leaving strict providers with an assistant-first request.
17. **SummarizationMiddleware** - *(optional, if enabled)* Context reduction when approaching token limits
18. **TodoListMiddleware** - *(optional, if `is_plan_mode`)* Task tracking with the `write_todos` tool
19. **TokenUsageMiddleware** - *(optional, if `token_usage.enabled`)* Records token usage metrics; subagent usage is merged back into the dispatching AIMessage by message position
20. **TitleMiddleware** - Auto-generates the thread title after the first complete exchange and normalizes structured message content before prompting the title model. If a first-turn run is interrupted before this middleware can write a title, `runtime/runs/worker.py` keeps the run in a finalizing state, persists a local fallback title from the latest checkpoint or original run input, and then syncs it to `threads_meta.display_name`. Replacement runs admitted by `multitask_strategy="interrupt"` / `"rollback"` wait for older same-thread finalization before entering the graph; the interrupted run only skips the fallback title write once a later run has started and may have advanced the checkpoint.
21. **MemoryMiddleware** - Queues conversations for async memory update (filters to user + final AI responses)
22. **ViewImageMiddleware** - *(optional, if the model supports vision)* Injects base64 image data before the LLM call
23. **McpRoutingMiddleware** - *(optional, if `tool_search.enabled` and PR1 MCP routing metadata produce a routing index)* Auto-promotes matching deferred MCP tool schemas before the model call by writing a minimal `promoted` state update. It matches only the latest real `HumanMessage`, uses the global `tool_search.auto_promote_top_k` limit (default 3, clamped to 1..5), never executes tools, and must be installed before `DeferredToolFilterMiddleware`
24. **DeferredToolFilterMiddleware** - *(optional, if `tool_search.enabled`)* Hides deferred (MCP) tool schemas from the bound model until `tool_search` or `McpRoutingMiddleware` promotes them (reads per-thread promotions from `ThreadState.promoted`, hash-scoped)
25. **SystemMessageCoalescingMiddleware** - Merges every SystemMessage into a single leading SystemMessage per request; provider-agnostic fix for strict backends (vLLM/SGLang/Qwen/Anthropic) that reject non-leading system messages. Touches the per-request payload only (checkpoint state unchanged); on midnight crossings only the latest `dynamic_context_reminder` SystemMessage survives
26. **SubagentLimitMiddleware** - *(optional, if `subagent_enabled`)* Truncates excess `task` tool calls to enforce both the per-response concurrency limit (`max_concurrent_subagents`, clamped to 2-4) and the per-run total delegation cap (`max_total_subagents` runtime override or `subagents.max_total_per_run`, default 6, clamped to 1-50). The total cap counts current-run entries in the durable delegation ledger (entries are tagged with `run_id` when captured), so repeated planning checkpoints in one run cannot keep launching legal-sized batches indefinitely, while later user turns in the same thread get a fresh run budget. If the cap is exhausted, the middleware strips remaining `task` calls, forces `finish_reason="stop"`, and appends a visible limit note so the run can synthesize existing results instead of ending with an empty tool-call response.
27. **LoopDetectionMiddleware** - *(optional, if `loop_detection.enabled`)* Detects repeated tool-call loops; hard-stop clears both structured `tool_calls` and raw provider tool-call metadata before forcing a final text answer; stamps `loop_capped` via `consume_stop_reason` (#3875 Phase 2), symmetric to `TokenBudgetMiddleware`
28. **TokenBudgetMiddleware** - *(optional, if `token_budget.enabled`)* Enforces per-run token limits
29. **Custom middlewares** - *(optional)* Any `custom_middlewares` passed to `build_middlewares` are injected here, before the terminal-response/safety/clarification tail
30. **TerminalResponseMiddleware** - When a provider returns an empty terminal `AIMessage` after tool execution, injects a hidden recovery prompt and retries the model once; a second empty response is replaced in checkpoint state by a visible error fallback marked for the run worker, so the run finishes as an error instead of a silent success
31. **SafetyFinishReasonMiddleware** - *(optional, if `safety_finish_reason.enabled`)* Suppresses tool execution when the provider safety-terminated the response (e.g. `finish_reason=content_filter`); registered after terminal-response/custom middlewares so LangChain's reverse-order `after_model` dispatch runs it first
32. **ClarificationMiddleware** - Intercepts `ask_clarification` tool calls, writes a readable `ToolMessage.content` fallback plus structured `ToolMessage.artifact.human_input` request payload, and interrupts via `Command(goto=END)` (must be last). Because this middleware can short-circuit tool execution before LangChain emits `on_tool_end`, `RunJournal` performs a root-run final reconciliation for allowlisted clarification `ToolMessage`s whose `tool_call_id` was produced by the current run, so human-input request cards remain recoverable from `run_events` after checkpoint compaction. Human Input Card replies are submitted as `hide_from_ui` `HumanMessage`s with `additional_kwargs.human_input_response`; `RunJournal` persists only allowlisted hidden response sources (currently `ask_clarification`) as `llm.human.input`, which preserves answered-card state after compaction without exposing generic internal hidden context.
16. **SkillToolPolicyMiddleware** - Applies `allowed-tools` only after real activation; passive enabled skills and a custom agent's configured skill allowlist do not clamp the lead toolset. A run-scoped slash activation is authoritative and suppresses `skill_context` as a policy source, so reading another skill cannot widen the explicit skill's tools; without slash activation, skills captured after configured `read_file` loads retain the existing union semantics. The middleware filters model-visible schemas and blocks unauthorized execution, resolving canonical paths against the live enabled/agent-allowed registry on every model call, then stores a versioned, JSON-safe, middleware-token-bound decision signed by policy source plus active paths in run context for the resulting tool calls to reuse. The next model call always refreshes it, and malformed, foreign, stale, or unmatched decisions fall back to live resolution. `tool_search` and `describe_skill` remain framework-safe discovery tools under a restrictive policy; they may reveal or promote metadata, but a deferred business tool must still be declared by the active policy before its schema or execution can survive the policy middleware. The decision's owner token is authorization-sensitive, so its reserved context key is owned by `runtime.secret_context` and included in `REDACTED_CONTEXT_KEYS` for observable and persisted context copies. Registry load failures and a non-empty active set with no authorized skill fail closed to framework-safe tools; an individual stale path is skipped only when at least one valid active skill remains. This is best-effort behavioral scoping rather than a hard security boundary: alternate loads such as `bash cat` are not captured, and bounded autonomous `skill_context` can evict old entries. `task` is not framework-exempt, so a restricted skill cannot delegate around its policy. The middleware must remain immediately after `SkillActivationMiddleware` (which publishes the slash source through `runtime.secret_context`'s public path helpers authenticated by a required token shared only within the assembled middleware chain) and immediately before `DurableContextMiddleware`; assembly and compiled-graph tests pin ordering, token sharing, schema filtering, and execution blocking.
17. **DurableContextMiddleware** - Captures `task` delegations into `ThreadState.delegations` (including in-progress dispatches and terminal result summaries) and loaded skill-file references (name/path/description, parsed in-memory - not the body) into `ThreadState.skill_context` before summarization can compact the paired tool-call/result messages, then projects durable context into each model request. Static authority rules are injected as a `SystemMessage`; untrusted field values (`summary_text`, delegation results, skill descriptions) are injected separately as a hidden `HumanMessage` data block so compressed history, delegated work, and which skills are active stay visible without being stored as `messages` or promoted to system-role instructions. `build_subagent_runtime_middlewares` also attaches this middleware immediately before subagent summarization so a compacted `summary_text` is projected ahead of a preserved assistant/tool tail instead of leaving strict providers with an assistant-first request.
18. **SummarizationMiddleware** - *(optional, if enabled)* Context reduction when approaching token limits
19. **TodoListMiddleware** - *(optional, if `is_plan_mode`)* Task tracking with the `write_todos` tool
20. **TokenUsageMiddleware** - *(optional, if `token_usage.enabled`)* Records token usage metrics; subagent usage is merged back into the dispatching AIMessage by message position
21. **TitleMiddleware** - Auto-generates the thread title after the first complete exchange and normalizes structured message content before prompting the title model. If a first-turn run is interrupted before this middleware can write a title, `runtime/runs/worker.py` keeps the run in a finalizing state, persists a local fallback title from the latest checkpoint or original run input, and then syncs it to `threads_meta.display_name`. Replacement runs admitted by `multitask_strategy="interrupt"` / `"rollback"` wait for older same-thread finalization before entering the graph; the interrupted run only skips the fallback title write once a later run has started and may have advanced the checkpoint.
22. **MemoryMiddleware** - Queues conversations for async memory update (filters to user + final AI responses)
23. **ViewImageMiddleware** - *(optional, if the model supports vision)* Injects base64 image data before the LLM call
24. **McpRoutingMiddleware** - *(optional, if `tool_search.enabled` and PR1 MCP routing metadata produce a routing index)* Auto-promotes matching deferred MCP tool schemas before the model call by writing a minimal `promoted` state update. It matches only the latest real `HumanMessage`, uses the global `tool_search.auto_promote_top_k` limit (default 3, clamped to 1..5), never executes tools, and must be installed before `DeferredToolFilterMiddleware`
25. **DeferredToolFilterMiddleware** - *(optional, if `tool_search.enabled`)* Hides deferred (MCP) tool schemas from the bound model until `tool_search` or `McpRoutingMiddleware` promotes them (reads per-thread promotions from `ThreadState.promoted`, hash-scoped)
26. **SystemMessageCoalescingMiddleware** - Merges every SystemMessage into a single leading SystemMessage per request; provider-agnostic fix for strict backends (vLLM/SGLang/Qwen/Anthropic) that reject non-leading system messages. Touches the per-request payload only (checkpoint state unchanged); on midnight crossings only the latest `dynamic_context_reminder` SystemMessage survives
27. **SubagentLimitMiddleware** - *(optional, if `subagent_enabled`)* Truncates excess `task` tool calls to enforce both the per-response concurrency limit (`max_concurrent_subagents`, clamped to 2-4) and the per-run total delegation cap (`max_total_subagents` runtime override or `subagents.max_total_per_run`, default 6, clamped to 1-50). The total cap counts current-run entries in the durable delegation ledger (entries are tagged with `run_id` when captured), so repeated planning checkpoints in one run cannot keep launching legal-sized batches indefinitely, while later user turns in the same thread get a fresh run budget. If the cap is exhausted, the middleware strips remaining `task` calls, forces `finish_reason="stop"`, and appends a visible limit note so the run can synthesize existing results instead of ending with an empty tool-call response.
28. **LoopDetectionMiddleware** - *(optional, if `loop_detection.enabled`)* Detects repeated tool-call loops; hard-stop clears both structured `tool_calls` and raw provider tool-call metadata before forcing a final text answer; stamps `loop_capped` via `consume_stop_reason` (#3875 Phase 2), symmetric to `TokenBudgetMiddleware`
29. **TokenBudgetMiddleware** - *(optional, if `token_budget.enabled`)* Enforces per-run token limits
30. **Custom middlewares** - *(optional)* Any `custom_middlewares` passed to `build_middlewares` are injected here, before the terminal-response/safety/clarification tail
31. **TerminalResponseMiddleware** - When a provider returns an empty terminal `AIMessage` after tool execution, injects a hidden recovery prompt and retries the model once; a second empty response is replaced in checkpoint state by a visible error fallback marked for the run worker, so the run finishes as an error instead of a silent success
32. **SafetyFinishReasonMiddleware** - *(optional, if `safety_finish_reason.enabled`)* Suppresses tool execution when the provider safety-terminated the response (e.g. `finish_reason=content_filter`); registered after terminal-response/custom middlewares so LangChain's reverse-order `after_model` dispatch runs it first
33. **ClarificationMiddleware** - Intercepts `ask_clarification` tool calls, writes a readable `ToolMessage.content` fallback plus structured `ToolMessage.artifact.human_input` request payload, and interrupts via `Command(goto=END)` (must be last). Because this middleware can short-circuit tool execution before LangChain emits `on_tool_end`, `RunJournal` performs a root-run final reconciliation for allowlisted clarification `ToolMessage`s whose `tool_call_id` was produced by the current run, so human-input request cards remain recoverable from `run_events` after checkpoint compaction. Human Input Card replies are submitted as `hide_from_ui` `HumanMessage`s with `additional_kwargs.human_input_response`; `RunJournal` persists only allowlisted hidden response sources (currently `ask_clarification`) as `llm.human.input`, which preserves answered-card state after compaction without exposing generic internal hidden context.
### Configuration System
@@ -449,7 +450,8 @@ Additional providers also live here (`boxlite`, `brave`, `browserless`, `crawl4a
- **Location**: `deer-flow/skills/{public,custom}/`
- **Format**: Directory with `SKILL.md` (YAML frontmatter: name, description, license, allowed-tools, required-secrets)
- **Loading**: `load_skills()` recursively scans `skills/{public,custom}` for `SKILL.md`, parses metadata, and reads enabled state from extensions_config.json
- **Loading**: `load_skills()` recursively scans namespace directories under `skills/{public,custom}`, but stops descending once it finds a `SKILL.md`; that directory is a package boundary, so no nested `SKILL.md` is registered as a runtime skill. SkillScan has a deliberately narrower packaging rule: known eval fixtures are permitted as support data, while other nested `SKILL.md` files are reported as package defects. It parses runtime metadata and reads enabled state from extensions_config.json.
- **Tool policy**: Lead-agent `allowed-tools` declarations apply dynamically only to slash-activated skills and skills captured in `ThreadState.skill_context` through configured `read_file` loads; passive enabled skills and custom-agent skill allowlists remain discoverable without clamping the global toolset. Slash policy is dominant for its run, preventing subsequently read skills from widening explicit authority; autonomous captured skills use the existing union only when no slash source exists. `tool_search` and `describe_skill` stay available as framework discovery infrastructure, while every discovered or promoted business tool still requires active-policy permission for schema visibility and execution; `task` likewise requires an explicit declaration. Each active model call intentionally reloads the full live registry so enable/disable changes, frontmatter edits, and custom/public name-shadow winners take effect without a stale TTL or unsafe direct-path cache; all tool calls produced by that model step reuse the resulting source-and-path-signed decision. Registry failures and all-invalid active sets fail closed, while stale individual paths are skipped when another valid skill remains. This is best-effort behavioral scoping, not a hard security boundary: alternate loading paths are not captured and bounded autonomous context may evict entries. Subagents still filter statically because their configured skills are all loaded into the session at startup.
- **Injection (legacy / default)**: Enabled skills are listed in the agent system prompt with full metadata and container paths (`<available_skills>` block). Controlled by `skills.deferred_discovery: false` (default).
- **Deferred discovery** (`skills.deferred_discovery: true`): Skills are listed by name only in a compact `<skill_index>` block, keeping the system prompt prefix-cache friendly. The agent calls the `describe_skill` tool at runtime to fetch full metadata for skills it wants to use, then loads the SKILL.md via `read_file`. Two new modules support this path:
- `skills/catalog.py``SkillCatalog` (immutable, searchable; query forms: `select:a,b`, `+prefix`, free-text regex); `select:` returns all requested skills without a result cap; other modes cap at `MAX_RESULTS=5`.
+2
View File
@@ -564,6 +564,8 @@ Custom agents can restrict which skills they load by defining a `skills` field i
- **`[]` (empty list)**: Disables all skills for this specific agent.
- **`["skill-name"]`**: Loads only the explicitly specified skills.
This field is a discovery and activation allowlist; it does not activate every listed skill's `allowed-tools` policy when the agent is constructed. Use `tool_groups` to define the agent's baseline tools. A listed skill's policy applies only after slash activation or an actual `SKILL.md` load.
### Title Generation
Automatic conversation title generation:
@@ -21,6 +21,7 @@ middleware, and the async path inside ``TitleMiddleware``. Any new in-graph
from __future__ import annotations
import logging
import secrets
from langchain.agents import create_agent
from langchain.agents.middleware import AgentMiddleware
@@ -45,7 +46,6 @@ from deerflow.config.app_config import AppConfig, get_app_config
from deerflow.config.memory_config import should_use_memory_tools
from deerflow.config.subagents_config import DEFAULT_MAX_TOTAL_SUBAGENTS_PER_RUN
from deerflow.models import create_chat_model
from deerflow.skills.tool_policy import ALWAYS_AVAILABLE_BUILTIN_TOOL_NAMES, filter_tools_by_skill_allowed_tools
from deerflow.skills.types import Skill
from deerflow.tracing import build_tracing_callbacks
@@ -284,7 +284,28 @@ def build_middlewares(
# explicit user activation priority over model-side relevance guessing.
from deerflow.agents.middlewares.skill_activation_middleware import SkillActivationMiddleware
middlewares.append(SkillActivationMiddleware(available_skills=available_skills, app_config=resolved_app_config, user_id=user_id))
slash_source_owner_token = secrets.token_urlsafe(24)
middlewares.append(
SkillActivationMiddleware(
available_skills=available_skills,
app_config=resolved_app_config,
user_id=user_id,
slash_source_owner_token=slash_source_owner_token,
)
)
# Enabled skills are only discoverable metadata. Apply allowed-tools at
# runtime after explicit slash activation or an actual skill-file load.
from deerflow.agents.middlewares.skill_tool_policy_middleware import SkillToolPolicyMiddleware
middlewares.append(
SkillToolPolicyMiddleware(
available_skills=available_skills,
app_config=resolved_app_config,
user_id=user_id,
slash_source_owner_token=slash_source_owner_token,
)
)
# Capture completed task delegations and loaded skill files before
# summarization can compact them, then inject durable context channels
@@ -404,13 +425,13 @@ def _available_skill_names(agent_config, is_bootstrap: bool) -> set[str] | None:
return None
def _load_enabled_skills_for_tool_policy(available_skills: set[str] | None, *, app_config: AppConfig, user_id: str | None = None) -> list[Skill]:
def _load_enabled_available_skills(available_skills: set[str] | None, *, app_config: AppConfig, user_id: str | None = None) -> list[Skill]:
try:
from deerflow.agents.lead_agent.prompt import get_enabled_skills_for_config
skills = get_enabled_skills_for_config(app_config, user_id=user_id)
except Exception:
logger.exception("Failed to load skills for allowed-tools policy")
logger.exception("Failed to load enabled skills")
raise
if available_skills is None:
@@ -511,7 +532,7 @@ def _make_lead_agent(config: RunnableConfig, *, app_config: AppConfig):
existing = list(existing)
config["callbacks"] = [*existing, *tracing_callbacks]
skills_for_tool_policy = _load_enabled_skills_for_tool_policy(available_skills, app_config=resolved_app_config, user_id=resolved_user_id)
enabled_skills = _load_enabled_available_skills(available_skills, app_config=resolved_app_config, user_id=resolved_user_id)
# Build skill search setup (deferred skill discovery).
# Controlled by skills.deferred_discovery — independent from tool_search.enabled.
@@ -524,17 +545,17 @@ def _make_lead_agent(config: RunnableConfig, *, app_config: AppConfig):
# Special bootstrap agent with minimal prompt for initial custom agent creation flow
# Keep the bootstrap skill set intentionally narrow so agent creation
# remains deterministic before the custom agent's own config exists.
bootstrap_skills = [s for s in skills_for_tool_policy if s.name in _BOOTSTRAP_SKILL_NAMES]
bootstrap_skills = [s for s in enabled_skills if s.name in _BOOTSTRAP_SKILL_NAMES]
skill_setup = build_skill_search_setup(
bootstrap_skills,
enabled=skill_search_enabled,
container_base_path=container_base_path,
)
raw_tools = get_available_tools(model_name=model_name, subagent_enabled=subagent_enabled, app_config=resolved_app_config) + [setup_agent]
filtered = filter_tools_by_skill_allowed_tools(raw_tools, skills_for_tool_policy, always_allowed_tool_names=ALWAYS_AVAILABLE_BUILTIN_TOOL_NAMES)
configured_tools = raw_tools
if non_interactive:
filtered = [tool for tool in filtered if tool.name not in _NON_INTERACTIVE_DISABLED_TOOL_NAMES]
final_tools, setup = assemble_deferred_tools(filtered, enabled=resolved_app_config.tool_search.enabled)
configured_tools = [tool for tool in configured_tools if tool.name not in _NON_INTERACTIVE_DISABLED_TOOL_NAMES]
final_tools, setup = assemble_deferred_tools(configured_tools, enabled=resolved_app_config.tool_search.enabled)
mcp_routing_middleware = build_mcp_routing_middleware(
final_tools,
setup,
@@ -571,10 +592,11 @@ def _make_lead_agent(config: RunnableConfig, *, app_config: AppConfig):
# Custom agents can update their own SOUL.md / config via update_agent.
# The default agent (no agent_name) does not see this tool.
# Build skill search setup from policy-filtered skills (same list used for
# tool-policy filtering), so describe_skill only exposes allowed skills.
# Build skill search setup from the agent-available skills. The same
# allowlist is enforced by the runtime policy resolver, so describe_skill
# cannot expose a skill this custom agent is not allowed to activate.
skill_setup = build_skill_search_setup(
skills_for_tool_policy,
enabled_skills,
enabled=skill_search_enabled,
container_base_path=container_base_path,
)
@@ -596,16 +618,16 @@ def _make_lead_agent(config: RunnableConfig, *, app_config: AppConfig):
extra_tools = [update_agent] if agent_name and not is_webhook_channel else []
# Default lead agent (unchanged behavior)
raw_tools = get_available_tools(model_name=model_name, groups=agent_config.tool_groups if agent_config else None, subagent_enabled=subagent_enabled, app_config=resolved_app_config)
filtered = filter_tools_by_skill_allowed_tools(raw_tools + extra_tools, skills_for_tool_policy, always_allowed_tool_names=ALWAYS_AVAILABLE_BUILTIN_TOOL_NAMES)
configured_tools = raw_tools + extra_tools
if non_interactive:
filtered = [tool for tool in filtered if tool.name not in _NON_INTERACTIVE_DISABLED_TOOL_NAMES]
final_tools, setup = assemble_deferred_tools(filtered, enabled=resolved_app_config.tool_search.enabled)
configured_tools = [tool for tool in configured_tools if tool.name not in _NON_INTERACTIVE_DISABLED_TOOL_NAMES]
final_tools, setup = assemble_deferred_tools(configured_tools, enabled=resolved_app_config.tool_search.enabled)
mcp_routing_middleware = build_mcp_routing_middleware(
final_tools,
setup,
top_k=resolved_app_config.tool_search.auto_promote_top_k,
)
mcp_routing_hints_section = get_mcp_routing_hints_prompt_section(filtered, deferred_names=setup.deferred_names)
mcp_routing_hints_section = get_mcp_routing_hints_prompt_section(configured_tools, deferred_names=setup.deferred_names)
if skill_setup.describe_skill_tool:
final_tools.append(skill_setup.describe_skill_tool)
if should_use_memory_tools(resolved_app_config.memory):
@@ -19,10 +19,11 @@ from langchain_core.messages import AIMessage, HumanMessage
from deerflow.runtime.secret_context import (
_SECRETS_BINDING_AUDIT_KEY,
_SLASH_SECRET_SOURCE_KEY,
_SLASH_SKILL_ACTIVATION_RUN_KEY,
ACTIVE_SECRETS_CONTEXT_KEY,
extract_request_secrets,
read_slash_skill_source_path,
write_slash_skill_source_path,
)
from deerflow.skills.slash import parse_slash_skill_reference, resolve_slash_skill
from deerflow.skills.storage import get_or_new_skill_storage, get_or_new_user_skill_storage
@@ -40,7 +41,7 @@ _SLASH_SKILL_ACTIVATION_TARGET_ID_KEY = "slash_skill_activation_target_id"
# _SECRETS_BINDING_AUDIT_KEY: last audited binding (skill and secret names only,
# never values) so unchanged bindings are not re-recorded each call.
# _SLASH_SECRET_SOURCE_KEY: latest slash activation as a secret source, holding
# The shared slash-source context contract holds the latest slash activation,
# ONLY the activated skill's canonical container path (never its declared
# secrets — those are read from the live registry on each call, #3938). The
# injection set is recomputed every model call, but a slash-activated skill must
@@ -92,11 +93,15 @@ class SkillActivationMiddleware(AgentMiddleware):
available_skills: set[str] | None = None,
app_config: AppConfig | None = None,
user_id: str | None = None,
slash_source_owner_token: str,
) -> None:
super().__init__()
if not isinstance(slash_source_owner_token, str) or not slash_source_owner_token:
raise ValueError("slash_source_owner_token must be a non-empty string")
self._available_skills = set(available_skills) if available_skills is not None else None
self._app_config = app_config
self._user_id = user_id
self._slash_source_owner_token = slash_source_owner_token
def _storage(self) -> SkillStorage:
if self._user_id is not None:
@@ -380,13 +385,16 @@ Follow this skill before choosing a general workflow. Load supporting resources
if not isinstance(context, dict):
return
# The slash source records only the canonical container path of the
# activated skill — never its declared secrets. Both sources resolve the
# live registry skill by path on read, so a caller-forged source (the
# context is caller-mergeable) can never inject secrets a real, enabled,
# allowlisted skill did not declare (#3938).
# The slash source records the canonical container path plus a
# middleware-chain-local owner token — never declared secrets. Both
# consumers authenticate the source and resolve the live registry skill
# by path, so caller-mergeable context cannot forge an activation.
if activation is not None:
context[_SLASH_SECRET_SOURCE_KEY] = {"path": activation.container_file_path}
write_slash_skill_source_path(
context,
activation.container_file_path,
owner_token=self._slash_source_owner_token,
)
request_secrets = extract_request_secrets(context)
sources: list[tuple[str, tuple[SecretRequirement, ...]]] = []
@@ -395,8 +403,7 @@ Follow this skill before choosing a general workflow. Load supporting resources
if registry is not None:
# Slash source: exempt from the ``secrets-autonomous`` opt-out
# (explicit ceremony), but still enabled + allowlist checked.
slash_source = context.get(_SLASH_SECRET_SOURCE_KEY)
slash_path = slash_source.get("path") if isinstance(slash_source, dict) else None
slash_path = read_slash_skill_source_path(context, owner_token=self._slash_source_owner_token)
slash_skill = self._resolve_registry_skill(registry, slash_path, require_autonomous=False)
if slash_skill is not None:
sources.append((slash_skill.name, tuple(slash_skill.required_secrets)))
@@ -0,0 +1,364 @@
"""Apply skill ``allowed-tools`` only to skills active in lead-agent context."""
from __future__ import annotations
import asyncio
import json
import logging
import posixpath
import secrets
from collections.abc import Awaitable, Callable, Mapping
from typing import TYPE_CHECKING, override
from langchain.agents import AgentState
from langchain.agents.middleware import AgentMiddleware
from langchain.agents.middleware.types import ModelCallResult, ModelRequest, ModelResponse
from langchain_core.messages import ToolMessage
from langgraph.prebuilt.tool_node import ToolCallRequest
from langgraph.types import Command
from deerflow.runtime.secret_context import SKILL_TOOL_POLICY_DECISION_CONTEXT_KEY, read_slash_skill_source_path
from deerflow.skills.storage import get_or_new_skill_storage, get_or_new_user_skill_storage
from deerflow.skills.tool_policy import ALWAYS_AVAILABLE_BUILTIN_TOOL_NAMES, allowed_tool_names_for_skills
from deerflow.skills.types import Skill
if TYPE_CHECKING:
from deerflow.config.app_config import AppConfig
from deerflow.skills.storage.skill_storage import SkillStorage
logger = logging.getLogger(__name__)
_POLICY_DECISION_VERSION = 2
_POLICY_SOURCE_PASSIVE = "passive"
_POLICY_SOURCE_SLASH = "slash"
_POLICY_SOURCE_SKILL_CONTEXT = "skill_context"
_POLICY_SOURCES = frozenset({_POLICY_SOURCE_PASSIVE, _POLICY_SOURCE_SLASH, _POLICY_SOURCE_SKILL_CONTEXT})
_MISSING_POLICY_DECISION = object()
_TOOL_SEARCH_NAME = "tool_search"
type _PolicySignature = tuple[str, tuple[str, ...]]
class SkillToolPolicyMiddleware(AgentMiddleware[AgentState]):
"""Restrict lead tools to declarations from slash/in-context skills.
Merely enabling a skill makes it discoverable; it does not activate its
authority policy. A skill becomes policy-active when the user slash-activates
it for the run or after the model loads it into ``skill_context``. Explicit
slash activation dominates for the rest of that run: passively reading a
second skill cannot widen the slash skill's authority.
"""
def __init__(
self,
*,
available_skills: set[str] | None = None,
app_config: AppConfig | None = None,
user_id: str | None = None,
slash_source_owner_token: str,
) -> None:
super().__init__()
if not isinstance(slash_source_owner_token, str) or not slash_source_owner_token:
raise ValueError("slash_source_owner_token must be a non-empty string")
self._available_skills = set(available_skills) if available_skills is not None else None
self._app_config = app_config
self._user_id = user_id
self._slash_source_owner_token = slash_source_owner_token
self._decision_owner_token = secrets.token_urlsafe(24)
def _storage(self) -> SkillStorage:
if self._user_id is not None:
return get_or_new_user_skill_storage(self._user_id, app_config=self._app_config)
if self._app_config is not None:
return get_or_new_skill_storage(app_config=self._app_config)
return get_or_new_skill_storage()
def _active_policy(self, request: ModelRequest | ToolCallRequest) -> _PolicySignature:
context = getattr(getattr(request, "runtime", None), "context", None)
slash_path = read_slash_skill_source_path(context, owner_token=self._slash_source_owner_token)
if slash_path is not None:
return _POLICY_SOURCE_SLASH, (slash_path,)
paths: list[str] = []
state = getattr(request, "state", None)
if state is None:
state = {}
if isinstance(state, Mapping):
entries = state.get("skill_context") or []
elif hasattr(state, "skill_context"):
entries = getattr(state, "skill_context") or []
else:
logger.warning("Unsupported agent state shape for skill tool policy: %s", type(state).__name__)
entries = []
if not isinstance(entries, (list, tuple)):
logger.warning("Invalid skill_context shape for skill tool policy: %s", type(entries).__name__)
entries = []
for entry in entries:
if isinstance(entry, dict) and isinstance(entry.get("path"), str):
paths.append(entry["path"])
if paths:
return _POLICY_SOURCE_SKILL_CONTEXT, tuple(paths)
return _POLICY_SOURCE_PASSIVE, ()
def _active_skills_for_paths(self, paths: tuple[str, ...]) -> tuple[list[Skill], bool]:
if not paths:
return [], False
try:
storage = self._storage()
skills = storage.load_skills(enabled_only=False)
container_root = storage.get_container_root()
except Exception:
logger.exception("Failed to load active skills for allowed-tools policy")
# A real active reference exists but cannot be authorized. Signal a
# policy failure so callers retain only framework-safe tools.
return [], True
registry = {posixpath.normpath(skill.get_container_file_path(container_root)): skill for skill in skills}
active: list[Skill] = []
seen: set[str] = set()
for path in paths:
skill = registry.get(posixpath.normpath(path))
if skill is None:
logger.warning("Active skill path could not be resolved for allowed-tools policy: %s", path)
continue
if not skill.enabled:
logger.warning("Active skill is disabled for allowed-tools policy: %s", path)
continue
if self._available_skills is not None and skill.name not in self._available_skills:
logger.warning("Active skill is outside the agent allowlist for allowed-tools policy: %s", path)
continue
if skill.name in seen:
continue
seen.add(skill.name)
active.append(skill)
if not active:
logger.warning("No active skill references could be authorized for allowed-tools policy; failing closed")
return [], True
return active, False
def _allowed_names_for_paths(self, paths: tuple[str, ...]) -> set[str] | None:
active_skills, policy_failed = self._active_skills_for_paths(paths)
if policy_failed:
return set(ALWAYS_AVAILABLE_BUILTIN_TOOL_NAMES)
allowed = allowed_tool_names_for_skills(active_skills)
if allowed is None:
return None
return allowed | set(ALWAYS_AVAILABLE_BUILTIN_TOOL_NAMES)
@staticmethod
def _runtime_context(request: ModelRequest | ToolCallRequest) -> dict | None:
context = getattr(getattr(request, "runtime", None), "context", None)
return context if isinstance(context, dict) else None
def _store_policy_decision(self, request: ModelRequest, policy: _PolicySignature, allowed: set[str] | None) -> None:
context = self._runtime_context(request)
if context is not None:
source, paths = policy
context[SKILL_TOOL_POLICY_DECISION_CONTEXT_KEY] = {
"version": _POLICY_DECISION_VERSION,
"owner_token": self._decision_owner_token,
"source": source,
"active_paths": list(paths),
"allowed_names": None if allowed is None else sorted(allowed),
}
def _read_policy_decision(self, context: dict | None, policy: _PolicySignature) -> set[str] | None | object:
if context is None:
return _MISSING_POLICY_DECISION
decision = context.get(SKILL_TOOL_POLICY_DECISION_CONTEXT_KEY)
if not isinstance(decision, dict):
return _MISSING_POLICY_DECISION
if type(decision.get("version")) is not int or decision["version"] != _POLICY_DECISION_VERSION:
return _MISSING_POLICY_DECISION
if not isinstance(decision.get("owner_token"), str) or decision["owner_token"] != self._decision_owner_token:
return _MISSING_POLICY_DECISION
source, paths = policy
stored_source = decision.get("source")
if not isinstance(stored_source, str) or stored_source not in _POLICY_SOURCES or stored_source != source:
return _MISSING_POLICY_DECISION
stored_paths = decision.get("active_paths")
if not isinstance(stored_paths, list) or not all(isinstance(path, str) for path in stored_paths) or tuple(stored_paths) != paths:
return _MISSING_POLICY_DECISION
allowed = decision.get("allowed_names")
if allowed is None:
return None
if not isinstance(allowed, list) or not all(isinstance(name, str) for name in allowed):
return _MISSING_POLICY_DECISION
return set(allowed)
def _allowed_names(
self,
request: ModelRequest | ToolCallRequest,
*,
policy: _PolicySignature | None = None,
) -> set[str] | None:
resolved_policy = self._active_policy(request) if policy is None else policy
_, paths = resolved_policy
context = self._runtime_context(request)
decision = self._read_policy_decision(context, resolved_policy)
if decision is not _MISSING_POLICY_DECISION:
return decision
return self._allowed_names_for_paths(paths)
def _filter_model_request(
self,
request: ModelRequest,
*,
policy: _PolicySignature | None = None,
refresh_decision: bool = False,
) -> ModelRequest:
resolved_policy = self._active_policy(request) if policy is None else policy
_, paths = resolved_policy
allowed = self._allowed_names_for_paths(paths) if refresh_decision else self._allowed_names(request, policy=resolved_policy)
if refresh_decision:
self._store_policy_decision(request, resolved_policy, allowed)
if allowed is None:
return request
tools = [tool for tool in request.tools if getattr(tool, "name", None) in allowed]
if len(tools) < len(request.tools):
logger.debug("Skill policy filtered %d lead tool schema(s)", len(request.tools) - len(tools))
return request.override(tools=tools)
def _blocked_tool_message(
self,
request: ToolCallRequest,
*,
allowed: set[str] | None,
) -> ToolMessage | None:
name = str(request.tool_call.get("name") or "")
if allowed is None or not name or name in allowed:
return None
return ToolMessage(
content=f"Error: Tool '{name}' is not allowed by the active skill policy.",
tool_call_id=str(request.tool_call.get("id") or "missing_tool_call_id"),
name=name,
status="error",
)
@staticmethod
def _tool_search_policy_error(request: ToolCallRequest) -> ToolMessage:
return ToolMessage(
content="Error: tool_search returned a result that could not be validated against the active skill policy.",
tool_call_id=str(request.tool_call.get("id") or "missing_tool_call_id"),
name=_TOOL_SEARCH_NAME,
status="error",
)
def _filter_tool_search_result(
self,
request: ToolCallRequest,
result: ToolMessage | Command,
*,
allowed: set[str] | None,
) -> ToolMessage | Command:
"""Remove denied schemas and promotions from tool_search output.
Keeping tool_search available is safe only if it cannot return a full
schema for a tool removed by the active policy. Deferred filtering still
controls when an allowed schema becomes model-visible; this method keeps
the discovery result itself within the same authorization boundary.
"""
name = str(request.tool_call.get("name") or "")
if name != _TOOL_SEARCH_NAME or allowed is None:
return result
if not isinstance(result, Command) or not isinstance(result.update, dict):
logger.warning("Active-policy tool_search returned an unsupported result shape")
return self._tool_search_policy_error(request)
promoted = result.update.get("promoted")
messages = result.update.get("messages")
if not isinstance(promoted, dict) or not isinstance(messages, list) or len(messages) != 1:
logger.warning("Active-policy tool_search command omitted promoted/messages updates")
return self._tool_search_policy_error(request)
raw_names = promoted.get("names")
if not isinstance(raw_names, list) or not all(isinstance(item, str) for item in raw_names):
logger.warning("Active-policy tool_search returned malformed promoted names")
return self._tool_search_policy_error(request)
permitted_names = [item for item in raw_names if item in allowed]
sanitized_messages: list[ToolMessage] = []
for message in messages:
if not isinstance(message, ToolMessage) or message.name != _TOOL_SEARCH_NAME:
logger.warning("Active-policy tool_search returned an unexpected message shape")
return self._tool_search_policy_error(request)
content = message.content
if raw_names:
try:
schemas = json.loads(content) if isinstance(content, str) else None
except json.JSONDecodeError:
schemas = None
if not isinstance(schemas, list):
logger.warning("Active-policy tool_search returned schemas that could not be filtered")
return self._tool_search_policy_error(request)
filtered_schemas = [schema for schema in schemas if isinstance(schema, dict) and (schema.get("name") in permitted_names or (isinstance(schema.get("function"), dict) and schema["function"].get("name") in permitted_names))]
content = json.dumps(filtered_schemas, indent=2, ensure_ascii=False) if filtered_schemas else "No tools found matching the active skill policy."
sanitized_messages.append(message.model_copy(update={"content": content}))
sanitized_update = dict(result.update)
sanitized_update["promoted"] = {**promoted, "names": permitted_names}
sanitized_update["messages"] = sanitized_messages
return Command(
graph=result.graph,
update=sanitized_update,
resume=result.resume,
goto=result.goto,
)
@override
def wrap_model_call(
self,
request: ModelRequest,
handler: Callable[[ModelRequest], ModelResponse],
) -> ModelCallResult:
policy = self._active_policy(request)
return handler(self._filter_model_request(request, policy=policy, refresh_decision=True))
@override
async def awrap_model_call(
self,
request: ModelRequest,
handler: Callable[[ModelRequest], Awaitable[ModelResponse]],
) -> ModelCallResult:
policy = self._active_policy(request)
_, paths = policy
if not paths:
self._store_policy_decision(request, policy, None)
return await handler(request)
filtered = await asyncio.to_thread(
self._filter_model_request,
request,
policy=policy,
refresh_decision=True,
)
return await handler(filtered)
@override
def wrap_tool_call(
self,
request: ToolCallRequest,
handler: Callable[[ToolCallRequest], ToolMessage | Command],
) -> ToolMessage | Command:
policy = self._active_policy(request)
if not policy[1]:
return handler(request)
allowed = self._allowed_names(request, policy=policy)
blocked = self._blocked_tool_message(request, allowed=allowed)
if blocked is not None:
return blocked
return self._filter_tool_search_result(request, handler(request), allowed=allowed)
@override
async def awrap_tool_call(
self,
request: ToolCallRequest,
handler: Callable[[ToolCallRequest], Awaitable[ToolMessage | Command]],
) -> ToolMessage | Command:
policy = self._active_policy(request)
if not policy[1]:
return await handler(request)
allowed = await asyncio.to_thread(self._allowed_names, request, policy=policy)
blocked = self._blocked_tool_message(request, allowed=allowed)
if blocked is not None:
return blocked
return self._filter_tool_search_result(request, await handler(request), allowed=allowed)
@@ -163,7 +163,8 @@ class AgentConfig(BaseModel):
description: str = ""
model: str | None = None
tool_groups: list[str] | None = None
# skills controls which skills are loaded into the agent's prompt:
# skills controls which skills are discoverable and may be activated by the
# agent. It does not activate their allowed-tools policies at construction:
# - None (or omitted): load all enabled skills (default fallback behavior)
# - [] (explicit empty list): disable all skills
# - ["skill1", "skill2"]: load only the specified skills
@@ -25,6 +25,12 @@ SECRETS_CONTEXT_KEY = "secrets"
# tool. Both reserved keys are stripped from trace payloads (see tracing redactor).
ACTIVE_SECRETS_CONTEXT_KEY = "__active_skill_secrets"
# Reserved sub-key holding the active skill tool-policy decision for one model
# step. The decision includes a middleware-instance owner token that prevents a
# caller from forging an allow-all decision in its mergeable run context, so the
# entire value must be stripped from every observable serialization surface.
SKILL_TOOL_POLICY_DECISION_CONTEXT_KEY = "__skill_tool_policy_decision"
def _string_pairs(raw: Any) -> dict[str, str]:
if not isinstance(raw, dict):
@@ -51,11 +57,36 @@ def read_active_secrets(context: Any) -> dict[str, str]:
return _string_pairs(context.get(ACTIVE_SECRETS_CONTEXT_KEY))
def write_slash_skill_source_path(context: Any, path: str, *, owner_token: str) -> None:
"""Persist an authenticated slash-activated skill path in a run context.
The source contains a path reference plus a middleware-chain-local token.
Consumers must authenticate the token and resolve the path against the live
skill registry before trusting any skill metadata.
"""
if isinstance(context, dict) and isinstance(path, str) and path and isinstance(owner_token, str) and owner_token:
context[_SLASH_SECRET_SOURCE_KEY] = {"path": path, "owner_token": owner_token}
def read_slash_skill_source_path(context: Any, *, owner_token: str) -> str | None:
"""Return the authenticated slash-activated skill path, if well formed."""
if not isinstance(context, dict):
return None
source = context.get(_SLASH_SECRET_SOURCE_KEY)
if not isinstance(source, dict):
return None
path = source.get("path")
source_owner_token = source.get("owner_token")
if not isinstance(owner_token, str) or not owner_token or source_owner_token != owner_token:
return None
return path if isinstance(path, str) and path else None
# Private run-context keys the skill-activation middleware uses to carry secret
# bindings across a run. Only ``secrets`` / ``__active_skill_secrets`` hold
# values; the binding-source and audit keys hold names only. All are listed so
# the redaction allowlist stays a complete guard even if a future edit starts
# storing a value under one of the name-only keys.
# secret values; the slash source holds a middleware-chain owner token, while
# the audit keys hold names only. All are listed so the redaction allowlist
# remains a complete guard.
_SLASH_SECRET_SOURCE_KEY = "__slash_skill_secret_source"
_SECRETS_BINDING_AUDIT_KEY = "__skill_secrets_binding_audit"
@@ -78,6 +109,7 @@ REDACTED_CONTEXT_KEYS = frozenset(
_SLASH_SECRET_SOURCE_KEY,
_SECRETS_BINDING_AUDIT_KEY,
_SLASH_SKILL_ACTIVATION_RUN_KEY,
SKILL_TOOL_POLICY_DECISION_CONTEXT_KEY,
}
)
@@ -83,6 +83,12 @@ class LocalSkillStorage(SkillStorage):
dir_names[:] = sorted(name for name in dir_names if not name.startswith("."))
if SKILL_MD_FILE not in file_names:
continue
# A directory containing SKILL.md is a package boundary. Any
# nested SKILL.md files belong to that package's supporting
# resources (for example eval fixtures), not to the runtime
# skill registry. Namespace directories without SKILL.md still
# recurse, preserving layouts such as public/team/helper.
dir_names.clear()
yield category, category_path, Path(current_root) / SKILL_MD_FILE
def read_custom_skill(self, name: str) -> str:
@@ -268,6 +268,7 @@ class UserScopedSkillStorage(LocalSkillStorage):
dir_names[:] = sorted(name for name in dir_names if not name.startswith("."))
if SKILL_MD_FILE not in file_names:
continue
dir_names.clear()
yield SkillCategory.PUBLIC, public_path, Path(current_root) / SKILL_MD_FILE
# 2. Custom skills: prefer user-level directory
@@ -278,6 +279,7 @@ class UserScopedSkillStorage(LocalSkillStorage):
dir_names[:] = sorted(name for name in dir_names if not name.startswith(".") and name != ".history")
if SKILL_MD_FILE not in file_names:
continue
dir_names.clear()
user_custom_exists = True
yield SkillCategory.CUSTOM, user_custom_path, Path(current_root) / SKILL_MD_FILE
@@ -293,6 +295,7 @@ class UserScopedSkillStorage(LocalSkillStorage):
dir_names[:] = sorted(name for name in dir_names if not name.startswith(".") and name != ".history")
if SKILL_MD_FILE not in file_names:
continue
dir_names.clear()
yield SkillCategory.LEGACY, global_custom_path, Path(current_root) / SKILL_MD_FILE
# ------------------------------------------------------------------
@@ -11,9 +11,18 @@ class NamedTool(Protocol):
# Framework built-ins that remain available even when an active skill declares
# allowed-tools. They support controlled framework workflows rather than
# extending the reviewed/activated skill's own tool authority.
ALWAYS_AVAILABLE_BUILTIN_TOOL_NAMES = frozenset({"read_file", "review_skill_package"})
# allowed-tools. They support controlled file/review/discovery workflows rather
# than extending the reviewed/activated skill's own business-tool authority.
# In particular, promotion through tool_search does not restore a tool removed
# by SkillToolPolicyMiddleware, and describe_skill only returns catalog metadata.
ALWAYS_AVAILABLE_BUILTIN_TOOL_NAMES = frozenset(
{
"describe_skill",
"read_file",
"review_skill_package",
"tool_search",
}
)
def allowed_tool_names_for_skills(skills: list[Skill]) -> set[str] | None:
@@ -4,10 +4,10 @@ Contains:
- DeferredToolCatalog: immutable, searchable catalog of deferred tools.
- build_tool_search_tool: builds the `tool_search` tool as a closure over a
catalog; it records promotions into graph state via ``Command``.
- build_deferred_tool_setup: assembles the catalog + tool from a
policy-filtered tool list (call AFTER tool-policy filtering).
- build_deferred_tool_setup: assembles the catalog + tool from the tools
configured for this agent build.
- build_mcp_routing_middleware: builds the PR2 auto-promote middleware from
serialized routing metadata on policy-filtered deferred tools.
serialized routing metadata on deferred tools available to the caller.
The agent sees deferred tool names in <available-deferred-tools> but cannot
call them until it fetches their full schema via the tool_search tool. The
@@ -124,7 +124,8 @@ class DeferredToolSetup:
The three fields move as a unit, so callers branch on ``tool_search_tool``:
- **Empty** ``(None, frozenset(), None)``: deferral is disabled, or no MCP
tool survived policy filtering. Nothing is deferred bind tools as-is.
tool is present in the candidate list. Nothing is deferred bind tools
as-is.
- **Populated**: ``tool_search_tool`` is appended to the agent's tools,
``deferred_names`` are withheld from the model until promoted, and
``catalog_hash`` scopes those promotions in graph state.
@@ -171,20 +172,24 @@ def build_tool_search_tool(catalog: DeferredToolCatalog) -> BaseTool:
return tool_search
def build_deferred_tool_setup(filtered_tools: list[BaseTool], *, enabled: bool) -> DeferredToolSetup:
"""Build the deferred-tool setup from a POLICY-FILTERED tool list.
def build_deferred_tool_setup(candidate_tools: list[BaseTool], *, enabled: bool) -> DeferredToolSetup:
"""Build deferred-tool setup from one agent build's candidate tools.
Must be called after skill/agent tool-policy filtering so the catalog never
exposes a tool the current agent is not allowed to use.
Lead agents pass their full configured tool list; ``SkillToolPolicyMiddleware``
later filters model-visible schemas, execution, and ``tool_search`` results
for the active skill while keeping the discovery tool itself available.
Subagents may pass a statically policy-filtered list because their configured
skills are loaded at startup. The downstream deferred-schema middleware still
hides unpromoted MCP schemas in either case.
Returns an empty setup (see :class:`DeferredToolSetup`) in two distinct
cases: deferral is disabled, or it is enabled but no MCP tool survived
filtering.
the caller's build-time selection.
"""
if not enabled:
# Deferral disabled: defer nothing; the model binds every tool as before.
return DeferredToolSetup(None, frozenset(), None)
deferred = [t for t in filtered_tools if is_mcp_tool(t)]
deferred = [t for t in candidate_tools if is_mcp_tool(t)]
if not deferred:
# Enabled, but no MCP tool to defer: same empty result, different reason.
return DeferredToolSetup(None, frozenset(), None)
@@ -192,21 +197,22 @@ def build_deferred_tool_setup(filtered_tools: list[BaseTool], *, enabled: bool)
return DeferredToolSetup(build_tool_search_tool(catalog), catalog.names, catalog.hash)
def assemble_deferred_tools(filtered_tools: list[BaseTool], *, enabled: bool) -> tuple[list[BaseTool], DeferredToolSetup]:
"""Build the final tool list + deferred setup from a POLICY-FILTERED list.
def assemble_deferred_tools(candidate_tools: list[BaseTool], *, enabled: bool) -> tuple[list[BaseTool], DeferredToolSetup]:
"""Build the final tool list and deferred setup from candidate tools.
Call AFTER tool-policy filtering so the deferred catalog never exposes a tool
the agent is not allowed to use. Fail-closed: if tool_search is enabled and
MCP tools survived filtering but no deferred set was recovered, raise rather
than silently binding their full schemas to the model.
Fail closed on deferral assembly itself: if tool_search is enabled and MCP
candidates exist but no deferred set was recovered, raise rather than silently
binding their full schemas to the model. Lead-agent authorization is enforced
separately at runtime by ``SkillToolPolicyMiddleware``; subagents may already
have applied their static skill policy to ``candidate_tools``.
Shared by every agent-build path (lead, embedded client, subagent) so they
all get the same fail-closed guarantee from one place.
"""
deferred_setup = build_deferred_tool_setup(filtered_tools, enabled=enabled)
if enabled and not deferred_setup.deferred_names and any(is_mcp_tool(t) for t in filtered_tools):
raise RuntimeError("tool_search enabled and MCP tools survived policy filtering, but no deferred set was recovered - refusing to bind MCP schemas (fail-closed).")
final_tools = list(filtered_tools)
deferred_setup = build_deferred_tool_setup(candidate_tools, enabled=enabled)
if enabled and not deferred_setup.deferred_names and any(is_mcp_tool(t) for t in candidate_tools):
raise RuntimeError("tool_search enabled and MCP candidates exist, but no deferred set was recovered - refusing to bind MCP schemas (fail-closed).")
final_tools = list(candidate_tools)
if deferred_setup.tool_search_tool:
final_tools.append(deferred_setup.tool_search_tool)
return final_tools, deferred_setup
@@ -236,7 +242,7 @@ def build_mcp_routing_middleware(
*,
top_k: int,
) -> "AgentMiddleware | None":
"""Build PR2 auto-promotion middleware from policy-filtered deferred tools.
"""Build PR2 auto-promotion middleware from the caller's deferred tools.
The builder may inspect ``BaseTool.metadata`` at construction time, but the
returned middleware receives only a flat serializable routing index.
@@ -3,14 +3,22 @@
from __future__ import annotations
import inspect
from pathlib import Path
from typing import Any
from unittest.mock import MagicMock
import pytest
from langchain.agents import create_agent
from langchain_core.language_models import BaseChatModel
from langchain_core.messages import AIMessage, HumanMessage, ToolMessage
from langchain_core.outputs import ChatGeneration, ChatResult
from langchain_core.tools import tool
from deerflow.agents.lead_agent import agent as lead_agent_module
from deerflow.agents.middlewares import summarization_middleware as summarization_middleware_module
from deerflow.agents.middlewares.loop_detection_middleware import LoopDetectionMiddleware
from deerflow.agents.middlewares.subagent_limit_middleware import SubagentLimitMiddleware
from deerflow.agents.thread_state import ThreadState
from deerflow.config.app_config import AppConfig
from deerflow.config.loop_detection_config import LoopDetectionConfig
from deerflow.config.memory_config import MemoryConfig
@@ -18,6 +26,60 @@ from deerflow.config.model_config import ModelConfig
from deerflow.config.sandbox_config import SandboxConfig
from deerflow.config.subagents_config import SubagentsAppConfig
from deerflow.config.summarization_config import SummarizationConfig
from deerflow.runtime.secret_context import write_slash_skill_source_path
from deerflow.skills.types import Skill, SkillCategory
_POLICY_INTEGRATION_TOOL_CALLS: list[str] = []
@tool
def policy_integration_dangerous_tool() -> str:
"""Record an invocation of a tool that the active skill does not allow."""
_POLICY_INTEGRATION_TOOL_CALLS.append("executed")
return "executed"
class _PolicyBypassModel(BaseChatModel):
"""Emit a forbidden call even when the bound schema omits it."""
call_count: int = 0
bound_tool_names: list[list[str]] = []
@property
def _llm_type(self) -> str:
return "policy-bypass-test"
def bind_tools(self, tools: Any, **kwargs: Any):
self.bound_tool_names.append([tool.name for tool in tools])
return self
def _generate(self, messages, stop=None, run_manager=None, **kwargs):
self.call_count += 1
if self.call_count == 1:
message = AIMessage(
content="",
tool_calls=[
{
"id": "forbidden-call",
"name": policy_integration_dangerous_tool.name,
"args": {},
}
],
)
else:
message = AIMessage(content="done")
return ChatResult(generations=[ChatGeneration(message=message)])
class _PolicyStorageStub:
def __init__(self, skills: list[Skill]):
self._skills = skills
def load_skills(self, *, enabled_only: bool = False) -> list[Skill]:
return [skill for skill in self._skills if skill.enabled or not enabled_only]
def get_container_root(self) -> str:
return "/mnt/skills"
def _make_app_config(models: list[ModelConfig], loop_detection: LoopDetectionConfig | None = None) -> AppConfig:
@@ -434,6 +496,101 @@ def test_build_middlewares_passes_explicit_app_config_to_shared_factory(monkeypa
assert middlewares[0] == "base-middleware"
def test_build_middlewares_orders_skill_activation_before_policy_and_durable_context(monkeypatch):
from deerflow.agents.middlewares.durable_context_middleware import DurableContextMiddleware
from deerflow.agents.middlewares.skill_activation_middleware import SkillActivationMiddleware
from deerflow.agents.middlewares.skill_tool_policy_middleware import SkillToolPolicyMiddleware
app_config = _make_app_config([_make_model("safe-model", supports_thinking=False)])
monkeypatch.setattr(lead_agent_module, "build_lead_runtime_middlewares", lambda *, app_config, lazy_init=True: [])
monkeypatch.setattr(lead_agent_module, "_create_summarization_middleware", lambda *, app_config=None: None)
monkeypatch.setattr(lead_agent_module, "_create_todo_list_middleware", lambda is_plan_mode: None)
middlewares = lead_agent_module.build_middlewares(
{"configurable": {"is_plan_mode": False, "subagent_enabled": False}},
model_name="safe-model",
app_config=app_config,
)
activation_idx = next(i for i, middleware in enumerate(middlewares) if isinstance(middleware, SkillActivationMiddleware))
policy_idx = next(i for i, middleware in enumerate(middlewares) if isinstance(middleware, SkillToolPolicyMiddleware))
durable_idx = next(i for i, middleware in enumerate(middlewares) if isinstance(middleware, DurableContextMiddleware))
assert policy_idx == activation_idx + 1
assert durable_idx == policy_idx + 1
assert middlewares[activation_idx]._slash_source_owner_token == middlewares[policy_idx]._slash_source_owner_token
@pytest.mark.parametrize("use_stale_path", [False, True], ids=["restrictive-skill", "stale-active-path"])
def test_compiled_skill_policy_chain_filters_schema_and_blocks_execution(monkeypatch, use_stale_path):
from deerflow.agents.middlewares.durable_context_middleware import DurableContextMiddleware
from deerflow.agents.middlewares.skill_activation_middleware import SkillActivationMiddleware
from deerflow.agents.middlewares.skill_tool_policy_middleware import SkillToolPolicyMiddleware
app_config = _make_app_config(
[_make_model("safe-model", supports_thinking=False)],
loop_detection=LoopDetectionConfig(enabled=False),
)
monkeypatch.setattr(lead_agent_module, "build_lead_runtime_middlewares", lambda *, app_config, lazy_init=True: [])
monkeypatch.setattr(lead_agent_module, "_create_summarization_middleware", lambda *, app_config=None: None)
monkeypatch.setattr(lead_agent_module, "_create_todo_list_middleware", lambda is_plan_mode: None)
middlewares = lead_agent_module.build_middlewares(
{"configurable": {"is_plan_mode": False, "subagent_enabled": False}},
model_name="safe-model",
app_config=app_config,
)
activation_idx = next(i for i, middleware in enumerate(middlewares) if isinstance(middleware, SkillActivationMiddleware))
durable_idx = next(i for i, middleware in enumerate(middlewares) if isinstance(middleware, DurableContextMiddleware))
compiled_slice = middlewares[activation_idx : durable_idx + 1]
assert [type(middleware) for middleware in compiled_slice] == [SkillActivationMiddleware, SkillToolPolicyMiddleware, DurableContextMiddleware]
skill_dir = Path("/tmp/skills/public/restricted")
restricted = Skill(
name="restricted",
description="Restrictive integration skill",
license="MIT",
skill_dir=skill_dir,
skill_file=skill_dir / "SKILL.md",
relative_path=Path("restricted"),
category=SkillCategory.PUBLIC,
allowed_tools=("read_file",),
enabled=True,
)
policy = compiled_slice[1]
policy._storage = lambda: _PolicyStorageStub([] if use_stale_path else [restricted])
context: dict[str, object] = {}
active_path = "/mnt/skills/public/missing/SKILL.md" if use_stale_path else restricted.get_container_file_path()
write_slash_skill_source_path(
context,
active_path,
owner_token=policy._slash_source_owner_token,
)
model = _PolicyBypassModel()
_POLICY_INTEGRATION_TOOL_CALLS.clear()
graph = create_agent(
model=model,
tools=[policy_integration_dangerous_tool],
middleware=compiled_slice,
state_schema=ThreadState,
)
result = graph.invoke(
{"messages": [HumanMessage(content="continue under the active skill")]},
context=context,
)
# LangChain skips ``bind_tools`` entirely when middleware filters the
# request to zero schemas. If the forbidden schema survived, this list
# would contain a binding with ``policy_integration_dangerous_tool``.
assert model.bound_tool_names == []
assert _POLICY_INTEGRATION_TOOL_CALLS == []
blocked = [message for message in result["messages"] if isinstance(message, ToolMessage) and message.tool_call_id == "forbidden-call"]
assert len(blocked) == 1
assert blocked[0].status == "error"
assert "not allowed by the active skill policy" in blocked[0].content
def test_build_middlewares_places_mcp_routing_before_deferred_filter(monkeypatch):
from deerflow.agents.middlewares.deferred_tool_filter_middleware import DeferredToolFilterMiddleware
from deerflow.agents.middlewares.mcp_routing_middleware import McpRoutingMiddleware
+13 -12
View File
@@ -265,7 +265,7 @@ def test_make_lead_agent_empty_skills_passed_correctly(monkeypatch):
monkeypatch.setattr(lead_agent_module, "_resolve_model_name", lambda x=None, **kwargs: "default-model")
monkeypatch.setattr(lead_agent_module, "create_chat_model", lambda **kwargs: "model")
monkeypatch.setattr("deerflow.tools.get_available_tools", lambda **kwargs: [])
monkeypatch.setattr(lead_agent_module, "_load_enabled_skills_for_tool_policy", lambda available_skills, *, app_config, user_id=None: [])
monkeypatch.setattr(lead_agent_module, "_load_enabled_available_skills", lambda available_skills, *, app_config, user_id=None: [])
monkeypatch.setattr(lead_agent_module, "build_middlewares", lambda *args, **kwargs: [])
monkeypatch.setattr(lead_agent_module, "create_agent", lambda **kwargs: kwargs)
@@ -300,7 +300,7 @@ def test_make_lead_agent_empty_skills_passed_correctly(monkeypatch):
assert captured_skills[-1] == {"skill1"}
def test_make_lead_agent_filters_tools_from_available_skills(monkeypatch):
def test_make_lead_agent_custom_skill_allowlist_does_not_activate_tool_policy(monkeypatch):
from unittest.mock import MagicMock
from deerflow.agents.lead_agent import agent as lead_agent_module
@@ -311,8 +311,8 @@ def test_make_lead_agent_filters_tools_from_available_skills(monkeypatch):
monkeypatch.setattr(lead_agent_module, "apply_prompt_template", lambda **kwargs: "mock_prompt")
monkeypatch.setattr(lead_agent_module, "create_agent", lambda **kwargs: kwargs)
monkeypatch.setattr(lead_agent_module, "load_agent_config", lambda x: AgentConfig(name="test", skills=["restricted", "legacy"]))
monkeypatch.setattr(lead_agent_module, "_load_enabled_skills_for_tool_policy", lambda available_skills, *, app_config, user_id=None: [_make_skill("restricted", ["read_file", "web_search"]), _make_skill("legacy", None)])
monkeypatch.setattr("deerflow.tools.get_available_tools", lambda **kwargs: [NamedTool("bash"), NamedTool("read_file"), NamedTool("web_search")])
monkeypatch.setattr(lead_agent_module, "_load_enabled_available_skills", lambda available_skills, *, app_config, user_id=None: [_make_skill("restricted", ["read_file", "web_search"]), _make_skill("legacy", None)])
monkeypatch.setattr("deerflow.tools.get_available_tools", lambda **kwargs: [NamedTool("task"), NamedTool("bash"), NamedTool("read_file"), NamedTool("web_search")])
mock_app_config = MagicMock()
mock_app_config.get_model_config.return_value = SimpleNamespace(supports_thinking=False, supports_vision=False)
@@ -323,8 +323,10 @@ def test_make_lead_agent_filters_tools_from_available_skills(monkeypatch):
agent_kwargs = lead_agent_module.make_lead_agent({"configurable": {"agent_name": "test"}})
# With skills.deferred_discovery=True, describe_skill is added to tools
# The custom-agent skill list controls discovery/activation, not baseline
# tools. With deferred discovery, describe_skill is added as well.
tool_names = [tool.name for tool in agent_kwargs["tools"]]
assert "task" in tool_names
assert "read_file" in tool_names
assert "describe_skill" in tool_names
@@ -351,7 +353,7 @@ def test_make_lead_agent_all_legacy_skills_preserve_all_tools(monkeypatch):
monkeypatch.setattr(lead_agent_module, "apply_prompt_template", lambda **kwargs: "mock_prompt")
monkeypatch.setattr(lead_agent_module, "create_agent", lambda **kwargs: kwargs)
monkeypatch.setattr(lead_agent_module, "load_agent_config", lambda x: AgentConfig(name="test", skills=None))
monkeypatch.setattr(lead_agent_module, "_load_enabled_skills_for_tool_policy", lambda available_skills, *, app_config, user_id=None: [_make_skill("legacy", None)])
monkeypatch.setattr(lead_agent_module, "_load_enabled_available_skills", lambda available_skills, *, app_config, user_id=None: [_make_skill("legacy", None)])
monkeypatch.setattr("deerflow.tools.get_available_tools", lambda **kwargs: [NamedTool("bash"), NamedTool("read_file")])
mock_app_config = MagicMock()
@@ -360,12 +362,12 @@ def test_make_lead_agent_all_legacy_skills_preserve_all_tools(monkeypatch):
agent_kwargs = lead_agent_module.make_lead_agent({"configurable": {"agent_name": "test"}})
# describe_skill is appended after skill-allowed-tools filtering (it bypasses policy).
# No skill is active yet, so the configured lead tools remain available.
tool_names = [tool.name for tool in agent_kwargs["tools"]]
assert tool_names == ["bash", "read_file", "update_agent", "describe_skill"]
def test_make_lead_agent_enforces_allowed_tools_when_skill_cache_is_cold(monkeypatch):
def test_make_lead_agent_does_not_apply_passive_skill_policy_when_cache_is_cold(monkeypatch):
from unittest.mock import MagicMock
from deerflow.agents.lead_agent import agent as lead_agent_module
@@ -391,9 +393,8 @@ def test_make_lead_agent_enforces_allowed_tools_when_skill_cache_is_cold(monkeyp
agent_kwargs = lead_agent_module.make_lead_agent({"configurable": {"agent_name": "test"}})
# describe_skill is appended after skill-allowed-tools filtering (it bypasses policy).
tool_names = [tool.name for tool in agent_kwargs["tools"]]
assert tool_names == ["read_file", "describe_skill"]
assert {"bash", "read_file", "web_search", "describe_skill"} <= set(tool_names)
def test_make_lead_agent_fails_closed_when_skill_policy_load_fails(monkeypatch):
@@ -452,7 +453,7 @@ def test_make_lead_agent_drops_update_agent_on_github_channel(monkeypatch):
monkeypatch.setattr(lead_agent_module, "apply_prompt_template", lambda **kwargs: "mock_prompt")
monkeypatch.setattr(lead_agent_module, "create_agent", lambda **kwargs: kwargs)
monkeypatch.setattr(lead_agent_module, "load_agent_config", lambda x: AgentConfig(name="test", skills=None))
monkeypatch.setattr(lead_agent_module, "_load_enabled_skills_for_tool_policy", lambda available_skills, *, app_config, user_id=None: [_make_skill("legacy", None)])
monkeypatch.setattr(lead_agent_module, "_load_enabled_available_skills", lambda available_skills, *, app_config, user_id=None: [_make_skill("legacy", None)])
monkeypatch.setattr("deerflow.tools.get_available_tools", lambda **kwargs: [NamedTool("bash"), NamedTool("read_file")])
mock_app_config = MagicMock()
@@ -488,7 +489,7 @@ def test_make_lead_agent_keeps_update_agent_on_non_webhook_channels(monkeypatch)
monkeypatch.setattr(lead_agent_module, "apply_prompt_template", lambda **kwargs: "mock_prompt")
monkeypatch.setattr(lead_agent_module, "create_agent", lambda **kwargs: kwargs)
monkeypatch.setattr(lead_agent_module, "load_agent_config", lambda x: AgentConfig(name="test", skills=None))
monkeypatch.setattr(lead_agent_module, "_load_enabled_skills_for_tool_policy", lambda available_skills, *, app_config, user_id=None: [_make_skill("legacy", None)])
monkeypatch.setattr(lead_agent_module, "_load_enabled_available_skills", lambda available_skills, *, app_config, user_id=None: [_make_skill("legacy", None)])
monkeypatch.setattr("deerflow.tools.get_available_tools", lambda **kwargs: [NamedTool("bash")])
mock_app_config = MagicMock()
+2 -4
View File
@@ -500,8 +500,7 @@ class TestModeGating:
"load_agent_config",
lambda name: SimpleNamespace(model=None, skills=None, tool_groups=None),
)
monkeypatch.setattr(lead_agent_module, "_load_enabled_skills_for_tool_policy", lambda available_skills, *, app_config, user_id=None: [])
monkeypatch.setattr(lead_agent_module, "filter_tools_by_skill_allowed_tools", lambda tools, skills, always_allowed_tool_names=(): tools)
monkeypatch.setattr(lead_agent_module, "_load_enabled_available_skills", lambda available_skills, *, app_config, user_id=None: [])
monkeypatch.setattr("deerflow.tools.get_available_tools", lambda **kwargs: [_NamedTool("memory_search"), _NamedTool("bash")])
app_config = SimpleNamespace(
@@ -533,8 +532,7 @@ class TestModeGating:
"load_agent_config",
lambda name: SimpleNamespace(model=None, skills=None, tool_groups=None),
)
monkeypatch.setattr(lead_agent_module, "_load_enabled_skills_for_tool_policy", lambda available_skills, *, app_config, user_id=None: [])
monkeypatch.setattr(lead_agent_module, "filter_tools_by_skill_allowed_tools", lambda tools, skills, always_allowed_tool_names=(): tools)
monkeypatch.setattr(lead_agent_module, "_load_enabled_available_skills", lambda available_skills, *, app_config, user_id=None: [])
monkeypatch.setattr("deerflow.tools.get_available_tools", lambda **kwargs: [_NamedTool("bash"), _NamedTool("bash")])
app_config = SimpleNamespace(
@@ -0,0 +1,341 @@
import asyncio
import json
from pathlib import Path
from langchain.agents import create_agent
from langchain_core.language_models.fake_chat_models import GenericFakeChatModel
from langchain_core.messages import AIMessage, HumanMessage, ToolMessage
from langchain_core.tools import tool
from pydantic import Field
from deerflow.agents.middlewares.deferred_tool_filter_middleware import DeferredToolFilterMiddleware
from deerflow.agents.middlewares.mcp_routing_middleware import McpRoutingMiddleware
from deerflow.agents.middlewares.skill_tool_policy_middleware import SkillToolPolicyMiddleware
from deerflow.agents.thread_state import ThreadState
from deerflow.runtime.secret_context import SKILL_TOOL_POLICY_DECISION_CONTEXT_KEY, write_slash_skill_source_path
from deerflow.runtime.serialization import serialize
from deerflow.skills.describe import build_skill_search_setup
from deerflow.skills.types import Skill, SkillCategory
from deerflow.tools.builtins.tool_search import build_deferred_tool_setup
from deerflow.tools.mcp_metadata import tag_mcp_tool
_SLASH_SOURCE_OWNER_TOKEN = "test-slash-source-owner"
_CALC_CALLS: list[str] = []
_DENIED_CALLS: list[str] = []
@tool
def calc(expression: str) -> str:
"""Evaluate an arithmetic expression."""
_CALC_CALLS.append(expression)
return "4"
@tool
def denied_lookup(query: str) -> str:
"""Run a lookup the active skill does not authorize."""
_DENIED_CALLS.append(query)
return "denied data"
class _StorageStub:
def __init__(self, skills: list[Skill]):
self._skills = skills
def load_skills(self, *, enabled_only: bool = False) -> list[Skill]:
return [skill for skill in self._skills if skill.enabled or not enabled_only]
def get_container_root(self) -> str:
return "/mnt/skills"
class _RecordingModel(GenericFakeChatModel):
bound_tool_names: list[list[str]] = Field(default_factory=list)
def __init__(self, responses: list[AIMessage]):
super().__init__(messages=iter(responses))
def bind_tools(self, tools, **kwargs):
self.bound_tool_names.append([getattr(candidate, "name", "") for candidate in tools])
return self
def _skill(name: str, allowed_tools: list[str]) -> Skill:
skill_dir = Path(f"/tmp/skills/public/{name}")
return Skill(
name=name,
description=f"Description for {name}",
license="MIT",
skill_dir=skill_dir,
skill_file=skill_dir / "SKILL.md",
relative_path=Path(name),
category=SkillCategory.PUBLIC,
allowed_tools=tuple(allowed_tools),
enabled=True,
)
def _active_policy(skill: Skill) -> tuple[SkillToolPolicyMiddleware, dict[str, object]]:
middleware = SkillToolPolicyMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
middleware._storage = lambda: _StorageStub([skill])
context: dict[str, object] = {}
write_slash_skill_source_path(
context,
skill.get_container_file_path(),
owner_token=_SLASH_SOURCE_OWNER_TOKEN,
)
return middleware, context
def _deferred_setup():
return build_deferred_tool_setup(
[tag_mcp_tool(calc), tag_mcp_tool(denied_lookup)],
enabled=True,
)
def test_active_skill_can_search_promote_and_call_allowed_deferred_tool():
restricted = _skill("restricted", ["calc"])
policy, context = _active_policy(restricted)
setup = _deferred_setup()
model = _RecordingModel(
[
AIMessage(
content="",
tool_calls=[
{
"name": "tool_search",
"args": {"query": "select:calc,denied_lookup"},
"id": "search-call",
"type": "tool_call",
}
],
),
AIMessage(
content="",
tool_calls=[
{
"name": "calc",
"args": {"expression": "2 + 2"},
"id": "calc-call",
"type": "tool_call",
}
],
),
AIMessage(content="done"),
]
)
_CALC_CALLS.clear()
graph = create_agent(
model=model,
tools=[calc, denied_lookup, setup.tool_search_tool],
middleware=[
policy,
DeferredToolFilterMiddleware(setup.deferred_names, setup.catalog_hash),
],
state_schema=ThreadState,
)
result = asyncio.run(
graph.ainvoke(
{"messages": [HumanMessage(content="use the allowed calculator")]},
context=context,
)
)
assert model.bound_tool_names[0] == ["tool_search"]
assert "calc" in model.bound_tool_names[1]
assert "denied_lookup" not in model.bound_tool_names[1]
assert _CALC_CALLS == ["2 + 2"]
assert result["promoted"] == {"catalog_hash": setup.catalog_hash, "names": ["calc"]}
search_result = [message for message in result["messages"] if isinstance(message, ToolMessage) and message.tool_call_id == "search-call"]
assert len(search_result) == 1
assert '"name": "calc"' in search_result[0].content
assert "denied_lookup" not in search_result[0].content
def test_tool_search_promotion_cannot_expose_or_execute_denied_deferred_tool():
restricted = _skill("restricted", ["calc"])
policy, context = _active_policy(restricted)
setup = _deferred_setup()
model = _RecordingModel(
[
AIMessage(
content="",
tool_calls=[
{
"name": "tool_search",
"args": {"query": "select:denied_lookup"},
"id": "search-denied",
"type": "tool_call",
}
],
),
AIMessage(
content="",
tool_calls=[
{
"name": "denied_lookup",
"args": {"query": "secret"},
"id": "denied-call",
"type": "tool_call",
}
],
),
AIMessage(content="done"),
]
)
_DENIED_CALLS.clear()
graph = create_agent(
model=model,
tools=[calc, denied_lookup, setup.tool_search_tool],
middleware=[
policy,
DeferredToolFilterMiddleware(setup.deferred_names, setup.catalog_hash),
],
state_schema=ThreadState,
)
result = graph.invoke(
{"messages": [HumanMessage(content="try the denied lookup")]},
context=context,
)
assert all("denied_lookup" not in names for names in model.bound_tool_names)
assert _DENIED_CALLS == []
assert result["promoted"] == {
"catalog_hash": setup.catalog_hash,
"names": [],
}
search_result = [message for message in result["messages"] if isinstance(message, ToolMessage) and message.tool_call_id == "search-denied"]
assert len(search_result) == 1
assert "denied_lookup" not in search_result[0].content
blocked = [message for message in result["messages"] if isinstance(message, ToolMessage) and message.tool_call_id == "denied-call"]
assert len(blocked) == 1
assert blocked[0].status == "error"
assert "not allowed by the active skill policy" in blocked[0].content
def test_auto_promotion_cannot_expose_or_execute_denied_deferred_tool():
restricted = _skill("restricted", ["calc"])
policy, context = _active_policy(restricted)
setup = _deferred_setup()
routing = McpRoutingMiddleware(
{"denied_lookup": {"priority": 100, "keywords": ["denied lookup"]}},
setup.catalog_hash,
3,
)
model = _RecordingModel(
[
AIMessage(
content="",
tool_calls=[
{
"name": "denied_lookup",
"args": {"query": "secret"},
"id": "auto-denied-call",
"type": "tool_call",
}
],
),
AIMessage(content="done"),
]
)
_DENIED_CALLS.clear()
graph = create_agent(
model=model,
tools=[calc, denied_lookup, setup.tool_search_tool],
middleware=[
policy,
routing,
DeferredToolFilterMiddleware(setup.deferred_names, setup.catalog_hash),
],
state_schema=ThreadState,
)
result = graph.invoke(
{"messages": [HumanMessage(content="run the denied lookup")]},
context=context,
)
assert all("denied_lookup" not in names for names in model.bound_tool_names)
assert _DENIED_CALLS == []
assert result["promoted"] == {
"catalog_hash": setup.catalog_hash,
"names": ["denied_lookup"],
}
blocked = [message for message in result["messages"] if isinstance(message, ToolMessage) and message.tool_call_id == "auto-denied-call"]
assert len(blocked) == 1
assert blocked[0].status == "error"
assert "not allowed by the active skill policy" in blocked[0].content
def test_restrictive_skill_keeps_deferred_skill_discovery_available():
restricted = _skill("restricted", [])
hidden = _skill("hidden", ["write_file"])
skill_setup = build_skill_search_setup(
[restricted],
enabled=True,
)
policy, context = _active_policy(restricted)
model = _RecordingModel(
[
AIMessage(
content="",
tool_calls=[
{
"name": "describe_skill",
"args": {"name": f"select:{restricted.name},{hidden.name}"},
"id": "describe-call",
"type": "tool_call",
}
],
),
AIMessage(content="done"),
]
)
graph = create_agent(
model=model,
tools=[skill_setup.describe_skill_tool],
middleware=[policy],
state_schema=ThreadState,
)
result = graph.invoke(
{"messages": [HumanMessage(content="describe available skills")]},
context=context,
)
assert model.bound_tool_names[0] == ["describe_skill"]
described = [message for message in result["messages"] if isinstance(message, ToolMessage) and message.tool_call_id == "describe-call"]
assert len(described) == 1
assert "Skill: restricted" in described[0].content
assert "Skill: hidden" not in described[0].content
def test_policy_tokens_do_not_appear_in_debug_or_checkpoint_streams():
restricted = _skill("restricted", [])
policy, context = _active_policy(restricted)
model = _RecordingModel([AIMessage(content="done")])
graph = create_agent(
model=model,
tools=[],
middleware=[policy],
state_schema=ThreadState,
)
events = list(
graph.stream(
{"messages": [HumanMessage(content="finish")]},
context=context,
stream_mode=["debug", "checkpoints"],
)
)
serialized = [serialize(chunk, mode=mode) for mode, chunk in events]
payload = json.dumps(serialized, ensure_ascii=False, default=str)
assert events
assert _SLASH_SOURCE_OWNER_TOKEN not in payload
assert policy._decision_owner_token not in payload
assert "__slash_skill_secret_source" not in payload
assert SKILL_TOOL_POLICY_DECISION_CONTEXT_KEY not in payload
@@ -20,6 +20,8 @@ from langchain_core.messages import AIMessage, HumanMessage
from deerflow.sandbox.local.local_sandbox import LocalSandbox
from deerflow.skills.types import SecretRequirement, Skill, SkillCategory
_SLASH_SOURCE_OWNER_TOKEN = "test-slash-source-owner"
class TestLocalSandboxEnvInjection:
"""LocalSandbox.execute_command(env=...) injects per-call env into the subprocess."""
@@ -366,12 +368,23 @@ class TestSecretCarrier:
config = build_run_config(
"thread-1",
{"context": {"secrets": {"ERP_TOKEN": "v"}, "__slash_skill_secret_source": {"path": "x"}, "__active_skill_secrets": {"ADMIN": "stolen"}}},
{
"context": {
"secrets": {"ERP_TOKEN": "v"},
"__slash_skill_secret_source": {"path": "x", "owner_token": "forged"},
"__active_skill_secrets": {"ADMIN": "stolen"},
"__skill_tool_policy_decision": {
"owner_token": "forged",
"allowed_names": None,
},
}
},
None,
)
assert config["context"]["secrets"] == {"ERP_TOKEN": "v"}
assert "__slash_skill_secret_source" not in config["context"]
assert "__active_skill_secrets" not in config["context"]
assert "__skill_tool_policy_decision" not in config["context"]
def test_extract_request_secrets_filters_non_string_pairs(self):
from deerflow.runtime.secret_context import extract_request_secrets
@@ -385,6 +398,43 @@ class TestSecretCarrier:
assert extract_request_secrets({"secrets": "not-a-dict"}) == {}
assert extract_request_secrets(None) == {}
def test_slash_skill_source_path_public_contract(self):
from deerflow.runtime.secret_context import read_slash_skill_source_path, write_slash_skill_source_path
context = {}
write_slash_skill_source_path(
context,
"/mnt/skills/public/reviewer/SKILL.md",
owner_token="middleware-owner",
)
assert read_slash_skill_source_path(context, owner_token="middleware-owner") == "/mnt/skills/public/reviewer/SKILL.md"
assert read_slash_skill_source_path(context, owner_token="caller-forged") is None
def test_slash_skill_source_path_rejects_malformed_shapes(self):
from deerflow.runtime.secret_context import read_slash_skill_source_path
malformed = [
None,
"path",
[],
{"path": None, "owner_token": "middleware-owner"},
{"path": "", "owner_token": "middleware-owner"},
{"path": 7, "owner_token": "middleware-owner"},
{"path": "/mnt/skills/public/reviewer/SKILL.md"},
{"path": "/mnt/skills/public/reviewer/SKILL.md", "owner_token": ""},
]
for value in malformed:
assert (
read_slash_skill_source_path(
{"__slash_skill_secret_source": value},
owner_token="middleware-owner",
)
is None
)
assert read_slash_skill_source_path({}, owner_token="middleware-owner") is None
assert read_slash_skill_source_path(None, owner_token="middleware-owner") is None
def _make_secret_skill(tmp_path: Path, name: str, required_secrets, *, enabled: bool = True, secrets_autonomous: bool = True):
skill_dir = tmp_path / name
@@ -418,7 +468,7 @@ class TestActivationBindsSecrets:
get_skills_root_path=lambda: tmp_path,
)
monkeypatch.setattr(mw, "get_or_new_skill_storage", lambda **kwargs: storage)
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
request = ModelRequest(
model=object(),
messages=[HumanMessage(content=f"/{skill.name} do it", id="m1")],
@@ -515,7 +565,7 @@ class TestActivationBindsSecrets:
set_app_config(AppConfig.model_validate({"sandbox": {"use": "deerflow.sandbox.local:LocalSandboxProvider"}}))
try:
sanitizer = InputSanitizationMiddleware()
skill_mw = SkillActivationMiddleware()
skill_mw = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
# Compose in real order: sanitizer (outer) -> skill activation (inner) -> model.
def skill_layer(req):
@@ -549,7 +599,7 @@ class TestActivationBindsSecrets:
context = {"secrets": {"A_TOKEN": "v-a"}}
monkeypatch.setattr(mw, "get_or_new_skill_storage", lambda **kwargs: _storage([skill_a]))
SkillActivationMiddleware().wrap_model_call(
SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN).wrap_model_call(
ModelRequest(
model=object(),
messages=[HumanMessage(content="/skill-a go", id="m1")],
@@ -561,7 +611,7 @@ class TestActivationBindsSecrets:
assert read_active_secrets(context) == {"A_TOKEN": "v-a"}
monkeypatch.setattr(mw, "get_or_new_skill_storage", lambda **kwargs: _storage([skill_b]))
SkillActivationMiddleware().wrap_model_call(
SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN).wrap_model_call(
ModelRequest(
model=object(),
messages=[HumanMessage(content="/skill-b go", id="m2")],
@@ -590,7 +640,7 @@ class TestActivationBindsSecrets:
# Turn 1: caller supplies ERP_TOKEN → injected.
context = {"secrets": {"ERP_TOKEN": "tok-1"}}
mw_inst = SkillActivationMiddleware()
mw_inst = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
mw_inst.wrap_model_call(
ModelRequest(
model=object(),
@@ -644,7 +694,7 @@ class TestInContextBindsSecrets:
get_skills_root_path=lambda: tmp_path,
)
monkeypatch.setattr(mw, "get_or_new_skill_storage", lambda **kwargs: storage)
mw_inst = middleware or SkillActivationMiddleware(available_skills=available_skills)
mw_inst = middleware or SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN, available_skills=available_skills)
mw_inst.wrap_model_call(
ModelRequest(
model=object(),
@@ -1015,7 +1065,7 @@ class TestLeakSurfaces:
runtime=SimpleNamespace(context=context),
)
captured = {}
SkillActivationMiddleware().wrap_model_call(request, lambda r: captured.setdefault("messages", r.messages) or AIMessage(content="ok"))
SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN).wrap_model_call(request, lambda r: captured.setdefault("messages", r.messages) or AIMessage(content="ok"))
return context, captured["messages"], journal_records
def test_prompt_surface_has_no_secret(self, tmp_path, monkeypatch):
@@ -1048,9 +1098,23 @@ class TestLeakSurfaces:
assert _SECRET not in str(config.get("configurable", {}))
def test_redact_helper_strips_secret_keys(self):
from deerflow.runtime.secret_context import redact_secret_context_keys
from deerflow.runtime.secret_context import SKILL_TOOL_POLICY_DECISION_CONTEXT_KEY, redact_secret_context_keys
ctx = {"thread_id": "t", "secrets": {"ERP_TOKEN": _SECRET}, "__active_skill_secrets": {"ERP_TOKEN": _SECRET}}
ctx = {
"thread_id": "t",
"secrets": {"ERP_TOKEN": _SECRET},
"__active_skill_secrets": {"ERP_TOKEN": _SECRET},
"__slash_skill_secret_source": {
"path": "/mnt/skills/public/reviewer/SKILL.md",
"owner_token": "slash-owner-token",
},
SKILL_TOOL_POLICY_DECISION_CONTEXT_KEY: {
"version": 1,
"owner_token": "policy-owner-token",
"active_paths": ["/mnt/skills/public/reviewer/SKILL.md"],
"allowed_names": None,
},
}
redacted = redact_secret_context_keys(ctx)
assert redacted == {"thread_id": "t"}
assert _SECRET not in str(redacted)
@@ -1059,14 +1123,32 @@ class TestLeakSurfaces:
# The run-record persistence + run API echo the raw request config; the
# stored/echoed copy must not carry secrets (verifier blocker), while the
# live config used to drive the run keeps them.
from deerflow.runtime.secret_context import redact_config_secrets
from deerflow.runtime.secret_context import SKILL_TOOL_POLICY_DECISION_CONTEXT_KEY, redact_config_secrets
config = {"context": {"secrets": {"ERP_TOKEN": _SECRET}, "thread_id": "t", "model_name": "m"}, "recursion_limit": 100}
config = {
"context": {
"secrets": {"ERP_TOKEN": _SECRET},
"thread_id": "t",
"model_name": "m",
"__slash_skill_secret_source": {
"path": "/mnt/skills/public/reviewer/SKILL.md",
"owner_token": "slash-owner-token",
},
SKILL_TOOL_POLICY_DECISION_CONTEXT_KEY: {
"version": 1,
"owner_token": "forged-or-leaked-token",
"active_paths": ["/mnt/skills/public/reviewer/SKILL.md"],
"allowed_names": None,
},
},
"recursion_limit": 100,
}
redacted = redact_config_secrets(config)
assert _SECRET not in str(redacted)
assert redacted["context"]["thread_id"] == "t"
assert redacted["context"]["model_name"] == "m"
assert "secrets" not in redacted["context"]
assert SKILL_TOOL_POLICY_DECISION_CONTEXT_KEY not in redacted["context"]
# Original is untouched (live config still has secrets).
assert config["context"]["secrets"] == {"ERP_TOKEN": _SECRET}
@@ -1133,7 +1215,7 @@ class TestEndToEndRealSubprocess:
state={"messages": []},
runtime=SimpleNamespace(context=context),
)
SkillActivationMiddleware().wrap_model_call(request, lambda r: AIMessage(content="ok"))
SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN).wrap_model_call(request, lambda r: AIMessage(content="ok"))
injected = read_active_secrets(context)
assert injected == {"ERP_TOKEN": _SECRET}
@@ -0,0 +1,775 @@
import asyncio
import json
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import MagicMock
import pytest
from langchain.agents.middleware.types import ModelRequest
from langchain.tools import ToolRuntime
from langchain_core.messages import HumanMessage
from langgraph.prebuilt.tool_node import ToolCallRequest
from langgraph.runtime import Runtime
from deerflow.runtime.secret_context import SKILL_TOOL_POLICY_DECISION_CONTEXT_KEY, write_slash_skill_source_path
from deerflow.skills.types import Skill, SkillCategory
_SLASH_SOURCE_OWNER_TOKEN = "test-slash-source-owner"
class NamedTool:
def __init__(self, name: str):
self.name = name
class ModelRequestStub:
def __init__(self, tools, *, state=None, context=None, messages=None):
self.tools = tools
self.state = state or {}
self.runtime = SimpleNamespace(context={} if context is None else context)
self.messages = messages or []
def override(self, **updates):
return ModelRequestStub(
updates.get("tools", self.tools),
state=updates.get("state", self.state),
context=self.runtime.context,
messages=updates.get("messages", self.messages),
)
class ToolRequestStub:
def __init__(self, name: str, *, state=None, context=None):
self.tool_call = {"name": name, "id": "call-1", "args": {}}
self.state = state or {}
self.runtime = SimpleNamespace(context={} if context is None else context)
class StorageStub:
def __init__(self, skills):
self._skills = skills
self.load_calls = 0
def load_skills(self, *, enabled_only=False):
self.load_calls += 1
return [skill for skill in self._skills if skill.enabled or not enabled_only]
def get_container_root(self):
return "/mnt/skills"
def _skill(name: str, allowed_tools, *, enabled=True):
skill_dir = Path(f"/tmp/skills/public/{name}")
return Skill(
name=name,
description=f"Description for {name}",
license="MIT",
skill_dir=skill_dir,
skill_file=skill_dir / "SKILL.md",
relative_path=Path(name),
category=SkillCategory.PUBLIC,
allowed_tools=None if allowed_tools is None else tuple(allowed_tools),
enabled=enabled,
)
def _middleware(skills, *, available_skills=None):
from deerflow.agents.middlewares.skill_tool_policy_middleware import SkillToolPolicyMiddleware
middleware = SkillToolPolicyMiddleware(
available_skills=available_skills,
slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN,
)
middleware._storage = lambda: StorageStub(skills)
return middleware
def _tool_names(request):
return [tool.name for tool in request.tools]
@pytest.mark.parametrize(
"middleware_class_path",
[
"deerflow.agents.middlewares.skill_activation_middleware.SkillActivationMiddleware",
"deerflow.agents.middlewares.skill_tool_policy_middleware.SkillToolPolicyMiddleware",
],
)
def test_skill_policy_middlewares_require_shared_slash_source_token(middleware_class_path):
module_name, class_name = middleware_class_path.rsplit(".", 1)
module = __import__(module_name, fromlist=[class_name])
middleware_class = getattr(module, class_name)
with pytest.raises(TypeError, match="slash_source_owner_token"):
middleware_class()
@pytest.mark.parametrize("invalid_token", [None, "", 7])
def test_skill_policy_middlewares_reject_invalid_slash_source_tokens(invalid_token):
from deerflow.agents.middlewares.skill_activation_middleware import SkillActivationMiddleware
from deerflow.agents.middlewares.skill_tool_policy_middleware import SkillToolPolicyMiddleware
for middleware_class in (SkillActivationMiddleware, SkillToolPolicyMiddleware):
with pytest.raises(ValueError, match="non-empty string"):
middleware_class(slash_source_owner_token=invalid_token)
def test_passive_enabled_skill_does_not_filter_lead_tools():
middleware = _middleware([_skill("reviewer", ["review_skill_package"])])
request = ModelRequestStub([NamedTool("task"), NamedTool("web_search"), NamedTool("review_skill_package")])
filtered = middleware._filter_model_request(request)
assert _tool_names(filtered) == ["task", "web_search", "review_skill_package"]
def test_sync_passive_model_call_skips_storage():
middleware = _middleware([])
def fail_storage():
raise AssertionError("passive model calls must not load skill storage")
middleware._storage = fail_storage
request = ModelRequestStub([NamedTool("task")])
assert middleware.wrap_model_call(request, lambda model_request: model_request) is request
def test_async_passive_model_call_skips_storage_and_thread_offload():
middleware = _middleware([])
def fail_storage():
raise AssertionError("passive model calls must not load skill storage")
middleware._storage = fail_storage
request = ModelRequestStub([NamedTool("task")])
async def handler(model_request):
return model_request
assert asyncio.run(middleware.awrap_model_call(request, handler)) is request
def test_slash_activated_skill_filters_first_model_call_and_task():
skill = _skill("reviewer", ["review_skill_package"])
context = {}
write_slash_skill_source_path(
context,
skill.get_container_file_path(),
owner_token=_SLASH_SOURCE_OWNER_TOKEN,
)
middleware = _middleware([skill])
request = ModelRequestStub(
[NamedTool("task"), NamedTool("read_file"), NamedTool("review_skill_package")],
context=context,
)
filtered = middleware._filter_model_request(request)
assert _tool_names(filtered) == ["read_file", "review_skill_package"]
@pytest.mark.parametrize("active_source", ["slash", "skill_context"])
def test_restrictive_skill_explicitly_allows_task_schema_and_execution(active_source):
skill = _skill("delegating", ["task"])
context = {}
state = {}
if active_source == "slash":
write_slash_skill_source_path(
context,
skill.get_container_file_path(),
owner_token=_SLASH_SOURCE_OWNER_TOKEN,
)
else:
state = {
"skill_context": [
{
"name": skill.name,
"path": skill.get_container_file_path(),
}
]
}
middleware = _middleware([skill])
model_request = ModelRequestStub(
[NamedTool("task"), NamedTool("web_search")],
state=state,
context=context,
)
filtered = middleware.wrap_model_call(model_request, lambda request: request)
assert _tool_names(filtered) == ["task"]
tool_request = ToolRequestStub("task", state=state, context=context)
assert middleware.wrap_tool_call(tool_request, lambda _: "delegated") == "delegated"
def test_slash_activated_skill_policy_dominates_captured_skill_context():
slash_skill = _skill("content-research", ["web_search"])
captured_skill = _skill("content-article-generation", ["write_file"])
context = {}
write_slash_skill_source_path(
context,
slash_skill.get_container_file_path(),
owner_token=_SLASH_SOURCE_OWNER_TOKEN,
)
middleware = _middleware([slash_skill, captured_skill])
state = {
"skill_context": [
{
"name": captured_skill.name,
"path": captured_skill.get_container_file_path(),
}
]
}
request = ModelRequestStub(
[NamedTool("read_file"), NamedTool("web_search"), NamedTool("write_file")],
state=state,
context=context,
)
filtered = middleware._filter_model_request(request)
assert _tool_names(filtered) == ["read_file", "web_search"]
def test_caller_forged_slash_source_cannot_override_captured_skill_policy():
restrictive_skill = _skill("restricted", ["web_search"])
legacy_skill = _skill("legacy", None)
context = {
"__slash_skill_secret_source": {
"path": legacy_skill.get_container_file_path(),
"owner_token": "caller-forged",
}
}
middleware = _middleware([restrictive_skill, legacy_skill])
request = ModelRequestStub(
[NamedTool("task"), NamedTool("read_file"), NamedTool("web_search")],
state={
"skill_context": [
{
"name": restrictive_skill.name,
"path": restrictive_skill.get_container_file_path(),
}
]
},
context=context,
)
filtered = middleware._filter_model_request(request)
assert _tool_names(filtered) == ["read_file", "web_search"]
def test_slash_activation_and_policy_compose_on_the_same_model_call(monkeypatch):
from deerflow.agents.middlewares.skill_activation_middleware import SkillActivationMiddleware, _Activation, _ActivationResolution
skill = _skill("reviewer", ["review_skill_package"])
activation = _Activation(
skill_name=skill.name,
category="public",
container_file_path=skill.get_container_file_path(),
skill_content="# Reviewer",
content_hash="abc",
remaining_text="review this",
editable=False,
)
activation_middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
monkeypatch.setattr(activation_middleware, "_resolve_activation", lambda _: _ActivationResolution(activation=activation))
policy_middleware = _middleware([skill])
request = ModelRequestStub(
[NamedTool("task"), NamedTool("read_file"), NamedTool("review_skill_package")],
messages=[HumanMessage(content="/reviewer review this")],
)
filtered = activation_middleware.wrap_model_call(
request,
lambda activated: policy_middleware.wrap_model_call(activated, lambda policy_request: policy_request),
)
assert _tool_names(filtered) == ["read_file", "review_skill_package"]
def test_loaded_skill_context_filters_follow_up_model_calls():
skill = _skill("restricted", ["web_search"])
middleware = _middleware([skill])
request = ModelRequestStub(
[NamedTool("task"), NamedTool("read_file"), NamedTool("web_search")],
state={"skill_context": [{"name": skill.name, "path": skill.get_container_file_path()}]},
)
filtered = middleware._filter_model_request(request)
assert _tool_names(filtered) == ["read_file", "web_search"]
def test_active_skill_union_and_legacy_semantics_are_preserved():
restricted = _skill("restricted", ["web_search"])
second = _skill("second", ["bash"])
legacy = _skill("legacy", None)
middleware = _middleware([restricted, second, legacy])
state = {
"skill_context": [
{"path": restricted.get_container_file_path()},
{"path": second.get_container_file_path()},
{"path": legacy.get_container_file_path()},
]
}
request = ModelRequestStub([NamedTool("task"), NamedTool("bash"), NamedTool("web_search")], state=state)
filtered = middleware._filter_model_request(request)
assert _tool_names(filtered) == ["bash", "web_search"]
def test_only_legacy_active_skill_preserves_all_tools():
legacy = _skill("legacy", None)
middleware = _middleware([legacy])
request = ModelRequestStub(
[NamedTool("task"), NamedTool("bash")],
state={"skill_context": [{"path": legacy.get_container_file_path()}]},
)
assert _tool_names(middleware._filter_model_request(request)) == ["task", "bash"]
def test_explicit_empty_allowed_tools_keeps_only_framework_tools():
restricted = _skill("restricted", [])
middleware = _middleware([restricted])
request = ModelRequestStub(
[
NamedTool("task"),
NamedTool("read_file"),
NamedTool("review_skill_package"),
NamedTool("tool_search"),
NamedTool("describe_skill"),
],
state={"skill_context": [{"path": restricted.get_container_file_path()}]},
)
assert _tool_names(middleware._filter_model_request(request)) == [
"read_file",
"review_skill_package",
"tool_search",
"describe_skill",
]
def test_active_skill_keeps_framework_discovery_tools():
restricted = _skill("restricted", ["calc"])
middleware = _middleware([restricted])
request = ModelRequestStub(
[NamedTool("calc"), NamedTool("tool_search"), NamedTool("describe_skill")],
state={"skill_context": [{"path": restricted.get_container_file_path()}]},
)
assert _tool_names(middleware._filter_model_request(request)) == ["calc", "tool_search", "describe_skill"]
def test_custom_agent_allowlist_rejects_all_out_of_scope_active_skills():
restricted = _skill("restricted", ["web_search"])
middleware = _middleware([restricted], available_skills={"other"})
request = ModelRequestStub(
[NamedTool("task"), NamedTool("web_search")],
state={"skill_context": [{"path": restricted.get_container_file_path()}]},
)
assert _tool_names(middleware._filter_model_request(request)) == []
def test_unauthorized_tool_execution_is_blocked():
restricted = _skill("restricted", ["web_search"])
middleware = _middleware([restricted])
request = ToolRequestStub(
"task",
state={"skill_context": [{"path": restricted.get_container_file_path()}]},
)
result = middleware.wrap_tool_call(request, lambda _: "executed")
assert result.status == "error"
assert result.name == "task"
assert "not allowed" in result.content
def test_allowed_tool_execution_reaches_handler():
restricted = _skill("restricted", ["web_search"])
middleware = _middleware([restricted])
request = ToolRequestStub(
"web_search",
state={"skill_context": [{"path": restricted.get_container_file_path()}]},
)
assert middleware.wrap_tool_call(request, lambda _: "executed") == "executed"
def test_async_unauthorized_tool_execution_is_blocked():
restricted = _skill("restricted", ["web_search"])
middleware = _middleware([restricted])
request = ToolRequestStub(
"task",
state={"skill_context": [{"path": restricted.get_container_file_path()}]},
)
async def handler(_):
return "executed"
result = asyncio.run(middleware.awrap_tool_call(request, handler))
assert result.status == "error"
assert result.name == "task"
def test_unknown_skill_context_path_is_skipped_while_resolvable_skills_apply():
restricted = _skill("restricted", ["web_search"])
middleware = _middleware([restricted])
request = ModelRequestStub(
[NamedTool("task"), NamedTool("read_file"), NamedTool("web_search")],
state={
"skill_context": [
{"path": "/mnt/skills/public/missing/SKILL.md"},
{"path": restricted.get_container_file_path()},
]
},
)
assert _tool_names(middleware._filter_model_request(request)) == ["read_file", "web_search"]
def test_all_unknown_active_paths_fail_closed_to_framework_tools():
middleware = _middleware([])
request = ModelRequestStub(
[
NamedTool("task"),
NamedTool("read_file"),
NamedTool("review_skill_package"),
NamedTool("tool_search"),
NamedTool("describe_skill"),
],
state={"skill_context": [{"path": "/mnt/skills/public/missing/SKILL.md"}]},
)
assert _tool_names(middleware._filter_model_request(request)) == [
"read_file",
"review_skill_package",
"tool_search",
"describe_skill",
]
def test_all_disabled_active_paths_fail_closed_to_framework_tools():
disabled = _skill("disabled", ["task"], enabled=False)
middleware = _middleware([disabled])
request = ModelRequestStub(
[
NamedTool("task"),
NamedTool("read_file"),
NamedTool("review_skill_package"),
NamedTool("tool_search"),
NamedTool("describe_skill"),
],
state={"skill_context": [{"path": disabled.get_container_file_path()}]},
)
assert _tool_names(middleware._filter_model_request(request)) == [
"read_file",
"review_skill_package",
"tool_search",
"describe_skill",
]
def test_async_passive_tool_call_skips_storage_and_thread_offload():
middleware = _middleware([])
def fail_storage():
raise AssertionError("passive tool calls must not load skill storage")
middleware._storage = fail_storage
request = ToolRequestStub("task")
async def handler(_):
return "executed"
assert asyncio.run(middleware.awrap_tool_call(request, handler)) == "executed"
def test_sync_passive_tool_call_skips_policy_resolution():
middleware = _middleware([])
request = ToolRequestStub("task")
middleware._blocked_tool_message = MagicMock(side_effect=AssertionError("passive tool calls must bypass policy resolution"))
assert middleware.wrap_tool_call(request, lambda _: "executed") == "executed"
middleware._blocked_tool_message.assert_not_called()
def test_tool_calls_reuse_the_current_model_step_policy_decision():
restricted = _skill("restricted", ["web_search"])
storage = StorageStub([restricted])
middleware = _middleware([])
middleware._storage = lambda: storage
context = {}
state = {"skill_context": [{"path": restricted.get_container_file_path()}]}
model_request = ModelRequestStub(
[NamedTool("task"), NamedTool("web_search")],
state=state,
context=context,
)
filtered = middleware.wrap_model_call(model_request, lambda request: request)
assert _tool_names(filtered) == ["web_search"]
for _ in range(3):
tool_request = ToolRequestStub("web_search", state=state, context=context)
assert middleware.wrap_tool_call(tool_request, lambda _: "executed") == "executed"
assert storage.load_calls == 1
def test_async_tool_calls_reuse_the_current_model_step_policy_decision():
restricted = _skill("restricted", ["web_search"])
storage = StorageStub([restricted])
middleware = _middleware([])
middleware._storage = lambda: storage
context = {}
state = {"skill_context": [{"path": restricted.get_container_file_path()}]}
model_request = ModelRequestStub(
[NamedTool("task"), NamedTool("web_search")],
state=state,
context=context,
)
async def go():
filtered = await middleware.awrap_model_call(model_request, lambda request: asyncio.sleep(0, result=request))
assert _tool_names(filtered) == ["web_search"]
async def execute(_):
return "executed"
results = await asyncio.gather(*(middleware.awrap_tool_call(ToolRequestStub("web_search", state=state, context=context), execute) for _ in range(3)))
assert results == ["executed", "executed", "executed"]
asyncio.run(go())
assert storage.load_calls == 1
def test_real_model_and_tool_requests_share_the_model_step_policy_decision():
restricted = _skill("restricted", ["web_search"])
storage = StorageStub([restricted])
middleware = _middleware([])
middleware._storage = lambda: storage
context = {}
state = {
"messages": [],
"skill_context": [{"path": restricted.get_container_file_path()}],
}
model_request = ModelRequest(
model=MagicMock(),
messages=[],
tools=[NamedTool("task"), NamedTool("web_search")],
state=state,
runtime=Runtime(context=context),
)
filtered = middleware.wrap_model_call(model_request, lambda request: request)
assert _tool_names(filtered) == ["web_search"]
tool_runtime = ToolRuntime(
state=state,
context=context,
config={},
stream_writer=lambda _: None,
tools=[],
tool_call_id="call-1",
store=None,
)
tool_request = ToolCallRequest(
tool_call={"name": "web_search", "args": {}, "id": "call-1", "type": "tool_call"},
tool=None,
state=state,
runtime=tool_runtime,
)
assert middleware.wrap_tool_call(tool_request, lambda _: "executed") == "executed"
assert storage.load_calls == 1
def test_policy_decision_is_json_safe_and_survives_round_trip():
restricted = _skill("restricted", ["web_search"])
storage = StorageStub([restricted])
middleware = _middleware([])
middleware._storage = lambda: storage
state = {"skill_context": [{"path": restricted.get_container_file_path()}]}
context = {}
model_request = ModelRequestStub([NamedTool("web_search")], state=state, context=context)
middleware.wrap_model_call(model_request, lambda request: request)
round_tripped = json.loads(json.dumps(context))
decision = round_tripped[SKILL_TOOL_POLICY_DECISION_CONTEXT_KEY]
assert decision["version"] == 2
assert decision["source"] == "skill_context"
tool_request = ToolRequestStub("web_search", state=state, context=round_tripped)
assert middleware.wrap_tool_call(tool_request, lambda _: "executed") == "executed"
assert storage.load_calls == 1
def test_forged_or_malformed_policy_decisions_fall_back_to_live_resolution():
restricted = _skill("restricted", ["web_search"])
malformed_decisions = [
None,
[],
{"version": 999, "owner_token": "forged", "active_paths": [restricted.get_container_file_path()], "allowed_names": ["task"]},
{"version": True, "owner_token": "forged", "active_paths": [restricted.get_container_file_path()], "allowed_names": ["task"]},
{"version": 2, "owner_token": "forged", "source": "skill_context", "active_paths": [restricted.get_container_file_path()], "allowed_names": ["task"]},
{"version": 2, "owner_token": "forged", "active_paths": [restricted.get_container_file_path()], "allowed_names": ["task"]},
{"version": 2, "owner_token": "forged", "source": "unknown", "active_paths": [restricted.get_container_file_path()], "allowed_names": ["task"]},
{"version": 2, "owner_token": "forged", "source": "skill_context", "active_paths": "not-a-list", "allowed_names": ["task"]},
]
for decision in malformed_decisions:
storage = StorageStub([restricted])
middleware = _middleware([])
middleware._storage = lambda storage=storage: storage
context = {SKILL_TOOL_POLICY_DECISION_CONTEXT_KEY: decision}
request = ToolRequestStub(
"task",
state={"skill_context": [{"path": restricted.get_container_file_path()}]},
context=context,
)
result = middleware.wrap_tool_call(request, lambda _: "executed")
assert result.status == "error"
assert storage.load_calls == 1
def test_policy_decision_path_mismatch_falls_back_to_live_resolution():
first = _skill("first", ["web_search"])
second = _skill("second", ["bash"])
storage = StorageStub([first, second])
middleware = _middleware([])
middleware._storage = lambda: storage
context = {}
first_state = {"skill_context": [{"path": first.get_container_file_path()}]}
second_state = {"skill_context": [{"path": second.get_container_file_path()}]}
middleware.wrap_model_call(ModelRequestStub([NamedTool("web_search")], state=first_state, context=context), lambda request: request)
result = middleware.wrap_tool_call(ToolRequestStub("web_search", state=second_state, context=context), lambda _: "executed")
assert result.status == "error"
assert storage.load_calls == 2
def test_policy_decision_source_mismatch_falls_back_to_live_resolution():
restricted = _skill("restricted", ["web_search"])
storage = StorageStub([restricted])
middleware = _middleware([])
middleware._storage = lambda: storage
context = {}
state = {"skill_context": [{"path": restricted.get_container_file_path()}]}
middleware.wrap_model_call(ModelRequestStub([NamedTool("web_search")], state=state, context=context), lambda request: request)
write_slash_skill_source_path(
context,
restricted.get_container_file_path(),
owner_token=_SLASH_SOURCE_OWNER_TOKEN,
)
result = middleware.wrap_tool_call(ToolRequestStub("web_search", state=state, context=context), lambda _: "executed")
assert result == "executed"
assert storage.load_calls == 2
def test_active_paths_support_attribute_based_state():
restricted = _skill("restricted", ["web_search"])
middleware = _middleware([restricted])
state = SimpleNamespace(skill_context=[{"path": restricted.get_container_file_path()}])
request = ModelRequestStub([NamedTool("task"), NamedTool("web_search")], state=state)
assert _tool_names(middleware._filter_model_request(request)) == ["web_search"]
def test_active_paths_support_falsey_attribute_based_state():
class FalseyState:
skill_context = []
def __bool__(self):
return False
restricted = _skill("restricted", ["web_search"])
state = FalseyState()
state.skill_context = [{"path": restricted.get_container_file_path()}]
middleware = _middleware([restricted])
request = ModelRequestStub([NamedTool("task"), NamedTool("web_search")])
request.state = state
assert _tool_names(middleware._filter_model_request(request)) == ["web_search"]
def test_unknown_state_shape_is_logged_instead_of_silently_ignored(caplog):
middleware = _middleware([])
request = ModelRequestStub([NamedTool("task")], state=object())
assert _tool_names(middleware._filter_model_request(request)) == ["task"]
assert "Unsupported agent state shape" in caplog.text
def test_next_model_call_refreshes_the_policy_decision():
restricted = _skill("restricted", ["web_search"])
storage = StorageStub([restricted])
middleware = _middleware([])
middleware._storage = lambda: storage
context = {}
state = {"skill_context": [{"path": restricted.get_container_file_path()}]}
first = ModelRequestStub([NamedTool("bash"), NamedTool("web_search")], state=state, context=context)
assert _tool_names(middleware.wrap_model_call(first, lambda request: request)) == ["web_search"]
storage._skills = [_skill("restricted", ["bash"])]
second = ModelRequestStub([NamedTool("bash"), NamedTool("web_search")], state=state, context=context)
assert _tool_names(middleware.wrap_model_call(second, lambda request: request)) == ["bash"]
assert storage.load_calls == 2
def test_tool_call_without_matching_model_decision_revalidates_registry():
restricted = _skill("restricted", ["web_search"])
storage = StorageStub([restricted])
middleware = _middleware([])
middleware._storage = lambda: storage
request = ToolRequestStub(
"task",
state={"skill_context": [{"path": restricted.get_container_file_path()}]},
context={},
)
result = middleware.wrap_tool_call(request, lambda _: "executed")
assert result.status == "error"
assert storage.load_calls == 1
def test_active_policy_load_failure_fails_closed_to_framework_tools():
middleware = _middleware([])
def fail_storage():
raise RuntimeError("storage unavailable")
middleware._storage = fail_storage
request = ModelRequestStub(
[
NamedTool("task"),
NamedTool("read_file"),
NamedTool("review_skill_package"),
NamedTool("tool_search"),
NamedTool("describe_skill"),
],
state={"skill_context": [{"path": "/mnt/skills/public/restricted/SKILL.md"}]},
)
assert _tool_names(middleware._filter_model_request(request)) == [
"read_file",
"review_skill_package",
"tool_search",
"describe_skill",
]
+9
View File
@@ -11,6 +11,7 @@ from pathlib import Path
import pytest
from deerflow.skills.package_paths import is_eval_fixture_skill_md
from deerflow.skills.storage import get_or_new_skill_storage
from deerflow.skills.validation import _validate_skill_frontmatter
SKILLS_PUBLIC_DIR = Path(__file__).resolve().parents[2] / "skills" / "public"
@@ -32,3 +33,11 @@ def test_bundled_skill_frontmatter_is_valid(skill_dir: Path) -> None:
def test_skills_public_dir_has_skills() -> None:
assert BUNDLED_SKILL_DIRS, f"no SKILL.md found under {SKILLS_PUBLIC_DIR}"
def test_runtime_registry_excludes_skill_reviewer_eval_fixtures() -> None:
skills = get_or_new_skill_storage(skills_path=SKILLS_PUBLIC_DIR.parent).load_skills(enabled_only=False)
names = {skill.name for skill in skills}
assert "skill-reviewer" in names
assert all("evals/fixtures" not in skill.skill_file.as_posix() for skill in skills)
+21
View File
@@ -63,6 +63,27 @@ def test_load_skills_discovers_nested_skills_and_sets_container_paths(tmp_path:
assert team_skill.get_container_file_path() == "/mnt/skills/custom/team/helper/SKILL.md"
def test_load_skills_stops_at_skill_package_boundary(tmp_path: Path):
"""SKILL.md files inside an existing skill package are support data, not skills."""
skills_root = tmp_path / "skills"
_write_skill(skills_root / "public" / "reviewer", "reviewer", "Reviews skills")
_write_skill(
skills_root / "public" / "reviewer" / "evals" / "fixtures" / "injection",
"injection-example",
"Calibration fixture",
)
_write_skill(
skills_root / "public" / "reviewer" / "examples" / "helper",
"nested-example",
"Nested package example",
)
skills = get_or_new_skill_storage(skills_path=skills_root).load_skills(enabled_only=False)
assert {skill.name for skill in skills} == {"reviewer"}
def test_load_skills_skips_hidden_directories(tmp_path: Path):
"""Hidden directories should be excluded from recursive discovery."""
skills_root = tmp_path / "skills"
+12
View File
@@ -60,6 +60,18 @@ def test_native_scan_reports_structured_secret_finding(tmp_path: Path) -> None:
assert result["blocked"] is True
def test_native_scan_allows_eval_fixture_but_flags_other_nested_skill_markdown(tmp_path: Path) -> None:
skill_dir = tmp_path / "demo-skill"
_write_skill(skill_dir)
_write_skill(skill_dir / "evals" / "fixtures" / "calibration")
_write_skill(skill_dir / "examples" / "helper")
findings = scan_skill_dir(skill_dir)["findings"]
nested = [finding for finding in findings if finding["rule_id"] == "package-nested-skill-md"]
assert [finding["file"] for finding in nested] == ["examples/helper/SKILL.md"]
def test_secret_evidence_is_redacted_everywhere(tmp_path: Path) -> None:
token = "ghp_" + "a1B2c3D4e5F6g7H8i9J0k1L2m3N4"
skill_dir = tmp_path / "demo-skill"
+25 -23
View File
@@ -13,6 +13,8 @@ from deerflow.skills.slash import RESERVED_SLASH_SKILL_NAMES, parse_slash_skill_
from deerflow.skills.types import Skill, SkillCategory
from deerflow.utils.messages import ORIGINAL_USER_CONTENT_KEY
_SLASH_SOURCE_OWNER_TOKEN = "test-slash-source-owner"
def _make_skill(tmp_path: Path, name: str, content: str = "skill body") -> Skill:
skill_dir = tmp_path / name
@@ -117,7 +119,7 @@ def test_skill_activation_middleware_injects_hidden_human_context_for_model_call
skill = _make_skill(tmp_path, "data-analysis", content="# Data Analysis\nUse pandas.")
monkeypatch.setattr(middleware_module, "get_or_new_skill_storage", lambda **kwargs: _make_storage(tmp_path, [skill]))
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
original = HumanMessage(content="/data-analysis analyze uploads/foo.csv", id="msg-1")
request = _make_model_request([original])
captured = {}
@@ -143,7 +145,7 @@ def test_skill_activation_middleware_does_not_duplicate_existing_activation(monk
skill = _make_skill(tmp_path, "data-analysis", content="# Data Analysis\nUse pandas.")
monkeypatch.setattr(middleware_module, "get_or_new_skill_storage", lambda **kwargs: _make_storage(tmp_path, [skill]))
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
original = HumanMessage(content="/data-analysis analyze uploads/foo.csv", id="msg-1")
first_capture = {}
@@ -174,7 +176,7 @@ def test_skill_activation_middleware_does_not_duplicate_activation_separated_by_
skill = _make_skill(tmp_path, "data-analysis", content="# Data Analysis\nUse pandas.")
monkeypatch.setattr(middleware_module, "get_or_new_skill_storage", lambda **kwargs: _make_storage(tmp_path, [skill]))
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
original = HumanMessage(content="/data-analysis analyze uploads/foo.csv", id="msg-1")
first_capture = {}
@@ -202,7 +204,7 @@ def test_skill_activation_middleware_dedupes_immediately_previous_activation_wit
skill = _make_skill(tmp_path, "data-analysis", content="# Data Analysis\nUse pandas.")
monkeypatch.setattr(middleware_module, "get_or_new_skill_storage", lambda **kwargs: _make_storage(tmp_path, [skill]))
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
legacy_activation_msg = SkillActivationMiddleware._make_activation_message(
HumanMessage(content="/data-analysis analyze uploads/foo.csv"),
"existing activation context",
@@ -244,7 +246,7 @@ def test_skill_activation_middleware_activates_once_across_tool_loop(monkeypatch
journal = SimpleNamespace(record_middleware=lambda *args, **kwargs: recorded.append(kwargs))
# One run context object, shared across every model call of the turn.
runtime = SimpleNamespace(context={"__run_journal": journal})
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
user = HumanMessage(content="/data-analysis analyze uploads/foo.csv", id="msg-1")
# --- model call 1: the single activation call ---
@@ -291,7 +293,7 @@ def test_skill_activation_middleware_reactivates_on_new_user_slash_command(monke
recorded = []
journal = SimpleNamespace(record_middleware=lambda *args, **kwargs: recorded.append(kwargs))
runtime = SimpleNamespace(context={"__run_journal": journal})
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
first_msg = HumanMessage(content="/data-analysis analyze uploads/foo.csv", id="msg-1")
first_capture = {}
@@ -336,7 +338,7 @@ def test_skill_activation_middleware_activates_per_call_when_run_context_is_none
skill = _make_skill(tmp_path, "data-analysis", content="# Data Analysis\nUse pandas.")
monkeypatch.setattr(middleware_module, "get_or_new_skill_storage", lambda **kwargs: _make_storage(tmp_path, [skill]))
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
original = HumanMessage(content="/data-analysis analyze uploads/foo.csv", id="msg-1")
runtime = SimpleNamespace(context=None)
captured = {}
@@ -359,7 +361,7 @@ def test_skill_activation_middleware_async_injects_hidden_human_context_for_mode
skill = _make_skill(tmp_path, "data-analysis", content="# Data Analysis\nUse pandas.")
monkeypatch.setattr(middleware_module, "get_or_new_skill_storage", lambda **kwargs: _make_storage(tmp_path, [skill]))
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
original = HumanMessage(content="/data-analysis analyze uploads/foo.csv", id="msg-1")
request = _make_model_request([original])
captured = {}
@@ -385,7 +387,7 @@ def test_skill_activation_middleware_uses_fallback_when_task_text_is_empty(monke
skill = _make_skill(tmp_path, "data-analysis", content="# Data Analysis\nUse pandas.")
monkeypatch.setattr(middleware_module, "get_or_new_skill_storage", lambda **kwargs: _make_storage(tmp_path, [skill]))
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
original = HumanMessage(content="/data-analysis", id="msg-1")
captured = {}
@@ -404,7 +406,7 @@ def test_skill_activation_middleware_uses_original_user_content_when_uploads_are
skill = _make_skill(tmp_path, "data-analysis", content="# Data Analysis\nUse pandas.")
monkeypatch.setattr(middleware_module, "get_or_new_skill_storage", lambda **kwargs: _make_storage(tmp_path, [skill]))
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
original = HumanMessage(
content="<uploaded_files>\n- report.pdf\n</uploaded_files>\n\n/data-analysis 分析这个文档",
id="msg-1",
@@ -432,7 +434,7 @@ def test_skill_activation_middleware_activates_from_list_content(monkeypatch, tm
skill = _make_skill(tmp_path, "data-analysis", content="# Data Analysis\nUse pandas.")
monkeypatch.setattr(middleware_module, "get_or_new_skill_storage", lambda **kwargs: _make_storage(tmp_path, [skill]))
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
original = HumanMessage(content=[{"type": "text", "text": "/data-analysis analyze uploads/foo.csv"}], id="msg-1")
captured = {}
@@ -456,7 +458,7 @@ def test_skill_activation_middleware_records_activation_audit_event(monkeypatch,
recorded = []
journal = SimpleNamespace(record_middleware=lambda *args, **kwargs: recorded.append((args, kwargs)))
runtime = SimpleNamespace(context={"__run_journal": journal})
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
original = HumanMessage(content="/data-analysis analyze uploads/foo.csv", id="msg-1")
def handler(model_request: ModelRequest):
@@ -486,7 +488,7 @@ def test_skill_activation_middleware_async_records_activation_audit_event(monkey
recorded = []
journal = SimpleNamespace(record_middleware=lambda *args, **kwargs: recorded.append((args, kwargs)))
runtime = SimpleNamespace(context={"__run_journal": journal})
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
original = HumanMessage(content="/data-analysis analyze uploads/foo.csv", id="msg-1")
async def handler(model_request: ModelRequest):
@@ -509,7 +511,7 @@ def test_skill_activation_middleware_ignores_activation_audit_errors(monkeypatch
journal = SimpleNamespace(record_middleware=lambda *args, **kwargs: (_ for _ in ()).throw(RuntimeError("db down")))
runtime = SimpleNamespace(context={"__run_journal": journal})
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
original = HumanMessage(content="/data-analysis analyze uploads/foo.csv", id="msg-1")
def handler(model_request: ModelRequest):
@@ -525,7 +527,7 @@ def test_skill_activation_middleware_activates_only_latest_real_user_message(mon
skill = _make_skill(tmp_path, "data-analysis", content="# Data Analysis\nUse pandas.")
monkeypatch.setattr(middleware_module, "get_or_new_skill_storage", lambda **kwargs: _make_storage(tmp_path, [skill]))
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
old_slash = HumanMessage(content="/data-analysis old request", id="msg-1")
latest_user = HumanMessage(content="continue normally", id="msg-2")
request = _make_model_request([old_slash, AIMessage(content="done"), latest_user])
@@ -546,7 +548,7 @@ def test_skill_activation_middleware_ignores_hidden_user_messages(monkeypatch, t
skill = _make_skill(tmp_path, "data-analysis", content="# Data Analysis\nUse pandas.")
monkeypatch.setattr(middleware_module, "get_or_new_skill_storage", lambda **kwargs: _make_storage(tmp_path, [skill]))
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
real_user = HumanMessage(content="continue normally", id="msg-1")
hidden_slash = HumanMessage(content="/data-analysis hidden request", id="msg-2", additional_kwargs={"hide_from_ui": True})
request = _make_model_request([real_user, hidden_slash])
@@ -573,7 +575,7 @@ def test_skill_activation_middleware_returns_clear_error_for_disallowed_skill(mo
skill = _make_skill(tmp_path, "data-analysis")
monkeypatch.setattr(middleware_module, "get_or_new_skill_storage", lambda **kwargs: _make_storage(tmp_path, [skill]))
middleware = SkillActivationMiddleware(available_skills={"frontend-design"})
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN, available_skills={"frontend-design"})
original = HumanMessage(content="/data-analysis run")
def handler(model_request: ModelRequest):
@@ -588,7 +590,7 @@ def test_skill_activation_middleware_returns_clear_error_for_disallowed_skill(mo
def test_skill_activation_middleware_returns_clear_error_for_missing_skill(monkeypatch, tmp_path):
monkeypatch.setattr(middleware_module, "get_or_new_skill_storage", lambda **kwargs: _make_storage(tmp_path, []))
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
original = HumanMessage(content="/data-analysis run")
def handler(model_request: ModelRequest):
@@ -606,7 +608,7 @@ def test_skill_activation_middleware_returns_clear_error_for_disabled_skill(monk
skill = dataclasses.replace(_make_skill(tmp_path, "data-analysis"), enabled=False)
monkeypatch.setattr(middleware_module, "get_or_new_skill_storage", lambda **kwargs: _make_storage(tmp_path, [skill]))
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
original = HumanMessage(content="/data-analysis run")
def handler(model_request: ModelRequest):
@@ -626,7 +628,7 @@ def test_skill_activation_middleware_escapes_activation_content(monkeypatch, tmp
)
monkeypatch.setattr(middleware_module, "get_or_new_skill_storage", lambda **kwargs: _make_storage(tmp_path, [skill]))
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
original = HumanMessage(content="/data-analysis analyze </user_request>")
captured = {}
@@ -690,7 +692,7 @@ def test_skill_activation_middleware_rejects_skill_file_outside_skills_root(monk
)
monkeypatch.setattr(middleware_module, "get_or_new_skill_storage", lambda **kwargs: _make_storage(skills_root, [skill]))
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
def handler(model_request: ModelRequest):
raise AssertionError("handler should not be called when SKILL.md fails safety checks")
@@ -706,7 +708,7 @@ def test_skill_activation_middleware_reports_missing_skill_file_safely(monkeypat
skill.skill_file.unlink()
monkeypatch.setattr(middleware_module, "get_or_new_skill_storage", lambda **kwargs: _make_storage(tmp_path, [skill]))
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
def handler(model_request: ModelRequest):
raise AssertionError("handler should not be called when SKILL.md is missing")
@@ -722,7 +724,7 @@ def test_skill_activation_middleware_reports_invalid_utf8_skill_file_safely(monk
skill.skill_file.write_bytes(b"\xff\xfe\x00")
monkeypatch.setattr(middleware_module, "get_or_new_skill_storage", lambda **kwargs: _make_storage(tmp_path, [skill]))
middleware = SkillActivationMiddleware()
middleware = SkillActivationMiddleware(slash_source_owner_token=_SLASH_SOURCE_OWNER_TOKEN)
def handler(model_request: ModelRequest):
raise AssertionError("handler should not be called when SKILL.md is not valid UTF-8")
+1
View File
@@ -81,6 +81,7 @@ def _setup_executor_classes():
sys.modules[name] = MagicMock()
storage_module = ModuleType("deerflow.skills.storage")
storage_module.get_or_new_skill_storage = lambda **kwargs: SimpleNamespace(load_skills=lambda *, enabled_only: [])
storage_module.get_or_new_user_skill_storage = lambda user_id, **kwargs: SimpleNamespace(load_skills=lambda *, enabled_only: [])
sys.modules["deerflow.skills.storage"] = storage_module
# Import real classes inside fixture
@@ -136,6 +136,17 @@ class TestSkillLoading:
assert len(public_skills) == 1
assert public_skills[0].name == "deep-research"
def test_public_skill_package_children_are_not_registered(self, user_storage: UserScopedSkillStorage, skills_root: Path):
public_dir = skills_root / "public" / "reviewer"
fixture_dir = public_dir / "evals" / "fixtures" / "injection"
fixture_dir.mkdir(parents=True)
(public_dir / "SKILL.md").write_text(_skill_content("reviewer"), encoding="utf-8")
(fixture_dir / "SKILL.md").write_text(_skill_content("injection-example"), encoding="utf-8")
names = {skill.name for skill in user_storage.load_skills(enabled_only=False)}
assert names == {"reviewer"}
def test_custom_skills_loaded_from_user_dir(self, user_storage: UserScopedSkillStorage, base_dir: Path):
user_storage.write_custom_skill("my-skill", "SKILL.md", _skill_content("my-skill"))