mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-07-21 02:05:45 +00:00
* feat(skills): add skill review quality gate * fix(skills): skip review eval fixtures in CI * fix(skills): ignore review eval fixtures in bundled scans * fix(skill-review): harden review gate boundaries * fix(skills): address skill review gate feedback
1.8 KiB
1.8 KiB
Skill Review Checklist
Use this checklist to keep reviews concrete and repeatable.
Deterministic Facts
- Root
SKILL.mdexists, is UTF-8 text, and has valid YAML frontmatter. nameanddescriptionare non-empty and valid for the selected profile.allowed-tools,required-secrets, andsecrets-autonomousare structurally valid.- Package digest is present.
SkillScanfindings are visible and severity is preserved.- Reader errors, truncation, and analyzer errors are reported as limitations.
Trigger Boundary
- Description says what the skill does.
- Description says when to invoke it.
- Description does not claim broad ownership over adjacent tasks.
- Sibling collision cases are named when relevant.
- Suggested replacement text is concise enough for catalog display.
Instructions
- Inputs are named.
- Ordered actions are explicit.
- Branches and fallback behavior are clear.
- Stop conditions are clear.
- The skill tells the agent what not to do when that matters.
Resources And Scripts
- Required references are reachable from
SKILL.md. - Unreferenced files are either intentional or removed.
- Scripts have documented inputs and outputs.
- Scripts are not required unless the instructions say when to run them.
- Templates and assets have read-when guidance.
Safety
- Side effects are named.
- Destructive or externally visible actions require confirmation.
- Secrets are declared rather than hardcoded.
- Network access is justified.
- Retry and idempotency behavior is bounded where relevant.
Evidence
- Trigger evals include positive and negative cases when routing is fuzzy.
- Behavior evals tie outputs to the reviewed digest.
- Baselines are frozen before claiming improvement.
- Runtime, model, prompts, outputs, and graders are retained for verified claims.