docs(release): bound release recovery loops (#106852)

* docs(release): keep release work scoped

* style(release): format workflow test
This commit is contained in:
Vincent Koc
2026-07-14 06:44:23 +08:00
committed by GitHub
parent 0f9cc945e3
commit 85cf09a3e1
9 changed files with 213 additions and 45 deletions
+46 -6
View File
@@ -10,6 +10,16 @@ Use this with `$release-openclaw-maintainer` and `$openclaw-testing` when a rele
## Guardrails
- No version bump, tag, npm publish, GitHub release, or release promotion without explicit operator approval.
- Hold the release scope once a release branch or Code SHA exists. Validate and
ship that exact release; do not turn moving `main` into a second work queue.
- Use trusted `main` workflow revisions as immutable dispatch sources. Do not
adopt newer main code, repair unrelated main CI, wait for broad main health,
or expand a release fix because the workflow source lives on `main`.
- Touch `main` only for an operator-requested change or the smallest critical
main-owned blocker that prevents this release and cannot be handled from the
release branch. If the required main landing policy is blocked by unrelated
main failures, report that blocker and keep independent release work moving
instead of healing broader main.
- Validate provider secrets before dispatching expensive full release matrices.
- Do not set GitHub secrets from unvalidated 1Password candidates. If a candidate returns 401/403, leave the existing secret alone and report the exact missing provider.
- Use `$one-password` for secret reads/writes: one persistent tmux session, targeted items only, no secret output.
@@ -52,6 +62,30 @@ Use this with `$release-openclaw-maintainer` and `$openclaw-testing` when a rele
on source sync to overlay committed branch changes onto the workflow's
default ref.
## Run identity and retry budget
Record the target SHA, target ref, parent run id, attempt, and effective
workflow SHA before watching or recovering Full Release Validation.
- One target SHA has one active canonical `rerun_group=all` parent run by
default.
- Use GitHub's failed-job rerun on the same parent when its original inputs
still select the correct work.
- A new parent for the same target SHA is allowed only when no usable parent
exists, the existing run cannot consume a required trusted-workflow fix, its
evidence identity is invalid, or the operator explicitly requests a fresh
run. The replacement must also use `rerun_group=all`; record which terminal
parent it supersedes and why.
- A narrow `rerun_group` dispatch is supplemental diagnostic proof only. It
never supersedes the canonical parent and cannot satisfy publish evidence.
Run it only after the canonical parent is terminal, then obtain green
`rerun_group=all` evidence before publish.
- Never keep two parents active for the same SHA. Cancel only the superseded
task-owned run after the replacement is identified.
- After two unchanged retries of the same failure, stop repeating it. Recheck
classification and report one precise blocker or the missing evidence needed
for a different action.
## Preflight
Before full release validation:
@@ -102,9 +136,10 @@ Prefer an immutable trusted-main workflow revision, target the exact Code SHA:
- Keep trusted-workflow checks compatible with frozen release targets. If
`main` adds a target-owned guard script or package command after the release
branch cut, make the trusted workflow skip only when that target surface is
absent. Heal the trusted workflow before rerunning validation; do not port an
unrelated runtime refactor or mutate the release candidate just to satisfy a
newer `main`-only check.
absent. Repair the smallest trusted-workflow compatibility issue only when it
blocks the release, then rerun validation. Do not port an unrelated runtime
refactor, heal other main failures, or mutate the release candidate just to
satisfy a newer `main`-only check.
```bash
node scripts/full-release-validation-at-sha.mjs \
@@ -168,8 +203,12 @@ Stop watchers before ending the turn or switching strategy.
6. Classify before editing:
- product/code failure: fix the release branch, freeze a new Code SHA, run
focused proof, then obtain green full validation for that new SHA
- workflow/harness/infrastructure/credential failure: fix that owner
separately and rerun the same Code SHA
- workflow/harness/infrastructure/credential failure: fix the smallest
owning surface and rerun failed jobs on the current parent when its inputs
still select the correct work; otherwise supersede the terminal parent once
with a new `rerun_group=all` parent on the required trusted-workflow
revision. Keep the same Code SHA; touching `main` still requires the active
release scope lock
- changelog/release-note failure: change only `CHANGELOG.md`, keep Code SHA
evidence, and repeat Release SHA proof
- publish child/registry selector failure: keep Release SHA and resume the
@@ -201,7 +240,8 @@ Record:
- Code SHA and Release SHA
- evidence-reuse policy and complete changed-path set
- full parent run URL
- active full parent run URL, attempt, workflow SHA, and any superseded parent
with the exact replacement reason
- child run IDs and conclusions: CI, Release Checks, Plugin Prerelease, NPM Telegram, Product Performance
- performance comparison result versus earlier releases when available
- targeted local proof commands
@@ -1,4 +1,4 @@
interface:
display_name: "OpenClaw Release CI"
short_description: "Verify and debug OpenClaw release validation runs"
default_prompt: "Use $release-openclaw-ci to preflight provider secrets, watch full release validation, summarize child runs, and triage only failing release lanes."
default_prompt: "Use $release-openclaw-ci to validate one frozen release SHA with one active parent run, recover only the smallest failed lane, and stop on a precise external blocker."
@@ -6,6 +6,8 @@
- GitHub secret presence was confused with key validity.
- Repeated `gh run view` and log fetches exhausted REST quota.
- Parent run state was less useful than child run evidence.
- Replacement parent runs were dispatched while an existing parent was still
recoverable, multiplying polling, cancellation, and identity checks.
- Live-cache failures needed structured classification: invalid key, empty provider output, timeout, or real cache regression.
- Background watchers accumulated and made interruption recovery harder.
@@ -14,7 +16,16 @@
- Run provider-secret preflight first. Require real `/models` or equivalent endpoint checks for release-blocking providers.
- Keep one watcher open. Use child summaries every few minutes, not every few seconds.
- Fetch failed-job logs only after a job reaches a terminal failing state.
- Prefer narrow `rerun_group` recovery after a focused fix.
- Prefer same-parent failed-job reruns when the original inputs still select the
right work.
- Keep one active parent per target SHA. Create a replacement only when the
existing parent cannot consume a required workflow fix or its evidence
identity is invalid; record the superseded run and reason. The replacement
remains `rerun_group=all`.
- Treat narrow `rerun_group` dispatches as supplemental diagnostics only; they
do not satisfy publish evidence.
- Stop after two unchanged retries and report the exact blocker instead of
creating another verification loop.
- Leave bad secrets unset. A 401 candidate from 1Password should not overwrite GitHub.
- Make the final release evidence note durable: parent URL, child run URLs, SHA, command proof, and gaps.
@@ -64,6 +64,11 @@ Do not update these from mixed sources. All three ASC fields must come from the
## Dispatch
The public handoff workflow validates the tag, source, build, and package
metadata before publication. It does not require a GitHub release page because
it does not upload assets. Keep this validation before the real publish
workflow; the publisher owns draft creation and final undraft.
Public handoff validation:
```bash
@@ -11,6 +11,25 @@ approved backport set for an extended-stable maintenance release. Load
locators or private host topology. Keep ordinary development changes and
GHSA-specific advisory work outside this skill.
## Freeze the release state
Before validation or publication, write one compact state record and keep it
current:
- goal and terminal success criteria
- release version, tag, branch, cut SHA, Code SHA, and Release SHA
- active Full Release Validation parent run id and attempt
- npm preflight and publish parent run ids
- completed phases and immutable child artifacts
- approved backports or main changes
- current phase, next action, and one precise blocker if stopped
Use `references/release-handoff-template.md` when starting a release session,
recovering after compaction, or handing the release to another operator.
Completed phases stay complete. Reopen one only when a named event invalidates
its evidence, such as a Code SHA change, a non-changelog Release SHA change, or
a workflow fix that the existing parent run cannot consume.
## Respect release guardrails
- Do not change version numbers without explicit operator approval.
@@ -31,23 +50,44 @@ GHSA-specific advisory work outside this skill.
user-facing documentation contract changed.
- Normal release work happens on a branch cut from `main`, not directly on
`main`. Use `release/YYYY.M.PATCH` for the branch name.
- Hold release scope from cut-SHA selection through publish and verification.
The active release is the work queue; moving `main` is only a trusted
workflow and provenance source unless the operator explicitly requests main
work.
- Touch `main` during the active release only for an operator-requested change
or a critical main-owned blocker that prevents this release and cannot be
fixed or proven from the release branch. Examples include a live installer or
trusted workflow that is sourced only from `main`. A red unrelated `main`
check, baseline, refactor, cleanup, or later improvement is not a release
blocker.
- Do not broaden a release-critical main fix to make moving `main` green. Keep
the change to the exact blocker, run focused proof, follow the required main
landing policy, then return immediately to the release branch. If unrelated
main health blocks that landing, report the blocker and continue independent
release work instead of adopting the failure.
- Defer normal forward-ports and main closeout until after publication.
Forward-port before publish only when the operator requests it or `main`
itself owns the exact release-critical runtime or workflow surface.
- If the operator asks for a release without saying stable/full, default to
beta only. Continue from beta to stable only when the operator explicitly asks
for the full release or an automated beta-and-stable train.
- Before release branching, pull latest `main` and confirm current `main` CI is
green. Then branch from that commit so regular development can continue on
`main` while release validation runs.
- Before release branching, commit any dirty files in coherent groups, push,
pull/rebase, and create the release branch from the intended `main` head.
- Resolve the intended cut SHA once. If the operator supplies a SHA, use it
exactly; do not pull, rebase, or advance it to newer `main`. Otherwise fetch
`origin/main` once and record the selected full SHA plus its CI state. A red
unrelated main check does not authorize healing main.
- Create a clean release worktree and `release/YYYY.M.PATCH` from that selected
SHA. Do not commit or absorb unrelated dirty files as release preparation.
Finish version preparation plus any operator-selected backports,
release-only fixes, and required forward-ports. Backports are optional.
release-only fixes, and explicitly required pre-publish main changes.
Backports are optional.
Freeze this product-complete tree as the **Code SHA** without changing the
release changelog.
- Full product validation belongs to the Code SHA. If validation finds a code
defect, fix it, freeze a new Code SHA, and validate that SHA. If the failure
belongs to trusted workflow tooling, the harness, credentials, or
infrastructure, repair that surface separately and rerun against the same
Code SHA. Never mutate the release candidate to satisfy newer tooling.
infrastructure, repair the smallest owning surface and rerun against the same
Code SHA. Touch `main` only under the active release scope lock above. Never
mutate the release candidate to satisfy newer tooling or heal unrelated main.
- Generate `CHANGELOG.md` only after the Code SHA is green. The resulting
**Release SHA** must be a descendant whose complete diff from the Code SHA is
exactly `CHANGELOG.md`. Release-note checks, npm preflight/package bytes,
@@ -86,10 +126,11 @@ GHSA-specific advisory work outside this skill.
against its exact bytes. If a product defect appears, return to a new Code
SHA; if a release-tooling or publication child fails, repair/resume that child
without changing the candidate. After a published beta needs a code fix,
forward-port it, increment the beta number, and repeat. A post-beta scan of
current `main` may suggest optional low-risk backports; it does not require
them. Operators may authorize up to 4 autonomous beta attempts; after 4
failed beta attempts, stop and report.
increment the beta number and repeat. Defer its forward-port until after
publication unless the operator requests it. Do not scan moving `main` for
extra fixes during an active release unless the operator explicitly asks for
that audit. Operators may authorize up to 4 autonomous beta attempts; after
4 failed beta attempts, stop and report.
- As soon as the Code SHA exists, dispatch `OpenClaw Performance`
with `target_ref=<code-sha>` in parallel with the other release work. Do
not wait for full release validation to start the performance signal.
@@ -99,11 +140,13 @@ GHSA-specific advisory work outside this skill.
or clawgrit reports. Report regressions explicitly. A major regression is a
release blocker unless the operator waives it or the data clearly proves
infrastructure noise.
- Heal CI before changelog, tagging, or publishing. The exact Code SHA must have
green `Full Release Validation`, including the root Dockerfile/install-smoke path.
- Heal release-owned CI before changelog, tagging, or publishing. The exact
Code SHA must have green `Full Release Validation`, including the root
Dockerfile/install-smoke path.
Treat a red Docker, package, or release workflow lane as a release-branch
defect until the smallest correct fix is landed and proven; do not waive it
because npm preflight or another sibling lane passed.
because npm preflight or another sibling lane passed. Unrelated moving-main
failures are not part of this gate.
- Keep the canonical `scripts/pr` runner authoritative for prepare and merge
artifacts. A release-gate policy change may use focused candidate tests and
exact-SHA hosted CI for proof, but never route `prepare-*` or `merge-*`
@@ -233,7 +276,9 @@ on pinned current `main` as the exact command and validation contract.
## Close stable releases on main
Stable publication is not complete until `main` carries the actual shipped release state.
This gate starts only after stable publication. It is a narrow shipped-state
closeout, not permission to heal broader `main`. Stable publication is not
complete until `main` carries the actual shipped release state.
1. Start from fresh latest `main`. Audit `release/YYYY.M.PATCH` against it and
forward-port real fixes that are absent from `main`. Do not blindly merge
@@ -567,9 +612,10 @@ pnpm test:install:smoke
or packed-artifact failures to postpublish verification.
- Before generating the changelog, require green CI for the exact Code SHA, not
an earlier branch SHA. Heal every related red CI, release-check, packaging,
or root-Dockerfile lane on the release branch, forward-port product fixes to
`main`, and rerun the affected exact-SHA gates. Never waive a red Docker lane
because npm preflight passed.
or root-Dockerfile lane on the release branch and rerun the affected exact-SHA
gates. Defer product-fix forward-ports until after publication unless the
active release scope lock requires an exact main-owned blocker fix. Never
waive a red Docker lane because npm preflight passed.
- Root Dockerfile proof is mandatory on the Code SHA before every beta and
stable tag. The changelog-only Release SHA reuses that product proof, while
exact Release SHA npm preflight and package/install acceptance prove the
@@ -921,14 +967,16 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts <published-version>
publish, or complete the named release as continuing authorization through
publish and verification. Reconfirm only if the release identity, channel,
publish scope, or material risk changes.
2. Choose the exact target version and git tag.
3. Commit any dirty files in coherent groups, push, pull/rebase, and verify the
worktree is clean.
4. Pull latest `main` and confirm current `main` CI is green.
5. Create `release/YYYY.M.PATCH` from the intended `main` commit.
2. Choose the exact target version, git tag, and npm dist-tag (`beta` or
`latest`).
3. Start from a clean release worktree. Do not absorb unrelated dirty files.
4. Resolve the cut SHA once. Use an operator-supplied SHA exactly; otherwise
fetch `origin/main` once, record its full SHA and CI state, and do not chase a
newer moving head.
5. Create `release/YYYY.M.PATCH` from that selected commit.
6. Make every repo version location match the beta tag. Apply only explicitly
selected backports or release fixes, and forward-port fixes to `main` where
required. Freeze the result as the Code SHA.
selected backports or release fixes. Make a pre-publish main change only
under the active release scope lock. Freeze the result as the Code SHA.
7. Immediately dispatch Actions > `OpenClaw Performance` from the pinned
trusted workflow source with `target_ref=<code-sha>`, `profile=release`,
`repeat=3`, deep profiling
@@ -1013,11 +1061,10 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts <published-version>
with the original child run IDs and an evidence output path before manually
recreating the workflow's draft, dependency evidence asset, proof section,
and publish step.
27. Run the post-published beta verification roster. First scan current `main`
for critical fixes that landed after the release branch cut; backport only
operator-selected low-risk fixes before starting expensive lanes, or
increment to the next beta if the fix must change the already-published
package. If any
27. Run the post-published beta verification roster. Do not scan current `main`
for extra fixes unless the operator explicitly requests a backport audit.
Apply only operator-selected backports, and increment to the next beta if a
selected fix must change the already-published package. If any
lane fails after the beta package is published, fix, commit/push/pull,
increment to the next beta tag, and rerun the affected beta evidence. Once
the beta is live, start remote/manual rosters where they
@@ -0,0 +1,52 @@
# Release Handoff
Use this compact record to start or resume one release session. Replace every
placeholder with current live state. Omit completed detail that is already
captured by a durable run or artifact URL.
## Goal
Ship `<version>` on `<channel>` and stop when `<terminal success criteria>`.
## Immutable state
- branch: `release/<YYYY.M.PATCH>`
- cut SHA: `<full sha>`
- Code SHA: `<full sha or not frozen>`
- Release SHA: `<full sha or not frozen>`
- tag: `v<version>`
- approved backports: `<none or exact PRs/commits>`
- approved main changes: `<none or exact blocker>`
## Active evidence
- Full Release Validation parent: `<run id / attempt / URL or none>`
- npm preflight: `<run id / URL or none>`
- publish parent: `<run id / URL or none>`
- immutable successful children: `<run ids / artifacts or none>`
## Phase
- completed: `<phases that stay complete>`
- current: `<one phase>`
- next action: `<one concrete action>`
## Failure policy
- product/code failure: fix the release branch, freeze a new Code SHA, and
invalidate downstream product evidence
- changelog-only failure: change only `CHANGELOG.md`, freeze a new Release SHA,
and reuse green Code SHA evidence after delta proof
- workflow/tooling/credential failure: keep the candidate frozen and recover
the smallest owning surface
- external approval or permission blocker: stop with the exact job, URL,
missing permission, and required operator action
Do not scan moving `main`, add optional backports, dispatch a replacement
validation parent, or repeat completed phases unless a named invalidating event
requires it.
## Stop conditions
- success: `<exact published and verified state>`
- blocked: `<one precise external action that only the operator can complete>`
-6
View File
@@ -54,12 +54,6 @@ jobs:
node-version: ${{ env.NODE_VERSION }}
install-bun: "false"
- name: Ensure matching GitHub release exists
env:
GH_TOKEN: ${{ github.token }}
RELEASE_TAG: ${{ inputs.tag }}
run: gh release view "$RELEASE_TAG" --repo "$GITHUB_REPOSITORY" >/dev/null
- name: Build
run: pnpm build
+7
View File
@@ -300,6 +300,13 @@ Skills own workflows; root owns hard policy and routing.
- Lockfiles/shrinkwrap are security surface: review `pnpm-lock.yaml`, `npm-shrinkwrap.json`, `package-lock.json`; root/plugin npm packages ship shrinkwrap, not package-lock.
- Carbon pins owner-only: do not change `@buape/carbon` unless Shadow (`@thewilloftheshadow`, verified by `gh`) asks.
- Releases/publish/version bumps need explicit approval. Use `$release-openclaw-maintainer`.
- Active release scope lock: freeze the operator-selected cut SHA and release
identity through publish and verification. Moving `main`, unrelated CI,
optional backports, refactors, cleanup, and normal forward-ports are not part
of the release work queue.
- Touch `main` during a release only when the operator requests it or the
smallest critical main-owned blocker prevents that release. Return to the
release branch immediately; defer broader main work until closeout.
- Release versions use `YYYY.M.PATCH`, where `PATCH` is a sequential monthly release-train number, never the calendar day. Stable and beta tags determine the current train; alpha-only tags do not consume or advance the beta/stable patch number. After `2026.6.5`, the next beta train is `2026.6.6-beta.1` even if higher alpha-only tags exist.
- Alpha/nightly versions use the next unreleased train plus an incrementing prerelease number. Repeated nightlies for the same train increment only `alpha.N`; they must not mint a new patch number from the date.
- Backports are optional. Apply only the operator-selected set; when requested without a target, use the newest open `release/` branch.
@@ -3949,6 +3949,18 @@ wait_for_run plugin-clawhub-new.yml 123 "${expectedSha}" || status=$?
expect(readFileSync(LIVE_E2E_WORKFLOW, "utf8")).toContain("live-cache attempt ${attempt}/2");
});
it("validates the macOS release handoff before the GitHub release page exists", () => {
const macosRelease = readWorkflow(".github/workflows/macos-release.yml");
const validateJob = workflowJob(
".github/workflows/macos-release.yml",
"validate_macos_release_request",
);
const stepNames = validateJob.steps?.map((step) => step.name) ?? [];
expect(stepNames).not.toContain("Ensure matching GitHub release exists");
expect(macosRelease.jobs?.validate_macos_release_request).toBeDefined();
});
it("keeps every tracked repository skill visible to Git-aware syncs", () => {
const gitignore = readFileSync(".gitignore", "utf8");
const skillFiles = execFileSync("git", ["ls-files", ".agents/skills/*/SKILL.md"], {