fix: harden live updater snapshot recovery (#105007)

* fix(updater): harden snapshot recovery

* fix(updater): avoid snapshot code execution

.agents/skills/openclaw-live-updater/SKILL.md

@@ -31,6 +31,7 @@ Keep `/Users/steipete/openclaw` a read-only-to-the-agent deployment mirror: clea
    - Missing, invalid, or stale build output also forces a build, even when Git did not move.
    - A dependency-input change, absent `node_modules`, or missing/invalid build provenance requires `pnpm install --frozen-lockfile`. When a build is required, do not install before acquiring the maintenance suspension and stopping the managed Gateway.
    - Before mutating any entrypoint currently executed by the Gateway, invoke an exact trusted source CLI to acquire `gateway.suspend.prepare`, binding both prepare and resume to this checkout's managed LaunchAgent loopback port and service auth even when normal CLI configuration points at a remote Gateway. The LaunchAgent may execute either this checkout's `dist/index.js` or a clean detached canonical snapshot under `~/.openclaw/runtime/gateway-<sha>` whose commit is an ancestor of the checkout; reject every other entrypoint. Never execute snapshot code: capture an exact source control build before the Git fast-forward for its prepare and failure-resume calls, preserve any validated generated service-environment wrapper, and stop the managed LaunchAgent with native launchd bootout. If that control build is missing, first accept native proof that the snapshot job is already booted out with its port free; when the isolated snapshot is still running, build the verified clean source checkout to obtain an exact suspension client, then use only that source client for prepare and failure-resume. Never use this recovery build while launchd targets the source checkout. This atomically pauses cron scheduling, closes new work admission, and refuses while active work remains. A busy result defers further mutation to the next heartbeat; never replace this fence with `cron list` polling. Once ready, stop directly without a source launcher, install frozen dependencies when required, then build unless the exact recovery build already produced the deployment artifact; source launchers can auto-build stale output before dispatching the stop. Resume the suspension if stop fails. If suspension RPC is unavailable on macOS, proceed only when native inspection proves this checkout's managed LaunchAgent is booted out and its configured port has no listener; never accept a loaded KeepAlive job's transient stopped state. On other platforms, require the existing CLI to prove the managed service is stopped with no PID, listener, or RPC. This preserves retry after a post-stop failure without weakening the live-work fence. Preserve `dist/OpenClaw.app` outside `dist` for the build and restore it even when the build fails, because the JS build cleans `dist` regardless of Mac impact classification. Never mutate the live `dist` tree while an old Gateway can dynamically import from it. `pnpm build` must leave both canonical stamp heads and `dist/build-info.json.commit` equal to post-update `afterSha`; any missing/mismatched stamp or required artifact blocks restart.
+   - Snapshot ownership validation must never invoke Git inside the snapshot. Treat its worktree, local Git configuration, filters, hooks, attributes, and build artifacts as untrusted; prove only that the regular owned LaunchAgent entrypoint is under the canonical detached ancestor snapshot path. Snapshot validation authorizes native bootout and retargeting only. Every CLI path must reject snapshot execution and use the trusted source checkout build instead.
    - Only after exact-SHA build proof may it restart the managed Gateway and require `gateway status --deep --require-rpc --json` plus `health --verbose --json`. A validated ancestor snapshot is suspension-only: prove the old launchd job is booted out with its port free, then atomically retarget only the owned LaunchAgent entrypoint to this checkout's exact `dist/index.js`, including on a retry where the build is already current. Preserve all other service arguments and environment unchanged. After restart, prove the loaded launchd PID owns the configured listener.
    - After every managed restart, query Gateway logs through RPC, restrict the audit to entries emitted since that restart began, report warning summaries, and fail the pass on any error/fatal entry. If RPC verification or log retrieval fails, still inspect the local structured log for that restart window. Never accept supervisor or RPC health without this restart-window log audit.
 

.agents/skills/openclaw-live-updater/scripts/update-main.mjs

@@ -123,6 +123,15 @@ function commitExists(checkout, sha) {
   }
 }
 
+function isAncestorCommit(checkout, ancestor, descendant = "HEAD") {
+  try {
+    git(checkout, ["merge-base", "--is-ancestor", ancestor, descendant]);
+    return true;
+  } catch {
+    return false;
+  }
+}
+
 export function classifyActions(
   changedPaths,
   { buildProvenanceKnown, buildRequired, nodeModulesPresent },
@@ -530,6 +539,54 @@ function defaultRunCommand(command, args, checkout) {
   });
 }
 
+function readSnapshotMetadata(snapshotRoot) {
+  const gitDir = path.join(snapshotRoot, ".git");
+  const headPath = path.join(gitDir, "HEAD");
+  const configPath = path.join(gitDir, "config");
+  const gitDirStat = lstatSync(gitDir);
+  const headStat = lstatSync(headPath);
+  const configStat = lstatSync(configPath);
+  const owner = typeof process.getuid === "function" ? process.getuid() : null;
+  const isTrustedMetadataFile = (filePath, fileStat) =>
+    fileStat.isFile() &&
+    !fileStat.isSymbolicLink() &&
+    (owner === null || fileStat.uid === owner) &&
+    (fileStat.mode & 0o022) === 0 &&
+    realpathSync(filePath) === filePath;
+  if (
+    !gitDirStat.isDirectory() ||
+    gitDirStat.isSymbolicLink() ||
+    realpathSync(gitDir) !== gitDir ||
+    !isTrustedMetadataFile(headPath, headStat) ||
+    !isTrustedMetadataFile(configPath, configStat)
+  ) {
+    return null;
+  }
+
+  const head = readFileSync(headPath, "utf8").trim().toLowerCase();
+  if (!FULL_SHA_RE.test(head)) {
+    return null;
+  }
+  let inOrigin = false;
+  let originUrl = null;
+  for (const rawLine of readFileSync(configPath, "utf8").split("\n")) {
+    const line = rawLine.trim();
+    if (line.startsWith("[") && line.endsWith("]")) {
+      inOrigin = /^\[remote\s+"origin"\]$/u.test(line);
+      continue;
+    }
+    if (!inOrigin) {
+      continue;
+    }
+    const urlMatch = line.match(/^url\s*=\s*(.+)$/u);
+    if (urlMatch) {
+      originUrl = urlMatch[1].trim().replace(/^"(.*)"$/u, "$1");
+      break;
+    }
+  }
+  return originUrl ? { head, originUrl } : null;
+}
+
 export function isOwnedGatewayEntrypoint(checkout, home, entrypoint) {
   const sourceEntrypoint = path.join(checkout, "dist/index.js");
   if (entrypoint === sourceEntrypoint) {
@@ -549,30 +606,30 @@ export function isOwnedGatewayEntrypoint(checkout, home, entrypoint) {
   }
 
   try {
+    const metadata = readSnapshotMetadata(snapshotRoot);
+    const entrypointStat = lstatSync(entrypoint);
+    const owner = typeof process.getuid === "function" ? process.getuid() : null;
     if (
       realpathSync(snapshotRoot) !== snapshotRoot ||
       realpathSync(entrypoint) !== entrypoint ||
-      !originMatches(gitText(snapshotRoot, ["remote", "get-url", "origin"])) ||
-      gitText(snapshotRoot, ["status", "--porcelain"]) !== ""
+      !entrypointStat.isFile() ||
+      entrypointStat.isSymbolicLink() ||
+      (owner !== null && entrypointStat.uid !== owner) ||
+      (entrypointStat.mode & 0o022) !== 0 ||
+      !metadata ||
+      !originMatches(metadata.originUrl)
     ) {
       return false;
     }
-    try {
-      git(snapshotRoot, ["symbolic-ref", "-q", "HEAD"]);
-      return false;
-    } catch {
-      // Immutable runtime snapshots are detached from every mutable branch.
-    }
-    const snapshotHead = gitText(snapshotRoot, ["rev-parse", "HEAD"]);
+    const snapshotHead = metadata.head;
     if (
       !FULL_SHA_RE.test(snapshotHead) ||
       snapshotName !== `gateway-${snapshotHead.slice(0, 7)}` ||
       !commitExists(checkout, snapshotHead) ||
-      !inspectBuildState(snapshotRoot, snapshotHead).current
+      !isAncestorCommit(checkout, snapshotHead)
     ) {
       return false;
     }
-    git(checkout, ["merge-base", "--is-ancestor", snapshotHead, "HEAD"]);
     return true;
   } catch {
     return false;
@@ -926,7 +983,24 @@ export function parseLaunchctlArguments(output) {
 }
 
 function runBuiltGatewayCli(checkout, args, deployment) {
-  const managedDeployment = deployment ?? readManagedGatewayLaunchAgent(checkout);
+  const observedDeployment = deployment ?? readManagedGatewayLaunchAgent(checkout);
+  const sourceEntrypoint = path.join(checkout, "dist/index.js");
+  let managedDeployment = observedDeployment;
+  if (observedDeployment.entrypoint !== sourceEntrypoint) {
+    const currentHead = gitText(checkout, ["rev-parse", "HEAD"]);
+    managedDeployment = resolveGatewayControlDeployment(
+      checkout,
+      observedDeployment,
+      inspectBuildState(checkout, currentHead),
+      currentHead,
+    );
+    if (!managedDeployment) {
+      throw new UpdateInvariantError(
+        "gateway_snapshot_control_unavailable",
+        "refusing to execute a managed Gateway runtime snapshot without a trusted source control build",
+      );
+    }
+  }
   const {
     configPath,
     entrypoint,
@@ -1326,6 +1400,15 @@ function restartGateway(
   return startedAtMs;
 }
 
+function isManagedGatewayLoaded(deployment) {
+  const result = spawnSync(
+    "/bin/launchctl",
+    ["print", `gui/${process.getuid()}/${deployment.label}`],
+    { encoding: "utf8" },
+  );
+  return result.status === 0;
+}
+
 function verifyGateway(runCommand, checkout, expectedSha, deployment = null) {
   assertExactBuild(checkout, expectedSha);
   if (deployment) {
@@ -1555,6 +1638,9 @@ export function maintainMain(options, dependencies = {}) {
     const replaceGatewayEntrypoint =
       dependencies.replaceGatewayEntrypoint ?? replaceLaunchAgentEntrypoint;
     const verifyGatewayRuntime = dependencies.verifyGatewayRuntime ?? verifyManagedGatewayRuntime;
+    const verifyGatewayProbe = dependencies.verifyGateway ?? verifyGateway;
+    const verifyGatewayAfterRestart = dependencies.verifyAndAuditGateway ?? verifyAndAuditGateway;
+    const isGatewayLoaded = dependencies.isGatewayLoaded ?? isManagedGatewayLoaded;
     const prepareSuspension =
       dependencies.prepareGatewaySuspension ??
       ((checkout, deployment) =>
@@ -1760,7 +1846,7 @@ export function maintainMain(options, dependencies = {}) {
         gatewayDeployment,
         gatewayDeployment !== null,
       );
-      gatewayLogAudit = verifyAndAuditGateway({
+      gatewayLogAudit = verifyGatewayAfterRestart({
         runCommand,
         auditGatewayLogs,
         checkout: update.checkout,
@@ -1772,19 +1858,22 @@ export function maintainMain(options, dependencies = {}) {
       gatewayRuntime = verifyGatewayRuntime(update.checkout, update.afterSha);
     } else {
       try {
-        verifyGateway(runCommand, update.checkout, update.afterSha, gatewayControlDeployment);
+        verifyGatewayProbe(runCommand, update.checkout, update.afterSha, gatewayControlDeployment);
         gatewayRuntime = verifyGatewayRuntime(update.checkout, update.afterSha);
       } catch {
         actions.gatewayRestart = true;
         actions.gatewaySelfHeal = true;
+        const bootstrap =
+          gatewayControlDeployment !== null && !isGatewayLoaded(gatewayControlDeployment);
         const restartStartedAt = restartGateway(
           runCommand,
           update.checkout,
           update.afterSha,
           Date.now(),
           gatewayControlDeployment,
+          bootstrap,
         );
-        gatewayLogAudit = verifyAndAuditGateway({
+        gatewayLogAudit = verifyGatewayAfterRestart({
           runCommand,
           auditGatewayLogs,
           checkout: update.checkout,
@@ -1821,7 +1910,7 @@ export function maintainMain(options, dependencies = {}) {
           update.checkout,
         );
         const macTarget = verifyMacTarget(update.checkout);
-        verifyGateway(
+        verifyGatewayProbe(
           runCommand,
           update.checkout,
           update.afterSha,

test/scripts/openclaw-live-updater.test.ts

@@ -372,17 +372,69 @@ describe("openclaw live updater", () => {
     expect(isOwnedGatewayEntrypoint(mirror, home, entrypoint)).toBe(true);
     expect(isOwnedGatewayEntrypoint(mirror, home, path.join(mirror, "dist/index.js"))).toBe(true);
 
+    const fsmonitorMarker = path.join(root, "fsmonitor-ran");
+    const fsmonitorHook = path.join(root, "fsmonitor.sh");
+    writeFileSync(fsmonitorHook, `#!/bin/sh\ntouch ${fsmonitorMarker}\n`);
+    chmodSync(fsmonitorHook, 0o755);
+    git(snapshot, "config", "core.fsmonitor", fsmonitorHook);
+    rmSync(fsmonitorMarker, { force: true });
+    expect(isOwnedGatewayEntrypoint(mirror, home, entrypoint)).toBe(true);
+    expect(existsSync(fsmonitorMarker)).toBe(false);
+
     git(snapshot, "switch", "-c", "mutable");
     expect(isOwnedGatewayEntrypoint(mirror, home, entrypoint)).toBe(false);
     git(snapshot, "checkout", "--detach", head);
+
+    const filterMarker = path.join(root, "filter-ran");
+    const filterHook = path.join(root, "filter.sh");
+    writeFileSync(filterHook, `#!/bin/sh\ntouch ${filterMarker}\ncat\n`);
+    chmodSync(filterHook, 0o755);
+    git(snapshot, "config", "filter.untrusted.clean", filterHook);
+    mkdirSync(path.join(snapshot, ".git/info"), { recursive: true });
+    writeFileSync(path.join(snapshot, ".git/info/attributes"), "README.md filter=untrusted\n");
     writeFileSync(path.join(snapshot, "README.md"), "dirty\n");
-    expect(isOwnedGatewayEntrypoint(mirror, home, entrypoint)).toBe(false);
+    rmSync(filterMarker, { force: true });
+    expect(isOwnedGatewayEntrypoint(mirror, home, entrypoint)).toBe(true);
+    expect(existsSync(filterMarker)).toBe(false);
     git(snapshot, "checkout", "--", "README.md");
     writeFileSync(
       path.join(snapshot, "dist", BUILD_STAMP_FILE),
       `${JSON.stringify({ head: "0".repeat(40) })}\n`,
     );
-    expect(isOwnedGatewayEntrypoint(mirror, home, entrypoint)).toBe(false);
+    const snapshotExecutionMarker = path.join(root, "snapshot-executed");
+    writeFileSync(
+      entrypoint,
+      `import fs from "node:fs"; fs.writeFileSync(${JSON.stringify(snapshotExecutionMarker)}, "ran");\n`,
+    );
+    expect(isOwnedGatewayEntrypoint(mirror, home, entrypoint)).toBe(true);
+
+    writeBuild(mirror);
+    const sourceExecutionMarker = path.join(root, "source-executed");
+    const sourceEntrypoint = path.join(mirror, "dist/index.js");
+    writeFileSync(
+      sourceEntrypoint,
+      `import fs from "node:fs"; fs.writeFileSync(${JSON.stringify(sourceExecutionMarker)}, "ran"); console.log("{}");\n`,
+    );
+    const configPath = path.join(root, "openclaw.json");
+    writeFileSync(configPath, "{}\n");
+    expect(
+      runBuiltGatewayCall(
+        mirror,
+        "gateway.suspend.prepare",
+        { requestId: "request-1" },
+        {
+          configPath,
+          entrypoint,
+          executable: process.execPath,
+          invocationPrefix: [entrypoint],
+          port: 19001,
+          serviceEnvironment: {},
+          wrapperPath: null,
+        },
+      ),
+    ).toContain("{}");
+    expect(existsSync(sourceExecutionMarker)).toBe(true);
+    expect(existsSync(snapshotExecutionMarker)).toBe(false);
   });
 
   test("accepts owned entrypoints only in supported LaunchAgent command layouts", () => {
@@ -1271,6 +1323,58 @@ describe("openclaw live updater", () => {
     ]);
   });
 
+  test("bootstraps an owned LaunchAgent left unloaded by a failed restart", () => {
+    const uid = process.getuid?.() ?? 501;
+    const { root, mirror } = makeFixture();
+    mkdirSync(path.join(mirror, "node_modules"));
+    writeBuild(mirror);
+    const commands = fakeCommands(mirror);
+    const source = path.join(mirror, "dist/index.js");
+    const plistPath = path.join(root, "ai.openclaw.gateway.plist");
+    writeFileSync(plistPath, "plist\n", { mode: 0o600 });
+    const deployment = {
+      configPath: path.join(root, "openclaw.json"),
+      entrypoint: source,
+      entrypointIndex: 1,
+      executable: process.execPath,
+      invocationPrefix: [source],
+      label: "ai.openclaw.gateway",
+      plistPath,
+      port: 18789,
+      runtime: process.execPath,
+    };
+
+    const output = maintainFixture(
+      { checkout: mirror, remote: "origin", lockPath: path.join(root, "maintenance.lock") },
+      {
+        runCommand: commands.runCommand,
+        inspectGatewayDeployment: () => deployment,
+        isGatewayLoaded: () => false,
+        verifyGateway: () => {
+          throw new Error("managed job is unloaded");
+        },
+        verifyAndAuditGateway: () => ({
+          entries: 0,
+          errorCount: 0,
+          warningCount: 0,
+          errors: [],
+          warnings: [],
+        }),
+        verifyGatewayRuntime: () => ({ entrypoint: source, pid: 123, port: 18789 }),
+      },
+    );
+
+    expect(output.actions).toMatchObject({
+      gatewayBuild: false,
+      gatewayRestart: true,
+      gatewaySelfHeal: true,
+    });
+    expect(commands.calls).toEqual([
+      `/bin/launchctl enable gui/${uid}/ai.openclaw.gateway`,
+      `/bin/launchctl bootstrap gui/${uid} ${plistPath}`,
+    ]);
+  });
+
   test("keeps successful CLI stdout as one machine-readable JSON object", () => {
     const { root, mirror, origin } = makeFixture();
     mkdirSync(path.join(mirror, "node_modules"));