fix(replay-e2e): match by conversation, not the living system prompt (#3436)

* fix(replay-e2e): match by conversation, not the living system prompt

The model-replay match key hashed the full input including the lead-agent
system prompt. That prompt is edited frequently (e.g. #3195 added a "File
Editing Workflow" section), so the committed fixture went stale the moment
the prompt changed on main — turning the Layer-2 render gate RED on every
unrelated PR (#3430, #3432, ...). This was a self-inflicted false positive.

Root-cause fix:
- replay_provider._canonical_messages now EXCLUDES the system message from
  the hash. The conversation (human/ai/tool) is the stable contract that
  identifies a recorded turn; the system prompt is an internal detail not
  part of the front-back contract under test. (Mirrors how open-design keys
  its mock picker on the user prompt, not the system internals.) Proven
  robust: injecting a prompt edit no longer causes a replay miss.
- Layer-1 golden was BLIND to replay misses: the gateway swallows a miss
  into an assistant error message, so the shape-only golden stayed green on
  a stale fixture. It now inspects replay_provider.replay_misses() and fails
  loud. (Layer-2 already fails on a miss.)
- Re-recorded write_read_file.ultra fixture + regenerated golden under the
  new conversation-only hash.
- Layer-2 render spec: assert the in-graph auto-title (deterministic); the
  follow-up suggestion is fired async and depends on a clean JSON model
  output, so assert it only when the fixture captured one — never gate on
  its absence (recording flakiness must not block CI).
- docs: REPLAY_E2E.md updated.

Verified: Layer-1 golden green (no miss), Layer-2 both specs green,
CI=true make test 4033 passed / 0 failed, frontend pnpm check clean.

* test(replay-e2e): restore suggestions coverage with a reliable capture

Addresses review feedback (the suggestion path was dropped from Layer-2):

- record spec now waits for the `/suggestions` response before checking
  capture stability, so the recorded fixture reliably includes the
  frontend-fired suggestions turn (previously the stability window could
  return before suggestions fired, yielding a fixture without it).
- Re-recorded write_read_file.ultra: 5 turns (write_file, auto-title,
  read_file, answer, suggestions). Golden unchanged — suggestions is a
  separate /suggestions call, not part of the /runs/stream SSE sequence.
- Layer-2 spec: restore the hard `EXPECTED_SUGGESTION` assertion. With the
  record spec now waiting for /suggestions, a fixture missing the suggestion
  turn means a broken recording and must fail loud, not pass silently.

Verified: Layer-1 golden green (no miss), Layer-2 both specs green
(auto-title + suggestion render), frontend pnpm check clean.

* ci: re-trigger (flaky Docker Hub image pull in sandbox e2e, unrelated)

backend-unit-tests failed only in test_sandbox_orphan_reconciliation_e2e.py
with 'docker pull busybox:latest ... context deadline exceeded' — a CI-runner
network flake reaching Docker Hub, not related to this docs/tests-only change.
Empty commit to re-run CI.

---------

Co-authored-by: DanielWalnut <45447813+hetaoBackend@users.noreply.github.com>

frontend/tests/e2e-real-backend/real-backend-render.spec.ts

@@ -85,17 +85,21 @@ test.describe("real backend render (replay, no API key)", () => {
     await textarea.fill(PROMPT);
     await textarea.press("Enter");
 
-    // Replay-only DOM assertions (derived from the fixture): they render only if
+    // Replay-only DOM assertions (derived from the fixture): both are
+    // model-generated strings absent from the user prompt, so they render only if
     // the recorded turns replayed AND the real frontend rendered them — the
     // in-graph auto-title and the post-answer follow-up suggestion. Together they
-    // prove the whole pipeline (replay backend -> real frontend render).
+    // prove the whole pipeline (replay backend -> real frontend render). The
+    // record spec waits for the /suggestions response, so a re-recorded fixture
+    // always captures the suggestion turn — a missing one is a broken recording
+    // and must fail loud here, not pass silently.
     expect(
       EXPECTED_TITLE,
       "fixture should contain an auto-title turn",
     ).not.toBe("");
     expect(
       EXPECTED_SUGGESTION,
-      "fixture should contain a suggestions turn",
+      "fixture should contain a suggestions turn (re-record; the record spec waits for /suggestions)",
     ).not.toBe("");
     await expect(page.getByText(EXPECTED_TITLE)).toBeVisible({
       timeout: 60_000,

frontend/tests/e2e-record/record-write-read-file.spec.ts

@@ -104,6 +104,16 @@ test("record write/read-file run through the real frontend", async ({
   await textarea.fill(PROMPT);
   await textarea.press("Enter");
 
+  // Suggestions fire only AFTER the run completes (input-box.tsx POSTs
+  // /suggestions). Wait for that response so its model call lands in the capture
+  // before we check for stability — otherwise the stability window can return
+  // first and the recorded fixture would be missing the suggestions turn.
+  await page
+    .waitForResponse((r) => r.url().includes("/suggestions"), {
+      timeout: 90_000,
+    })
+    .catch(() => undefined);
+
   const captured = await waitForCaptureStable(out!);
   console.log(
     `[record] captures stabilized at ${captured} model call(s) -> ${out}`,