Compare commits

..
Author SHA1 Message Date
Kit Langton 3369c08ffa docs(plugin): update current API examples 2026-08-27 15:00:32 -04:00
Kit Langton 0c77f6ed5b refactor(core): remove unreachable permission branch (#45617)
Remove the impossible pre-hook deny aggregation branch while preserving configured denials and post-hook denial handling.
2026-08-27 14:48:11 -04:00
Kit Langton 284b222489 refactor(core): reuse read newline locator (#45616)
Reuse the reader newline locator for the terminal tree leaf while preserving accumulated offsets and the whole-tree fallback. Add chunk-boundary coverage.
2026-08-27 14:47:58 -04:00
Kit Langton 705606face refactor(core): reuse formatter executable helper (#45615)
Reuse the existing executable formatter helper for gofmt, mix, zig, and ktlint while preserving arguments, extensions, ordering, and lazy lookup.
2026-08-27 14:47:10 -04:00
Kit Langton b738ef970d test(core): stabilize Windows live I/O fixtures (#45595) 2026-08-27 14:45:07 -04:00
Kit Langton 4112698e72 refactor(core): simplify session runner control flow (#45614) 2026-08-27 14:42:12 -04:00
Aiden Cline a609174969 feat(core): expand tildes in tool path resolution (#45605) 2026-08-27 13:41:39 -05:00
Kit Langton d3694a5383 refactor(core): share read media types (#45597)
Reuse the reader-owned media MIME set in the read tool leaf while preserving both ingestion and unsupported-base64 validation boundaries.
2026-08-27 14:24:41 -04:00
Kit Langton fafcea42e6 refactor(core): share patch write path (#45588)
Share the identical file write, error mapping, and result recording path used by additions and non-moving updates. Keep deletion and moving-update behavior explicit.
2026-08-27 12:51:58 -04:00
opencode-agent[bot]andrekram1-node 95c3c3f962 fix(mcp): retry initial 404 without injected codemode (#45563)
Co-authored-by: rekram1-node <rekram1-node@users.noreply.github.com>
2026-08-27 11:51:34 -05:00
Aiden Cline 497a24c17d refactor(ai): consolidate provider error diagnostics (#45381) 2026-08-27 11:28:57 -05:00
opencode-agent[bot] f1521000ec chore: update nix node_modules hashes 2026-08-27 16:15:47 +00:00
Kit Langton 8a3e9890d2 chore(core): narrow websocket platform import (#45585) 2026-08-27 16:02:37 +00:00
Kit Langton 2142de87dd test(core): make websocket overflow fixture deterministic (#45586) 2026-08-27 16:00:39 +00:00
Kit Langton 367cf59612 refactor(core): simplify WebFetch error narrowing (#45582)
Use the typed HTTP error reason instead of manual object probing and an unchecked response cast. Preserve the single challenge-only retry and cover ordinary 403 failures without retry.
2026-08-27 11:56:34 -04:00
James Long 2b8a3e7ee0 fix(pty): own daemon lifetime and hand off restarts (#45551) 2026-08-27 11:52:48 -04:00
Kit Langton 6dbb5f5008 fix(core): reuse ignore checks during index refresh (#45459) 2026-08-27 11:44:44 -04:00
Kit Langton 8d1d1ee982 refactor(core): align model content helper names (#45458) 2026-08-27 11:44:37 -04:00
Kit Langton c33dc8808b refactor(core): rename model package predicate (#45457) 2026-08-27 11:44:30 -04:00
Kit Langton 5253603583 fix(core): preserve downloaded skill directory IDs (#45455) 2026-08-27 11:40:55 -04:00
Kit Langton 5fb3040505 test(core): scope skill discovery fixtures (#45464) 2026-08-27 11:25:17 -04:00
Kit Langton df7dae3706 refactor(plugin): reuse tool executor conversion (#45572)
Reuse the private executor/progress conversion for Promise tool list, get, and update while preserving captured executors, identifiers, and explicit optional-field clearing. Add focused adapter regression coverage.
2026-08-27 11:25:12 -04:00
Kit Langton 83ff4253f5 test: share session message expectations (#45454) 2026-08-27 10:58:12 -04:00
Kit Langton 89cb5146d4 test(cli): scope temporary directories to test lifetimes (#45467) 2026-08-27 10:57:06 -04:00
James Long b0b5bf6749 feat(tui): jump to background tool starts (#45554) 2026-08-27 10:56:51 -04:00
Kit Langton ef9b9fb6bf test(client): scope service subprocess fixtures per test (#45471) 2026-08-27 10:55:20 -04:00
James Long c05d481ec6 fix(pty): use server shell and session cwd for terminals (#45552) 2026-08-27 10:39:27 -04:00
Shoubhit Dash 8e726dc7d8 fix(core): preserve imported session parents (#45566) 2026-08-27 19:57:59 +05:30
Shoubhit Dash 23f0ee53cf feat(plugin): intercept user prompts before admission (#45550) 2026-08-27 18:58:00 +05:30
Shoubhit Dash 46aa25f236 feat(plugin): expose model generation options to session hooks (#45268) 2026-08-27 17:30:37 +05:30
Brendan Allan 803ead32e7 feat(app): use channel-specific favicon and PWA icons (#45543) 2026-08-27 11:17:11 +00:00
Luke Parker 42422a1e03 fix(app): show sessions while worktrees are created (#45529) 2026-08-27 19:44:23 +10:00
Brendan Allan b1374978c9 fix(app): prevent Safari shell content from collapsing 2026-08-27 16:59:43 +08:00
Luke Parker ed95fdaa27 fix(app): show MCP connection failures as toasts (#45522) 2026-08-27 08:46:54 +00:00
Luke Parker 53a4829672 fix(app): restore fullscreen settings version footer (#45419) 2026-08-27 17:57:04 +10:00
Luke Parker 7036294543 fix(app): align thinking states and reasoning settings (#45515) 2026-08-27 17:50:39 +10:00
opencode-agent[bot]andHona 23506b5fb4 fix(app): scope MCP picker toggles to workspace (#45509)
Co-authored-by: Hona <10430890+Hona@users.noreply.github.com>
2026-08-27 17:20:38 +10:00
Luke Parker 2af02d0ad7 fix(app): use worktree inventory for session accents (#45506) 2026-08-27 07:03:16 +00:00
opencode-agent[bot]andBrendonovich 26ee104829 fix(app): prevent renderer OOM on multiline paste (#45497)
Co-authored-by: Brendonovich <14191578+Brendonovich@users.noreply.github.com>
2026-08-27 14:31:28 +08:00
opencode-agent[bot]andBrendonovich 8252897a33 fix(app): precache complete builds before serving cached HTML (#45495)
Co-authored-by: Brendonovich <14191578+Brendonovich@users.noreply.github.com>
2026-08-27 14:23:31 +08:00
Luke Parker 7000607fd0 fix(app): stop shimmer when background shells finish (#45494) 2026-08-27 16:15:34 +10:00
opencode-agent[bot]andBrendonovich fa1ab5f8e1 fix(app): restore worktree message accent for further trial (#45486)
Co-authored-by: Brendonovich <14191578+Brendonovich@users.noreply.github.com>
2026-08-27 05:18:54 +00:00
Aiden Cline e288e0fc4d feat(plugin): allow tool call repair before lookup (#45453) 2026-08-27 00:10:43 -05:00
Luke Parker ff5b5d00f9 fix(app): preserve tool disclosures when groups update (#45474) 2026-08-27 15:02:39 +10:00
Luke Parker 990f7ee000 fix(app): keep timeline notices on one line (#45479) 2026-08-27 04:47:50 +00:00
Luke Parker e2d6c4bd4a fix(app): merge adjacent patches inside used groups (#45477) 2026-08-27 04:36:27 +00:00
Aiden Cline fcc6568fcb feat(core): adjust bg shell completion msg (include output file) (#45461) 2026-08-26 23:04:11 -05:00
opencode-agent[bot]andrekram1-node 1c66cd7832 fix(tui): clarify tool grouping setting (#45470)
Co-authored-by: rekram1-node <rekram1-node@users.noreply.github.com>
2026-08-26 23:02:27 -05:00
Luke Parker 5cc81a497c fix(app): keep active tools in existing groups (#45462) 2026-08-27 03:54:38 +00:00
Kit Langton 10786cb60c refactor(core): consolidate runner capability reads (#45448) 2026-08-26 23:45:01 -04:00
Kit Langton 1e7c60adce fix(server): wait for plugins before text generation (#45447)
Wait for bounded plugin readiness in the generation location before resolving explicit or default models. Add deterministic cold-first-request regressions through the embedded SDK.
2026-08-26 23:30:52 -04:00
Aiden Cline 1c4f8c40a8 feat(plugin): add tool draft reads (#45443) 2026-08-26 22:16:27 -05:00
Luke Parker 2ca55b479d fix(app): reduce tab switch rendering work (#45428) 2026-08-27 13:10:27 +10:00
opencode-agent[bot]andthdxr 8d7caa178b fix(core): route session events to location subscribers (#45411)
Co-authored-by: thdxr <826656+thdxr@users.noreply.github.com>
2026-08-26 23:04:27 -04:00
Aiden Cline 2bcb67a71e feat(plugin): add tool updates and removal (#45436) 2026-08-26 22:00:01 -05:00
Luke Parker 48d4e52143 fix(app): keep pending steers after assistant work (#45435) 2026-08-27 02:44:18 +00:00
Aiden Cline 40cbea3c19 refactor(core): use shared state for tool registry (#45414) 2026-08-26 21:33:00 -05:00
Luke Parker 51065122d8 fix(app): keep project extensions inside settings (#45432) 2026-08-27 02:30:36 +00:00
Luke Parker 7507f19a00 fix(app): prevent settings loading flicker (#45427) 2026-08-27 02:01:36 +00:00
opencode-agent[bot] 71706577c4 chore: update nix node_modules hashes 2026-08-27 01:53:01 +00:00
Kit Langton 66f326ddec fix(tui): restore default Kitty keyboard reporting (#45423) 2026-08-26 21:41:19 -04:00
opencode-agent[bot]andBrendonovich 83b81b39f1 fix(app): remove worktree user message accent (#45250)
Co-authored-by: Brendonovich <14191578+Brendonovich@users.noreply.github.com>
2026-08-27 09:36:35 +08:00
Brendan Allan b20e4971db test(app): run CI e2e against production builds (#45422) 2026-08-27 09:28:23 +08:00
James Long c3378f2d34 fix(pty): upgrade opencode-pty to 0.1.12 (#45420) 2026-08-26 21:15:09 -04:00
Kit Langton d68031bc0d fix(core): preserve renamed files when undoing changes (#45383) 2026-08-26 20:42:40 -04:00
Kit Langton 4ac19d744a feat(tui): make session tab status explicit (#45347) 2026-08-26 20:12:15 -04:00
Aiden Cline b5cb5e2a55 refactor(plugin): manage MCP servers through transforms (#45408) 2026-08-26 18:42:28 -05:00
Luke Parker 53d171781a fix(app): reduce catalog retention and status animation work (#45406) 2026-08-26 23:30:49 +00:00
opencode-agent[bot]andthdxr 3184427c03 feat(plugin): expose the current location in plugin context (#45403)
Co-authored-by: thdxr <826656+thdxr@users.noreply.github.com>
2026-08-26 19:09:16 -04:00
James Long 6b0613ac6f fix(tui): restore session focus during permission prompts (#45391) 2026-08-26 19:02:56 -04:00
Aiden Cline 5ac4156eb2 fix(core): preserve reasoning across model switches (#45401) 2026-08-26 17:52:41 -05:00
Luke Parker 954cdc7bc8 fix(desktop): restore compact Windows titlebar spacing (#45397) 2026-08-26 22:18:05 +00:00
Aiden Cline 767115cd69 fix(ai): default native Mantle to Responses (#45394) 2026-08-26 16:47:39 -05:00
James Long 5b25ee8430 style(tui): brighten embedded terminal palette (#45393) 2026-08-26 17:30:23 -04:00
James Long df6317d23c refactor(tui): simplify terminal selection state (#45387) 2026-08-26 17:18:44 -04:00
opencode-agent[bot] 524e39db02 chore: update nix node_modules hashes 2026-08-26 20:58:41 +00:00
James Long 6170221e21 fix(pty): upgrade opencode-pty to 0.1.11 (#45380) 2026-08-26 16:39:24 -04:00
Kit Langton 74a0b2d0ad feat(core): make the portable shell scanner authoritative 2026-08-26 16:35:16 -04:00
opencode-agent[bot] 9d46941e70 chore: update nix node_modules hashes 2026-08-26 20:31:57 +00:00
Aiden Cline d48a5d16ff fix(core): avoid nested MCP code mode (#45377) 2026-08-26 15:23:14 -05:00
Kit Langton 6c392355dd feat(tui): render LaTeX as Unicode math (#45339) 2026-08-26 16:14:37 -04:00
Aiden Cline dd4ee94682 refactor(ai): remove derived HTTP request ID (#45376) 2026-08-26 15:12:53 -05:00
Aiden Cline 009eca8b5b test(core): shrink webfetch stress fixtures (#45375) 2026-08-26 15:03:49 -05:00
Aiden Cline 94bd9f6c8b fix(ai): isolate native provider metadata ownership (#45155) 2026-08-26 14:54:12 -05:00
Aiden Cline 33909f48d7 fix(core): label migrated credentials by auth type (#45369) 2026-08-26 14:41:31 -05:00
James Long 80653a0a1a fix(tui): preserve resolved server directory (#45354) 2026-08-26 15:40:00 -04:00
James Long f7913a04d2 fix(cli): stop PTY daemon on explicit service restart (#45373) 2026-08-26 15:39:29 -04:00
Kit Langton cf347cd5e4 refactor(core): advance sessions before running steps (#45358)
Centralize control dispatch and first-Step preparation in advanceToStep. Keep input delivery outside logical-Step retries and preserve queue ordering, Location handoff, context refresh, and durable settlement.
2026-08-26 15:25:13 -04:00
opencode-agent[bot] 6600d59635 chore: update nix node_modules hashes 2026-08-26 19:07:20 +00:00
Aiden Cline 8b6a2450d5 fix(core): isolate invalid tool registrations (#45325) 2026-08-26 13:49:56 -05:00
opencode-agent[bot]andBrendonovich 1aa4046f02 test(app): isolate component coverage with storybook (#45142)
Co-authored-by: Brendonovich <14191578+Brendonovich@users.noreply.github.com>
2026-08-27 02:42:27 +08:00
opencode-agent[bot] 9bd69fe847 chore: update nix node_modules hashes 2026-08-26 18:32:55 +00:00
Kit Langton ded8a492d1 fix(core): recover background jobs after restart
Persist background Job ownership and terminal results across server restarts. Resume existing subagent Sessions, admit shell cancellation notices without waking idle parents, and preserve explicit cancellation.
2026-08-26 14:17:43 -04:00
439 changed files with 26139 additions and 7464 deletions
+5
View File
@@ -0,0 +1,5 @@
---
"@opencode-ai/core": patch
---
Propagate fatal Git ignore-check errors during index refresh so best-effort snapshot capture returns unavailable instead of treating the check as having no matches.
+5
View File
@@ -0,0 +1,5 @@
---
"@opencode-ai/core": patch
---
Rename the write, patch, and question tool formatting helpers from `toModelOutput` to `toModelContent` to match the result field they populate. Direct imports of these helpers must use the new name; generated content and declared machine output are unchanged.
+5
View File
@@ -0,0 +1,5 @@
---
"@opencode-ai/core": minor
---
Rename ModelResolver.supported to ModelResolver.hasPackage. Consumers of the old export must update the name; the predicate remains Boolean(model.package), checking only whether a catalog model declares a provider package, not whether it can be loaded. Default-model selection behavior is unchanged.
-6
View File
@@ -1,6 +0,0 @@
---
"@opencode-ai/core": patch
"@opencode-ai/server": patch
---
Keep the live models.dev catalog independent of persistence so failed cache reads or writes cannot prevent model updates. Cache downloaded catalogs in local files on Bun and Node, and use the bundled snapshot plus in-memory refreshes on workerd instead of storing the catalog in each Durable Object's database. Explicit catalog files refresh locally without fetching or writing an implicit cache.
+5
View File
@@ -0,0 +1,5 @@
---
"@opencode-ai/core": patch
---
Include both paths of renamed files in new snapshot change lists so undo restores the original file instead of only deleting the renamed file.
+10
View File
@@ -0,0 +1,10 @@
---
"@opencode-ai/core": patch
---
Make the experimental portable shell scanner authoritative, with no Tree-sitter
fallback. Scan common Bash and PowerShell control flow, heredocs, functions,
expressions, quoting, and substitutions natively. Preserve existing redirect and
declaration permission matching, and make PowerShell saved approvals cover the
original command spelling. Parser failures remain visible without changing the
permission engine. The default Tree-sitter path is unchanged.
+5
View File
@@ -0,0 +1,5 @@
---
"@opencode-ai/core": patch
---
Use parent directory names for SKILL.md IDs so individually downloaded skills do not overwrite each other.
+8 -2
View File
@@ -237,13 +237,19 @@ jobs:
working-directory: packages/app
run: bunx playwright install chromium
- name: Run app e2e tests
- name: Run app e2e tests against production build
if: env.E2E_ENABLED == 'true'
run: bun --cwd packages/app test:e2e:local
run: bun --cwd packages/app test:e2e:built
env:
CI: true
timeout-minutes: 30
- name: Verify service worker precaching and upgrades
if: env.E2E_ENABLED == 'true'
working-directory: packages/app
run: bunx playwright test --config e2e/service-worker/playwright.config.ts
timeout-minutes: 5
- name: Upload Playwright artifacts
if: always() && env.E2E_ENABLED == 'true'
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
+28 -9
View File
@@ -125,7 +125,7 @@
"@effect/platform-node": "catalog:",
"@opencode-ai/client": "workspace:*",
"@opencode-ai/plugin": "workspace:*",
"@opencode-ai/pty": "0.1.10",
"@opencode-ai/pty": "0.1.13",
"@opencode-ai/schema": "workspace:*",
"@opencode-ai/server": "workspace:*",
"@opencode-ai/tui": "workspace:*",
@@ -364,7 +364,7 @@
"@opencode-ai/ai": "workspace:*",
"@opencode-ai/codemode": "workspace:*",
"@opencode-ai/plugin": "workspace:*",
"@opencode-ai/pty": "0.1.10",
"@opencode-ai/pty": "0.1.13",
"@opencode-ai/schema": "workspace:*",
"@opencode-ai/util": "workspace:*",
"@parcel/watcher": "2.5.1",
@@ -553,6 +553,20 @@
"@typescript/native-preview": "catalog:",
},
},
"packages/latex": {
"name": "@opencode-ai/latex",
"version": "0.0.0",
"dependencies": {
"@opencode-ai/plugin": "workspace:*",
"@opentui/core": "catalog:",
"string-width": "catalog:",
},
"devDependencies": {
"@tsconfig/bun": "catalog:",
"@types/bun": "catalog:",
"@typescript/native-preview": "catalog:",
},
},
"packages/merman": {
"name": "@opencode-ai/merman",
"version": "0.0.0",
@@ -731,6 +745,7 @@
},
"devDependencies": {
"@happy-dom/global-registrator": "20.0.11",
"@playwright/test": "catalog:",
"@tsconfig/node22": "catalog:",
"@types/bun": "catalog:",
"@types/luxon": "catalog:",
@@ -836,6 +851,7 @@
"@opencode-ai/client": "workspace:*",
"@opencode-ai/session-ui": "workspace:*",
"@opencode-ai/ui": "workspace:*",
"@playwright/test": "catalog:",
"@solidjs/meta": "catalog:",
"@storybook/addon-a11y": "10.4.4",
"@storybook/addon-docs": "10.4.4",
@@ -877,6 +893,7 @@
"dependencies": {
"@opencode-ai/client": "workspace:*",
"@opencode-ai/core": "workspace:*",
"@opencode-ai/latex": "workspace:*",
"@opencode-ai/merman": "workspace:*",
"@opencode-ai/plugin": "workspace:*",
"@opencode-ai/schema": "workspace:*",
@@ -2148,6 +2165,8 @@
"@opencode-ai/httpapi-codegen": ["@opencode-ai/httpapi-codegen@workspace:packages/httpapi-codegen"],
"@opencode-ai/latex": ["@opencode-ai/latex@workspace:packages/latex"],
"@opencode-ai/merman": ["@opencode-ai/merman@workspace:packages/merman"],
"@opencode-ai/plugin": ["@opencode-ai/plugin@workspace:packages/plugin"],
@@ -2156,19 +2175,19 @@
"@opencode-ai/protocol": ["@opencode-ai/protocol@workspace:packages/protocol"],
"@opencode-ai/pty": ["@opencode-ai/pty@0.1.10", "", { "optionalDependencies": { "@opencode-ai/pty-darwin-arm64": "0.1.10", "@opencode-ai/pty-darwin-x64": "0.1.10", "@opencode-ai/pty-linux-arm64-gnu": "0.1.10", "@opencode-ai/pty-linux-arm64-musl": "0.1.10", "@opencode-ai/pty-linux-x64-gnu": "0.1.10", "@opencode-ai/pty-linux-x64-musl": "0.1.10" }, "bin": { "opencode-pty": "bin/opencode-pty.js" } }, "sha512-cEJT1ADtmnb+df2wrlUcsGny6Q7pTe9Sa7keISzCO0xN1FrL1aS6+eleBPpDimHjgM/sXqvLwJv0UiAeiAvgxQ=="],
"@opencode-ai/pty": ["@opencode-ai/pty@0.1.13", "", { "optionalDependencies": { "@opencode-ai/pty-darwin-arm64": "0.1.13", "@opencode-ai/pty-darwin-x64": "0.1.13", "@opencode-ai/pty-linux-arm64-gnu": "0.1.13", "@opencode-ai/pty-linux-arm64-musl": "0.1.13", "@opencode-ai/pty-linux-x64-gnu": "0.1.13", "@opencode-ai/pty-linux-x64-musl": "0.1.13" }, "bin": { "opencode-pty": "bin/opencode-pty.js" } }, "sha512-WPCN8h8HaZhhUcrMG0zu+4D9vco0EZiEg/gCF1K3JPRN6UsHMiXq1HVIy5IlyfcoyjfViRmQmXYE4AuU3laBjA=="],
"@opencode-ai/pty-darwin-arm64": ["@opencode-ai/pty-darwin-arm64@0.1.10", "", { "os": "darwin", "cpu": "arm64" }, "sha512-j7aszDFRwCIazGUT9eIy4PZwh4rltjvRmoicPRTK3kONN3v0MMflstkmAFDYYpqDPTNh3qJ6xkQmB+DugEbhAg=="],
"@opencode-ai/pty-darwin-arm64": ["@opencode-ai/pty-darwin-arm64@0.1.13", "", { "os": "darwin", "cpu": "arm64" }, "sha512-fVtQZqVLBuJx/aB+5ojfmQifS1KMc9gxlxpFQ6bxEFU8tn8xHQTiFPaNroZgOtaw7I4ceGyx/eXieK1wp68yAA=="],
"@opencode-ai/pty-darwin-x64": ["@opencode-ai/pty-darwin-x64@0.1.10", "", { "os": "darwin", "cpu": "x64" }, "sha512-UAMP/E4lo9RGQF7xrfIwpW2ZEemj308rCogJy14ruKYJt5MwHeGNTynGiHE/1JlDLRy+21wV50jpugADgT71ag=="],
"@opencode-ai/pty-darwin-x64": ["@opencode-ai/pty-darwin-x64@0.1.13", "", { "os": "darwin", "cpu": "x64" }, "sha512-b/tAEm0hCMXraPM9cxR8Rg7X1UBZInRTaxWAS4Ht9eH1nWj1rANOLvHWiWX/vVh5TB0Ubg8bWPu4B0nZkEHROQ=="],
"@opencode-ai/pty-linux-arm64-gnu": ["@opencode-ai/pty-linux-arm64-gnu@0.1.10", "", { "os": "linux", "cpu": "arm64" }, "sha512-lTPlZNQ66koFHZqoPmvvq0SetlepKVQYgnLryhlVfYtcryWDJM7gV4+P66V12RwqWQTjt2u8j12mtg3axSKg2w=="],
"@opencode-ai/pty-linux-arm64-gnu": ["@opencode-ai/pty-linux-arm64-gnu@0.1.13", "", { "os": "linux", "cpu": "arm64" }, "sha512-I124aSYBBjpGZnYExHfIajkvVK1FiK+//OJBGdqqFp5pas2Oruq4O8tv+pMoxomZIYh2ce/QhOOYLHRwXsthTg=="],
"@opencode-ai/pty-linux-arm64-musl": ["@opencode-ai/pty-linux-arm64-musl@0.1.10", "", { "os": "linux", "cpu": "arm64" }, "sha512-IDmWHRylMR/ZfMw9/AAktO/Edi4TITPC+Tq7Xx3JZHsDgSba3QdyE11uNL0zM1myTGdk6Yrt4rpdAzaItPnDjw=="],
"@opencode-ai/pty-linux-arm64-musl": ["@opencode-ai/pty-linux-arm64-musl@0.1.13", "", { "os": "linux", "cpu": "arm64" }, "sha512-feWsfKpaDytGJzutoK43GqQwVghG2vHZt6BE/ydPZNuqIrySQ/6JfliUAMwn5BWs/Ky7ouSwKHCyAVeukusSvg=="],
"@opencode-ai/pty-linux-x64-gnu": ["@opencode-ai/pty-linux-x64-gnu@0.1.10", "", { "os": "linux", "cpu": "x64" }, "sha512-Q1yob0/8X2JoJZzFmNKUc32XDRAe0avKQ8PLKkpJr30qWXSrGmhltgcDmn94Q70zW9Ght9on84T7cmge9brvdQ=="],
"@opencode-ai/pty-linux-x64-gnu": ["@opencode-ai/pty-linux-x64-gnu@0.1.13", "", { "os": "linux", "cpu": "x64" }, "sha512-jliNgsevGuxfIeX7eyzjHhrJkF8uEUPnDLbF2v16uv69FhEHrraf7jyWkxazMP6rNvn2CGtwMMc4BXPS5pzjhg=="],
"@opencode-ai/pty-linux-x64-musl": ["@opencode-ai/pty-linux-x64-musl@0.1.10", "", { "os": "linux", "cpu": "x64" }, "sha512-7RLHWQxX/wfUKJJP2ZMMtkXaPsrgoMNKzE6PL/LbnYbMBtkqfld9EDcMv1RFZ0CqjNFgI0Hg4eRk6x+ZNc/wyQ=="],
"@opencode-ai/pty-linux-x64-musl": ["@opencode-ai/pty-linux-x64-musl@0.1.13", "", { "os": "linux", "cpu": "x64" }, "sha512-rXDpidW66gz2b2M/NbUN8ZKmAxaJcASnuHATeXevlrFdiPUv8uJwvkRd6Pla1fp01Q65MkBmgRa7Q9c+H1PlzA=="],
"@opencode-ai/schema": ["@opencode-ai/schema@workspace:packages/schema"],
+4 -4
View File
@@ -1,8 +1,8 @@
{
"nodeModules": {
"x86_64-linux": "sha256-QWLIdvu985FH5I9cZJOAuoeFeXU+4Jx9RzBB9RPoeeQ=",
"aarch64-linux": "sha256-SSzGD5hMj2vFvyw+dUPR9g/ZH6qhs0ZyZ/DnltZt3N8=",
"aarch64-darwin": "sha256-CeFUxiV+e8pKho+YcSclC3soQBogoxNMxwyIMztAExU=",
"x86_64-darwin": "sha256-FYwcACzU72y0+KtOpFfU7ndak8vMasqMgd5NLS6+XtY="
"x86_64-linux": "sha256-agxF1Ab8P84tijHvw7jaG+9UZm/dJZhHw8CGNlQQqYc=",
"aarch64-linux": "sha256-Kg6CkhoEEsb+ZOGPN/rnNoirLdGSyQl6HU/Fy8t9Mqs=",
"aarch64-darwin": "sha256-SnxBpmcfPL2p3OYmSuG5Y/aFW8PeEOLhfQrxAs/53fs=",
"x86_64-darwin": "sha256-Dnxl1EcfiE7/rlHmPkko0BKc6hHVk6rQ/2h/PWuLfNo="
}
}
+8
View File
@@ -19,6 +19,14 @@ Per-type constructors live on the type, not as top-level re-exports. Use `Messag
- Use `testEffect(...)` from `test/lib/effect.ts` for tests requiring Effect layers.
- Keep provider tests fixture-first. Live provider calls must stay behind `RECORD=true` and required API-key checks.
## Errors
- `AIError` wraps a union of tagged reason errors. It stores only `reason`, derives `message` from the reason, and exposes the reason as its `cause`.
- Each reason owns its readable `message`, category-specific fields, and optional `body`, `http`, and underlying exception in `cause`.
- `reason.body` is the sole original-response or triggering-event payload field. Preserve original text before schema decoding removes fields; do not replace the complete event with only its nested error.
- `reason.http` describes an observed HTTP response with required `url`, `status`, and response `headers`. Do not invent status codes or derive a separate request ID from headers.
- Reclassification and transport recovery must preserve the reason's body, HTTP context, and underlying cause. Error `message` and `cause` are non-enumerable: copy them explicitly when constructing an enriched reason with its constructor or `AIErrorReason.make`.
## Architecture
This package is an Effect Schema-first LLM core. The Schema classes in `src/schema/` are the canonical runtime data model. Convenience functions in `src/llm.ts` are thin constructors that return those same Schema class instances; they should improve callsites without creating a second model.
+5 -4
View File
@@ -1,7 +1,7 @@
import { Effect, Schema } from "effect"
import {
HttpOptions,
InvalidRequestReason,
InvalidRequestError,
AIError,
ModelID,
ProviderID,
@@ -158,9 +158,10 @@ export function generate(input: ImageRequest | ImageRequestInput) {
try: () => (input instanceof ImageRequest ? input : request(input)),
catch: (error) =>
new AIError({
module: "Image",
method: "generate",
reason: new InvalidRequestReason({ message: error instanceof Error ? error.message : String(error) }),
reason: new InvalidRequestError({
message: error instanceof Error ? error.message : String(error),
cause: error,
}),
}),
}).pipe(Effect.flatMap((request) => ImageClient.generate(request as unknown as ImageRequestFor<ImageOptions>)))
}
+4 -7
View File
@@ -3,7 +3,7 @@ import { LLMClient, Service } from "./route/client.js"
import {
GenerationOptions,
HttpOptions,
InvalidProviderOutputReason,
InvalidProviderOutputError,
AIError,
LLMEvent,
LLMRequest,
@@ -116,9 +116,7 @@ const runGenerateObject = Effect.fn("LLM.generateObject")(function* (
)
if (!call || !LLMEvent.is.toolCall(call))
return yield* new AIError({
module: "LLM",
method: "generateObject",
reason: new InvalidProviderOutputReason({
reason: new InvalidProviderOutputError({
message: `generateObject: model did not call the forced \`${GENERATE_OBJECT_TOOL_NAME}\` tool`,
}),
})
@@ -126,10 +124,9 @@ const runGenerateObject = Effect.fn("LLM.generateObject")(function* (
Effect.mapError(
(error) =>
new AIError({
module: "LLM",
method: "generateObject",
reason: new InvalidProviderOutputReason({
reason: new InvalidProviderOutputError({
message: `generateObject: tool input failed schema decode: ${error.message}`,
cause: error,
}),
}),
),
+48 -37
View File
@@ -406,6 +406,7 @@ const AnthropicEvent = Schema.Struct({
type AnthropicEvent = Schema.Schema.Type<typeof AnthropicEvent>
interface ParserState {
readonly providerMetadataKey: string
readonly tools: ToolStream.State<number>
readonly reasoningSignatures: Readonly<Record<number, string>>
readonly usage?: Usage
@@ -440,18 +441,18 @@ const cacheControl = (breakpoints: Cache.Breakpoints, cache: CacheHint | undefin
return Cache.ttlBucket(cache.ttlSeconds) === "1h" ? EPHEMERAL_1H : EPHEMERAL_5M
}
const anthropicMetadata = (metadata: Record<string, unknown>): ProviderMetadata => ({ anthropic: metadata })
const providerMetadata = (key: string, metadata: Record<string, unknown>): ProviderMetadata => ({ [key]: metadata })
const signatureFromMetadata = (metadata: ProviderMetadata | undefined): string | undefined => {
const anthropic = metadata?.anthropic
if (!ProviderShared.isRecord(anthropic)) return undefined
return typeof anthropic.signature === "string" ? anthropic.signature : undefined
const signatureFromMetadata = (metadata: ProviderMetadata | undefined, key: string): string | undefined => {
const provider = metadata?.[key]
if (!ProviderShared.isRecord(provider)) return undefined
return typeof provider.signature === "string" ? provider.signature : undefined
}
const redactedDataFromMetadata = (metadata: ProviderMetadata | undefined): string | undefined => {
const anthropic = metadata?.anthropic
if (!ProviderShared.isRecord(anthropic)) return undefined
return typeof anthropic.redactedData === "string" ? anthropic.redactedData : undefined
const redactedDataFromMetadata = (metadata: ProviderMetadata | undefined, key: string): string | undefined => {
const provider = metadata?.[key]
if (!ProviderShared.isRecord(provider)) return undefined
return typeof provider.redactedData === "string" ? provider.redactedData : undefined
}
const lowerTool = (breakpoints: Cache.Breakpoints, tool: ToolDefinition, inputSchema: JsonSchema): AnthropicTool => ({
@@ -511,13 +512,16 @@ const serverToolResultType = (name: string): AnthropicServerToolResultType | und
return undefined
}
const lowerServerToolResult = Effect.fn("AnthropicMessages.lowerServerToolResult")(function* (part: ToolResultPart) {
const lowerServerToolResult = Effect.fn("AnthropicMessages.lowerServerToolResult")(function* (
part: ToolResultPart,
providerMetadataKey: string,
) {
const wireType = serverToolResultType(part.name)
if (!wireType)
return yield* invalid(`Anthropic Messages does not know how to round-trip server tool result for ${part.name}`)
// Prefer the provider-owned replay payload; fall back to the result value for
// histories constructed directly from provider events.
const payload = part.providerMetadata?.anthropic?.["result"] ?? part.result.value
const payload = part.providerMetadata?.[providerMetadataKey]?.["result"] ?? part.result.value
return {
type: wireType,
tool_use_id: scrubToolCallID(part.id),
@@ -804,6 +808,7 @@ const lowerMessages = Effect.fn("AnthropicMessages.lowerMessages")(function* (
breakpoints: Cache.Breakpoints,
) {
const messages: AnthropicMessage[] = []
const providerMetadataKey = request.model.route.providerMetadataKey ?? String(request.model.provider)
for (const [index, message] of request.messages.entries()) {
if (message.role === "system") {
@@ -849,8 +854,8 @@ const lowerMessages = Effect.fn("AnthropicMessages.lowerMessages")(function* (
if (part.type === "reasoning") {
// A signature marks visible thinking; only signature-less parts carrying
// redactedData round-trip as opaque redacted_thinking blocks.
const signature = part.encrypted ?? signatureFromMetadata(part.providerMetadata)
const redactedData = redactedDataFromMetadata(part.providerMetadata)
const signature = part.encrypted ?? signatureFromMetadata(part.providerMetadata, providerMetadataKey)
const redactedData = redactedDataFromMetadata(part.providerMetadata, providerMetadataKey)
if (signature === undefined && redactedData !== undefined) {
content.push({ type: "redacted_thinking", data: redactedData })
continue
@@ -879,7 +884,7 @@ const lowerMessages = Effect.fn("AnthropicMessages.lowerMessages")(function* (
continue
}
if (part.type === "tool-result" && part.providerExecuted) {
content.push(yield* lowerServerToolResult(part))
content.push(yield* lowerServerToolResult(part, providerMetadataKey))
continue
}
return yield* invalid(
@@ -1069,7 +1074,7 @@ const mapFinishReason = (reason: string | null | undefined): FinishReason => {
// inclusive `inputTokens` the rest of the contract expects. Extended
// thinking tokens are included in `output_tokens`; newer responses also
// expose that subset through `output_tokens_details.thinking_tokens`.
const mapUsage = (usage: AnthropicUsage | undefined): Usage | undefined => {
const mapUsage = (usage: AnthropicUsage | undefined, providerMetadataKey: string): Usage | undefined => {
if (!usage) return undefined
const nonCached = usage.input_tokens ?? undefined
const cacheRead = usage.cache_read_input_tokens ?? undefined
@@ -1083,7 +1088,7 @@ const mapUsage = (usage: AnthropicUsage | undefined): Usage | undefined => {
cacheWriteInputTokens: cacheWrite,
reasoningTokens: usage.output_tokens_details?.thinking_tokens,
totalTokens: ProviderShared.totalTokens(inputTokens, usage.output_tokens, undefined),
providerMetadata: { anthropic: usage },
providerMetadata: { [providerMetadataKey]: usage },
})
}
@@ -1092,7 +1097,7 @@ const mapUsage = (usage: AnthropicUsage | undefined): Usage | undefined => {
// field prefers `right` when defined, falls back to `left`. `inputTokens` is
// recomputed from the merged breakdown so the inclusive total stays
// consistent with `nonCached + cacheRead + cacheWrite`.
const mergeUsage = (left: Usage | undefined, right: Usage | undefined) => {
const mergeUsage = (left: Usage | undefined, right: Usage | undefined, providerMetadataKey: string) => {
if (!left) return right
if (!right) return left
const nonCachedInputTokens = right.nonCachedInputTokens ?? left.nonCachedInputTokens
@@ -1110,7 +1115,9 @@ const mergeUsage = (left: Usage | undefined, right: Usage | undefined) => {
reasoningTokens,
totalTokens: ProviderShared.totalTokens(inputTokens, outputTokens, undefined),
providerMetadata: {
anthropic: mergeJsonRecords(left.providerMetadata?.["anthropic"], right.providerMetadata?.["anthropic"]) ?? {},
[providerMetadataKey]:
mergeJsonRecords(left.providerMetadata?.[providerMetadataKey], right.providerMetadata?.[providerMetadataKey]) ??
{},
},
})
}
@@ -1128,7 +1135,7 @@ const SERVER_TOOL_RESULT_NAMES: Record<AnthropicServerToolResultType, string> =
const isServerToolResultType = (type: string): type is AnthropicServerToolResultType => type in SERVER_TOOL_RESULT_NAMES
const serverToolResultEvent = (block: AnthropicStreamBlock): LLMEvent | undefined => {
const serverToolResultEvent = (block: AnthropicStreamBlock, providerMetadataKey: string): LLMEvent | undefined => {
if (!block.type || !isServerToolResultType(block.type)) return undefined
const errorPayload =
typeof block.content === "object" && block.content !== null && "type" in block.content
@@ -1142,7 +1149,7 @@ const serverToolResultEvent = (block: AnthropicStreamBlock): LLMEvent | undefine
providerExecuted: true,
// The complete payload is irreducible provider replay state: subsequent
// stateless requests must round-trip the typed result block verbatim.
providerMetadata: anthropicMetadata({ blockType: block.type, result: block.content }),
providerMetadata: providerMetadata(providerMetadataKey, { blockType: block.type, result: block.content }),
})
}
@@ -1151,8 +1158,8 @@ type StepResult = readonly [ParserState, ReadonlyArray<LLMEvent>]
const NO_EVENTS: StepResult["1"] = []
const onMessageStart = (state: ParserState, event: AnthropicEvent): StepResult => {
const usage = mapUsage(event.message?.usage)
return [usage ? { ...state, usage: mergeUsage(state.usage, usage) } : state, NO_EVENTS]
const usage = mapUsage(event.message?.usage, state.providerMetadataKey)
return [usage ? { ...state, usage: mergeUsage(state.usage, usage, state.providerMetadataKey) } : state, NO_EVENTS]
}
const onContentBlockStart = (
@@ -1204,14 +1211,16 @@ const onContentBlockStart = (
if (block.type === "thinking" && block.thinking !== undefined) {
const events: LLMEvent[] = []
const id = `reasoning-${event.index ?? 0}`
const providerMetadata =
block.signature === undefined ? undefined : anthropicMetadata({ signature: block.signature })
const lifecycle = Lifecycle.reasoningStart(state.lifecycle, events, id, providerMetadata)
const metadata =
block.signature === undefined
? undefined
: providerMetadata(state.providerMetadataKey, { signature: block.signature })
const lifecycle = Lifecycle.reasoningStart(state.lifecycle, events, id, metadata)
return [
{
...state,
lifecycle: block.thinking
? Lifecycle.reasoningDelta(lifecycle, events, id, block.thinking, providerMetadata)
? Lifecycle.reasoningDelta(lifecycle, events, id, block.thinking, metadata)
: lifecycle,
reasoningSignatures:
event.index === undefined || block.signature === undefined
@@ -1234,14 +1243,14 @@ const onContentBlockStart = (
state.lifecycle,
events,
`reasoning-${event.index ?? 0}`,
anthropicMetadata({ redactedData: block.data }),
providerMetadata(state.providerMetadataKey, { redactedData: block.data }),
),
},
events,
]
}
const result = serverToolResultEvent(block)
const result = serverToolResultEvent(block, state.providerMetadataKey)
if (!result) return [state, NO_EVENTS]
const events: LLMEvent[] = []
return [{ ...state, lifecycle: Lifecycle.stepStart(state.lifecycle, events) }, [...events, result]]
@@ -1321,7 +1330,7 @@ const onContentBlockStop = Effect.fn("AnthropicMessages.onContentBlockStop")(fun
Lifecycle.textEnd(state.lifecycle, events, `text-${event.index}`),
events,
`reasoning-${event.index}`,
signature === undefined ? undefined : anthropicMetadata({ signature }),
signature === undefined ? undefined : providerMetadata(state.providerMetadataKey, { signature }),
)
events.push(...resultEvents)
const reasoningSignatures = { ...state.reasoningSignatures }
@@ -1333,7 +1342,7 @@ const onMessageDelta = (
state: ParserState,
event: AnthropicEvent & { readonly delta?: AnthropicStreamDelta },
): StepResult => {
const usage = mergeUsage(state.usage, mapUsage(event.usage))
const usage = mergeUsage(state.usage, mapUsage(event.usage, state.providerMetadataKey), state.providerMetadataKey)
return [
{
...state,
@@ -1346,7 +1355,7 @@ const onMessageDelta = (
providerMetadata:
event.delta?.stop_sequence === null || event.delta?.stop_sequence === undefined
? undefined
: anthropicMetadata({ stopSequence: event.delta.stop_sequence }),
: providerMetadata(state.providerMetadataKey, { stopSequence: event.delta.stop_sequence }),
},
},
NO_EVENTS,
@@ -1378,14 +1387,15 @@ const providerErrorMessage = (event: AnthropicEvent): string => {
return message || type || "Anthropic Messages stream error"
}
const onError = (event: AnthropicEvent) =>
Effect.fail(
const onError = (event: AnthropicEvent) => {
const message = providerErrorMessage(event)
const body = ProviderShared.encodeJson(event)
return Effect.fail(
new AIError({
module: ADAPTER,
method: "stream",
reason: classifyProviderFailure({ message: providerErrorMessage(event), code: event.error?.type }),
reason: classifyProviderFailure({ message, rawBody: body }),
}),
)
}
const isKnownStreamBlockType = (type: string) =>
type === "text" ||
@@ -1472,7 +1482,8 @@ export const protocol = Protocol.make({
},
stream: {
event: Protocol.jsonEvent(AnthropicEvent),
initial: () => ({
initial: (request) => ({
providerMetadataKey: request.model.route.providerMetadataKey ?? String(request.model.provider),
tools: ToolStream.empty<number>(),
reasoningSignatures: {},
lifecycle: Lifecycle.initial(),
+29 -32
View File
@@ -258,19 +258,21 @@ const lowerToolChoice = (toolChoice: NonNullable<LLMRequest["toolChoice"]>) =>
tool: (name) => ({ tool: { name } }) as const,
})
const bedrockMetadata = (metadata: Record<string, unknown>): ProviderMetadata => ({ bedrock: metadata })
const providerMetadata = (key: string, metadata: Record<string, unknown>): ProviderMetadata => ({ [key]: metadata })
const reasoningSignature = (part: ReasoningPart) => {
const bedrock = part.providerMetadata?.bedrock
const reasoningSignature = (part: ReasoningPart, providerMetadataKey: string) => {
const metadata = part.providerMetadata?.[providerMetadataKey]
return (
part.encrypted ??
(ProviderShared.isRecord(bedrock) && typeof bedrock.signature === "string" ? bedrock.signature : undefined)
(ProviderShared.isRecord(metadata) && typeof metadata.signature === "string" ? metadata.signature : undefined)
)
}
const reasoningRedactedData = (part: ReasoningPart) => {
const bedrock = part.providerMetadata?.bedrock
return ProviderShared.isRecord(bedrock) && typeof bedrock.redactedData === "string" ? bedrock.redactedData : undefined
const reasoningRedactedData = (part: ReasoningPart, providerMetadataKey: string) => {
const metadata = part.providerMetadata?.[providerMetadataKey]
return ProviderShared.isRecord(metadata) && typeof metadata.redactedData === "string"
? metadata.redactedData
: undefined
}
const lowerToolCall = (part: ToolCallPart): BedrockToolUseBlock => ({
@@ -318,6 +320,7 @@ const lowerMessages = Effect.fn("BedrockConverse.lowerMessages")(function* (
breakpoints: BedrockCache.Breakpoints,
) {
const messages: BedrockMessage[] = []
const providerMetadataKey = request.model.route.providerMetadataKey ?? String(request.model.provider)
for (const message of request.messages) {
if (message.role === "system") {
@@ -365,8 +368,8 @@ const lowerMessages = Effect.fn("BedrockConverse.lowerMessages")(function* (
continue
}
if (part.type === "reasoning") {
const signature = reasoningSignature(part)
const redactedData = reasoningRedactedData(part)
const signature = reasoningSignature(part, providerMetadataKey)
const redactedData = reasoningRedactedData(part, providerMetadataKey)
if (signature === undefined && redactedData !== undefined) {
content.push({ reasoningContent: { redactedContent: redactedData } })
continue
@@ -466,7 +469,7 @@ const mapFinishReason = (reason: string): FinishReason => {
// AWS reports inputTokens separately from cache reads and writes.
// Bedrock does not break reasoning out of outputTokens for current models.
const mapUsage = (usage: BedrockUsageSchema | undefined): Usage | undefined => {
const mapUsage = (usage: BedrockUsageSchema | undefined, providerMetadataKey: string): Usage | undefined => {
if (!usage) return undefined
const inputTokens = ProviderShared.sumTokens(
usage.inputTokens,
@@ -480,11 +483,12 @@ const mapUsage = (usage: BedrockUsageSchema | undefined): Usage | undefined => {
cacheReadInputTokens: usage.cacheReadInputTokens,
cacheWriteInputTokens: usage.cacheWriteInputTokens,
totalTokens: ProviderShared.totalTokens(inputTokens, usage.outputTokens, usage.totalTokens),
providerMetadata: { bedrock: usage },
providerMetadata: { [providerMetadataKey]: usage },
})
}
interface ParserState {
readonly providerMetadataKey: string
readonly tools: ToolStream.State<number>
// Bedrock splits the finish into `messageStop` (carries `stopReason`) and
// `metadata` (carries usage). Hold the terminal event in state so `onHalt`
@@ -541,20 +545,14 @@ const step = (state: ParserState, event: BedrockEvent) =>
const reasoning = event.contentBlockDelta.delta.reasoningContent
const events: LLMEvent[] = []
const redactedData = reasoning.redactedContent ?? reasoning.data
const providerMetadata = reasoning.signature
? bedrockMetadata({ signature: reasoning.signature })
const metadata = reasoning.signature
? providerMetadata(state.providerMetadataKey, { signature: reasoning.signature })
: redactedData !== undefined
? bedrockMetadata({ redactedData })
? providerMetadata(state.providerMetadataKey, { redactedData })
: undefined
const lifecycle =
reasoning.text !== undefined || providerMetadata !== undefined
? Lifecycle.reasoningDelta(
state.lifecycle,
events,
`reasoning-${index}`,
reasoning.text ?? "",
providerMetadata,
)
reasoning.text !== undefined || metadata !== undefined
? Lifecycle.reasoningDelta(state.lifecycle, events, `reasoning-${index}`, reasoning.text ?? "", metadata)
: state.lifecycle
return [
{
@@ -596,7 +594,7 @@ const step = (state: ParserState, event: BedrockEvent) =>
events,
`reasoning-${index}`,
state.reasoningSignatures[index]
? bedrockMetadata({ signature: state.reasoningSignatures[index] })
? providerMetadata(state.providerMetadataKey, { signature: state.reasoningSignatures[index] })
: undefined,
)
events.push(...resultEvents)
@@ -633,7 +631,7 @@ const step = (state: ParserState, event: BedrockEvent) =>
}
if (event.metadata) {
const usage = mapUsage(event.metadata.usage) ?? state.pendingFinish?.usage
const usage = mapUsage(event.metadata.usage, state.providerMetadataKey) ?? state.pendingFinish?.usage
return [
{
...state,
@@ -647,15 +645,13 @@ const step = (state: ParserState, event: BedrockEvent) =>
}
if (event.exception) {
const message =
event.exception.details.message ?? event.exception.details.originalMessage ?? "Bedrock Converse stream error"
const body = ProviderShared.encodeJson(event)
return yield* new AIError({
module: ADAPTER,
method: "stream",
reason: classifyProviderFailure({
message:
event.exception.details.message ??
event.exception.details.originalMessage ??
"Bedrock Converse stream error",
code: event.exception.type,
message,
rawBody: body,
}),
})
}
@@ -698,7 +694,8 @@ export const protocol = Protocol.make({
},
stream: {
event: BedrockEvent,
initial: () => ({
initial: (request) => ({
providerMetadataKey: request.model.route.providerMetadataKey ?? String(request.model.provider),
tools: ToolStream.empty<number>(),
pendingFinish: undefined,
hasToolCalls: false,
@@ -1,6 +1,7 @@
import { EventStreamCodec } from "@smithy/eventstream-codec"
import { fromUtf8, toUtf8 } from "@smithy/util-utf8"
import { Effect, Stream } from "effect"
import { Effect, Encoding, Stream } from "effect"
import { AIError, AIErrorReason } from "../schema/index.js"
import { Framing } from "../route/framing.js"
import { ProviderShared } from "./shared.js"
@@ -49,10 +50,14 @@ const consumeFrames = (route: string) => (state: FrameBufferState, chunk: Uint8A
`Failed to decode Bedrock Converse event-stream frame: ${
error instanceof Error ? error.message : String(error)
}`,
Encoding.encodeBase64(view.subarray(0, totalLength)),
error,
),
})
cursor = { buffer: cursor.buffer, offset: cursor.offset + totalLength }
const payload = utf8.decode(decoded.body)
const body = ProviderShared.encodeJson({ headers: decoded.headers, body: payload })
const messageType = decoded.headers[":message-type"]?.value
if (messageType === "error") {
const code = decoded.headers[":error-code"]?.value
@@ -61,6 +66,7 @@ const consumeFrames = (route: string) => (state: FrameBufferState, chunk: Uint8A
route,
[code, message].filter((value): value is string => typeof value === "string").join(": ") ||
"Bedrock Converse event-stream error",
body,
)
}
const eventType =
@@ -70,7 +76,6 @@ const consumeFrames = (route: string) => (state: FrameBufferState, chunk: Uint8A
? decoded.headers[":exception-type"]?.value
: undefined
if (typeof eventType !== "string") continue
const payload = utf8.decode(decoded.body)
if (!payload) continue
// The AWS event stream pads short payloads with a `p` field. Drop it
// before handing the object to the chunk schema. JSON decode goes
@@ -80,11 +85,21 @@ const consumeFrames = (route: string) => (state: FrameBufferState, chunk: Uint8A
route,
payload,
"Failed to parse Bedrock Converse event-stream payload",
).pipe(
Effect.mapError(
(error) =>
new AIError({
reason: AIErrorReason.make({ ...error.reason, message: error.message, cause: error.reason.cause, body }),
}),
),
)) as Record<string, unknown>
delete parsed.p
out.push(
messageType === "exception" ? { exception: { type: eventType, details: parsed } } : { [eventType]: parsed },
)
out.push({
...(messageType === "exception"
? { exception: { type: eventType, details: parsed } }
: { [eventType]: parsed }),
rawBody: body,
})
}
return [cursor, out] as const
})
@@ -97,6 +112,7 @@ const consumeFrames = (route: string) => (state: FrameBufferState, chunk: Uint8A
*/
export const framing = (route: string): Framing.Definition<object> => ({
id: "aws-event-stream",
body: (frame) => ("rawBody" in frame && typeof frame.rawBody === "string" ? frame.rawBody : undefined),
frame: (bytes) => bytes.pipe(Stream.mapAccumEffect(() => initialFrameBuffer, consumeFrames(route))),
})
+51 -21
View File
@@ -6,6 +6,7 @@ import { Endpoint } from "../route/endpoint.js"
import { Framing } from "../route/framing.js"
import { Protocol } from "../route/protocol.js"
import {
AIError,
LLMEvent,
Usage,
type FinishReason,
@@ -17,6 +18,7 @@ import {
type ToolCallPart,
type ToolDefinition,
} from "../schema/index.js"
import { classifyProviderFailure } from "../provider-error.js"
import { JsonObject, optionalArray, optionalNull, ProviderShared } from "./shared.js"
import { GeminiToolSchema } from "./utils/gemini-tool-schema.js"
import { Lifecycle } from "./utils/lifecycle.js"
@@ -221,6 +223,7 @@ const GeminiPromptFeedback = Schema.StructWithRest(
type GeminiPromptFeedback = Schema.Schema.Type<typeof GeminiPromptFeedback>
const GeminiEvent = Schema.Struct({
error: Schema.optional(Schema.Unknown),
candidates: optionalNull(Schema.Array(GeminiCandidate)),
promptFeedback: optionalNull(GeminiPromptFeedback),
usageMetadata: optionalNull(GeminiUsage),
@@ -229,6 +232,7 @@ type GeminiEvent = Schema.Schema.Type<typeof GeminiEvent>
interface ParserState {
readonly route: string
readonly providerMetadataKey: string
readonly finishReason?: string
readonly hasToolCalls: boolean
readonly promptFeedback?: GeminiPromptFeedback
@@ -285,22 +289,23 @@ const lowerUserPart = Effect.fn("Gemini.lowerUserPart")(function* (part: TextPar
return { inlineData: { mimeType: media.mime, data: media.base64 } }
})
const googleMetadata = (metadata: Record<string, unknown>): ProviderMetadata => ({ google: metadata })
const providerMetadata = (key: string, metadata: Record<string, unknown>): ProviderMetadata => ({ [key]: metadata })
const thoughtSignature = (providerMetadata: ProviderMetadata | undefined) => {
const google = providerMetadata?.google
return ProviderShared.isRecord(google) && typeof google.thoughtSignature === "string"
? google.thoughtSignature
const thoughtSignature = (metadata: ProviderMetadata | undefined, key: string) => {
const value = metadata?.[key]
return ProviderShared.isRecord(value) && typeof value.thoughtSignature === "string"
? value.thoughtSignature
: undefined
}
const lowerToolCall = (part: ToolCallPart, omitIds: boolean) => ({
const lowerToolCall = (part: ToolCallPart, omitIds: boolean, metadataKey: string) => ({
functionCall: { ...(omitIds ? {} : { id: part.id }), name: part.name, args: part.input },
thoughtSignature: thoughtSignature(part.providerMetadata),
thoughtSignature: thoughtSignature(part.providerMetadata, metadataKey),
})
const lowerMessages = Effect.fn("Gemini.lowerMessages")(function* (request: LLMRequest) {
const contents: GeminiContent[] = []
const metadataKey = request.model.route.providerMetadataKey ?? String(request.model.provider)
const omitCallIds = omitsFunctionCallIds(request.model.id)
const legacyToolMedia = routesLegacyToolMedia(request.model.id)
let pendingMedia: GeminiInlineDataPart[] | undefined
@@ -342,15 +347,19 @@ const lowerMessages = Effect.fn("Gemini.lowerMessages")(function* (request: LLMR
if (!ProviderShared.supportsContent(part, ["text", "reasoning", "tool-call"]))
return yield* ProviderShared.unsupportedContent("Gemini", "assistant", ["text", "reasoning", "tool-call"])
if (part.type === "text") {
parts.push({ text: part.text, thoughtSignature: thoughtSignature(part.providerMetadata) })
parts.push({ text: part.text, thoughtSignature: thoughtSignature(part.providerMetadata, metadataKey) })
continue
}
if (part.type === "reasoning") {
parts.push({ text: part.text, thought: true, thoughtSignature: thoughtSignature(part.providerMetadata) })
parts.push({
text: part.text,
thought: true,
thoughtSignature: thoughtSignature(part.providerMetadata, metadataKey),
})
continue
}
if (part.type === "tool-call") {
const lowered = lowerToolCall(part, omitCallIds)
const lowered = lowerToolCall(part, omitCallIds, metadataKey)
const signature = lowered.thoughtSignature
parts.push({
...lowered,
@@ -498,7 +507,7 @@ const fromRequest = Effect.fn("Gemini.fromRequest")(function* (request: LLMReque
// `cachedContentTokenCount` subset. `candidatesTokenCount` is *exclusive*
// of `thoughtsTokenCount` — visible-only, not a total — so we sum the two
// to produce the inclusive `outputTokens` the rest of the contract expects.
const mapUsage = (usage: GeminiUsage | undefined) => {
const mapUsage = (usage: GeminiUsage | undefined, metadataKey: string) => {
if (!usage) return undefined
// Explicit provider nulls decode as `null`; normalize to `undefined` so the
// token arithmetic below treats them like absent counts.
@@ -519,7 +528,7 @@ const mapUsage = (usage: GeminiUsage | undefined) => {
cacheReadInputTokens: cached,
reasoningTokens: thoughts,
totalTokens: ProviderShared.totalTokens(promptTokens, outputTokens, usage.totalTokenCount ?? undefined),
providerMetadata: { google: usage },
providerMetadata: providerMetadata(metadataKey, usage),
})
}
@@ -567,14 +576,14 @@ const finish = (state: ParserState): ReadonlyArray<LLMEvent> => {
lifecycle,
events,
"reasoning-0",
googleMetadata({ thoughtSignature: state.reasoningSignature }),
providerMetadata(state.providerMetadataKey, { thoughtSignature: state.reasoningSignature }),
)
if (state.textSignature !== undefined)
lifecycle = Lifecycle.textEnd(
lifecycle,
events,
"text-0",
googleMetadata({ thoughtSignature: state.textSignature }),
providerMetadata(state.providerMetadataKey, { thoughtSignature: state.textSignature }),
)
Lifecycle.finish(lifecycle, events, {
reason: {
@@ -584,16 +593,32 @@ const finish = (state: ParserState): ReadonlyArray<LLMEvent> => {
},
usage: state.usage,
providerMetadata:
state.promptFeedback === undefined ? undefined : googleMetadata({ promptFeedback: state.promptFeedback }),
state.promptFeedback === undefined
? undefined
: providerMetadata(state.providerMetadataKey, { promptFeedback: state.promptFeedback }),
})
return events
}
const step = (state: ParserState, event: GeminiEvent) => {
if (ProviderShared.isRecord(event.error) && typeof event.error.message === "string") {
const body = ProviderShared.encodeJson(event)
return Effect.fail(
new AIError({
reason: classifyProviderFailure({
message: event.error.message,
status: typeof event.error.code === "number" ? event.error.code : undefined,
rawBody: body,
}),
}),
)
}
const nextState = {
...state,
promptFeedback: event.promptFeedback ?? state.promptFeedback,
usage: event.usageMetadata ? (mapUsage(event.usageMetadata) ?? state.usage) : state.usage,
usage: event.usageMetadata
? (mapUsage(event.usageMetadata, state.providerMetadataKey) ?? state.usage)
: state.usage,
}
const candidate = event.candidates?.[0]
if (!candidate?.content)
@@ -637,7 +662,7 @@ const step = (state: ParserState, event: GeminiEvent) => {
events,
"reasoning-0",
part.text,
signature ? googleMetadata({ thoughtSignature: signature }) : undefined,
signature ? providerMetadata(state.providerMetadataKey, { thoughtSignature: signature }) : undefined,
)
continue
}
@@ -645,14 +670,16 @@ const step = (state: ParserState, event: GeminiEvent) => {
lifecycle,
events,
"reasoning-0",
reasoningSignature ? googleMetadata({ thoughtSignature: reasoningSignature }) : undefined,
reasoningSignature
? providerMetadata(state.providerMetadataKey, { thoughtSignature: reasoningSignature })
: undefined,
)
lifecycle = Lifecycle.textDelta(
lifecycle,
events,
"text-0",
part.text,
textSignature ? googleMetadata({ thoughtSignature: textSignature }) : undefined,
textSignature ? providerMetadata(state.providerMetadataKey, { thoughtSignature: textSignature }) : undefined,
)
textSignature = undefined
continue
@@ -672,7 +699,9 @@ const step = (state: ParserState, event: GeminiEvent) => {
lifecycle,
events,
"reasoning-0",
reasoningSignature ? googleMetadata({ thoughtSignature: reasoningSignature }) : undefined,
reasoningSignature
? providerMetadata(state.providerMetadataKey, { thoughtSignature: reasoningSignature })
: undefined,
)
lifecycle = Lifecycle.stepStart(lifecycle, events)
events.push(
@@ -681,7 +710,7 @@ const step = (state: ParserState, event: GeminiEvent) => {
name: part.functionCall.name,
input,
providerMetadata: part.thoughtSignature
? googleMetadata({ thoughtSignature: part.thoughtSignature })
? providerMetadata(state.providerMetadataKey, { thoughtSignature: part.thoughtSignature })
: undefined,
}),
)
@@ -720,6 +749,7 @@ export const protocol = Protocol.make({
event: Protocol.jsonEvent(GeminiEvent),
initial: (request) => ({
route: `${request.model.provider}/${request.model.route.id}`,
providerMetadataKey: request.model.route.providerMetadataKey ?? String(request.model.provider),
hasToolCalls: false,
lifecycle: Lifecycle.initial(),
}),
+11 -34
View File
@@ -9,15 +9,7 @@ import {
type ImageRoute,
} from "../image.js"
import { Auth, type Definition as AuthDefinition } from "../route/auth.js"
import {
InvalidProviderOutputReason,
AIError,
Usage,
mergeHttpOptions,
mergeJsonRecords,
type HttpOptions,
type ProviderMetadata,
} from "../schema/index.js"
import { AIError, Usage, mergeHttpOptions, mergeJsonRecords, type HttpOptions } from "../schema/index.js"
import { ProviderShared } from "./shared.js"
import { ImageInputs } from "./utils/image-input.js"
@@ -124,13 +116,6 @@ const nativeOptions = (options: GoogleImageOptions | undefined) => {
)
}
const invalidOutput = (message: string, providerMetadata?: ProviderMetadata) =>
new AIError({
module: ADAPTER,
method: "generate",
reason: new InvalidProviderOutputReason({ message, route: ADAPTER, providerMetadata }),
})
const applyQuery = (url: string, query: Record<string, string> | undefined) => {
if (!query) return url
const next = new URL(url)
@@ -169,11 +154,9 @@ export const model = (input: ModelInput) => {
HttpClientRequest.bodyText(text, "application/json"),
),
)
const payload = yield* response.json.pipe(
Effect.mapError(() => invalidOutput("Failed to read the Google Images response")),
)
const decoded = yield* Schema.decodeUnknownEffect(GoogleImageResponse)(payload).pipe(
Effect.mapError(() => invalidOutput("Google Images returned an invalid response")),
const output = yield* ProviderShared.imageResponse(ADAPTER, "Google Images", response)
const decoded = yield* Schema.decodeUnknownEffect(Schema.fromJsonString(GoogleImageResponse))(output.body).pipe(
Effect.mapError((cause) => output.invalid("Google Images returned an invalid response", cause)),
)
const candidates = decoded.candidates ?? []
const candidateMetadata = candidates.map((candidate, candidateIndex) => ({
@@ -208,9 +191,10 @@ export const model = (input: ModelInput) => {
)
const images = yield* Effect.forEach(encoded, (item) =>
Effect.fromResult(Encoding.decodeBase64(item.inlineData.data)).pipe(
Effect.mapError(() =>
invalidOutput(
Effect.mapError((cause) =>
output.invalid(
`Google Images candidate ${item.candidateIndex} part ${item.partIndex} contains invalid base64 data`,
cause,
),
),
Effect.map(
@@ -237,16 +221,10 @@ export const model = (input: ModelInput) => {
const finishReasons = candidates.flatMap((candidate) =>
candidate.finishReason === undefined ? [] : [candidate.finishReason],
)
return yield* invalidOutput(
return yield* output.invalid(
`Google Images returned no final images${
finishReasons.length === 0 ? "" : ` (finish reasons: ${finishReasons.join(", ")})`
}; inspect reason.providerMetadata.google for prompt feedback and candidate details`,
{
google: {
promptFeedback: decoded.promptFeedback,
candidates: candidateMetadata,
},
},
}; inspect body for prompt feedback and candidate details`,
)
}
const usage = decoded.usageMetadata
@@ -290,12 +268,11 @@ const googleImagePart = (image: ImageInput): Effect.Effect<Record<string, unknow
return Effect.succeed({ inlineData: { mimeType: image.mediaType, data: Encoding.encodeBase64(image.data) } })
if (image.type === "file-uri") return Effect.succeed({ fileData: { mimeType: image.mediaType, fileUri: image.uri } })
if (image.type === "url")
return ImageInputs.decodeDataUrl(image.url, ADAPTER).pipe(
return ImageInputs.decodeDataUrl(image.url).pipe(
Effect.flatMap((decoded) => {
if (decoded === undefined)
return Effect.fail(
ImageInputs.invalid(
ADAPTER,
"Google generateContent does not fetch public image URLs; use bytes, a data URL, or a Gemini file URI",
),
)
@@ -305,7 +282,7 @@ const googleImagePart = (image: ImageInput): Effect.Effect<Record<string, unknow
}),
)
return Effect.fail(
ImageInputs.invalid(ADAPTER, "Google generateContent requires Gemini file URIs rather than provider file IDs"),
ImageInputs.invalid("Google generateContent requires Gemini file URIs rather than provider file IDs"),
)
}
@@ -1,6 +1,7 @@
import { Effect, Schema, Stream } from "effect"
import { Headers } from "effect/unstable/http"
import { Framing } from "../route/framing.js"
import type { HttpContext } from "../schema/index.js"
import {
HttpTransport,
WebSocketTransport,
@@ -60,8 +61,8 @@ const driver = (options: Options, body: string): WebSocketChannelDriver => {
observe: (_create, frame) =>
Effect.gen(function* () {
const event = yield* decodeEvent(frame).pipe(
Effect.mapError(() =>
ProviderShared.eventError(options.id, `Invalid ${options.name} WebSocket event`, frame),
Effect.mapError((cause) =>
ProviderShared.eventError(options.id, `Invalid ${options.name} WebSocket event`, frame, cause),
),
)
if (terminal)
@@ -73,13 +74,13 @@ const driver = (options: Options, body: string): WebSocketChannelDriver => {
if (event.type === "error") {
terminal = true
yield* OpenResponses.decodeKnownErrorEvent(event).pipe(
Effect.mapError(() =>
ProviderShared.eventError(options.id, `${options.name} returned a malformed error event`, frame),
Effect.mapError((cause) =>
ProviderShared.eventError(options.id, `${options.name} returned a malformed error event`, frame, cause),
),
)
return {
type: "provider-failure",
error: OpenResponses.providerFailure(options.id, event, `${options.name} stream error`),
error: OpenResponses.providerFailure(event, `${options.name} stream error`, frame),
}
}
if (event.type === "response.failed") {
@@ -92,7 +93,7 @@ const driver = (options: Options, body: string): WebSocketChannelDriver => {
)
return {
type: "provider-failure",
error: OpenResponses.providerFailure(options.id, event, `${options.name} response failed`),
error: OpenResponses.providerFailure(event, `${options.name} response failed`, frame),
}
}
if (event.type === "response.created") {
@@ -175,23 +176,37 @@ export const transport = <Body>(options: Options): Transport<Body, Prepared, str
channel,
}
}),
execute: (prepared, request, runtime, executeOptions) => {
if (!executeOptions?.webSocket || !prepared.channel) return http.execute(prepared.http, request, runtime)
const exchange: WebSocketChannelExchange = {
id: request.id ?? "request",
connect: {
url: prepared.channel.url,
headers: prepared.channel.headers,
rotateAfterMs: prepared.channel.rotateAfterMs,
},
fallback: () =>
Stream.unwrap(
http.execute(prepared.http, request, runtime).pipe(Effect.map((execution) => execution.frames)),
),
driver: prepared.channel.driver,
}
return executeOptions.webSocket.execute(exchange)
},
execute: (prepared, request, runtime, executeOptions) =>
Effect.gen(function* () {
if (!executeOptions?.webSocket || !prepared.channel) return yield* http.execute(prepared.http, request, runtime)
let fallbackHttp: HttpContext | undefined
const exchange: WebSocketChannelExchange = {
id: request.id ?? "request",
connect: {
url: prepared.channel.url,
headers: prepared.channel.headers,
rotateAfterMs: prepared.channel.rotateAfterMs,
},
fallback: () =>
Stream.unwrap(
http.execute(prepared.http, request, runtime).pipe(
Effect.map((execution) => {
fallbackHttp = execution.http
return execution.frames
}),
),
),
driver: prepared.channel.driver,
}
const execution = yield* executeOptions.webSocket.execute(exchange)
return {
frames: execution.frames,
complete: execution.complete,
get http() {
return fallbackHttp ?? execution.http
},
}
}),
}
}
@@ -1,4 +1,4 @@
import { AIError, TransportReason } from "../schema/index.js"
import { AIError, TransportError } from "../schema/index.js"
import type { ChannelCheckpoint, ChannelObservation, WebSocketChannelDriver } from "../route/transport/index.js"
import { Effect, Option, Schema } from "effect"
import * as ProviderShared from "./shared.js"
@@ -99,17 +99,17 @@ const incremental = (
const code = (event: OpenResponses.Event) => event.code || event.error?.code || event.response?.error?.code || undefined
const rejected = (
input: DriverInput,
observation: Extract<ChannelObservation, { readonly type: "provider-failure" }>,
recovery: "retry-full" | "rotate-and-retry-full",
): ChannelObservation => ({
type: "rejected",
recovery,
error: new AIError({
module: input.id,
method: "stream",
reason: new TransportReason({
reason: new TransportError({
message: observation.error.message,
body: observation.error.reason.body,
http: observation.error.reason.http,
cause: observation.error.reason.cause,
transport: "websocket",
operation: "read",
phase: "receive",
@@ -137,15 +137,16 @@ export const driver = (input: DriverInput): WebSocketChannelDriver => {
observe: (create, frame) =>
Effect.gen(function* () {
const event = yield* decodeEvent(frame).pipe(
Effect.mapError(() => ProviderShared.eventError(input.id, `Invalid ${input.name} WebSocket event`, frame)),
Effect.mapError((cause) =>
ProviderShared.eventError(input.id, `Invalid ${input.name} WebSocket event`, frame, cause),
),
)
const observation = yield* input.base.observe(create, frame)
if (event.type === "response.output_item.done" && event.item) output.push(event.item)
if (observation.type === "provider-failure") {
const rejection = code(event)
if (rejection === "previous_response_not_found") return rejected(input, observation, "retry-full")
if (rejection === "websocket_connection_limit_reached")
return rejected(input, observation, "rotate-and-retry-full")
if (rejection === "previous_response_not_found") return rejected(observation, "retry-full")
if (rejection === "websocket_connection_limit_reached") return rejected(observation, "rotate-and-retry-full")
}
if (observation.type !== "completed") return observation
const responseID = event.response?.id
+15 -18
View File
@@ -5,7 +5,7 @@ import { Protocol } from "../route/protocol.js"
import {
AIError,
LLMEvent,
ProviderInternalReason,
ProviderInternalError,
Usage,
type FinishReason,
type JsonSchema,
@@ -1188,11 +1188,8 @@ const providerErrorMessage = (event: Event, nested: OpenResponsesErrorPayload |
return message || code
}
export const providerFailure = (id: string, event: Event, fallback: string) => {
export const providerFailure = (event: Event, fallback: string, body = ProviderShared.encodeJson(event)) => {
const nested = event.error ?? event.response?.error ?? undefined
const code = event.code || nested?.code || undefined
// Keep the full raw payload on the error even when the message is a summary.
const body = JSON.stringify(nested ?? event) ?? ""
const summary = providerErrorMessage(event, nested)
const message = summary ?? (body === "{}" ? fallback : body)
const status =
@@ -1207,18 +1204,11 @@ export const providerFailure = (id: string, event: Event, fallback: string) => {
event.response === undefined &&
summary === undefined &&
status === undefined
? new ProviderInternalReason({ message })
: classifyProviderFailure({ message, code, status, rawBody: body })
return new AIError({
module: id,
method: "stream",
body,
reason,
})
? new ProviderInternalError({ message, body })
: classifyProviderFailure({ message, status, rawBody: body })
return new AIError({ reason })
}
const providerError = (state: ParserState, event: Event, fallback: string) => providerFailure(state.id, event, fallback)
export const step = (state: ParserState, input: Event) => {
// The OpenAPI requires string IDs but imposes no minLength; empty is not missing.
const event =
@@ -1286,11 +1276,18 @@ export const step = (state: ParserState, input: Event) => {
return onOutputItemDone(state, event)
}
if (event.type === "response.completed" || event.type === "response.incomplete") return onResponseFinish(state, event)
if (event.type === "response.failed") return providerError(state, event, `${state.name} response failed`)
if (event.type === "response.failed") return providerFailure(event, `${state.name} response failed`)
if (event.type === "error")
return decodeKnownErrorEvent(event).pipe(
Effect.mapError(() => ProviderShared.eventError(state.id, `${state.name} returned a malformed error event`)),
Effect.flatMap(() => providerError(state, event, `${state.name} stream error`)),
Effect.mapError((cause) =>
ProviderShared.eventError(
state.id,
`${state.name} returned a malformed error event`,
ProviderShared.encodeJson(event),
cause,
),
),
Effect.flatMap(() => providerFailure(event, `${state.name} stream error`)),
)
return Effect.succeed<StepResult>([state, NO_EVENTS])
}
+75 -47
View File
@@ -7,10 +7,11 @@ import { HttpTransport } from "../route/transport/index.js"
import { Protocol } from "../route/protocol.js"
import {
AIError,
InvalidProviderOutputReason,
AIErrorReason,
InvalidProviderOutputError,
LLMEvent,
ProviderInternalReason,
UnknownProviderReason,
ProviderInternalError,
UnknownProviderError,
Usage,
type FinishReason,
type FinishReasonDetails,
@@ -253,6 +254,7 @@ interface PendingToolDelta {
}
export interface ParserState {
readonly providerMetadataKey: string
readonly tools: ToolStream.State<number>
readonly pendingTools: Partial<Record<number, PendingToolDelta>>
readonly toolCallEvents: ReadonlyArray<LLMEvent>
@@ -324,17 +326,18 @@ const lowerMedia = Effect.fn("OpenAIChat.lowerMedia")(function* (part: MediaPart
const openAICompatibleReasoningContent = (native: unknown) =>
isRecord(native) && typeof native.reasoning_content === "string" ? native.reasoning_content : undefined
const reasoningField = (part: ReasoningPart) => {
const field = part.providerMetadata?.openai?.reasoningField
const reasoningField = (part: ReasoningPart, providerMetadataKey: string) => {
const field = part.providerMetadata?.[providerMetadataKey]?.reasoningField
return typeof field === "string" ? field : undefined
}
const reasoningDetails = (parts: ReadonlyArray<ReasoningPart>, native: unknown) => {
const reasoningDetails = (parts: ReadonlyArray<ReasoningPart>, native: unknown, providerMetadataKey: string) => {
const observed = parts.flatMap((part) => {
const details = part.providerMetadata?.openai?.reasoningDetails
const details = part.providerMetadata?.[providerMetadataKey]?.reasoningDetails
return Array.isArray(details) ? details : []
})
if (parts.some((part) => Array.isArray(part.providerMetadata?.openai?.reasoningDetails))) return observed
if (parts.some((part) => Array.isArray(part.providerMetadata?.[providerMetadataKey]?.reasoningDetails)))
return observed
if (isRecord(native) && Array.isArray(native.reasoning_details)) return native.reasoning_details
}
@@ -366,7 +369,7 @@ const lowerAssistantMessage = Effect.fn("OpenAIChat.lowerAssistantMessage")(func
message: OpenAIChatRequestMessage,
configuredField: string | undefined,
requireReasoning: boolean,
options: LoweringOptions,
options: LoweringOptions & { readonly providerMetadataKey: string },
) {
const content: TextPart[] = []
const reasoning: ReasoningPart[] = []
@@ -388,10 +391,14 @@ const lowerAssistantMessage = Effect.fn("OpenAIChat.lowerAssistantMessage")(func
}
}
const text = reasoning.map((part) => part.text).join("")
const details = reasoningDetails(reasoning, message.native?.openaiCompatible)
const observedField = reasoning.map(reasoningField).find((value) => value !== undefined)
const details = reasoningDetails(reasoning, message.native?.openaiCompatible, options.providerMetadataKey)
const observedField = reasoning
.map((part) => reasoningField(part, options.providerMetadataKey))
.find((value) => value !== undefined)
const nativeReasoning = openAICompatibleReasoningContent(message.native?.openaiCompatible)
const fullyStructured = reasoning.every((part) => Array.isArray(part.providerMetadata?.openai?.reasoningDetails))
const fullyStructured = reasoning.every((part) =>
Array.isArray(part.providerMetadata?.[options.providerMetadataKey]?.reasoningDetails),
)
const field = (() => {
if (configuredField !== undefined && (requireReasoning || reasoning.length > 0 || nativeReasoning !== undefined))
return configuredField
@@ -459,7 +466,7 @@ const lowerMessage = Effect.fn("OpenAIChat.lowerMessage")(function* (
message: OpenAIChatRequestMessage,
reasoningField: string | undefined,
requireReasoning: boolean,
options: LoweringOptions,
options: LoweringOptions & { readonly providerMetadataKey: string },
) {
if (message.role === "user") return [yield* lowerUserMessage(message, options)]
if (message.role === "assistant")
@@ -495,6 +502,7 @@ const lowerMessages = Effect.fn("OpenAIChat.lowerMessages")(function* (request:
const mistral = ["mistral", "devstral", "codestral", "pixtral", "mixtral"].some((family) => modelID.includes(family))
const lowering = {
...options,
providerMetadataKey: request.model.route.providerMetadataKey ?? String(request.model.provider),
toolCallID: (id: string) => {
if (mistral)
return id
@@ -777,26 +785,22 @@ export const fromRequest = Effect.fn("OpenAIChat.fromRequest")(function* (
// Streaming parsers are small state machines: every event returns a new state
// plus the common `LLMEvent`s produced by that event. Tool calls are accumulated
// because OpenAI streams JSON arguments across multiple deltas.
const finishReasonError = (event: OpenAIChatEvent, reason: AIError["reason"]) =>
new AIError({
module: ADAPTER,
method: "stream",
body: ProviderShared.encodeJson(event),
reason,
})
const mapFinishReason = Effect.fn("OpenAIChat.mapFinishReason")(function* (event: OpenAIChatEvent, reason: string) {
switch (reason) {
case "error":
return yield* finishReasonError(
event,
new UnknownProviderReason({ message: "Provider reported an error (finish_reason: error)" }),
)
return yield* new AIError({
reason: new UnknownProviderError({
message: "Provider reported an error (finish_reason: error)",
body: ProviderShared.encodeJson(event),
}),
})
case "network_error":
return yield* finishReasonError(
event,
new ProviderInternalReason({ message: "Provider reported a network error (finish_reason: network_error)" }),
)
return yield* new AIError({
reason: new ProviderInternalError({
message: "Provider reported a network error (finish_reason: network_error)",
body: ProviderShared.encodeJson(event),
}),
})
case "stop":
case "end":
return "stop" as const
@@ -820,7 +824,7 @@ const mapFinishReason = Effect.fn("OpenAIChat.mapFinishReason")(function* (event
// Providers differ on cache-hit location: OpenAI uses
// `prompt_tokens_details.cached_tokens`, DeepSeek uses
// `prompt_cache_hit_tokens`, and Zai uses top-level `cached_tokens`.
const mapUsage = (usage: OpenAIChatEvent["usage"]): Usage | undefined => {
const mapUsage = (usage: OpenAIChatEvent["usage"], providerMetadataKey: string): Usage | undefined => {
if (!usage) return undefined
const input = usage.prompt_tokens ?? undefined
const output = usage.completion_tokens ?? undefined
@@ -839,7 +843,7 @@ const mapUsage = (usage: OpenAIChatEvent["usage"]): Usage | undefined => {
cacheWriteInputTokens: cacheWrite,
reasoningTokens: reasoning,
totalTokens: ProviderShared.totalTokens(input, output, usage.total_tokens ?? undefined),
providerMetadata: { openai: usage },
providerMetadata: { [providerMetadataKey]: usage },
})
}
@@ -913,8 +917,12 @@ const conflictingReasoningTextDetails = (previous: Record<string, unknown>, curr
const conflictingDetailValue = (previous: unknown, current: unknown) =>
previous !== undefined && previous !== null && current !== undefined && current !== null && previous !== current
const reasoningMetadata = (field: ParserState["reasoningField"], details?: ReadonlyArray<unknown>) => ({
openai: {
const reasoningMetadata = (
providerMetadataKey: string,
field: ParserState["reasoningField"],
details?: ReadonlyArray<unknown>,
) => ({
[providerMetadataKey]: {
...(field ? { reasoningField: field } : {}),
...(details ? { reasoningDetails: details } : {}),
},
@@ -925,12 +933,8 @@ const step = (state: ParserState, event: OpenAIChatEvent) =>
if (event.error) {
const body = ProviderShared.encodeJson(event)
return yield* new AIError({
module: ADAPTER,
method: "stream",
body,
reason: classifyProviderFailure({
message: event.error.message,
code: event.error.code === undefined || event.error.code === null ? undefined : String(event.error.code),
status: typeof event.error.code === "number" ? event.error.code : undefined,
rawBody: body,
}),
@@ -941,7 +945,10 @@ const step = (state: ParserState, event: OpenAIChatEvent) =>
// Moonshot (and a few other OpenAI-compatible providers) attach usage to
// `choice.usage` instead of the top-level `usage` field.
const choiceUsage = (choice as unknown as { usage?: OpenAIChatEvent["usage"] })?.usage
const usage = mapUsage(event.usage) ?? (choiceUsage ? mapUsage(choiceUsage) : undefined) ?? state.usage
const usage =
mapUsage(event.usage, state.providerMetadataKey) ??
(choiceUsage ? mapUsage(choiceUsage, state.providerMetadataKey) : undefined) ??
state.usage
const rawFinishReason = choice?.finish_reason
const finishReason = rawFinishReason
? {
@@ -979,7 +986,7 @@ const step = (state: ParserState, event: OpenAIChatEvent) =>
const detailDelta = Array.isArray(delta?.reasoning_details) ? delta.reasoning_details : undefined
if (detailDelta !== undefined) appendReasoningDetails(state.reasoningDetails, detailDelta)
const reasoningDetailsObserved = state.reasoningDetailsObserved || detailDelta !== undefined
const deltaMetadata = reasoningMetadata(reasoningField)
const deltaMetadata = reasoningMetadata(state.providerMetadataKey, reasoningField)
const text = detailDelta?.length ? (detailText(detailDelta) ?? reasoning?.text) : reasoning?.text
if (text !== undefined) lifecycle = Lifecycle.reasoningDelta(lifecycle, events, "reasoning-0", text, deltaMetadata)
else if (
@@ -995,7 +1002,11 @@ const step = (state: ParserState, event: OpenAIChatEvent) =>
lifecycle,
events,
"reasoning-0",
reasoningMetadata(reasoningField, reasoningDetailsObserved ? state.reasoningDetails : undefined),
reasoningMetadata(
state.providerMetadataKey,
reasoningField,
reasoningDetailsObserved ? state.reasoningDetails : undefined,
),
)
lifecycle = Lifecycle.textDelta(lifecycle, events, "text-0", delta.content)
}
@@ -1005,7 +1016,11 @@ const step = (state: ParserState, event: OpenAIChatEvent) =>
lifecycle,
events,
"reasoning-0",
reasoningMetadata(reasoningField, reasoningDetailsObserved ? state.reasoningDetails : undefined),
reasoningMetadata(
state.providerMetadataKey,
reasoningField,
reasoningDetailsObserved ? state.reasoningDetails : undefined,
),
)
lifecycle = Lifecycle.textDelta(lifecycle, events, "text-0", delta.refusal)
}
@@ -1044,7 +1059,14 @@ const step = (state: ParserState, event: OpenAIChatEvent) =>
"OpenAI Chat tool call delta is missing id or name",
)
if (ToolStream.isError(result))
return yield* ProviderShared.eventError(ADAPTER, result.reason.message, ProviderShared.encodeJson(event))
return yield* new AIError({
reason: AIErrorReason.make({
...result.reason,
message: result.message,
cause: result.reason.cause,
body: ProviderShared.encodeJson(event),
}),
})
tools = result.tools
if (result.events.length) lifecycle = Lifecycle.stepStart(lifecycle, events)
events.push(...result.events)
@@ -1066,6 +1088,7 @@ const step = (state: ParserState, event: OpenAIChatEvent) =>
return [
{
providerMetadataKey: state.providerMetadataKey,
tools: finished?.tools ?? tools,
pendingTools,
toolCallEvents: finished?.events ?? state.toolCallEvents,
@@ -1087,11 +1110,9 @@ const step = (state: ParserState, event: OpenAIChatEvent) =>
const finishEvents = Effect.fn("OpenAIChat.finishEvents")(function* (state: ParserState) {
if (state.finishReason === undefined && state.requireFinishReason)
return yield* new AIError({
module: ADAPTER,
method: "stream",
reason: new InvalidProviderOutputReason({
classification: "incomplete-stream",
reason: new InvalidProviderOutputError({
message: "OpenAI Chat stream ended without finish_reason",
classification: "incomplete-stream",
route: ADAPTER,
}),
})
@@ -1109,12 +1130,18 @@ const finishEvents = Effect.fn("OpenAIChat.finishEvents")(function* (state: Pars
}
: { normalized: hasToolCalls ? ("tool-calls" as const) : ("stop" as const) }
const metadata = reasoningMetadata(
state.providerMetadataKey,
state.reasoningField,
state.reasoningDetailsObserved ? state.reasoningDetails : undefined,
)
const started =
state.reasoningDetailsObserved && !state.reasoningEmitted
? Lifecycle.reasoningStart(state.lifecycle, events, "reasoning-0", reasoningMetadata(state.reasoningField))
? Lifecycle.reasoningStart(
state.lifecycle,
events,
"reasoning-0",
reasoningMetadata(state.providerMetadataKey, state.reasoningField),
)
: state.lifecycle
const ended = Lifecycle.reasoningEnd(started, events, "reasoning-0", metadata)
const lifecycle = toolCallEvents.length ? Lifecycle.stepStart(ended, events) : ended
@@ -1141,6 +1168,7 @@ export const protocol = Protocol.make({
stream: {
event: Protocol.jsonEvent(OpenAIChatEvent),
initial: (request) => ({
providerMetadataKey: request.model.route.providerMetadataKey ?? String(request.model.provider),
tools: ToolStream.empty<number>(),
pendingTools: {},
toolCallEvents: [],
+12 -28
View File
@@ -9,14 +9,7 @@ import {
type ImageRoute,
} from "../image.js"
import { Auth, type Definition as AuthDefinition } from "../route/auth.js"
import {
InvalidProviderOutputReason,
AIError,
Usage,
mergeHttpOptions,
mergeJsonRecords,
type HttpOptions,
} from "../schema/index.js"
import { Usage, mergeHttpOptions, mergeJsonRecords, type HttpOptions } from "../schema/index.js"
import { ProviderShared } from "./shared.js"
import { ImageInputs } from "./utils/image-input.js"
import { OpenAIImage } from "./utils/openai-image.js"
@@ -84,13 +77,6 @@ const nativeOptions = (options: OpenAIImageOptions | undefined) => {
}
}
const invalidOutput = (message: string) =>
new AIError({
module: ADAPTER,
method: "generate",
reason: new InvalidProviderOutputReason({ message, route: ADAPTER }),
})
const applyQuery = (url: string, query: Record<string, string> | undefined) => {
if (!query) return url
const next = new URL(url)
@@ -104,12 +90,12 @@ export const model = (input: ModelInput) => {
generate: Effect.fn("OpenAIImages.generate")(function* (request: ImageRequestFor<OpenAIImageOptions>, execute) {
const mask = request.options?.mask
if (mask !== undefined && (request.images?.length ?? 0) === 0)
return yield* ImageInputs.invalid(ADAPTER, "An OpenAI image mask requires at least one input image")
return yield* ImageInputs.invalid("An OpenAI image mask requires at least one input image")
const http = mergeHttpOptions(request.model.http, request.http)
const sourceImages = request.images ?? []
const multipartImages = yield* Effect.forEach(sourceImages, (image) => {
if (image.type === "bytes") return Effect.succeed({ data: image.data, mediaType: image.mediaType })
if (image.type === "url") return ImageInputs.decodeDataUrl(image.url, ADAPTER)
if (image.type === "url") return ImageInputs.decodeDataUrl(image.url)
return Effect.undefined
})
const multipartMask =
@@ -118,7 +104,7 @@ export const model = (input: ModelInput) => {
: mask.type === "bytes"
? { data: mask.data, mediaType: mask.mediaType }
: mask.type === "url"
? yield* ImageInputs.decodeDataUrl(mask.url, ADAPTER)
? yield* ImageInputs.decodeDataUrl(mask.url)
: undefined
const useMultipart =
sourceImages.length > 0 &&
@@ -161,7 +147,7 @@ export const model = (input: ModelInput) => {
return undefined
})
if (references.some((image) => image === undefined))
return yield* ImageInputs.invalid(ADAPTER, "OpenAI Images accepts image URLs, data URLs, bytes, and file IDs")
return yield* ImageInputs.invalid("OpenAI Images accepts image URLs, data URLs, bytes, and file IDs")
const maskReference =
mask === undefined
? undefined
@@ -173,7 +159,7 @@ export const model = (input: ModelInput) => {
? { file_id: mask.id }
: undefined
if (mask !== undefined && maskReference === undefined)
return yield* ImageInputs.invalid(ADAPTER, "OpenAI Images accepts masks as URLs, data URLs, bytes, or file IDs")
return yield* ImageInputs.invalid("OpenAI Images accepts masks as URLs, data URLs, bytes, or file IDs")
const requestBody = mergeJsonRecords(
{
model: request.model.id,
@@ -209,11 +195,9 @@ const parseResponse = Effect.fn("OpenAIImages.parseResponse")(function* (
options: OpenAIImageOptions | undefined,
overlay: Record<string, unknown> | undefined,
) {
const payload = yield* response.json.pipe(
Effect.mapError(() => invalidOutput("Failed to read the OpenAI Images response")),
)
const decoded = yield* Schema.decodeUnknownEffect(OpenAIImageResponse)(payload).pipe(
Effect.mapError(() => invalidOutput("OpenAI Images returned an invalid response")),
const output = yield* ProviderShared.imageResponse(ADAPTER, "OpenAI Images", response)
const decoded = yield* Schema.decodeUnknownEffect(Schema.fromJsonString(OpenAIImageResponse))(output.body).pipe(
Effect.mapError((cause) => output.invalid("OpenAI Images returned an invalid response", cause)),
)
const requestBody = mergeJsonRecords(nativeOptions(options), overlay)
const format =
@@ -221,7 +205,7 @@ const parseResponse = Effect.fn("OpenAIImages.parseResponse")(function* (
const images = yield* Effect.forEach(decoded.data, (item, index) => {
if (item.b64_json)
return Effect.fromResult(Encoding.decodeBase64(item.b64_json)).pipe(
Effect.mapError(() => invalidOutput(`OpenAI Images result ${index} contains invalid base64 data`)),
Effect.mapError((cause) => output.invalid(`OpenAI Images result ${index} contains invalid base64 data`, cause)),
Effect.map(
(data) =>
new GeneratedImage({
@@ -241,9 +225,9 @@ const parseResponse = Effect.fn("OpenAIImages.parseResponse")(function* (
item.revised_prompt === undefined ? undefined : { openai: { revisedPrompt: item.revised_prompt } },
}),
)
return Effect.fail(invalidOutput(`OpenAI Images result ${index} has neither image data nor a URL`))
return Effect.fail(output.invalid(`OpenAI Images result ${index} has neither image data nor a URL`))
})
if (images.length === 0) return yield* invalidOutput("OpenAI Images returned no images")
if (images.length === 0) return yield* output.invalid("OpenAI Images returned no images")
return new ImageResponse({
images,
usage:
@@ -149,7 +149,9 @@ const hostedToolResult = Effect.fn("OpenAIResponses.hostedToolResult")(function*
const isError = item.error !== undefined && item.error !== null
if (item.type === "image_generation_call" && item.result) {
yield* Effect.fromResult(Encoding.decodeBase64(item.result)).pipe(
Effect.mapError(() => ProviderShared.eventError(ADAPTER, "OpenAI Responses returned invalid image base64")),
Effect.mapError((cause) =>
ProviderShared.eventError(ADAPTER, "OpenAI Responses returned invalid image base64", undefined, cause),
),
)
const format = item.output_format ?? "png"
return {
+39 -14
View File
@@ -2,11 +2,12 @@ import { Buffer } from "node:buffer"
import { Tool } from "@opencode-ai/schema/tool"
import { Effect, Schema, Stream } from "effect"
import * as Sse from "effect/unstable/encoding/Sse"
import { Headers, HttpClientRequest } from "effect/unstable/http"
import { Headers, HttpClientRequest, HttpClientResponse } from "effect/unstable/http"
import {
InvalidProviderOutputReason,
InvalidRequestReason,
InvalidProviderOutputError,
InvalidRequestError,
AIError,
HttpContext,
type ContentPart,
type LLMRequest,
type MediaPart,
@@ -96,17 +97,15 @@ export const sumTokens = (...values: ReadonlyArray<number | undefined>): number
return values.reduce((acc: number, value) => acc + (value ?? 0), 0)
}
export const eventError = (route: string, message: string, raw?: string) =>
export const eventError = (route: string, message: string, body?: string, cause?: unknown) =>
new AIError({
module: "ProviderShared",
method: "stream",
reason: new InvalidProviderOutputReason({ route, message, raw }),
reason: new InvalidProviderOutputError({ route, message, body, cause }),
})
export const parseJson = (route: string, input: string, message: string) =>
Effect.try({
try: () => decodeJson(input),
catch: () => eventError(route, message, input),
catch: (cause) => eventError(route, message, input, cause),
})
/**
@@ -233,7 +232,7 @@ export const sseFraming = (
(state, chunk) =>
Effect.gen(function* () {
const error = state.parser.feed(chunk)
if (error) return yield* eventError("sse", error.message)
if (error) return yield* eventError("sse", error.message, chunk, error)
return [state, state.output.splice(0)] as const
}),
),
@@ -249,13 +248,39 @@ export const sseFraming = (
/**
* Canonical invalid-request constructor shared by protocol lowering.
*/
export const invalidRequest = (message: string) =>
export const invalidRequest = (message: string, cause?: unknown) =>
new AIError({
module: "ProviderShared",
method: "request",
reason: new InvalidRequestReason({ message }),
reason: new InvalidRequestError({ message, cause }),
})
export const imageResponse = Effect.fn("ProviderShared.imageResponse")(function* (
route: string,
name: string,
response: HttpClientResponse.HttpClientResponse,
) {
const http = new HttpContext({ url: response.request.url, status: response.status, headers: response.headers })
const body = yield* response.text.pipe(
Effect.mapError(
(cause) =>
new AIError({
reason: new InvalidProviderOutputError({
route,
message: `Failed to read the ${name} response`,
http,
cause,
}),
}),
),
)
return {
body,
invalid: (message: string, cause?: unknown) =>
new AIError({
reason: new InvalidProviderOutputError({ route, message, body, http, cause }),
}),
}
})
export const matchToolChoice = <Auto, None, Required, Tool>(
route: string,
toolChoice: NonNullable<LLMRequest["toolChoice"]>,
@@ -302,7 +327,7 @@ export const unsupportedContent = (
export const validateWith =
<A, I, E extends { readonly message: string }>(decode: (input: I) => Effect.Effect<A, E>) =>
(payload: I) =>
decode(payload).pipe(Effect.mapError((error) => invalidRequest(error.message)))
decode(payload).pipe(Effect.mapError((error) => invalidRequest(error.message, error)))
/**
* Build an HTTP POST with a JSON body. Sets `content-type: application/json`
@@ -1,12 +1,10 @@
import { Effect, Encoding } from "effect"
import type { ImageInput } from "../../image.js"
import { InvalidRequestReason, AIError } from "../../schema/index.js"
import { InvalidRequestError, AIError } from "../../schema/index.js"
const invalid = (module: string, message: string) =>
const invalid = (message: string, cause?: unknown) =>
new AIError({
module,
method: "generate",
reason: new InvalidRequestReason({ message }),
reason: new InvalidRequestError({ message, cause }),
})
export const dataUrl = (input: Extract<ImageInput, { readonly type: "bytes" }>) =>
@@ -14,13 +12,12 @@ export const dataUrl = (input: Extract<ImageInput, { readonly type: "bytes" }>)
export const decodeDataUrl = (
url: string,
module: string,
): Effect.Effect<{ readonly mediaType: string; readonly data: Uint8Array } | undefined, AIError> => {
if (!url.startsWith("data:")) return Effect.undefined
const match = /^data:([^;,]+);base64,(.*)$/s.exec(url)
if (!match) return Effect.fail(invalid(module, "Image data URLs must contain a MIME type and base64 data"))
if (!match) return Effect.fail(invalid("Image data URLs must contain a MIME type and base64 data"))
return Effect.fromResult(Encoding.decodeBase64(match[2])).pipe(
Effect.mapError(() => invalid(module, "Image data URL contains invalid base64 data")),
Effect.mapError((cause) => invalid("Image data URL contains invalid base64 data", cause)),
Effect.map((data) => ({ mediaType: match[1], data })),
)
}
+10 -24
View File
@@ -2,14 +2,7 @@ import { Effect, Encoding, Schema } from "effect"
import { Headers, HttpClientRequest } from "effect/unstable/http"
import { GeneratedImage, ImageModel, ImageResponse, type ImageRequestFor, type ImageRoute } from "../image.js"
import { Auth, type Definition as AuthDefinition } from "../route/auth.js"
import {
InvalidProviderOutputReason,
AIError,
Usage,
mergeHttpOptions,
mergeJsonRecords,
type HttpOptions,
} from "../schema/index.js"
import { Usage, mergeHttpOptions, mergeJsonRecords, type HttpOptions } from "../schema/index.js"
import { ProviderShared, optionalNull } from "./shared.js"
import { ImageInputs } from "./utils/image-input.js"
@@ -94,13 +87,6 @@ const nativeOptions = (options: XAIImageOptions | undefined) => {
}
}
const invalidOutput = (message: string) =>
new AIError({
module: ADAPTER,
method: "generate",
reason: new InvalidProviderOutputReason({ message, route: ADAPTER }),
})
const applyQuery = (url: string, query: Record<string, string> | undefined) => {
if (!query) return url
const next = new URL(url)
@@ -120,7 +106,7 @@ export const model = (input: ModelInput) => {
return undefined
})
if (imageReferences.some((image) => image === undefined))
return yield* ImageInputs.invalid(ADAPTER, "xAI Images accepts image URLs, data URLs, bytes, and file IDs")
return yield* ImageInputs.invalid("xAI Images accepts image URLs, data URLs, bytes, and file IDs")
const requestBody = mergeJsonRecords(
{
model: request.model.id,
@@ -149,17 +135,17 @@ export const model = (input: ModelInput) => {
HttpClientRequest.bodyText(text, "application/json"),
),
)
const payload = yield* response.json.pipe(
Effect.mapError(() => invalidOutput("Failed to read the xAI Images response")),
)
const decoded = yield* Schema.decodeUnknownEffect(XAIImageResponse)(payload).pipe(
Effect.mapError(() => invalidOutput("xAI Images returned an invalid response")),
const output = yield* ProviderShared.imageResponse(ADAPTER, "xAI Images", response)
const decoded = yield* Schema.decodeUnknownEffect(Schema.fromJsonString(XAIImageResponse))(output.body).pipe(
Effect.mapError((cause) => output.invalid("xAI Images returned an invalid response", cause)),
)
const images = yield* Effect.forEach(decoded.data, (item, index) => {
const mediaType = item.mime_type ?? "application/octet-stream"
if (item.b64_json)
return Effect.fromResult(Encoding.decodeBase64(item.b64_json)).pipe(
Effect.mapError(() => invalidOutput(`xAI Images result ${index} contains invalid base64 data`)),
Effect.mapError((cause) =>
output.invalid(`xAI Images result ${index} contains invalid base64 data`, cause),
),
Effect.map(
(data) =>
new GeneratedImage({
@@ -183,9 +169,9 @@ export const model = (input: ModelInput) => {
: { xai: { revisedPrompt: item.revised_prompt } },
}),
)
return Effect.fail(invalidOutput(`xAI Images result ${index} has neither image data nor a URL`))
return Effect.fail(output.invalid(`xAI Images result ${index} has neither image data nor a URL`))
})
if (images.length === 0) return yield* invalidOutput("xAI Images returned no images")
if (images.length === 0) return yield* output.invalid("xAI Images returned no images")
const usage = ProviderShared.isRecord(decoded.usage) ? decoded.usage : undefined
return new ImageResponse({
images,
+6 -21
View File
@@ -2,13 +2,7 @@ import { Effect, Schema } from "effect"
import { Headers, HttpClientRequest } from "effect/unstable/http"
import { GeneratedImage, ImageModel, ImageResponse, type ImageRequestFor, type ImageRoute } from "../image.js"
import { Auth, type Definition as AuthDefinition } from "../route/auth.js"
import {
InvalidProviderOutputReason,
AIError,
mergeHttpOptions,
mergeJsonRecords,
type HttpOptions,
} from "../schema/index.js"
import { mergeHttpOptions, mergeJsonRecords, type HttpOptions } from "../schema/index.js"
import { ProviderShared } from "./shared.js"
import { ImageInputs } from "./utils/image-input.js"
@@ -63,13 +57,6 @@ const nativeOptions = (options: ZAIImageOptions | undefined) => {
}
}
const invalidOutput = (message: string) =>
new AIError({
module: ADAPTER,
method: "generate",
reason: new InvalidProviderOutputReason({ message, route: ADAPTER }),
})
const applyQuery = (url: string, query: Record<string, string> | undefined) => {
if (!query) return url
const next = new URL(url)
@@ -82,7 +69,7 @@ export const model = (input: ModelInput) => {
id: ADAPTER,
generate: Effect.fn("ZAIImages.generate")(function* (request: ImageRequestFor<ZAIImageOptions>, execute) {
if ((request.images?.length ?? 0) > 0)
return yield* ImageInputs.invalid(ADAPTER, "Z.ai hosted image generation does not support image inputs")
return yield* ImageInputs.invalid("Z.ai hosted image generation does not support image inputs")
const http = mergeHttpOptions(request.model.http, request.http)
const requestBody = mergeJsonRecords(
{ model: request.model.id, prompt: request.prompt },
@@ -104,13 +91,11 @@ export const model = (input: ModelInput) => {
HttpClientRequest.bodyText(text, "application/json"),
),
)
const payload = yield* response.json.pipe(
Effect.mapError(() => invalidOutput("Failed to read the Z.ai Images response")),
const output = yield* ProviderShared.imageResponse(ADAPTER, "Z.ai Images", response)
const decoded = yield* Schema.decodeUnknownEffect(Schema.fromJsonString(ZAIImageResponse))(output.body).pipe(
Effect.mapError((cause) => output.invalid("Z.ai Images returned an invalid response", cause)),
)
const decoded = yield* Schema.decodeUnknownEffect(ZAIImageResponse)(payload).pipe(
Effect.mapError(() => invalidOutput("Z.ai Images returned an invalid response")),
)
if (decoded.data.length === 0) return yield* invalidOutput("Z.ai Images returned no images")
if (decoded.data.length === 0) return yield* output.invalid("Z.ai Images returned no images")
return new ImageResponse({
images: decoded.data.map(
(item) =>
+46 -43
View File
@@ -1,17 +1,16 @@
import { Option, Schema } from "effect"
import {
AuthenticationReason,
ContentPolicyReason,
InvalidRequestReason,
AuthenticationError,
ContentPolicyError,
InvalidRequestError,
AIError,
ProviderErrorEvent,
ProviderInternalReason,
QuotaExceededReason,
RateLimitReason,
UnknownProviderReason,
ProviderInternalError,
QuotaExceededError,
RateLimitError,
UnknownProviderError,
type HttpContext,
type HttpRateLimitDetails,
type ProviderMetadata,
} from "./schema/index.js"
const patterns = [
@@ -80,27 +79,28 @@ const NETWORK_ERROR_TEXT = /network[-_\s]error/i
export interface ProviderFailure {
readonly message: string
readonly status?: number | undefined
readonly code?: string | undefined
// Raw wire payload, scanned for failure signals (codes, overflow phrases)
// that the summary message does not carry. Not shown to users.
readonly rawBody?: string | undefined
// Some SDKs supply parsed error data separately from the original response text.
readonly data?: unknown
readonly http?: HttpContext | undefined
readonly cause?: unknown
readonly retryAfterMs?: number | undefined
readonly rateLimit?: HttpRateLimitDetails | undefined
readonly http?: HttpContext | undefined
readonly providerMetadata?: ProviderMetadata | undefined
}
// Keep HTTP failures and provider-reported stream failures on one typed path so
// session retry policy never needs provider-specific string matching.
export function classifyProviderFailure(input: ProviderFailure): AIError["reason"] {
const body = input.http?.body ?? input.rawBody ?? ""
const codes = [input.code, ...providerCodes(body), ...providerCodes(input.message)]
.filter((code): code is string => code !== undefined)
.map((code) => code.toLowerCase())
const details = { message: input.message, body: input.rawBody, http: input.http, cause: input.cause }
const body = input.rawBody ?? ""
const codes = [...providerCodes(input.data), ...providerCodes(body), ...providerCodes(input.message)].map((code) =>
code.toLowerCase(),
)
// Scan the raw payload too so signals missing from the summary message
// (e.g. overflow phrases nested in a JSON error body) still classify.
const text = [input.message, body].filter((value) => value.length > 0).join("\n")
const common = { message: input.message, providerMetadata: input.providerMetadata, http: input.http }
const clientScoped = input.status === undefined || (input.status >= 400 && input.status < 500)
if (
@@ -110,62 +110,65 @@ export function classifyProviderFailure(input: ProviderFailure): AIError["reason
codes.includes("request_too_large") ||
isContextOverflow(text))
)
return new InvalidRequestReason({ ...common, classification: "context-overflow" })
return new InvalidRequestError({ ...details, classification: "context-overflow" })
if (input.status === 413 || isPayloadTooLarge(text))
return new InvalidRequestReason({ ...common, classification: "payload-too-large" })
if (CONTENT_POLICY_TEXT.test(text)) return new ContentPolicyReason(common)
return new InvalidRequestError({ ...details, classification: "payload-too-large" })
if (CONTENT_POLICY_TEXT.test(text)) return new ContentPolicyError(details)
if (codes.some((code) => QUOTA_CODES.has(code)) || (input.status === 429 && QUOTA_TEXT.test(text)))
return new QuotaExceededReason(common)
if (input.status === 401) return new AuthenticationReason({ ...common, kind: "invalid" })
if (input.status === 403) return new AuthenticationReason({ ...common, kind: "insufficient-permissions" })
if (codes.includes("authentication_error")) return new AuthenticationReason({ ...common, kind: "invalid" })
return new QuotaExceededError(details)
if (input.status === 401) return new AuthenticationError({ ...details, kind: "invalid" })
if (input.status === 403) return new AuthenticationError({ ...details, kind: "insufficient-permissions" })
if (codes.includes("authentication_error")) return new AuthenticationError({ ...details, kind: "invalid" })
if (codes.includes("permission_error"))
return new AuthenticationReason({ ...common, kind: "insufficient-permissions" })
return new AuthenticationError({ ...details, kind: "insufficient-permissions" })
if (
codes.some((code) => code.includes("rate_limit") || code === "too_many_requests" || code === "throttlingexception")
)
return new RateLimitReason({
...common,
return new RateLimitError({
...details,
retryAfterMs: input.retryAfterMs,
rateLimit: input.rateLimit,
})
if (RATE_LIMIT_TEXT.test(text))
return new RateLimitReason({
...common,
return new RateLimitError({
...details,
retryAfterMs: input.retryAfterMs,
rateLimit: input.rateLimit,
})
if (NETWORK_ERROR_TEXT.test(text)) return new ProviderInternalReason({ ...common, status: input.status })
if (NETWORK_ERROR_TEXT.test(text)) return new ProviderInternalError(details)
if (codes.some((code) => SERVER_CODES.has(code) || code.includes("exhausted") || code.includes("unavailable")))
return new ProviderInternalReason({
...common,
status: input.status,
return new ProviderInternalError({
...details,
retryAfterMs: input.retryAfterMs,
})
if (input.status === 429) {
return new RateLimitReason({
...common,
return new RateLimitError({
...details,
retryAfterMs: input.retryAfterMs,
rateLimit: input.rateLimit,
})
}
if (input.status === 408 || input.status === 409 || (input.status !== undefined && input.status >= 500))
return new ProviderInternalReason({
...common,
status: input.status,
return new ProviderInternalError({
...details,
retryAfterMs: input.retryAfterMs,
})
if (codes.some((code) => INVALID_REQUEST_CODES.has(code))) return new InvalidRequestReason(common)
if (codes.some((code) => INVALID_REQUEST_CODES.has(code))) return new InvalidRequestError(details)
if (input.status === 400 || input.status === 404 || input.status === 413 || input.status === 422)
return new InvalidRequestReason(common)
return new UnknownProviderReason({ ...common, status: input.status })
return new InvalidRequestError(details)
return new UnknownProviderError(details)
}
function providerCodes(value: string) {
const decoded = Option.getOrUndefined(decodeJson(value))
function providerCodes(value: unknown) {
const decoded = typeof value === "string" ? Option.getOrUndefined(decodeJson(value)) : value
if (!isRecord(decoded)) return []
const error = isRecord(decoded.error) ? decoded.error : undefined
return [decoded.code, error?.code, error?.type].filter((value): value is string => typeof value === "string")
const response = isRecord(decoded.response) ? decoded.response : undefined
const responseError = response && isRecord(response.error) ? response.error : undefined
const exception = isRecord(decoded.exception) ? decoded.exception : undefined
return [decoded.code, error?.code, error?.type, error?.status, responseError?.code, exception?.type].filter(
(value): value is string => typeof value === "string",
)
}
function isRecord(value: unknown): value is Record<string, unknown> {
@@ -23,13 +23,14 @@ export interface Settings extends ProviderPackage.Settings {
readonly baseURL?: string
readonly credentials?: Credentials
readonly region?: string
readonly topP?: number
readonly providerOptions?: OpenAIProviderOptionsInput
}
const responsesRoute = Route.make({
id: "bedrock-mantle-responses",
provider: id,
providerMetadataKey: OpenAIResponses.route.providerMetadataKey,
providerMetadataKey: "mantle",
protocol: OpenAIResponses.protocol,
endpoint: OpenAIResponses.route.endpoint,
auth: OpenAIResponses.route.auth,
@@ -40,6 +41,7 @@ const responsesRoute = Route.make({
const chatRoute = OpenAIChat.route.with({
id: "bedrock-mantle-chat",
provider: id,
providerMetadataKey: "mantle",
})
export const routes = [responsesRoute, chatRoute]
@@ -76,7 +78,7 @@ export const configure = (input: Config = {}) => {
return {
id,
model: chat,
model: responses,
chat,
responses,
configure,
@@ -94,6 +96,7 @@ const config = (settings: Settings): Config => {
apiKey: settings.auth === "sigv4" ? undefined : settings.apiKey,
baseURL: settings.baseURL,
credentials: settings.credentials,
generation: settings.topP === undefined ? undefined : { topP: settings.topP },
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
providerOptions: settings.providerOptions,
@@ -109,4 +112,4 @@ export const responsesModel: ProviderPackage.Definition<Settings, OpenAIProvider
modelID,
settings,
) => configure(config(settings)).responses(modelID)
export const model = chatModel
export const model = responsesModel
@@ -35,6 +35,7 @@ const configuredRoute = (input: Config) => {
return BedrockConverse.route.with({
...rest,
provider: id,
providerMetadataKey: "bedrock",
endpoint: { baseURL: baseURL ?? bedrockBaseURL(resolvedRegion) },
auth: apiKey === undefined ? BedrockConverse.sigV4Auth(credentials) : Auth.bearer(apiKey),
})
@@ -1,2 +1,2 @@
export { chatModel as model } from "../amazon-bedrock-mantle.js"
export { responsesModel as model } from "../amazon-bedrock-mantle.js"
export type { Settings } from "../amazon-bedrock-mantle.js"
@@ -27,6 +27,7 @@ export interface Settings extends ProviderPackage.Settings {
const route = OpenAICompatibleChat.route.with({
id: "google-vertex-chat",
provider: id,
providerMetadataKey: "vertex",
})
export const routes = [route]
@@ -27,6 +27,7 @@ export interface Settings extends ProviderPackage.Settings {
const route = OpenAICompatibleResponses.route.with({
id: "google-vertex-responses",
provider: id,
providerMetadataKey: "vertex",
providerOptions: { store: false },
})
+1 -1
View File
@@ -68,7 +68,7 @@ const protocol = {
const route = Route.make({
id: "google-vertex-gemini",
provider: id,
providerMetadataKey: "google",
providerMetadataKey: "vertex",
protocol,
endpoint: Endpoint.path(({ request }) => {
const model = String(request.model.id)
+1
View File
@@ -164,6 +164,7 @@ const bodyOptions = (input: unknown) => {
export const route = Route.make({
id: ADAPTER,
provider: profile.provider,
providerMetadataKey: "openrouter",
protocol,
endpoint: Endpoint.path("/chat/completions", { baseURL: profile.baseURL }),
framing: Framing.sse,
+3 -5
View File
@@ -1,6 +1,6 @@
import { Config, Effect, Redacted } from "effect"
import { Headers } from "effect/unstable/http"
import { AuthenticationReason, InvalidRequestReason, AIError, type HttpOptions } from "../schema/index.js"
import { AuthenticationError, InvalidRequestError, AIError, type HttpOptions } from "../schema/index.js"
export class MissingCredentialError extends Error {
readonly _tag = "MissingCredentialError"
@@ -137,12 +137,10 @@ export function bearerHeader(name: string, source?: Secret | Credential) {
const toAIError = (error: AuthError): AIError => {
if (error instanceof MissingCredentialError || error instanceof Config.ConfigError) {
return new AIError({
module: "Auth",
method: "apply",
reason:
error instanceof MissingCredentialError
? new AuthenticationReason({ message: error.message, kind: "missing" })
: new InvalidRequestReason({ message: `Failed to resolve auth config: ${error.message}` }),
? new AuthenticationError({ message: error.message, cause: error, kind: "missing" })
: new InvalidRequestError({ message: `Failed to resolve auth config: ${error.message}`, cause: error }),
})
}
return error
+57 -11
View File
@@ -12,13 +12,14 @@ import * as ProviderShared from "../protocols/shared.js"
import type { ProtocolID, ProviderOptions } from "../schema/index.js"
import {
AIError,
AIErrorReason,
GenerationOptions,
HttpOptions,
LLMRequest,
LLMResponse,
LanguageModel,
LLMEvent,
InvalidProviderOutputReason,
InvalidProviderOutputError,
ProviderID,
mergeGenerationOptions,
mergeHttpOptions,
@@ -89,6 +90,7 @@ export interface RouteDefaultsInput {
export interface RoutePatch<Body, Prepared> extends RouteDefaultsInput {
readonly id?: string
readonly provider?: string | ProviderID
readonly providerMetadataKey?: string
readonly auth?: Auth.Definition
readonly transport?: Transport<Body, Prepared, unknown>
readonly endpoint?: EndpointPatch<Body>
@@ -226,16 +228,14 @@ export interface MakeTransportInput<Body, Prepared, Frame, Event, State> {
const streamError = (route: string, message: string, cause: Cause.Cause<unknown>) => {
const failed = cause.reasons.find(Cause.isFailReason)?.error
if (failed instanceof AIError) return failed
return ProviderShared.eventError(route, message, Cause.pretty(cause))
return ProviderShared.eventError(route, message, undefined, cause)
}
const incompleteStreamError = (route: string) =>
new AIError({
module: "LLMClient",
method: "stream",
reason: new InvalidProviderOutputReason({
classification: "incomplete-stream",
reason: new InvalidProviderOutputError({
message: "The provider response ended unexpectedly.",
classification: "incomplete-stream",
route,
}),
})
@@ -264,11 +264,12 @@ function makeFromTransport<Body, Prepared, Frame, Event, State>(
const decodeEventEffect = Schema.decodeUnknownEffect(protocol.stream.event)
const decodeEvent = (route: string) => (frame: Frame) =>
decodeEventEffect(frame).pipe(
Effect.mapError(() =>
Effect.mapError((cause) =>
ProviderShared.eventError(
input.id,
`Invalid ${route} stream event`,
typeof frame === "string" ? frame : ProviderShared.encodeJson(frame),
cause,
),
),
)
@@ -289,11 +290,16 @@ function makeFromTransport<Body, Prepared, Frame, Event, State>(
defaults: routeInput.defaults ?? {},
body: protocol.body,
with: (patch: RoutePatch<Body, Prepared>) => {
const { id, provider, auth, transport, endpoint, ...defaults } = patch
const { id, provider, providerMetadataKey, auth, transport, endpoint, ...defaults } = patch
return build({
...routeInput,
id: id ?? routeInput.id,
provider: provider ?? routeInput.provider,
providerMetadataKey:
providerMetadataKey ??
(provider !== undefined && String(provider) !== String(routeInput.provider)
? String(provider)
: routeInput.providerMetadataKey),
auth: auth ?? routeInput.auth,
endpoint: endpoint ? Endpoint.merge(routeInput.endpoint, endpoint) : routeInput.endpoint,
transport: (transport as Transport<Body, Prepared, Frame> | undefined) ?? routeInput.transport,
@@ -318,19 +324,48 @@ function makeFromTransport<Body, Prepared, Frame, Event, State>(
return Stream.unwrap(
routeInput.transport.execute(prepared, request, runtime, options).pipe(
Effect.map((execution) => {
const terminal = protocol.stream.terminal
// Preserve assembled inputs; replace only serialized event fallbacks with their original wire data.
const frameError =
(frame: Frame, event: Frame | Event = frame) =>
(error: AIError) =>
new AIError({
reason: AIErrorReason.make({
...error.reason,
message: error.reason.message,
cause: error.reason.cause,
body:
error.reason.body !== undefined && error.reason.body !== ProviderShared.encodeJson(event)
? error.reason.body
: (execution.body?.(frame) ??
(typeof frame === "string" ? frame : ProviderShared.encodeJson(frame))),
}),
})
const events = execution.frames.pipe(
Stream.mapEffect(decodeEvent(route)),
protocol.stream.terminal ? Stream.takeUntil(protocol.stream.terminal) : (stream) => stream,
Stream.mapEffect((frame) =>
decodeEvent(route)(frame).pipe(
Effect.catchCause((cause) =>
Effect.fail(streamError(route, `Failed to decode ${route} event`, cause)),
),
Effect.map((event) => ({ event, frame })),
Effect.mapError(frameError(frame)),
),
),
terminal ? Stream.takeUntil(({ event }) => terminal(event)) : (stream) => stream,
)
const stream = Stream.suspend(() => {
let state = protocol.stream.initial(request)
const parsed = events.pipe(
Stream.mapEffect((event) =>
Stream.mapEffect(({ event, frame }) =>
protocol.stream.step(state, event).pipe(
Effect.catchCause((cause) =>
Effect.fail(streamError(route, `Failed to parse ${route} event`, cause)),
),
Effect.map(([next, output]) => {
state = next
return output
}),
Effect.mapError(frameError(frame, event)),
),
),
Stream.flatMap(Stream.fromIterable),
@@ -346,6 +381,17 @@ function makeFromTransport<Body, Prepared, Frame, Event, State>(
}).pipe(
Stream.catchCause((cause) => Stream.fail(streamError(route, `Failed to read ${route} stream`, cause))),
requireTerminalEvent(route),
Stream.mapError(
(error) =>
new AIError({
reason: AIErrorReason.make({
...error.reason,
message: error.reason.message,
cause: error.reason.cause,
http: error.reason.http ?? execution.http,
}),
}),
),
)
return execution.complete ? stream.pipe(Stream.onEnd(execution.complete)) : stream
}),
+48 -107
View File
@@ -7,14 +7,7 @@ import {
HttpClientRequest,
HttpClientResponse,
} from "effect/unstable/http"
import {
HttpContext,
HttpRateLimitDetails,
HttpRequestDetails,
HttpResponseDetails,
AIError,
TransportReason,
} from "../schema/index.js"
import { HttpContext, HttpRateLimitDetails, AIError, TransportError } from "../schema/index.js"
import { classifyProviderFailure } from "../provider-error.js"
export interface Interface {
@@ -40,17 +33,6 @@ const headerDetails = (headers: Headers.Headers) =>
const normalizedHeaders = (headers: Headers.Headers) =>
Object.fromEntries(Object.entries(headers).map(([key, value]) => [key.toLowerCase(), value]))
const requestId = (headers: Record<string, string>) => {
return (
headers["x-request-id"] ??
headers["request-id"] ??
headers["x-amzn-requestid"] ??
headers["x-amz-request-id"] ??
headers["x-goog-request-id"] ??
headers["cf-ray"]
)
}
const retryAfterMs = (headers: Record<string, string>) => {
const millis = Number(headers["retry-after-ms"])
if (Number.isFinite(millis)) return Math.max(0, millis)
@@ -108,24 +90,13 @@ const rateLimitDetails = (headers: Record<string, string>, retryAfter: number |
})
}
const requestDetails = (request: HttpClientRequest.HttpClientRequest) =>
new HttpRequestDetails({
method: request.method,
url: request.url,
headers: headerDetails(request.headers),
})
const responseDetails = (response: HttpClientResponse.HttpClientResponse) =>
new HttpResponseDetails({
export const responseHttp = (response: HttpClientResponse.HttpClientResponse) =>
new HttpContext({
url: response.request.url,
status: response.status,
headers: headerDetails(response.headers),
})
const responseBody = (body: string | void) => {
if (body === undefined) return {}
return { body }
}
const decodeProviderBody = Schema.decodeUnknownOption(
Schema.fromJsonString(
Schema.Struct({
@@ -143,81 +114,46 @@ const providerMessage = (status: number, body: string | void) => {
)
}
const responseHttp = (input: {
readonly request: HttpClientRequest.HttpClientRequest
readonly response: HttpClientResponse.HttpClientResponse
readonly body: ReturnType<typeof responseBody>
readonly requestId?: string | undefined
readonly rateLimit?: HttpRateLimitDetails | undefined
}) =>
new HttpContext({
request: requestDetails(input.request),
response: responseDetails(input.response),
...input.body,
requestId: input.requestId,
rateLimit: input.rateLimit,
const statusError = (response: HttpClientResponse.HttpClientResponse) =>
Effect.gen(function* () {
if (response.status < 400) return response
const result = yield* response.text.pipe(Effect.result)
return yield* httpFailure({
message: providerMessage(response.status, result._tag === "Success" ? result.success : undefined),
url: response.request.url,
status: response.status,
responseHeaders: headerDetails(response.headers),
responseBody: result._tag === "Success" ? result.success : undefined,
cause: result._tag === "Failure" ? (result.failure.cause ?? result.failure) : undefined,
})
})
const statusError =
(request: HttpClientRequest.HttpClientRequest) => (response: HttpClientResponse.HttpClientResponse) =>
Effect.gen(function* () {
if (response.status < 400) return response
const body = yield* response.text.pipe(Effect.catch(() => Effect.void))
const headers = normalizedHeaders(response.headers)
const retryAfter = retryAfterMs(headers)
const rateLimit = rateLimitDetails(headers, retryAfter)
const details = responseBody(body)
return yield* new AIError({
module: "RequestExecutor",
method: "execute",
reason: classifyProviderFailure({
status: response.status,
message: providerMessage(response.status, body),
retryAfterMs: retryAfter,
rateLimit,
http: responseHttp({
request,
response,
body: details,
requestId: requestId(headers),
rateLimit,
}),
}),
})
})
// Classifies an HTTP failure captured outside the executor (for example by the
// AI SDK's own fetch) onto the same reason types and HttpContext that
// executor-driven requests produce. The originating request is not available on
// that path, so the method is assumed (language model calls are always POST),
// request headers are empty.
export const classifyHttpFailure = (input: {
/** Preserve HTTP diagnostics for executor and externally captured failures alike. */
export const httpFailure = (input: {
readonly message: string
readonly url: string
readonly url?: string | undefined
readonly status?: number | undefined
readonly code?: string | undefined
readonly data?: unknown
readonly responseHeaders?: Record<string, string> | undefined
readonly responseBody?: string | undefined
readonly cause?: unknown
}) => {
const headers = normalizedHeaders(Headers.fromInput(input.responseHeaders))
const retryAfter = retryAfterMs(headers)
const rateLimit = rateLimitDetails(headers, retryAfter)
const details = responseBody(input.responseBody)
return classifyProviderFailure({
message: input.message,
status: input.status,
code: input.code,
retryAfterMs: retryAfter,
rateLimit,
http: new HttpContext({
request: new HttpRequestDetails({ method: "POST", url: input.url, headers: {} }),
response:
input.status === undefined
? undefined
: new HttpResponseDetails({ status: input.status, headers: headerDetails(Headers.fromInput(headers)) }),
...details,
requestId: requestId(headers),
return new AIError({
reason: classifyProviderFailure({
message: input.message,
status: input.status,
data: input.data,
rawBody: input.responseBody,
retryAfterMs: retryAfter,
rateLimit,
cause: input.cause,
http:
input.status === undefined || input.url === undefined
? undefined
: new HttpContext({ url: input.url, status: input.status, headers }),
}),
})
}
@@ -244,25 +180,25 @@ const httpError = (input: {
readonly error: unknown
readonly request: HttpClientRequest.HttpClientRequest
readonly operation: HttpOperation
readonly http?: HttpContext
}) => {
const request = HttpClientError.isHttpClientError(input.error) ? input.error.request : input.request
const transportError = (failure: { readonly message: string; readonly code?: string | undefined }) =>
new AIError({
module: "RequestExecutor",
method: input.operation,
reason: new TransportReason({
reason: new TransportError({
message: failure.message,
cause: source,
http: input.http,
transport: "http",
operation: input.operation,
code: failure.code,
url: request.url,
http: new HttpContext({ request: requestDetails(request) }),
}),
})
const source =
HttpClientError.isHttpClientError(input.error) && "cause" in input.error.reason
? input.error.reason.cause
? (input.error.reason.cause ?? input.error)
: input.error
const native = nativeTransportFailure(source)
const code = native?.code
@@ -286,6 +222,13 @@ const httpError = (input: {
})
}
export const responseStream = (response: HttpClientResponse.HttpClientResponse): Stream.Stream<Uint8Array, AIError> =>
response.stream.pipe(
Stream.mapError((error) =>
httpError({ error, request: response.request, operation: "read", http: responseHttp(response) }),
),
)
export const stream = (
executor: Interface,
request: HttpClientRequest.HttpClientRequest,
@@ -294,9 +237,7 @@ export const stream = (
Stream.unwrap(
Effect.gen(function* () {
const response = yield* executor.execute(request, middleware)
return response.stream.pipe(
Stream.mapError((error) => httpError({ error, request: response.request, operation: "read" })),
)
return responseStream(response)
}),
)
@@ -309,7 +250,7 @@ export const layer: Layer.Layer<Service, never, HttpClient.HttpClient> = Layer.e
if (!middleware)
return yield* http.execute(request).pipe(
Effect.mapError((error) => httpError({ error, request, operation: "request" })),
Effect.flatMap(statusError(request)),
Effect.flatMap(statusError),
)
const response = yield* middleware(request, (input) =>
@@ -317,7 +258,7 @@ export const layer: Layer.Layer<Service, never, HttpClient.HttpClient> = Layer.e
.execute(input)
.pipe(Effect.mapError((cause) => (cause instanceof Error ? cause : new Error(String(cause))))),
).pipe(Effect.mapError((error) => httpError({ error, request, operation: "request" })))
return yield* statusError(response.request)(response)
return yield* statusError(response)
})
return Service.of({
execute: executeOnce,
+2
View File
@@ -19,6 +19,8 @@ import type { AIError } from "../schema/index.js"
export interface Definition<Frame> {
readonly id: string
readonly frame: (bytes: Stream.Stream<Uint8Array, AIError>) => Stream.Stream<Frame, AIError>
/** Original wire representation when framing transforms the provider payload. */
readonly body?: (frame: Frame) => string | undefined
}
/** Server-Sent Events framing. Used by every JSON-streaming HTTP provider. */
+7 -2
View File
@@ -88,8 +88,13 @@ export const httpJson = <Body, Frame>(input: HttpJsonInput<Body, Frame>): HttpJs
}
}),
execute: (prepared, _request, runtime) =>
Effect.succeed({
frames: prepared.framing.frame(RequestExecutor.stream(runtime.http, prepared.request, prepared.middleware)),
Effect.gen(function* () {
const response = yield* runtime.http.execute(prepared.request, prepared.middleware)
return {
frames: prepared.framing.frame(RequestExecutor.responseStream(response)),
http: RequestExecutor.responseHttp(response),
body: prepared.framing.body,
}
}),
})
+3 -1
View File
@@ -3,7 +3,7 @@ import { Endpoint } from "../endpoint.js"
import { Auth } from "../auth.js"
import type { HttpMiddleware, Interface as RequestExecutorInterface } from "../executor.js"
import type { WebSocketChannelExecutor } from "./websocket-channel.js"
import type { AIError, LLMRequest } from "../../schema/index.js"
import type { AIError, HttpContext, LLMRequest } from "../../schema/index.js"
export interface TransportRuntime {
readonly http: RequestExecutorInterface
@@ -11,6 +11,8 @@ export interface TransportRuntime {
export interface TransportExecution<Frame> {
readonly frames: Stream.Stream<Frame, AIError>
readonly http?: HttpContext
body?(frame: Frame): string | undefined
/** Optional successful-consumption acknowledgement. HTTP leaves this absent. */
readonly complete?: Effect.Effect<void>
}
@@ -1,6 +1,6 @@
import type { Effect, Scope, Stream } from "effect"
import type { Headers } from "effect/unstable/http"
import type { AIError } from "../../schema/index.js"
import type { AIError, HttpContext } from "../../schema/index.js"
export interface WebSocketChannelExecutor {
readonly execute: (
@@ -10,6 +10,7 @@ export interface WebSocketChannelExecutor {
export interface WebSocketChannelExecution {
readonly frames: Stream.Stream<string, AIError>
readonly http?: HttpContext
/** Commits staged state after the decoded Route stream ends successfully. */
readonly complete: Effect.Effect<void>
}
+100 -33
View File
@@ -1,7 +1,13 @@
import { Cause, Effect, Queue, Stream } from "effect"
import { Headers } from "effect/unstable/http"
import { Socket } from "effect/unstable/socket"
import { AIError, TransportReason, type TransportOperation } from "../../schema/index.js"
import {
AIError,
AIErrorReason,
TransportError,
type HttpContext,
type TransportOperation,
} from "../../schema/index.js"
import * as HttpTransport from "./http.js"
import type { Transport } from "./index.js"
import type {
@@ -17,6 +23,7 @@ export interface WebSocketRequest {
}
export interface WebSocketConnection {
readonly http?: HttpContext
readonly sendText: (message: string) => Effect.Effect<void, AIError>
readonly messages: Stream.Stream<string | Uint8Array, AIError>
readonly close: Effect.Effect<void, never>
@@ -33,21 +40,22 @@ type WebSocketConstructorWithHeaders = (
const MAX_FRAME_BYTES = 16 * 1024 * 1024
const transportError = (
method: string,
message: string,
input: {
readonly operation: TransportOperation
readonly url?: string
readonly code?: string
readonly phase?: TransportReason["phase"]
readonly delivery?: TransportReason["delivery"]
readonly phase?: TransportError["phase"]
readonly delivery?: TransportError["delivery"]
readonly body?: string
readonly cause?: unknown
},
) =>
new AIError({
module: "WebSocketConnector",
method,
reason: new TransportReason({
reason: new TransportError({
message,
body: input.body,
cause: input.cause,
transport: "websocket",
operation: input.operation,
url: input.url,
@@ -59,22 +67,16 @@ const transportError = (
const annotateTransportError = (
error: AIError,
input: { readonly phase: TransportReason["phase"]; readonly delivery: TransportReason["delivery"] },
input: { readonly phase: TransportError["phase"]; readonly delivery: TransportError["delivery"] },
) =>
error.reason._tag === "Transport"
? new AIError({
module: error.module,
method: error.method,
reason: new TransportReason({
reason: new TransportError({
...error.reason,
message: error.reason.message,
transport: error.reason.transport,
operation: error.reason.operation,
code: error.reason.code,
url: error.reason.url,
http: error.reason.http,
cause: error.reason.cause,
phase: input.phase,
delivery: input.delivery,
recovery: error.reason.recovery,
}),
})
: error
@@ -95,7 +97,7 @@ const waitOpen = (ws: globalThis.WebSocket, input: WebSocketRequest) => {
if (ws.readyState === globalThis.WebSocket.OPEN) return Effect.void
if (ws.readyState === globalThis.WebSocket.CLOSING || ws.readyState === globalThis.WebSocket.CLOSED) {
return Effect.fail(
transportError("open", `WebSocket closed before opening (state ${ws.readyState})`, {
transportError(`WebSocket closed before opening (state ${ws.readyState})`, {
url: input.url,
operation: "request",
code: "closed",
@@ -124,7 +126,8 @@ const waitOpen = (ws: globalThis.WebSocket, input: WebSocketRequest) => {
cleanup()
resume(
Effect.fail(
transportError("open", `Failed to open WebSocket: ${eventMessage(event)}`, {
transportError(`Failed to open WebSocket: ${eventMessage(event)}`, {
cause: "error" in event ? (event.error ?? event) : event,
url: input.url,
operation: "request",
phase: "connect",
@@ -137,7 +140,9 @@ const waitOpen = (ws: globalThis.WebSocket, input: WebSocketRequest) => {
cleanup()
resume(
Effect.fail(
transportError("open", `WebSocket closed before opening with code ${event.code}`, {
transportError(`WebSocket closed before opening with code ${event.code}`, {
body: event.reason,
cause: event,
url: input.url,
operation: "request",
code: String(event.code),
@@ -169,7 +174,8 @@ export const toWebSocketUrl = (value: string) =>
throw new Error(`Unsupported WebSocket URL protocol ${url.protocol}`)
},
catch: (error) =>
transportError("prepare", error instanceof Error ? error.message : "Invalid WebSocket URL", {
transportError(error instanceof Error ? error.message : "Invalid WebSocket URL", {
cause: error,
url: value,
operation: "request",
code: "invalid-url",
@@ -189,7 +195,8 @@ export const open = (input: WebSocketRequest) =>
headers: input.headers,
}),
catch: (error) =>
transportError("open", error instanceof Error ? error.message : "Failed to construct WebSocket", {
transportError(error instanceof Error ? error.message : "Failed to construct WebSocket", {
cause: error,
url: input.url,
operation: "request",
phase: "connect",
@@ -214,7 +221,8 @@ export const fromWebSocket = (
Queue.failCauseUnsafe(
messages,
Cause.fail(
transportError("message", "WebSocket message exceeds the 16 MiB limit", {
transportError("WebSocket message exceeds the 16 MiB limit", {
body: typeof message === "string" ? message : new TextDecoder().decode(message),
url: input.url,
operation: "read",
code: "message-too-large",
@@ -231,7 +239,8 @@ export const fromWebSocket = (
Queue.failCauseUnsafe(
messages,
Cause.fail(
transportError("message", "WebSocket inbound queue overflow", {
transportError("WebSocket inbound queue overflow", {
body: typeof message === "string" ? message : new TextDecoder().decode(message),
url: input.url,
operation: "read",
code: "queue-overflow",
@@ -248,7 +257,8 @@ export const fromWebSocket = (
Queue.failCauseUnsafe(
messages,
Cause.fail(
transportError("message", "Unsupported WebSocket message payload", {
transportError("Unsupported WebSocket message payload", {
cause: event,
url: input.url,
operation: "read",
code: "message",
@@ -261,7 +271,8 @@ export const fromWebSocket = (
Queue.failCauseUnsafe(
messages,
Cause.fail(
transportError("message", `WebSocket error: ${eventMessage(event)}`, {
transportError(`WebSocket error: ${eventMessage(event)}`, {
cause: "error" in event ? (event.error ?? event) : event,
url: input.url,
operation: "read",
code: "message",
@@ -274,7 +285,9 @@ export const fromWebSocket = (
Queue.failCauseUnsafe(
messages,
Cause.fail(
transportError("message", `WebSocket closed with code ${event.code}`, {
transportError(`WebSocket closed with code ${event.code}`, {
body: event.reason,
cause: event,
url: input.url,
operation: "read",
code: String(event.code),
@@ -298,7 +311,7 @@ export const fromWebSocket = (
Effect.suspend(() => {
if (ws.readyState !== globalThis.WebSocket.OPEN)
return Effect.fail(
transportError("sendText", `WebSocket is not open (state ${ws.readyState})`, {
transportError(`WebSocket is not open (state ${ws.readyState})`, {
url: input.url,
operation: "write",
phase: "send",
@@ -308,7 +321,8 @@ export const fromWebSocket = (
return Effect.try({
try: () => ws.send(message),
catch: (error) =>
transportError("sendText", error instanceof Error ? error.message : "Failed to send WebSocket message", {
transportError(error instanceof Error ? error.message : "Failed to send WebSocket message", {
cause: error,
url: input.url,
operation: "write",
phase: "send",
@@ -349,10 +363,23 @@ export const makeDirect = (connector: WebSocketConnector): WebSocketChannelExecu
(connection) => connection.close,
)
const create = yield* exchange.driver.create(undefined)
yield* connection.sendText(create.message)
yield* connection.sendText(create.message).pipe(
Effect.mapError(
(error) =>
new AIError({
reason: AIErrorReason.make({
...error.reason,
message: error.reason.message,
cause: error.reason.cause,
http: error.reason.http ?? connection.http,
}),
}),
),
)
const decoder = new TextDecoder()
let observed = false
return {
http: connection.http,
frames: connection.messages.pipe(
Stream.map((message) => {
observed = true
@@ -364,9 +391,49 @@ export const makeDirect = (connector: WebSocketConnector): WebSocketChannelExecu
delivery: observed ? "accepted" : "ambiguous",
}),
),
Stream.mapEffect((frame) => exchange.driver.observe(create, frame)),
Stream.mapEffect((frame) =>
exchange.driver.observe(create, frame).pipe(
Effect.mapError(
(error) =>
new AIError({
reason: AIErrorReason.make({
...error.reason,
message: error.reason.message,
cause: error.reason.cause,
body: frame,
}),
}),
),
Effect.map((observation) =>
"error" in observation
? {
...observation,
error: new AIError({
reason: AIErrorReason.make({
...observation.error.reason,
message: observation.error.reason.message,
cause: observation.error.reason.cause,
body: frame,
}),
}),
}
: observation,
),
),
),
Stream.takeUntil(observationTerminal),
Stream.mapEffect(observationFrame),
Stream.mapError(
(error) =>
new AIError({
reason: AIErrorReason.make({
...error.reason,
message: error.reason.message,
cause: error.reason.cause,
http: error.reason.http ?? connection.http,
}),
}),
),
),
complete: Effect.void,
}
@@ -417,7 +484,7 @@ export const json = <Body, Message>(input: JsonInput<Body, Message>): JsonTransp
const webSocket = options?.webSocket
if (!webSocket) {
return Effect.fail(
transportError("json", "WebSocket JSON transport requires StreamOptions.webSocket", {
transportError("WebSocket JSON transport requires StreamOptions.webSocket", {
url: prepared.url,
operation: "request",
code: "unavailable",
@@ -435,7 +502,7 @@ export const json = <Body, Message>(input: JsonInput<Body, Message>): JsonTransp
connect: { url: prepared.url, headers: prepared.headers },
fallback: () =>
Stream.fail(
transportError("fallback", "WebSocket JSON transport does not provide HTTP fallback", {
transportError("WebSocket JSON transport does not provide HTTP fallback", {
url: prepared.url,
operation: "request",
code: "websocket",
+64 -96
View File
@@ -1,19 +1,13 @@
import { Schema } from "effect"
import { Tool } from "@opencode-ai/schema/tool"
import { ModelID, ProviderID, RouteID } from "./ids.js"
import { ProviderMetadata } from "./messages.js"
export const ProviderFailureClassification = Schema.Literals(["context-overflow", "payload-too-large"])
export type ProviderFailureClassification = typeof ProviderFailureClassification.Type
export class HttpRequestDetails extends Schema.Class<HttpRequestDetails>("AI.HttpRequestDetails")({
method: Schema.String,
export class HttpContext extends Schema.Class<HttpContext>("AI.HttpContext")({
url: Schema.String,
headers: Schema.Record(Schema.String, Schema.String),
}) {}
export class HttpResponseDetails extends Schema.Class<HttpResponseDetails>("AI.HttpResponseDetails")({
status: Schema.Number,
status: Schema.Int.check(Schema.isBetween({ minimum: 100, maximum: 599 })),
headers: Schema.Record(Schema.String, Schema.String),
}) {}
@@ -24,74 +18,61 @@ export class HttpRateLimitDetails extends Schema.Class<HttpRateLimitDetails>("AI
reset: Schema.optional(Schema.Record(Schema.String, Schema.String)),
}) {}
export class HttpContext extends Schema.Class<HttpContext>("AI.HttpContext")({
request: HttpRequestDetails,
response: Schema.optional(HttpResponseDetails),
body: Schema.optional(Schema.String),
bodyTruncated: Schema.optional(Schema.Boolean),
requestId: Schema.optional(Schema.String),
rateLimit: Schema.optional(HttpRateLimitDetails),
}) {}
export class InvalidRequestReason extends Schema.Class<InvalidRequestReason>("AI.Error.InvalidRequest")({
_tag: Schema.tag("InvalidRequest"),
const ReasonFields = {
message: Schema.String,
parameter: Schema.optional(Schema.String),
classification: Schema.optional(ProviderFailureClassification),
providerMetadata: Schema.optional(ProviderMetadata),
// Preserve the complete original response or triggering event before decoding narrows it.
body: Schema.optional(Schema.String),
http: Schema.optional(HttpContext),
}) {}
cause: Schema.optional(Schema.Defect({ includeStack: true })),
}
export class NoRouteReason extends Schema.Class<NoRouteReason>("AI.Error.NoRoute")({
_tag: Schema.tag("NoRoute"),
export class InvalidRequestError extends Schema.TaggedError<InvalidRequestError>("AI.Error.InvalidRequest")(
"InvalidRequest",
{
...ReasonFields,
parameter: Schema.optional(Schema.String),
classification: Schema.optional(ProviderFailureClassification),
},
) {}
export class NoRouteError extends Schema.TaggedError<NoRouteError>("AI.Error.NoRoute")("NoRoute", {
...ReasonFields,
route: RouteID,
provider: ProviderID,
model: ModelID,
}) {
get message() {
return `No AI route for ${this.provider}/${this.model} using ${this.route}`
}
}
export class AuthenticationReason extends Schema.Class<AuthenticationReason>("AI.Error.Authentication")({
_tag: Schema.tag("Authentication"),
message: Schema.String,
kind: Schema.Literals(["missing", "invalid", "expired", "insufficient-permissions", "unknown"]),
providerMetadata: Schema.optional(ProviderMetadata),
http: Schema.optional(HttpContext),
}) {}
export class RateLimitReason extends Schema.Class<RateLimitReason>("AI.Error.RateLimit")({
_tag: Schema.tag("RateLimit"),
message: Schema.String,
export class AuthenticationError extends Schema.TaggedError<AuthenticationError>("AI.Error.Authentication")(
"Authentication",
{
...ReasonFields,
kind: Schema.Literals(["missing", "invalid", "expired", "insufficient-permissions", "unknown"]),
},
) {}
export class RateLimitError extends Schema.TaggedError<RateLimitError>("AI.Error.RateLimit")("RateLimit", {
...ReasonFields,
retryAfterMs: Schema.optional(Schema.Number),
rateLimit: Schema.optional(HttpRateLimitDetails),
providerMetadata: Schema.optional(ProviderMetadata),
http: Schema.optional(HttpContext),
}) {}
export class QuotaExceededReason extends Schema.Class<QuotaExceededReason>("AI.Error.QuotaExceeded")({
_tag: Schema.tag("QuotaExceeded"),
message: Schema.String,
providerMetadata: Schema.optional(ProviderMetadata),
http: Schema.optional(HttpContext),
}) {}
export class QuotaExceededError extends Schema.TaggedError<QuotaExceededError>("AI.Error.QuotaExceeded")(
"QuotaExceeded",
ReasonFields,
) {}
export class ContentPolicyReason extends Schema.Class<ContentPolicyReason>("AI.Error.ContentPolicy")({
_tag: Schema.tag("ContentPolicy"),
message: Schema.String,
providerMetadata: Schema.optional(ProviderMetadata),
http: Schema.optional(HttpContext),
}) {}
export class ContentPolicyError extends Schema.TaggedError<ContentPolicyError>("AI.Error.ContentPolicy")(
"ContentPolicy",
ReasonFields,
) {}
export class ProviderInternalReason extends Schema.Class<ProviderInternalReason>("AI.Error.ProviderInternal")({
_tag: Schema.tag("ProviderInternal"),
message: Schema.String,
status: Schema.optional(Schema.Number),
retryAfterMs: Schema.optional(Schema.Number),
providerMetadata: Schema.optional(ProviderMetadata),
http: Schema.optional(HttpContext),
}) {}
export class ProviderInternalError extends Schema.TaggedError<ProviderInternalError>("AI.Error.ProviderInternal")(
"ProviderInternal",
{
...ReasonFields,
retryAfterMs: Schema.optional(Schema.Number),
},
) {}
export const TransportType = Schema.Literals(["http", "websocket"])
export type TransportType = typeof TransportType.Type
@@ -99,14 +80,12 @@ export type TransportType = typeof TransportType.Type
export const TransportOperation = Schema.Literals(["request", "read", "write"])
export type TransportOperation = typeof TransportOperation.Type
export class TransportReason extends Schema.Class<TransportReason>("AI.Error.Transport")({
_tag: Schema.tag("Transport"),
message: Schema.String,
export class TransportError extends Schema.TaggedError<TransportError>("AI.Error.Transport")("Transport", {
...ReasonFields,
transport: TransportType,
operation: TransportOperation,
code: Schema.optional(Schema.String),
url: Schema.optional(Schema.String),
http: Schema.optional(HttpContext),
phase: Schema.optional(
Schema.Literals(["prepare", "queue", "connect", "send", "receive", "decode", "complete", "fallback", "close"]),
),
@@ -116,51 +95,40 @@ export class TransportReason extends Schema.Class<TransportReason>("AI.Error.Tra
),
}) {}
export class InvalidProviderOutputReason extends Schema.Class<InvalidProviderOutputReason>(
export class InvalidProviderOutputError extends Schema.TaggedError<InvalidProviderOutputError>(
"AI.Error.InvalidProviderOutput",
)({
_tag: Schema.tag("InvalidProviderOutput"),
message: Schema.String,
)("InvalidProviderOutput", {
...ReasonFields,
classification: Schema.optional(Schema.Literals(["incomplete-stream"])),
route: Schema.optional(Schema.String),
raw: Schema.optional(Schema.String),
providerMetadata: Schema.optional(ProviderMetadata),
}) {}
export class UnknownProviderReason extends Schema.Class<UnknownProviderReason>("AI.Error.UnknownProvider")({
_tag: Schema.tag("UnknownProvider"),
message: Schema.String,
status: Schema.optional(Schema.Number),
providerMetadata: Schema.optional(ProviderMetadata),
http: Schema.optional(HttpContext),
}) {}
export class UnknownProviderError extends Schema.TaggedError<UnknownProviderError>("AI.Error.UnknownProvider")(
"UnknownProvider",
ReasonFields,
) {}
export const AIErrorReason = Schema.Union([
InvalidRequestReason,
NoRouteReason,
AuthenticationReason,
RateLimitReason,
QuotaExceededReason,
ContentPolicyReason,
ProviderInternalReason,
TransportReason,
InvalidProviderOutputReason,
UnknownProviderReason,
InvalidRequestError,
NoRouteError,
AuthenticationError,
RateLimitError,
QuotaExceededError,
ContentPolicyError,
ProviderInternalError,
TransportError,
InvalidProviderOutputError,
UnknownProviderError,
]).pipe(Schema.toTaggedUnion("_tag"))
export type AIErrorReason = Schema.Schema.Type<typeof AIErrorReason>
export class AIError extends Schema.TaggedError<AIError>()("AI.Error", {
module: Schema.String,
method: Schema.String,
reason: AIErrorReason,
// Raw provider payload as a string, so classified failures never lose the
// original error detail even when the pretty message is a summary.
body: Schema.optional(Schema.String),
}) {
override readonly cause = this.reason
override get message() {
return `${this.module}.${this.method}: ${this.reason.message}`
override get message(): string {
return this.reason.message
}
}
+255 -54
View File
@@ -1,7 +1,7 @@
import { describe, expect } from "bun:test"
import { Deferred, Effect, Fiber, Layer, Ref, Stream } from "effect"
import { Headers, HttpClient, HttpClientError, HttpClientRequest, HttpClientResponse } from "effect/unstable/http"
import { LLM, AIError } from "../src/index.js"
import { LLM, AIError, HttpContext, InvalidProviderOutputError, TransportError } from "../src/index.js"
import { LLMClient, RequestExecutor, WebSocketTransport, type WebSocketChannelExecutor } from "../src/route.js"
import * as OpenAIChat from "../src/protocols/openai-chat.js"
import * as OpenAI from "../src/providers/openai.js"
@@ -62,22 +62,85 @@ const countedResponsesLayer = (attempts: Ref.Ref<number>, responses: ReadonlyArr
const expectAIError = (error: unknown) => {
expect(error).toBeInstanceOf(AIError)
if (!(error instanceof AIError)) throw new Error("expected AIError")
expect(error.reason).toBeInstanceOf(Error)
expect(error.cause).toBe(error.reason)
return error
}
const errorHttp = (error: AIError) => ("http" in error.reason ? error.reason.http : undefined)
const largeProviderMessage = `Upstream request failed: ${"validation failed; ".repeat(1_000)}`
describe("RequestExecutor", () => {
it.effect("preserves externally captured HTTP errors without inventing response context", () =>
Effect.sync(() => {
const cause = new Error("upstream request failed")
const body = '{"error":{"message":"Rate limited","trace":"original"}}'
const error = RequestExecutor.httpFailure({
message: "Rate limited",
url: request.url,
status: 429,
responseHeaders: { "Retry-After": "2", "X-Request-ID": "req_external" },
responseBody: body,
cause,
})
expect(error.message).toBe("Rate limited")
expect(error.reason).toMatchObject({ _tag: "RateLimit", retryAfterMs: 2000 })
expect(error.reason.body).toBe(body)
expect(error.reason.cause).toBe(cause)
expect(error.reason.http).toEqual(
new HttpContext({
url: request.url,
status: 429,
headers: { "retry-after": "2", "x-request-id": "req_external" },
}),
)
expect(RequestExecutor.httpFailure({ message: "No response", url: request.url }).reason.http).toBeUndefined()
expect(RequestExecutor.httpFailure({ message: "No URL", status: 500 }).reason.http).toBeUndefined()
}),
)
it.effect("retains the original body-read failure on an HTTP status error", () =>
Effect.gen(function* () {
const cause = new Error("response body disconnected")
const error = yield* Effect.gen(function* () {
const executor = yield* RequestExecutor.Service
return yield* executor.execute(request).pipe(Effect.flip)
}).pipe(
Effect.provide(
responsesLayer([
new Response(
new ReadableStream({
start(controller) {
controller.error(cause)
},
}),
{
status: 503,
headers: { "x-request-id": "req_failed_body" },
},
),
]),
),
)
expect(error.reason._tag).toBe("ProviderInternal")
expect(error.reason.cause).toBe(cause)
expect(error.reason.body).toBeUndefined()
expect(error.reason.http).toMatchObject({ status: 503, headers: { "x-request-id": "req_failed_body" } })
}),
)
it.effect("parses response body failures at the executor seam", () =>
Effect.gen(function* () {
const executor = yield* RequestExecutor.Service
const error = yield* RequestExecutor.stream(executor, secretRequest).pipe(Stream.runDrain, Effect.flip)
expectAIError(error)
expect(error.message).toBe("ECONNRESET: disconnected query-secret-123 header-secret-456")
expect(error.reason.http).toMatchObject({ status: 200, url: secretRequest.url })
expect(error.reason.cause).toMatchObject({ code: "ECONNRESET" })
expect(error.reason).toMatchObject({
_tag: "Transport",
message: "ECONNRESET: disconnected query-secret-123 header-secret-456",
transport: "http",
operation: "read",
code: "ECONNRESET",
@@ -104,9 +167,10 @@ describe("RequestExecutor", () => {
const error = yield* RequestExecutor.stream(executor, secretRequest).pipe(Stream.runDrain, Effect.flip)
expectAIError(error)
expect(error.message).toBe("ECONNRESET: socket closed")
expect(error.reason.cause).toBeInstanceOf(TypeError)
expect(error.reason).toMatchObject({
_tag: "Transport",
message: "ECONNRESET: socket closed",
operation: "read",
code: "ECONNRESET",
})
@@ -133,7 +197,9 @@ describe("RequestExecutor", () => {
.pipe(Effect.flip)
expectAIError(error)
expect(error.reason.message).toBe("plugin rejected request")
expect(error.message).toBe("plugin rejected request")
expect(error.reason.cause).toBeInstanceOf(Error)
expect(error.reason.http).toBeUndefined()
}).pipe(Effect.provide(responsesLayer([]))),
)
@@ -152,16 +218,11 @@ describe("RequestExecutor", () => {
.pipe(Effect.flip)
expectAIError(error)
expect(error.message).toBe("ECONNRESET: proxy disconnected proxy-secret")
expect(error.reason.http).toBeUndefined()
expect(error.reason).toMatchObject({
_tag: "Transport",
message: "ECONNRESET: proxy disconnected proxy-secret",
url: "https://proxy.test/v1/chat?api_key=proxy-secret",
http: {
request: {
url: "https://proxy.test/v1/chat?api_key=proxy-secret",
headers: { authorization: "Bearer proxy-secret" },
},
},
})
}).pipe(
Effect.provide(
@@ -206,8 +267,8 @@ describe("RequestExecutor", () => {
expect(error.reason).toMatchObject({
_tag: "InvalidRequest",
classification: "payload-too-large",
http: { response: { status: 413 } },
})
expect(error.reason.http?.status).toBe(413)
}).pipe(Effect.provide(responsesLayer([new Response("request too large", { status: 413 })]))),
)
@@ -220,8 +281,8 @@ describe("RequestExecutor", () => {
expect(error.reason).toMatchObject({
_tag: "InvalidRequest",
classification: "context-overflow",
http: { response: { status: 413 } },
})
expect(error.reason.http?.status).toBe(413)
}).pipe(
Effect.provide(
responsesLayer([
@@ -241,7 +302,7 @@ describe("RequestExecutor", () => {
expectAIError(error)
expect(error.reason).toMatchObject({ _tag: "InvalidRequest" })
expect("classification" in error.reason ? error.reason.classification : undefined).toBeUndefined()
expect(error.reason.message).toBe("Provider request failed with HTTP 400")
expect(error.message).toBe("Provider request failed with HTTP 400")
}).pipe(Effect.provide(responsesLayer([new Response("invalid parameter", { status: 400 })]))),
)
@@ -251,9 +312,9 @@ describe("RequestExecutor", () => {
const error = yield* executor.execute(request).pipe(Effect.flip)
expectAIError(error)
expect(error.reason).toMatchObject({ _tag: "InvalidRequest", message: largeProviderMessage })
expect(errorHttp(error)?.body).toContain(largeProviderMessage)
expect(errorHttp(error)?.bodyTruncated).toBeUndefined()
expect(error.reason).toMatchObject({ _tag: "InvalidRequest" })
expect(error.message).toBe(largeProviderMessage)
expect(error.reason.body).toContain(largeProviderMessage)
}).pipe(
Effect.provide(
responsesLayer([
@@ -277,8 +338,8 @@ describe("RequestExecutor", () => {
expectAIError(error)
expect(error.reason).toMatchObject({
_tag: "InvalidRequest",
message: "Provider request failed with HTTP 400",
})
expect(error.message).toBe("Provider request failed with HTTP 400")
}).pipe(Effect.provide(responsesLayer([new Response('{"error":{"message":" "}}', { status: 400 })]))),
)
@@ -321,30 +382,21 @@ describe("RequestExecutor", () => {
const error = yield* executor.execute(request).pipe(Effect.flip)
expectAIError(error)
expect(error).toMatchObject({
reason: {
_tag: "RateLimit",
retryAfterMs: 0,
rateLimit: { retryAfterMs: 0 },
http: {
requestId: "req_123",
request: {
method: "POST",
url: "https://provider.test/v1/chat?api_key=secret&key=secret&debug=1",
headers: { authorization: "Bearer secret", "x-safe": "visible" },
},
response: {
status: 429,
headers: {
"retry-after-ms": "0",
"x-request-id": "req_123",
"x-api-key": "secret",
},
},
expect(error.reason).toMatchObject({
_tag: "RateLimit",
retryAfterMs: 0,
rateLimit: { retryAfterMs: 0 },
http: {
url: "https://provider.test/v1/chat?api_key=secret&key=secret&debug=1",
status: 429,
headers: {
"retry-after-ms": "0",
"x-request-id": "req_123",
"x-api-key": "secret",
},
},
})
expect(errorHttp(error)?.body).toBe("rate limited")
expect(error.reason.body).toBe("rate limited")
}).pipe(
Effect.provide(
responsesLayer([
@@ -363,8 +415,7 @@ describe("RequestExecutor", () => {
const error = yield* executor.execute(request).pipe(Effect.flip)
expectAIError(error)
expect(errorHttp(error)?.request.headers["x-safe"]).toBe("visible")
expect(errorHttp(error)?.response?.headers["x-safe"]).toBe("response-secret")
expect(error.reason.http?.headers["x-safe"]).toBe("response-secret")
}).pipe(
Effect.provide(responsesLayer([new Response("bad", { status: 400, headers: { "x-safe": "response-secret" } })])),
Effect.provideService(Headers.CurrentRedactedNames, ["x-safe"]),
@@ -410,8 +461,8 @@ describe("RequestExecutor", () => {
const error = yield* executor.execute(request).pipe(Effect.flip)
expectAIError(error)
expect(error.reason).toMatchObject({ _tag: "ProviderInternal" })
expect(errorHttp(error)?.rateLimit).toEqual({
expect(error.reason).toMatchObject({ _tag: "RateLimit" })
expect(error.reason._tag === "RateLimit" ? error.reason.rateLimit : undefined).toEqual({
retryAfterMs: 0,
limit: { requests: "100", "input-tokens": "10000" },
remaining: { requests: "12", "input-tokens": "9000" },
@@ -420,8 +471,8 @@ describe("RequestExecutor", () => {
}).pipe(
Effect.provide(
responsesLayer([
new Response("overloaded", {
status: 529,
new Response("rate limited", {
status: 429,
headers: {
"retry-after-ms": "0",
"anthropic-ratelimit-requests-limit": "100",
@@ -453,7 +504,8 @@ describe("RequestExecutor", () => {
)
expectAIError(error)
expect(error.reason).toMatchObject({ _tag: "ProviderInternal", status: 503 })
expect(error.reason).toMatchObject({ _tag: "ProviderInternal" })
expect(error.reason.http?.status).toBe(503)
expect(yield* Ref.get(attempts)).toBe(1)
}),
)
@@ -466,7 +518,8 @@ describe("RequestExecutor", () => {
const error = yield* executor.execute(request).pipe(Effect.flip)
expectAIError(error)
expect(error.reason).toMatchObject({ _tag: "ProviderInternal", status })
expect(error.reason).toMatchObject({ _tag: "ProviderInternal" })
expect(error.reason.http?.status).toBe(status)
}).pipe(
Effect.provide(
responsesLayer([
@@ -490,8 +543,7 @@ describe("RequestExecutor", () => {
expectAIError(error)
expect(error.reason).toMatchObject({ _tag: "Authentication" })
expect(errorHttp(error)?.bodyTruncated).toBeUndefined()
expect(errorHttp(error)?.body).toHaveLength(20_000)
expect(error.reason.body).toHaveLength(20_000)
}).pipe(
Effect.provide(
responsesLayer([
@@ -508,9 +560,7 @@ describe("RequestExecutor", () => {
const error = yield* executor.execute(request).pipe(Effect.flip)
expectAIError(error)
expect(errorHttp(error)?.body).toBe(
'{"error":{"message":"bad","key":"body-secret","detail":"api_key=query-secret"}}',
)
expect(error.reason.body).toBe('{"error":{"message":"bad","key":"body-secret","detail":"api_key=query-secret"}}')
}).pipe(
Effect.provide(
responsesLayer([
@@ -528,7 +578,7 @@ describe("RequestExecutor", () => {
const error = yield* executor.execute(secretRequest).pipe(Effect.flip)
expectAIError(error)
expect(errorHttp(error)?.body).toBe("provider echoed query-secret-123 and authorization header-secret-456")
expect(error.reason.body).toBe("provider echoed query-secret-123 and authorization header-secret-456")
}).pipe(
Effect.provide(
responsesLayer([
@@ -565,6 +615,9 @@ describe("RequestExecutor", () => {
expectAIError(error)
expect(error.reason).toMatchObject({ _tag: "InvalidProviderOutput" })
expect(error.reason.body).toBe("not-json")
expect(error.reason.cause).toBeDefined()
expect(error.reason.http).toMatchObject({ status: 200, headers: { "content-type": "text/event-stream" } })
expect(yield* Ref.get(attempts)).toBe(1)
}),
)
@@ -578,6 +631,154 @@ describe("WebSocket channel execution", () => {
JSON.stringify({ type: "response.completed", response: { id: "resp_1" } }),
]
it.effect("preserves close reasons and native event causes without fabricated HTTP metadata", () =>
Effect.gen(function* () {
class TestSocket extends EventTarget {
readyState = globalThis.WebSocket.OPEN
send() {}
close() {}
}
const socket = new TestSocket()
const connection = yield* WebSocketTransport.fromWebSocket(
// oxlint-disable-next-line typescript-eslint/no-unsafe-type-assertion
socket as unknown as globalThis.WebSocket,
{ url: "wss://provider.test/responses", headers: Headers.empty },
)
const event = new CloseEvent("close", { code: 1011, reason: "upstream trace: req_close" })
socket.dispatchEvent(event)
const error = yield* connection.messages.pipe(Stream.runDrain, Effect.flip)
expect(error.reason).toMatchObject({ _tag: "Transport", code: "1011", phase: "close" })
expect(error.message).toBe("WebSocket closed with code 1011")
expect(error.reason.body).toBe(event.reason)
expect(error.reason.cause).toBe(event)
expect(error.reason.http).toBeUndefined()
yield* connection.close
}),
)
it.effect("preserves opening event errors and native send exceptions", () =>
Effect.gen(function* () {
const cause = new Error("native send failed")
class TestSocket extends EventTarget {
readyState = globalThis.WebSocket.CONNECTING
send() {
throw cause
}
close() {}
}
const socket = new TestSocket()
const open = WebSocketTransport.fromWebSocket(
// oxlint-disable-next-line typescript-eslint/no-unsafe-type-assertion
socket as unknown as globalThis.WebSocket,
{ url: "wss://provider.test/responses", headers: Headers.empty },
)
const fiber = yield* open.pipe(Effect.flip, Effect.forkChild({ startImmediately: true }))
const event = new ErrorEvent("error", { message: "handshake rejected", error: cause })
socket.dispatchEvent(event)
const error = yield* Fiber.join(fiber)
expect(error.reason.cause).toBe(cause)
expect(error.message).toContain("handshake rejected")
expect(error.reason.http).toBeUndefined()
socket.readyState = globalThis.WebSocket.OPEN
const connection = yield* open
const sent = yield* connection.sendText("create").pipe(Effect.flip)
expect(sent.reason.cause).toBe(cause)
expect(sent.message).toBe(cause.message)
yield* connection.close
}),
)
it.effect("preserves raw driver failures and known upgrade metadata", () =>
Effect.gen(function* () {
const cause = new Error("driver validation failed")
const frame = '{ "error": "failed", "trace": "original" }'
const http = new HttpContext({
url: "https://provider.test/responses",
status: 101,
headers: { upgrade: "websocket" },
})
const executor = WebSocketTransport.makeDirect({
open: () =>
Effect.succeed({
http,
sendText: () => Effect.void,
messages: Stream.make(frame),
close: Effect.void,
}),
})
const execution = yield* executor.execute({
id: "exchange_error",
connect: { url: "wss://provider.test/responses", headers: Headers.empty },
fallback: () => Stream.empty,
driver: {
create: () => Effect.succeed({ message: "create", mode: "full" }),
observe: () =>
Effect.succeed({
type: "provider-failure",
error: new AIError({
reason: new InvalidProviderOutputError({
message: "Driver failed",
cause,
body: "narrowed",
}),
}),
}),
},
})
const error = yield* execution.frames.pipe(Stream.runDrain, Effect.flip)
expect(error.message).toBe("Driver failed")
expect(error.reason.body).toBe(frame)
expect(error.reason.cause).toBe(cause)
expect(error.reason.http).toBe(http)
expect(execution.http).toBe(http)
}),
)
it.effect("retains diagnostic fields when annotating transport delivery", () =>
Effect.gen(function* () {
const cause = new Error("connection closed")
const executor = WebSocketTransport.makeDirect({
open: () =>
Effect.succeed({
sendText: () => Effect.void,
messages: Stream.fail(
new AIError({
reason: new TransportError({
message: "Socket closed",
transport: "websocket",
operation: "read",
phase: "close",
recovery: "retry-full",
body: "server close detail",
cause,
}),
}),
),
close: Effect.void,
}),
})
const execution = yield* executor.execute({
id: "exchange_closed",
connect: { url: "wss://provider.test/responses", headers: Headers.empty },
fallback: () => Stream.empty,
driver: {
create: () => Effect.succeed({ message: "create", mode: "full" }),
observe: (_create, frame) => Effect.succeed({ type: "frame", frame }),
},
})
const error = yield* execution.frames.pipe(Stream.runDrain, Effect.flip)
expect(error.message).toBe("Socket closed")
expect(error.reason.body).toBe("server close detail")
expect(error.reason.cause).toBe(cause)
expect(error.reason).toMatchObject({ phase: "close", delivery: "ambiguous", recovery: "retry-full" })
expect(error.reason.http).toBeUndefined()
}),
)
it.effect("runs a channel driver through the direct executor", () =>
Effect.gen(function* () {
const sent = yield* Ref.make("")
+42 -16
View File
@@ -7,6 +7,37 @@ import { it } from "./lib/effect.js"
import { dynamicResponse } from "./lib/http.js"
describe("Image", () => {
for (const provider of [OpenAI, Google, XAI, ZAI]) {
const model = provider.configure({ apiKey: "test", baseURL: "https://image.test" }).image("image-model")
for (const body of ['{"data":42,"candidates":42,"opaque":{"nested":[1,2]},"trace":"outer"}', '{"invalid":']) {
it.effect(`retains ${model.provider} image response body and decode cause: ${body}`, () =>
Effect.gen(function* () {
const error = yield* Image.generate({ model, prompt: "hello" }).pipe(Effect.flip)
expect(error.reason._tag).toBe("InvalidProviderOutput")
expect(error.message).toContain("invalid response")
expect(error.reason.body).toBe(body)
expect(error.reason.http).toMatchObject({ status: 200, headers: { "x-image-trace": "trace-1" } })
expect(error.reason.http?.url).toStartWith("https://image.test/")
expect(error.reason.cause).toBeInstanceOf(Error)
}).pipe(
Effect.provide(
ImageClient.layer.pipe(
Layer.provide(
dynamicResponse((input) =>
Effect.succeed(
input.respond(body, {
headers: { "content-type": "application/json", "x-image-trace": "trace-1" },
}),
),
),
),
),
),
),
)
}
}
it.effect("generates images through the OpenAI Images API", () =>
Effect.gen(function* () {
const response = yield* Image.generate({
@@ -530,22 +561,17 @@ describe("Image", () => {
Effect.sync(() => {
expect(error.reason._tag).toBe("InvalidProviderOutput")
if (error.reason._tag !== "InvalidProviderOutput") return
expect(error.reason.message).toContain("finish reasons: IMAGE_SAFETY")
expect(error.reason.providerMetadata).toEqual({
google: {
promptFeedback: { blockReason: "SAFETY" },
candidates: [
{
index: 0,
finishReason: "IMAGE_SAFETY",
finishMessage: "The generated image was blocked by safety filters.",
safetyRatings: [{ category: "HARM_CATEGORY_DANGEROUS_CONTENT", blocked: true }],
citationMetadata: undefined,
groundingMetadata: undefined,
parts: [{ type: "text", text: "blocked", thought: false, thoughtSignature: undefined }],
},
],
},
expect(error.message).toContain("finish reasons: IMAGE_SAFETY")
expect(JSON.parse(error.reason.body ?? "")).toEqual({
promptFeedback: { blockReason: "SAFETY" },
candidates: [
{
finishReason: "IMAGE_SAFETY",
finishMessage: "The generated image was blocked by safety filters.",
safetyRatings: [{ category: "HARM_CATEGORY_DANGEROUS_CONTENT", blocked: true }],
content: { parts: [{ text: "blocked", thought: false }] },
},
],
})
}),
),
+33
View File
@@ -111,6 +111,39 @@ describe("provider error classification", () => {
})
describe("provider error rawBody classification", () => {
test("classifies provider envelopes without separate code inputs", () => {
const cases = [
['{"type":"error","error":{"type":"overloaded_error","message":"Try again"}}', "ProviderInternal"],
['{"error":{"code":"insufficient_quota","message":"Request failed"}}', "QuotaExceeded"],
[
'{"type":"response.failed","response":{"error":{"code":"authentication_error","message":"Denied"}}}',
"Authentication",
],
['{"error":{"code":429,"status":"RESOURCE_EXHAUSTED","message":"Try again"}}', "ProviderInternal"],
['{"exception":{"type":"throttlingException","details":{"message":"Try again"}}}', "RateLimit"],
] as const
for (const [rawBody, expected] of cases) {
const reason = classifyProviderFailure({ message: "Request failed", rawBody })
expect(reason._tag).toBe(expected)
expect(reason.body).toBe(rawBody)
expect(reason).not.toHaveProperty("code")
}
})
test("classifies separately supplied SDK data without replacing the response body", () => {
const data = { error: { code: "authentication_error" } }
for (const value of [data, JSON.stringify(data)]) {
const reason = classifyProviderFailure({
message: "Request failed",
status: 400,
rawBody: '{"message":"Request failed"}',
data: value,
})
expect(reason._tag).toBe("Authentication")
expect(reason.body).toBe('{"message":"Request failed"}')
}
})
test("classifies overflow signals buried in the raw payload when the summary is vague", () => {
const reason = classifyProviderFailure({
message: "Request failed",
@@ -39,7 +39,7 @@ describe("Anthropic Messages sad-path recorded", () => {
expect(error).toBeInstanceOf(AIError)
expect(error.reason).toMatchObject({ _tag: "InvalidRequest" })
expect(error.reason.message).toContain("`tool_use` ids were found without `tool_result` blocks")
expect(error.message).toContain("`tool_use` ids were found without `tool_result` blocks")
}),
)
})
@@ -2,7 +2,7 @@ import { describe, expect } from "bun:test"
import { Effect } from "effect"
import { HttpClientRequest } from "effect/unstable/http"
import { CacheHint, LLM, AIError, LLMRequest, Message, ToolCallPart, ToolDefinition, Usage } from "../../src/index.js"
import { Auth, LLMClient } from "../../src/route.js"
import { Auth, Endpoint, LLMClient, Route } from "../../src/route.js"
import { compileRequest } from "../../src/route/client.js"
import * as AnthropicMessages from "../../src/protocols/anthropic-messages.js"
import { GoogleVertexMessages } from "../../src/providers.js"
@@ -810,6 +810,99 @@ describe("Anthropic Messages route", () => {
}),
)
it.effect("round-trips compatible provider metadata in its own namespace", () =>
Effect.gen(function* () {
const compatible = Route.make({
id: "custom-anthropic-messages",
provider: "custom-anthropic",
protocol: AnthropicMessages.protocol,
endpoint: Endpoint.path("/messages", { baseURL: "https://compatible.test/v1" }),
auth: Auth.header("x-api-key", "test"),
framing: AnthropicMessages.framing,
}).model({ id: "custom-model" })
const result = [
{
type: "web_search_result",
url: "https://example.com",
citations: [{ type: "web_search_result_location", cited_text: "Example" }],
},
]
const response = yield* LLMClient.generate(LLM.request({ model: compatible, prompt: "Search." })).pipe(
Effect.provide(
fixedResponse(
sseEvents(
{ type: "message_start", message: { usage: { input_tokens: 5, custom_start: true } } },
{ type: "content_block_start", index: 0, content_block: { type: "thinking", thinking: "Thinking." } },
{ type: "content_block_delta", index: 0, delta: { type: "signature_delta", signature: "custom_sig" } },
{ type: "content_block_stop", index: 0 },
{
type: "content_block_start",
index: 1,
content_block: { type: "redacted_thinking", data: "custom_redacted" },
},
{ type: "content_block_stop", index: 1 },
{
type: "content_block_start",
index: 2,
content_block: {
type: "server_tool_use",
id: "custom_tool",
name: "web_search",
input: { query: "example" },
},
},
{ type: "content_block_stop", index: 2 },
{
type: "content_block_start",
index: 3,
content_block: { type: "web_search_tool_result", tool_use_id: "custom_tool", content: result },
},
{ type: "content_block_stop", index: 3 },
{
type: "message_delta",
delta: { stop_reason: "end_turn", stop_sequence: "custom_stop" },
usage: { output_tokens: 2, custom_terminal: true },
},
{ type: "message_stop" },
),
),
),
)
expect(response.message.content).toMatchObject([
{ type: "reasoning", text: "Thinking.", providerMetadata: { "custom-anthropic": { signature: "custom_sig" } } },
{ type: "reasoning", text: "", providerMetadata: { "custom-anthropic": { redactedData: "custom_redacted" } } },
{ type: "tool-call", id: "custom_tool", providerExecuted: true },
{
type: "tool-result",
providerExecuted: true,
providerMetadata: { "custom-anthropic": { blockType: "web_search_tool_result", result } },
},
])
expect(response.usage?.providerMetadata).toEqual({
"custom-anthropic": { input_tokens: 5, custom_start: true, output_tokens: 2, custom_terminal: true },
})
expect(response.events.at(-1)).toMatchObject({
providerMetadata: { "custom-anthropic": { stopSequence: "custom_stop" } },
})
const prepared = yield* compileRequest(
LLM.request({ model: compatible, messages: [response.message], cache: "none" }),
)
expect(prepared.body.messages).toEqual([
{
role: "assistant",
content: [
{ type: "thinking", thinking: "Thinking.", signature: "custom_sig" },
{ type: "redacted_thinking", data: "custom_redacted" },
{ type: "server_tool_use", id: "custom_tool", name: "web_search", input: { query: "example" } },
{ type: "web_search_tool_result", tool_use_id: "custom_tool", content: result },
],
},
])
}),
)
it.effect("parses text, reasoning, and usage stream fixtures", () =>
Effect.gen(function* () {
const body = sseEvents(
@@ -873,9 +966,8 @@ describe("Anthropic Messages route", () => {
Effect.flip,
)
expect(error.reason).toMatchObject({
_tag: "InvalidProviderOutput",
classification: "incomplete-stream",
expect(error).toMatchObject({
reason: { _tag: "InvalidProviderOutput", classification: "incomplete-stream" },
message: "The provider response ended unexpectedly.",
})
}),
@@ -964,8 +1056,8 @@ describe("Anthropic Messages route", () => {
Effect.flip,
)
expect(error.reason).toMatchObject({
_tag: "InvalidProviderOutput",
expect(error).toMatchObject({
reason: { _tag: "InvalidProviderOutput" },
message: "Invalid anthropic/anthropic-messages stream event",
})
}),
@@ -986,8 +1078,8 @@ describe("Anthropic Messages route", () => {
Effect.flip,
)
expect(error.reason).toMatchObject({
_tag: "InvalidProviderOutput",
expect(error).toMatchObject({
reason: { _tag: "InvalidProviderOutput" },
message: "Invalid anthropic/anthropic-messages stream event",
})
}),
@@ -1013,8 +1105,8 @@ describe("Anthropic Messages route", () => {
Effect.flip,
)
expect(error.reason).toMatchObject({
_tag: "InvalidProviderOutput",
expect(error).toMatchObject({
reason: { _tag: "InvalidProviderOutput" },
message: "Invalid anthropic/anthropic-messages stream event",
})
}),
@@ -1029,8 +1121,8 @@ describe("Anthropic Messages route", () => {
Effect.flip,
)
expect(error.reason).toMatchObject({
_tag: "InvalidProviderOutput",
expect(error).toMatchObject({
reason: { _tag: "InvalidProviderOutput" },
message: "Invalid anthropic/anthropic-messages stream event",
})
}),
@@ -1552,7 +1644,7 @@ describe("Anthropic Messages route", () => {
Effect.flip,
)
expect(error.reason).toMatchObject({ _tag: "ProviderInternal", message: "overloaded_error: Overloaded" })
expect(error).toMatchObject({ reason: { _tag: "ProviderInternal" }, message: "overloaded_error: Overloaded" })
}),
)
@@ -1570,10 +1662,9 @@ describe("Anthropic Messages route", () => {
Effect.flip,
)
expect(error.reason).toMatchObject({
_tag: "InvalidRequest",
expect(error).toMatchObject({
reason: { _tag: "InvalidRequest", classification: "context-overflow" },
message: "invalid_request_error: prompt is too long: 210000 tokens",
classification: "context-overflow",
})
}),
)
@@ -1585,7 +1676,7 @@ describe("Anthropic Messages route", () => {
Effect.flip,
)
expect(error.reason).toMatchObject({ _tag: "ProviderInternal", message: "overloaded_error" })
expect(error).toMatchObject({ reason: { _tag: "ProviderInternal" }, message: "overloaded_error" })
}),
)
@@ -1596,7 +1687,7 @@ describe("Anthropic Messages route", () => {
Effect.flip,
)
expect(error.reason).toMatchObject({ _tag: "UnknownProvider", message: "Anthropic Messages stream error" })
expect(error).toMatchObject({ reason: { _tag: "UnknownProvider" }, message: "Anthropic Messages stream error" })
}),
)
@@ -1613,7 +1704,7 @@ describe("Anthropic Messages route", () => {
)
expect(error).toBeInstanceOf(AIError)
expect(error.reason).toMatchObject({ _tag: "InvalidRequest", message: "Bad request" })
expect(error).toMatchObject({ reason: { _tag: "InvalidRequest" }, message: "Bad request" })
}),
)
@@ -569,6 +569,57 @@ describe("Bedrock Converse route", () => {
}),
)
it.effect("round-trips reassigned provider reasoning and usage metadata in its own namespace", () =>
Effect.gen(function* () {
const compatible = model.route.with({ provider: "custom-bedrock" }).model({ id: model.id })
const redactedData = "cmVkYWN0ZWQtdGhpbmtpbmc="
const response = yield* LLMClient.generate(LLMRequest.update(baseRequest, { model: compatible })).pipe(
Effect.provide(
fixedBytes(
eventStreamBody(
["messageStart", { role: "assistant" }],
["contentBlockDelta", { contentBlockIndex: 0, delta: { reasoningContent: { text: "Let me think." } } }],
["contentBlockDelta", { contentBlockIndex: 0, delta: { reasoningContent: { signature: "custom_sig" } } }],
["contentBlockStop", { contentBlockIndex: 0 }],
[
"contentBlockDelta",
{ contentBlockIndex: 1, delta: { reasoningContent: { redactedContent: redactedData } } },
],
["contentBlockStop", { contentBlockIndex: 1 }],
["messageStop", { stopReason: "end_turn" }],
["metadata", { usage: { inputTokens: 5, outputTokens: 2, totalTokens: 7 } }],
),
),
),
)
expect(response.message.content).toEqual([
{
type: "reasoning",
text: "Let me think.",
providerMetadata: { "custom-bedrock": { signature: "custom_sig" } },
},
{ type: "reasoning", text: "", providerMetadata: { "custom-bedrock": { redactedData } } },
])
expect(response.usage?.providerMetadata).toEqual({
"custom-bedrock": { inputTokens: 5, outputTokens: 2, totalTokens: 7 },
})
const prepared = yield* compileRequest(
LLM.request({ model: compatible, messages: [response.message], cache: "none" }),
)
expect(prepared.body.messages).toEqual([
{
role: "assistant",
content: [
{ reasoningContent: { reasoningText: { text: "Let me think.", signature: "custom_sig" } } },
{ reasoningContent: { redactedContent: redactedData } },
],
},
])
}),
)
it.effect("preserves reasoning signatures when contentBlockStop is missing", () =>
Effect.gen(function* () {
const response = yield* LLMClient.generate(baseRequest).pipe(
@@ -738,19 +789,32 @@ describe("Bedrock Converse route", () => {
])
const error = yield* LLMClient.generate(baseRequest).pipe(Effect.provide(fixedBytes(body)), Effect.flip)
expect(error.reason).toMatchObject({ _tag: "UnknownProvider", message: "A future provider failure" })
expect(error).toMatchObject({ reason: { _tag: "UnknownProvider" }, message: "A future provider failure" })
}),
)
it.effect("classifies throttlingException as a rate limit", () =>
Effect.gen(function* () {
const payload = { message: "Slow down", details: { opaque: [1, 2] }, trace: "outer", p: "padding" }
const body = concat([
eventFrame("messageStart", { role: "assistant" }),
exceptionFrame("throttlingException", { message: "Slow down" }),
exceptionFrame("throttlingException", payload),
])
const error = yield* LLMClient.generate(baseRequest).pipe(Effect.provide(fixedBytes(body)), Effect.flip)
expect(error.reason).toMatchObject({ _tag: "RateLimit", message: "Slow down" })
expect(error).toMatchObject({ reason: { _tag: "RateLimit" }, message: "Slow down" })
expect(JSON.parse(error.reason.body ?? "")).toEqual({
headers: {
":message-type": { type: "string", value: "exception" },
":exception-type": { type: "string", value: "throttlingException" },
":content-type": { type: "string", value: "application/json" },
},
body: JSON.stringify(payload),
})
expect(error.reason.http).toMatchObject({
status: 200,
headers: { "content-type": "application/vnd.amazon.eventstream" },
})
}),
)
@@ -763,10 +827,9 @@ describe("Bedrock Converse route", () => {
Effect.flip,
)
expect(error.reason).toMatchObject({
_tag: "InvalidRequest",
expect(error).toMatchObject({
reason: { _tag: "InvalidRequest", classification: "context-overflow" },
message: "Input is too long for requested model",
classification: "context-overflow",
})
}),
)
@@ -785,7 +848,7 @@ describe("Bedrock Converse route", () => {
Effect.flip,
)
expect(error.reason).toMatchObject({ _tag: "ProviderInternal", message: "Upstream model failed" })
expect(error).toMatchObject({ reason: { _tag: "ProviderInternal" }, message: "Upstream model failed" })
}),
)
@@ -796,10 +859,32 @@ describe("Bedrock Converse route", () => {
Effect.flip,
)
expect(error.reason).toMatchObject({
_tag: "InvalidProviderOutput",
expect(error).toMatchObject({
reason: { _tag: "InvalidProviderOutput" },
message: "BadStream: Stream failed",
})
expect(JSON.parse(error.reason.body ?? "")).toMatchObject({
headers: { ":error-code": { value: "BadStream" } },
body: "",
})
}),
)
it.effect("retains malformed AWS payloads with headers and decode cause", () =>
Effect.gen(function* () {
const headers = {
":message-type": { type: "string" as const, value: "event" },
":event-type": { type: "string" as const, value: "messageStart" },
}
const body = '{"malformed":'
const error = yield* LLMClient.generate(baseRequest).pipe(
Effect.provide(fixedBytes(codec.encode({ headers, body: utf8Encoder.encode(body) }))),
Effect.flip,
)
expect(error.reason._tag).toBe("InvalidProviderOutput")
expect(JSON.parse(error.reason.body ?? "")).toEqual({ headers, body })
expect(error.reason.cause).toBeInstanceOf(Error)
expect(error.reason.http?.status).toBe(200)
}),
)
@@ -3,6 +3,7 @@ import { Effect } from "effect"
import { HttpClientRequest } from "effect/unstable/http"
import { LLM, Message } from "../../src/index.js"
import { AmazonBedrockMantle } from "../../src/providers.js"
import { model } from "../../src/providers/amazon-bedrock/mantle.js"
import { OpenAIResponses } from "../../src/protocols/openai-responses.js"
import { compileRequest, LLMClient } from "../../src/route/client.js"
import { it } from "../lib/effect.js"
@@ -17,13 +18,16 @@ const credentials = {
}
describe("Amazon Bedrock Mantle provider", () => {
it.effect("uses Chat by default and exposes Responses", () =>
it.effect("uses Responses by default and exposes Chat explicitly", () =>
Effect.gen(function* () {
const provider = AmazonBedrockMantle.configure({ credentials })
expect(provider.responses("openai.gpt-oss-120b").route.transport).toBe(OpenAIResponses.httpTransport)
const chat = yield* compileRequest(LLM.request({ model: provider.model("openai.gpt-oss-120b"), prompt: "Hi" }))
expect(provider.model).toBe(provider.responses)
expect(AmazonBedrockMantle.model).toBe(AmazonBedrockMantle.responsesModel)
expect(model).toBe(AmazonBedrockMantle.responsesModel)
expect(provider.model("openai.gpt-oss-120b").route.transport).toBe(OpenAIResponses.httpTransport)
const chat = yield* compileRequest(LLM.request({ model: provider.chat("openai.gpt-oss-120b"), prompt: "Hi" }))
const responses = yield* compileRequest(
LLM.request({ model: provider.responses("openai.gpt-oss-120b"), prompt: "Hi" }),
LLM.request({ model: provider.model("openai.gpt-oss-120b"), prompt: "Hi" }),
)
expect(chat).toMatchObject({
@@ -36,6 +40,23 @@ describe("Amazon Bedrock Mantle provider", () => {
protocol: "openai-responses",
body: { model: "openai.gpt-oss-120b", store: false },
})
expect(provider.model("openai.gpt-oss-120b").route.providerMetadataKey).toBe("mantle")
expect(provider.chat("openai.gpt-oss-120b").route.providerMetadataKey).toBe("mantle")
}),
)
it.effect("preserves configured top-p generation defaults for Chat and Responses", () =>
Effect.gen(function* () {
const settings = { apiKey: "test-key", topP: 0.8 }
const chat = yield* compileRequest(
LLM.request({ model: AmazonBedrockMantle.chatModel("openai.gpt-oss-safeguard-20b", settings), prompt: "Hi" }),
)
const responses = yield* compileRequest(
LLM.request({ model: AmazonBedrockMantle.responsesModel("openai.gpt-oss-120b", settings), prompt: "Hi" }),
)
expect(chat.body.top_p).toBe(0.8)
expect(responses.body.top_p).toBe(0.8)
}),
)
@@ -107,6 +128,9 @@ describe("Amazon Bedrock Mantle provider", () => {
LLM.request({ model, messages: [response.message, Message.user("Continue.")] }),
)
expect(response.message.content.find((part) => part.type === "reasoning")?.providerMetadata).toEqual({
mantle: { itemId: "msg_95d4d0af4350432a", reasoningEncryptedContent: "mantle-state" },
})
expect(prepared.body.input).toEqual([
{
type: "reasoning",
+1 -1
View File
@@ -126,7 +126,7 @@ describe("Cloudflare", () => {
expect(response.reasoning).toBe("Thinking")
expect(response.events.filter(LLMEvent.is.reasoningDelta)).toHaveLength(2)
expect(response.message.content.find((part) => part.type === "reasoning")?.providerMetadata).toEqual({
openai: { reasoningField: "reasoning", reasoningDetails: merged },
"cloudflare-ai-gateway": { reasoningField: "reasoning", reasoningDetails: merged },
})
const replay = yield* compileRequest(LLM.request({ model, messages: [response.message] }))
@@ -0,0 +1,102 @@
import { describe, expect } from "bun:test"
import { Effect } from "effect"
import { LLM } from "../../src/index.js"
import { Anthropic, Google, OpenAI } from "../../src/providers.js"
import { LLMClient } from "../../src/route.js"
import { it } from "../lib/effect.js"
import { fixedResponse } from "../lib/http.js"
import { sseEvents } from "../lib/sse.js"
describe("provider error retention", () => {
const options = { apiKey: "test", baseURL: "https://provider.test" }
const cases = [
{
name: "Anthropic",
model: Anthropic.configure(options).model("claude"),
event: { type: "error", error: { type: "rate_limit_error", message: "Slow down", details: { opaque: [1, 2] } } },
},
{
name: "OpenAI Chat",
model: OpenAI.configure(options).chat("gpt"),
event: { error: { code: "rate_limit_exceeded", message: "Slow down", details: { opaque: [1, 2] } } },
},
{
name: "OpenAI Responses",
model: OpenAI.configure(options).responses("gpt"),
event: {
type: "response.failed",
response: {
id: "resp_error",
error: { code: "rate_limit_exceeded", message: "Slow down", details: { opaque: [1, 2] } },
opaque: { upstream: true },
},
},
},
{
name: "Gemini",
model: Google.configure(options).model("gemini"),
event: { error: { code: 429, status: "RESOURCE_EXHAUSTED", message: "Slow down", details: { opaque: [1, 2] } } },
},
]
for (const entry of cases) {
it.effect(`retains the complete ${entry.name} event and HTTP context`, () =>
Effect.gen(function* () {
const body = JSON.stringify({ ...entry.event, trace: { opaque: "outer" } })
const error = yield* LLMClient.generate(LLM.request({ model: entry.model, prompt: "hello" })).pipe(
Effect.provide(
fixedResponse(sseEvents(body), {
headers: { "content-type": "text/event-stream", "x-provider-trace": "trace-1" },
}),
),
Effect.flip,
)
expect(error.message).toContain("Slow down")
expect(error.reason._tag).toBe(entry.name === "Gemini" ? "ProviderInternal" : "RateLimit")
expect(error.reason.body).toBe(body)
expect(error.reason.http).toMatchObject({ status: 200, headers: { "x-provider-trace": "trace-1" } })
expect(error.reason.http?.url).toStartWith("https://provider.test/")
expect(error.reason.cause).toBeUndefined()
expect(error.cause).toBe(error.reason)
}),
)
}
it.effect("retains malformed provider frames and the original decode cause", () =>
Effect.gen(function* () {
const body = '{"type":"error","error":{"message":42,"opaque":{"nested":true}},"trace":"outer"}'
const error = yield* LLMClient.generate(
LLM.request({ model: Anthropic.configure(options).model("claude"), prompt: "hello" }),
).pipe(Effect.provide(fixedResponse(sseEvents(body))), Effect.flip)
expect(error.reason._tag).toBe("InvalidProviderOutput")
expect(error.reason.body).toBe(body)
expect(error.reason.cause).toBeInstanceOf(Error)
expect(error.reason.http?.status).toBe(200)
}),
)
it.effect("retains the HTTP response context when a channel falls back", () =>
Effect.gen(function* () {
const body = '{"type":"error","error":{"code":"rate_limit_exceeded","message":"Slow down","extra":42}}'
const error = yield* LLMClient.generate(
LLM.request({ model: OpenAI.configure(options).responses("gpt"), prompt: "hello" }),
{
webSocket: {
execute: (exchange) => Effect.succeed({ frames: exchange.fallback(), complete: Effect.void }),
},
},
).pipe(
Effect.provide(fixedResponse(sseEvents(body), { headers: { "x-provider-trace": "fallback-1" } })),
Effect.flip,
)
expect(error.reason._tag).toBe("RateLimit")
expect(error.reason.body).toBe(body)
expect(error.reason.http).toMatchObject({
url: "https://provider.test/responses",
status: 200,
headers: { "x-provider-trace": "fallback-1" },
})
}),
)
})
@@ -6,7 +6,7 @@ import { GoogleVertex, GoogleVertexChat, GoogleVertexMessages, GoogleVertexRespo
import { LLMClient } from "../../src/route.js"
import { compileRequest } from "../../src/route/client.js"
import { it } from "../lib/effect.js"
import { dynamicResponse } from "../lib/http.js"
import { dynamicResponse, fixedResponse } from "../lib/http.js"
import { deltaChunk, finishChunk } from "../lib/openai-chunks.js"
import { sseEvents } from "../lib/sse.js"
@@ -89,7 +89,7 @@ describe("Google Vertex providers", () => {
id: "call_1",
name: "lookup",
input: { query: "weather" },
providerMetadata: { google: { functionCallId: "provider_call_1" } },
providerMetadata: { vertex: { functionCallId: "provider_call_1" } },
}),
]),
Message.tool({
@@ -97,7 +97,7 @@ describe("Google Vertex providers", () => {
name: "lookup",
result: "sunny",
resultType: "text",
providerMetadata: { google: { functionCallId: "provider_call_1" } },
providerMetadata: { vertex: { functionCallId: "provider_call_1" } },
}),
],
}),
@@ -122,6 +122,91 @@ describe("Google Vertex providers", () => {
}),
)
it.effect("round-trips Vertex Gemini metadata through signed content, tool calls, and usage", () =>
Effect.gen(function* () {
const model = GoogleVertex.configure({
accessToken: "vertex-token",
project: "vertex-project",
}).model("gemini-3.5-flash")
const response = yield* LLMClient.generate(LLM.request({ model, prompt: "Check the weather." })).pipe(
Effect.provide(
fixedResponse(
sseEvents({
candidates: [
{
content: {
role: "model",
parts: [
{ text: "Thinking.", thought: true, thoughtSignature: "reasoning_sig" },
{ text: "Checking.", thoughtSignature: "text_sig" },
{
functionCall: { id: "provider_call_1", name: "lookup", args: { query: "weather" } },
thoughtSignature: "tool_sig",
},
],
},
finishReason: "STOP",
},
],
promptFeedback: { blockReasonMessage: "Reviewed" },
usageMetadata: { promptTokenCount: 5, candidatesTokenCount: 2, thoughtsTokenCount: 1 },
}),
),
),
)
const reasoning = response.events.find((event) => event.type === "reasoning-end")
const text = response.events.find((event) => event.type === "text-delta")
const toolCall = response.toolCalls[0]
expect(reasoning?.providerMetadata).toEqual({ vertex: { thoughtSignature: "reasoning_sig" } })
expect(text?.providerMetadata).toEqual({ vertex: { thoughtSignature: "text_sig" } })
expect(toolCall).toMatchObject({
id: "provider_call_1",
providerMetadata: { vertex: { thoughtSignature: "tool_sig" } },
})
expect(response.usage?.providerMetadata).toEqual({
vertex: { promptTokenCount: 5, candidatesTokenCount: 2, thoughtsTokenCount: 1 },
})
expect(response.events.at(-1)?.providerMetadata).toEqual({
vertex: { promptFeedback: { blockReasonMessage: "Reviewed" } },
})
const prepared = yield* compileRequest(
LLM.request({
model,
messages: [
Message.assistant([
{ type: "reasoning", text: "Thinking.", providerMetadata: reasoning?.providerMetadata },
{ type: "text", text: "Checking.", providerMetadata: text?.providerMetadata },
ToolCallPart.make({
id: toolCall.id,
name: toolCall.name,
input: toolCall.input,
providerMetadata: toolCall.providerMetadata,
}),
]),
Message.tool({ id: toolCall.id, name: toolCall.name, result: "sunny", resultType: "text" }),
],
}),
)
expect(prepared.body.contents).toEqual([
{
role: "model",
parts: [
{ text: "Thinking.", thought: true, thoughtSignature: "reasoning_sig" },
{ text: "Checking.", thoughtSignature: "text_sig" },
{ functionCall: { name: "lookup", args: { query: "weather" } }, thoughtSignature: "tool_sig" },
],
},
{
role: "user",
parts: [{ functionResponse: { name: "lookup", response: { name: "lookup", content: "sunny" } } }],
},
])
}),
)
it.effect("projects Anthropic Messages onto the Vertex raw-predict API", () =>
Effect.gen(function* () {
const model = GoogleVertexMessages.configure({
@@ -2,13 +2,80 @@ import { describe, expect } from "bun:test"
import { ConfigProvider, Effect } from "effect"
import { HttpClientRequest } from "effect/unstable/http"
import { LLM, Message, ToolDefinition } from "../../src/index.js"
import { Cerebras, DeepInfra, Groq, TogetherAI } from "../../src/providers/index.js"
import {
AmazonBedrock,
AmazonBedrockMantle,
Anthropic,
AnthropicCompatible,
Azure,
Cerebras,
CloudflareAIGateway,
CloudflareWorkersAI,
DeepInfra,
Google,
GoogleVertex,
GoogleVertexChat,
GoogleVertexMessages,
GoogleVertexResponses,
Groq,
OpenAI,
OpenAICompatible,
OpenAICompatibleResponses,
OpenRouter,
TogetherAI,
XAI,
} from "../../src/providers/index.js"
import { compileRequest } from "../../src/route/client.js"
import { it } from "../lib/effect.js"
import { dynamicResponse } from "../lib/http.js"
import { sseEvents } from "../lib/sse.js"
describe("native OpenAI-compatible providers", () => {
it.effect("assigns provider-owned metadata namespaces across native routes", () =>
Effect.gen(function* () {
const vertex = { project: "project", accessToken: "token" }
const providers = [
[OpenAI.configure({ apiKey: "test" }).chat("model"), "openai"],
[OpenAI.configure({ apiKey: "test" }).responses("model"), "openai"],
[Azure.configure({ resourceName: "resource", apiKey: "test" }).chat("model"), "azure"],
[Azure.configure({ resourceName: "resource", apiKey: "test" }).responses("model"), "azure"],
[AmazonBedrock.configure({ apiKey: "test" }).model("model"), "bedrock"],
[AmazonBedrockMantle.configure({ apiKey: "test" }).chat("model"), "mantle"],
[AmazonBedrockMantle.configure({ apiKey: "test" }).responses("model"), "mantle"],
[Google.configure({ apiKey: "test" }).model("model"), "google"],
[GoogleVertex.configure(vertex).model("model"), "vertex"],
[GoogleVertexChat.configure(vertex).model("model"), "vertex"],
[GoogleVertexResponses.configure(vertex).model("model"), "vertex"],
[GoogleVertexMessages.configure(vertex).model("model"), "anthropic"],
[Anthropic.configure({ apiKey: "test" }).model("model"), "anthropic"],
[
AnthropicCompatible.configure({ baseURL: "https://example.test/v1", provider: "minimax" }).model("model"),
"minimax",
],
[
OpenAICompatible.configure({ baseURL: "https://example.test/v1", provider: "custom" }).model("model"),
"custom",
],
[
OpenAICompatibleResponses.configure({ baseURL: "https://example.test/v1", provider: "custom" }).model(
"model",
),
"custom",
],
[Cerebras.configure({ apiKey: "test" }).model("model"), "cerebras"],
[DeepInfra.configure({ apiKey: "test" }).model("model"), "deepinfra"],
[TogetherAI.configure({ apiKey: "test" }).model("model"), "togetherai"],
[CloudflareAIGateway.configure({ accountId: "account" }).model("model"), "cloudflare-ai-gateway"],
[CloudflareWorkersAI.configure({ accountId: "account" }).model("model"), "cloudflare-workers-ai"],
[OpenRouter.configure({ apiKey: "test" }).model("model"), "openrouter"],
[XAI.configure({ apiKey: "test" }).chat("model"), "xai"],
[XAI.configure({ apiKey: "test" }).responses("model"), "xai"],
] as const
for (const [model, key] of providers) expect(model.route.providerMetadataKey).toBe(key)
}),
)
it.effect("preserves native Together AI and Cerebras provider and route identities", () =>
Effect.gen(function* () {
const together = TogetherAI.configure({ apiKey: "fixture" }).model("meta-llama/Llama-3.3-70B")
@@ -68,11 +68,13 @@ for (const item of cases) {
expect(response.text.replaceAll(",", "").trim()).toBe("37887")
expect(response.reasoning.length).toBeGreaterThan(0)
expect(response.events.some(LLMEvent.is.reasoningDelta)).toBe(true)
const metadata = response.message.content.find((part) => part.type === "reasoning")?.providerMetadata
expect(metadata?.openai?.reasoningField).toBe(item.structured ? "reasoning" : "reasoning_content")
expect(Array.isArray(metadata?.openai?.reasoningDetails)).toBe(item.structured)
const metadata = response.message.content.find((part) => part.type === "reasoning")?.providerMetadata?.[
item.model.route.providerMetadataKey ?? String(item.model.provider)
]
expect(metadata?.reasoningField).toBe(item.structured ? "reasoning" : "reasoning_content")
expect(Array.isArray(metadata?.reasoningDetails)).toBe(item.structured)
if (!item.structured) return
const details = metadata?.openai?.reasoningDetails
const details = metadata?.reasoningDetails
if (!Array.isArray(details)) return
expect(
details.some(
@@ -126,7 +128,11 @@ for (const item of cases) {
).toMatch(/^Paris is sunny\.?$/)
const details = events
.filter(LLMEvent.is.reasoningEnd)
.map((event) => event.providerMetadata?.openai?.reasoningDetails)
.map(
(event) =>
event.providerMetadata?.[item.model.route.providerMetadataKey ?? String(item.model.provider)]
?.reasoningDetails,
)
.find(Array.isArray)
expect(Array.isArray(details)).toBe(item.structured)
if (!item.structured || !Array.isArray(details)) return
+68 -4
View File
@@ -903,6 +903,70 @@ describe("OpenAI Chat route", () => {
}),
)
it.effect("uses the configured provider metadata namespace for reasoning and usage", () =>
Effect.gen(function* () {
const selected = LanguageModel.update(model, {
route: { ...model.route, providerMetadataKey: "vendor" },
})
const details = [{ type: "reasoning.text", text: "thinking", signature: "signed" }]
const response = yield* LLMClient.generate(LLMRequest.update(request, { model: selected })).pipe(
Effect.provide(
fixedResponse(
sseEvents(
{ choices: [{ delta: { reasoning: "thinking", reasoning_details: details } }] },
deltaChunk({ content: "Hello" }),
deltaChunk({}, "stop"),
usageChunk({ prompt_tokens: 5, completion_tokens: 2, total_tokens: 7 }),
),
),
),
)
expect(response.message.content.find((part) => part.type === "reasoning")?.providerMetadata).toEqual({
vendor: { reasoningField: "reasoning", reasoningDetails: details },
})
expect(response.usage?.providerMetadata).toEqual({
vendor: { prompt_tokens: 5, completion_tokens: 2, total_tokens: 7 },
})
const replay = yield* compileRequest(LLM.request({ model: selected, messages: [response.message] }))
expect(replay.body.messages).toEqual([
{ role: "assistant", content: "Hello", reasoning: "thinking", reasoning_details: details },
])
}),
)
it.effect("falls back to the selected provider for the metadata namespace", () =>
Effect.gen(function* () {
const compatible = model.route.with({ provider: "deepseek" }).model({ id: "deepseek-chat" })
const selected = LanguageModel.update(compatible, {
route: { ...compatible.route, providerMetadataKey: undefined },
})
const response = yield* LLMClient.generate(LLMRequest.update(request, { model: selected })).pipe(
Effect.provide(
fixedResponse(
sseEvents(
deltaChunk({ reasoning_content: "thinking" }),
deltaChunk({ content: "Hello" }),
deltaChunk({}, "stop"),
usageChunk({ prompt_tokens: 5, completion_tokens: 2, total_tokens: 7 }),
),
),
),
)
expect(response.message.content.find((part) => part.type === "reasoning")?.providerMetadata).toEqual({
deepseek: { reasoningField: "reasoning_content" },
})
expect(response.usage?.providerMetadata).toEqual({
deepseek: { prompt_tokens: 5, completion_tokens: 2, total_tokens: 7 },
})
const replay = yield* compileRequest(LLM.request({ model: selected, messages: [response.message] }))
expect(replay.body.messages).toEqual([{ role: "assistant", content: "Hello", reasoning_content: "thinking" }])
}),
)
it.effect("parses and replays a configured custom reasoning field", () =>
Effect.gen(function* () {
const custom = LanguageModel.update(model, { compatibility: { reasoningField: "vendor_reasoning" } })
@@ -1393,7 +1457,7 @@ describe("OpenAI Chat route", () => {
expect(error.message).toContain("OpenAI Chat tool call delta is missing id or name")
expect(error.reason._tag).toBe("InvalidProviderOutput")
if (error.reason._tag !== "InvalidProviderOutput") return
expect(decodeJson(error.reason.raw ?? "")).toMatchObject({
expect(decodeJson(error.reason.body ?? "")).toMatchObject({
choices: [{ finish_reason: "tool_calls" }],
})
}),
@@ -1470,9 +1534,9 @@ describe("OpenAI Chat route", () => {
)
expect((yield* Ref.get(events)).some((event) => event.type === "text-delta")).toBeTrue()
expect(error.message).toBe("ECONNRESET: socket closed unexpectedly")
expect(error.reason).toMatchObject({
_tag: "Transport",
message: "ECONNRESET: socket closed unexpectedly",
transport: "http",
operation: "read",
code: "ECONNRESET",
@@ -1488,9 +1552,9 @@ describe("OpenAI Chat route", () => {
Effect.flip,
)
expect(error.message).toBe("ECONNRESET: socket closed before output")
expect(error.reason).toMatchObject({
_tag: "Transport",
message: "ECONNRESET: socket closed before output",
transport: "http",
operation: "read",
code: "ECONNRESET",
@@ -1511,7 +1575,7 @@ describe("OpenAI Chat route", () => {
)
expect(error).toBeInstanceOf(AIError)
expect(error.reason).toMatchObject({ _tag: "InvalidRequest", message: "Bad request" })
expect(error).toMatchObject({ reason: { _tag: "InvalidRequest" }, message: "Bad request" })
}),
)
@@ -437,7 +437,7 @@ describe("OpenAI-compatible Chat route", () => {
outputTokens: undefined,
totalTokens: undefined,
providerMetadata: {
openai: {
deepseek: {
prompt_tokens: null,
completion_tokens: null,
total_tokens: null,
@@ -489,9 +489,8 @@ describe("OpenAI-compatible Chat route", () => {
Effect.flip,
)
expect(error.reason).toMatchObject({
_tag: "InvalidProviderOutput",
classification: "incomplete-stream",
expect(error).toMatchObject({
reason: { _tag: "InvalidProviderOutput", classification: "incomplete-stream" },
message: "OpenAI Chat stream ended without finish_reason",
})
}),
@@ -527,11 +526,11 @@ describe("OpenAI-compatible Chat route", () => {
Effect.flip,
)
expect(error.reason).toMatchObject({
_tag: "ProviderInternal",
expect(error).toMatchObject({
reason: { _tag: "ProviderInternal" },
message: "Provider reported a network error (finish_reason: network_error)",
})
expect(decodeJson(error.body ?? "")).toMatchObject({
expect(decodeJson(error.reason.body ?? "")).toMatchObject({
id: "chatcmpl_fixture",
choices: [{ finish_reason: "network_error" }],
})
@@ -540,8 +539,8 @@ describe("OpenAI-compatible Chat route", () => {
Effect.provide(fixedResponse(sseEvents(deltaChunk({}, "error")))),
Effect.flip,
)
expect(generic.reason).toMatchObject({
_tag: "UnknownProvider",
expect(generic).toMatchObject({
reason: { _tag: "UnknownProvider" },
message: "Provider reported an error (finish_reason: error)",
})
}),
@@ -562,8 +561,8 @@ describe("OpenAI-compatible Chat route", () => {
Effect.flip,
)
expect(error.reason).toMatchObject({ _tag: "ProviderInternal", message: "Provider disconnected", status: 502 })
expect(decodeJson(error.body ?? "")).toMatchObject({
expect(error).toMatchObject({ reason: { _tag: "ProviderInternal" }, message: "Provider disconnected" })
expect(decodeJson(error.reason.body ?? "")).toMatchObject({
id: "chatcmpl_error",
error: { code: 502, message: "Provider disconnected", details: { upstream: "vendor" } },
trace_id: "trace_1",
@@ -603,7 +602,7 @@ describe("OpenAI-compatible Chat route", () => {
expect(error.message).toContain("OpenAI Chat received content after the finish reason")
expect(error.reason._tag).toBe("InvalidProviderOutput")
if (error.reason._tag !== "InvalidProviderOutput") return
expect(decodeJson(error.reason.raw ?? "")).toMatchObject({
expect(decodeJson(error.reason.body ?? "")).toMatchObject({
choices: [{ delta: { tool_calls: [{ id: "call_1" }] } }],
})
}),
@@ -195,19 +195,19 @@ describe("Open Responses-compatible route", () => {
model,
messages: [
Message.assistant([
{ type: "text", text: "Kept.", providerMetadata: { openresponses: { itemId: "history_1" } } },
{ type: "text", text: "Kept.", providerMetadata: { "openai-compatible": { itemId: "history_1" } } },
{
type: "text",
text: "Long.",
providerMetadata: { openresponses: { itemId: `history_${"a".repeat(64)}` } },
providerMetadata: { "openai-compatible": { itemId: `history_${"a".repeat(64)}` } },
},
{
type: "text",
text: "Opaque.",
providerMetadata: { openresponses: { itemId: "provider_value/with+symbols" } },
providerMetadata: { "openai-compatible": { itemId: "provider_value/with+symbols" } },
},
{ type: "text", text: "No suffix.", providerMetadata: { openresponses: { itemId: "msg_" } } },
{ type: "text", text: "No prefix.", providerMetadata: { openresponses: { itemId: "_item" } } },
{ type: "text", text: "No suffix.", providerMetadata: { "openai-compatible": { itemId: "msg_" } } },
{ type: "text", text: "No prefix.", providerMetadata: { "openai-compatible": { itemId: "_item" } } },
]),
],
}),
@@ -267,7 +267,7 @@ describe("Open Responses-compatible route", () => {
name: item.type,
result: { type: "json", value: item },
providerExecuted: true,
providerMetadata: { openresponses: { itemId: item.id } },
providerMetadata: { example: { itemId: item.id } },
}),
),
}),
@@ -302,7 +302,7 @@ describe("Open Responses-compatible route", () => {
)
expect(response.message.content).toEqual([
{ type: "text", text: "Indexed", providerMetadata: { openresponses: { itemId: "msg_1" } } },
{ type: "text", text: "Indexed", providerMetadata: { "openai-compatible": { itemId: "msg_1" } } },
])
}),
)
@@ -368,7 +368,7 @@ describe("Open Responses-compatible route", () => {
),
)
const metadata = { openresponses: { itemId: routing.id } }
const metadata = { "openai-compatible": { itemId: routing.id } }
if (fixture.item.type === "function_call") {
expect(response.toolCalls).toEqual([
expect.objectContaining({
@@ -386,7 +386,7 @@ describe("Open Responses-compatible route", () => {
type: "reasoning",
text: "Preserved",
providerMetadata: {
openresponses: { itemId: routing.id, reasoningEncryptedContent: "encrypted-state" },
"openai-compatible": { itemId: routing.id, reasoningEncryptedContent: "encrypted-state" },
},
},
])
@@ -438,22 +438,26 @@ describe("Open Responses-compatible route", () => {
{
type: "reasoning",
text: "First.",
providerMetadata: { openresponses: { itemId: routing.id } },
providerMetadata: { "openai-compatible": { itemId: routing.id } },
},
{
type: "reasoning",
text: "Second.",
providerMetadata: { openresponses: { itemId: routing.id, reasoningEncryptedContent: "final-state" } },
providerMetadata: {
"openai-compatible": { itemId: routing.id, reasoningEncryptedContent: "final-state" },
},
},
])
expect(response.events.filter(LLMEvent.is.reasoningEnd)).toEqual([
expect.objectContaining({
id: `${routing.id}:0`,
providerMetadata: { openresponses: { itemId: routing.id } },
providerMetadata: { "openai-compatible": { itemId: routing.id } },
}),
expect.objectContaining({
id: `${routing.id}:1`,
providerMetadata: { openresponses: { itemId: routing.id, reasoningEncryptedContent: "final-state" } },
providerMetadata: {
"openai-compatible": { itemId: routing.id, reasoningEncryptedContent: "final-state" },
},
}),
])
}),
@@ -483,7 +487,7 @@ describe("Open Responses-compatible route", () => {
id: "call_1",
name: "lookup",
input: { query: "complete" },
providerMetadata: { openresponses: { itemId: "" } },
providerMetadata: { "openai-compatible": { itemId: "" } },
}),
])
}),
@@ -510,7 +514,7 @@ describe("Open Responses-compatible route", () => {
)
expect(response.message.content).toEqual([
{ type: "text", text: "Before after", providerMetadata: { openresponses: { itemId: "msg_1" } } },
{ type: "text", text: "Before after", providerMetadata: { "openai-compatible": { itemId: "msg_1" } } },
])
expect(response.events.map((event) => event.type)).toEqual([
"step-start",
@@ -662,7 +666,7 @@ describe("Open Responses-compatible route", () => {
expect(response.events.find(LLMEvent.is.toolCall)).toMatchObject({
input: { query: "complete" },
providerMetadata: { openresponses: { itemId: "item_1" } },
providerMetadata: { example: { itemId: "item_1" } },
})
}),
)
@@ -694,7 +698,7 @@ describe("Open Responses-compatible route", () => {
)
expect(response.events.find((event) => event.type === "reasoning-end")).toMatchObject({
providerMetadata: { openresponses: { itemId: "rs_raw", reasoningEncryptedContent: "raw-state" } },
providerMetadata: { "openai-compatible": { itemId: "rs_raw", reasoningEncryptedContent: "raw-state" } },
})
}),
)
@@ -743,7 +747,7 @@ describe("Open Responses-compatible route", () => {
Message.assistant({
type: "text",
text: "Unclassified.",
providerMetadata: { openresponses: { phase: null } },
providerMetadata: { "openai-compatible": { phase: null } },
}),
],
}),
@@ -802,7 +806,7 @@ describe("Open Responses-compatible route", () => {
{
type: "text",
text: "I can't help with that.",
providerMetadata: { openresponses: { itemId: "msg_refusal" } },
providerMetadata: { example: { itemId: "msg_refusal" } },
},
])
@@ -891,7 +895,7 @@ describe("Open Responses-compatible route", () => {
expect(response.toolCalls).toEqual([])
expect(response.events.find(LLMEvent.is.finish)).toMatchObject({
providerMetadata: { openresponses: { responseId: "resp_1" } },
providerMetadata: { example: { responseId: "resp_1" } },
})
}),
)
@@ -4,6 +4,7 @@ import { Headers, HttpClientRequest } from "effect/unstable/http"
import {
LLM,
AIError,
HttpContext,
HttpOptions,
LLMEvent,
LLMRequest,
@@ -12,7 +13,7 @@ import {
ToolCallPart,
ToolDefinition,
ToolResultPart,
TransportReason,
TransportError,
Usage,
} from "../../src/index.js"
import {
@@ -52,10 +53,11 @@ const baseChannelDriver = (message: string): WebSocketChannelDriver => ({
return Effect.succeed({
type: "provider-failure",
error: new AIError({
module: "test",
method: "stream",
reason: new TransportReason({
reason: new TransportError({
message: "provider rejected request",
body: frame,
cause: new Error("provider cause"),
http: new HttpContext({ url: "https://provider.test", status: 200, headers: { "x-trace": "trace-1" } }),
transport: "websocket",
operation: "read",
phase: "receive",
@@ -744,6 +746,18 @@ describe("OpenAI Responses route", () => {
reason: { _tag: "Transport", delivery: "rejected", recovery: "rotate-and-retry-full" },
},
})
for (const observation of [missing, limit]) {
expect(observation.type).toBe("rejected")
if (observation.type !== "rejected") continue
expect(observation.error.message).toBe("provider rejected request")
expect(observation.error.reason.cause).toBeInstanceOf(Error)
expect(observation.error.reason.cause).toMatchObject({ message: "provider cause" })
expect(observation.error.reason.http).toMatchObject({ status: 200, headers: { "x-trace": "trace-1" } })
expect(ProviderShared.decodeJson(observation.error.reason.body ?? "")).toMatchObject({
type: "error",
error: { code: expect.any(String) },
})
}
}),
)
@@ -1110,9 +1124,7 @@ describe("OpenAI Responses route", () => {
it.effect("marks post-send WebSocket failures with delivery state", () =>
Effect.gen(function* () {
const failure = new AIError({
module: "test",
method: "receive",
reason: new TransportReason({
reason: new TransportError({
message: "socket closed",
transport: "websocket",
operation: "read",
@@ -4435,7 +4447,7 @@ describe("OpenAI Responses route", () => {
)
expect(error).toBeInstanceOf(AIError)
expect(error.reason).toMatchObject({ _tag: "RateLimit", message: "rate_limit_exceeded: Slow down" })
expect(error).toMatchObject({ reason: { _tag: "RateLimit" }, message: "rate_limit_exceeded: Slow down" })
}),
)
@@ -4446,7 +4458,7 @@ describe("OpenAI Responses route", () => {
Effect.flip,
)
expect(error.reason).toMatchObject({ _tag: "ProviderInternal", message: "internal_error" })
expect(error).toMatchObject({ reason: { _tag: "ProviderInternal" }, message: "internal_error" })
}),
)
@@ -4457,7 +4469,7 @@ describe("OpenAI Responses route", () => {
Effect.flip,
)
expect(error.reason).toMatchObject({ _tag: "ProviderInternal", message: "internal_error" })
expect(error).toMatchObject({ reason: { _tag: "ProviderInternal" }, message: "internal_error" })
}),
)
@@ -4482,8 +4494,8 @@ describe("OpenAI Responses route", () => {
Effect.flip,
)
expect(error.reason).toMatchObject({
_tag: "ProviderInternal",
expect(error).toMatchObject({
reason: { _tag: "ProviderInternal" },
message: "server_error: Upstream model unavailable",
})
}),
@@ -4503,7 +4515,7 @@ describe("OpenAI Responses route", () => {
Effect.flip,
)
expect(error.reason).toMatchObject({ _tag: "InvalidRequest", message: "invalid_prompt" })
expect(error).toMatchObject({ reason: { _tag: "InvalidRequest" }, message: "invalid_prompt" })
}),
)
@@ -4526,10 +4538,9 @@ describe("OpenAI Responses route", () => {
Effect.flip,
)
expect(error.reason).toMatchObject({
_tag: "InvalidRequest",
expect(error).toMatchObject({
reason: { _tag: "InvalidRequest", classification: "context-overflow" },
message: "context_length_exceeded: prompt too long",
classification: "context-overflow",
})
}),
)
@@ -4554,10 +4565,9 @@ describe("OpenAI Responses route", () => {
Effect.flip,
)
expect(error.reason).toMatchObject({
_tag: "InvalidRequest",
expect(error).toMatchObject({
reason: { _tag: "InvalidRequest", classification: "context-overflow" },
message: "context_length_exceeded: prompt too long",
classification: "context-overflow",
})
}),
)
@@ -4579,7 +4589,7 @@ describe("OpenAI Responses route", () => {
Effect.flip,
)
expect(error.reason).toMatchObject({ _tag: "UnknownProvider", message: "Something went wrong" })
expect(error).toMatchObject({ reason: { _tag: "UnknownProvider" }, message: "Something went wrong" })
}),
)
@@ -4591,8 +4601,8 @@ describe("OpenAI Responses route", () => {
)
expect(error.reason).toMatchObject({ _tag: "UnknownProvider" })
expect(error.reason.message).toContain('"error":null')
expect(error.body).toBe(error.reason.message)
expect(error.message).toContain('"error":null')
expect(error.reason.body).toBe(error.message)
}),
)
@@ -4604,8 +4614,8 @@ describe("OpenAI Responses route", () => {
)
expect(error.reason).toMatchObject({ _tag: "ProviderInternal" })
expect(error.reason.message).toContain('"type":"error"')
expect(error.body).toBe(error.reason.message)
expect(error.message).toContain('"type":"error"')
expect(error.reason.body).toBe(error.message)
}),
)
@@ -4617,8 +4627,8 @@ describe("OpenAI Responses route", () => {
)
expect(error.reason).toMatchObject({ _tag: "UnknownProvider" })
expect(error.reason.message).toContain('"resp_failed_3"')
expect(error.body).toBe(error.reason.message)
expect(error.message).toContain('"resp_failed_3"')
expect(error.reason.body).toBe(error.message)
}),
)
@@ -4635,7 +4645,7 @@ describe("OpenAI Responses route", () => {
)
expect(error).toBeInstanceOf(AIError)
expect(error.reason).toMatchObject({ _tag: "InvalidRequest", message: "Bad request" })
expect(error).toMatchObject({ reason: { _tag: "InvalidRequest" }, message: "Bad request" })
}),
)
})
+3 -3
View File
@@ -295,7 +295,7 @@ describe("OpenRouter", () => {
{
type: "reasoning",
text: "Thinking",
providerMetadata: { openai: { reasoningField: "reasoning", reasoningDetails: details } },
providerMetadata: { openrouter: { reasoningField: "reasoning", reasoningDetails: details } },
},
]),
],
@@ -328,7 +328,7 @@ describe("OpenRouter", () => {
Message.assistant({
type: "reasoning",
text: "Thinking",
providerMetadata: { openai: { reasoningField: "reasoning", reasoningDetails: details } },
providerMetadata: { openrouter: { reasoningField: "reasoning", reasoningDetails: details } },
}),
],
}),
@@ -354,7 +354,7 @@ describe("OpenRouter", () => {
Message.assistant({
type: "reasoning",
text: "AB",
providerMetadata: { openai: { reasoningField: "reasoning", reasoningDetails: details } },
providerMetadata: { openrouter: { reasoningField: "reasoning", reasoningDetails: details } },
}),
],
}),
+188 -1
View File
@@ -1,6 +1,12 @@
import { describe, expect, test } from "bun:test"
import { Effect, Schema, Stream } from "effect"
import * as OpenAIChat from "../src/protocols/openai-chat.js"
import { Auth } from "../src/route.js"
import { AIError, HttpContext, InvalidProviderOutputError, LLM } from "../src/index.js"
import { Anthropic } from "../src/providers.js"
import { Auth, Framing, HttpTransport, LLMClient, Route } from "../src/route.js"
import { fixedResponse, truncatedStream } from "./lib/http.js"
import { sseEvents } from "./lib/sse.js"
import { it } from "./lib/effect.js"
describe("Route.with", () => {
test("merges endpoint query and header defaults while replacing auth and id", () => {
@@ -40,4 +46,185 @@ describe("Route.with", () => {
"x-patch": "patch",
})
})
test("assigns metadata ownership to a replacement provider and preserves explicit overrides", () => {
const route = OpenAIChat.route.with({ provider: "azure" })
const overridden = route.with({ providerMetadataKey: "custom-azure" }).with({ headers: { "x-test": "value" } })
expect(route.providerMetadataKey).toBe("azure")
expect(overridden.providerMetadataKey).toBe("custom-azure")
expect(overridden.defaults).not.toHaveProperty("providerMetadataKey")
})
})
describe("Route diagnostics", () => {
const route = OpenAIChat.route.with({ endpoint: { baseURL: "https://provider.test/v1" } })
const request = LLM.request({ model: route.model({ id: "test" }), prompt: "Hello" })
const headers = { "content-type": "text/event-stream", "x-request-id": "req_stream" }
it.effect("retains an entire invalid event and its validation cause", () =>
Effect.gen(function* () {
const frame = '{ "choices": "invalid", "diagnostic": { "detail": "original" } }'
const error = yield* LLMClient.generate(request).pipe(
Effect.provide(fixedResponse(sseEvents(frame), { headers })),
Effect.flip,
)
expect(error.reason._tag).toBe("InvalidProviderOutput")
expect(error.reason.body).toBe(frame)
expect(error.reason.cause).toMatchObject({ _tag: "SchemaError" })
expect(error.reason.http).toEqual(
new HttpContext({ url: "https://provider.test/v1/chat/completions", status: 200, headers }),
)
}),
)
it.effect("retains original provider error fields discarded by the event schema", () =>
Effect.gen(function* () {
const frame =
'{ "error": { "message": "Rate limit exceeded", "code": "rate_limit_exceeded", "debug": { "trace": "original" } }, "request_id": "req_original" }'
const error = yield* LLMClient.generate(request).pipe(
Effect.provide(fixedResponse(sseEvents(frame), { headers })),
Effect.flip,
)
expect(error.reason._tag).toBe("RateLimit")
expect(error.message).toBe("Rate limit exceeded")
expect(error.reason.body).toBe(frame)
expect(error.reason.http?.headers["x-request-id"]).toBe("req_stream")
}),
)
it.effect("preserves semantic parser bodies while replacing serialized event fallbacks", () =>
Effect.forEach([undefined, '{"query": BROKEN}', ""], (body) =>
Effect.gen(function* () {
const cause = new Error("parser failure")
const http = new HttpContext({
url: "https://upstream.test/v1",
status: 202,
headers: { "x-trace": "precise" },
})
const frame = '{ "type": "failure", "debug": "not in schema" }'
const failing = Route.make({
id: "diagnostics",
provider: "test",
endpoint: route.endpoint,
framing: Framing.sse,
protocol: {
...OpenAIChat.protocol,
stream: {
event: Schema.fromJsonString(Schema.Struct({ type: Schema.String })),
initial: () => undefined,
step: (_state, event) =>
Effect.fail(
new AIError({
reason: new InvalidProviderOutputError({
message: "Parser failed",
body: body ?? JSON.stringify(event),
http,
cause,
}),
}),
),
},
},
})
const error = yield* LLMClient.generate(
LLM.request({ model: failing.model({ id: "test" }), prompt: "Hello" }),
).pipe(Effect.provide(fixedResponse(sseEvents(frame), { headers })), Effect.flip)
expect(error.message).toBe("Parser failed")
expect(error.reason.body).toBe(body ?? frame)
expect(error.reason.cause).toBe(cause)
expect(error.reason.http).toBe(http)
}),
),
)
it.effect("retains malformed assembled Anthropic hosted-tool arguments", () =>
Effect.gen(function* () {
const body = '{"query": BROKEN}'
const error = yield* LLMClient.generate(
LLM.request({
model: Anthropic.configure({ apiKey: "test", baseURL: "https://provider.test" }).model("claude"),
prompt: "Hello",
}),
).pipe(
Effect.provide(
fixedResponse(
sseEvents(
{ type: "message_start", message: { usage: { input_tokens: 5 } } },
{
type: "content_block_start",
index: 0,
content_block: { type: "server_tool_use", id: "srv1", name: "web_search" },
},
{
type: "content_block_delta",
index: 0,
delta: { type: "input_json_delta", partial_json: '{"query": ' },
},
{ type: "content_block_delta", index: 0, delta: { type: "input_json_delta", partial_json: "BROKEN}" } },
{ type: "content_block_stop", index: 0 },
),
{ headers },
),
),
Effect.flip,
)
expect(error.reason._tag).toBe("InvalidProviderOutput")
expect(error.message).toContain("Invalid JSON input for anthropic-messages tool call web_search")
expect(error.reason.body).toBe(body)
expect(error.reason.cause).toBeInstanceOf(Error)
expect(error.reason.http).toMatchObject({ status: 200, headers })
}),
)
it.effect("adds successful HTTP metadata without replacing framing diagnostics", () =>
Effect.gen(function* () {
const cause = new Error("frame checksum mismatch")
const failure = new AIError({
reason: new InvalidProviderOutputError({
message: "Invalid frame",
body: "original frame representation",
cause,
}),
})
const failing = route.with({
transport: HttpTransport.httpJson({ framing: { id: "failure", frame: () => Stream.fail(failure) } }),
})
const error = yield* LLMClient.generate(
LLM.request({ model: failing.model({ id: "test" }), prompt: "Hello" }),
).pipe(Effect.provide(fixedResponse("wire bytes", { headers })), Effect.flip)
expect(error.reason.body).toBe(failure.reason.body)
expect(error.reason.cause).toBe(cause)
expect(error.message).toBe(failure.message)
expect(error.reason.http?.status).toBe(200)
expect(error.reason.http?.headers).toEqual(headers)
}),
)
it.effect("retains the original read error after successful response headers", () =>
Effect.gen(function* () {
const cause = new Error("socket disconnected")
const error = yield* LLMClient.generate(request).pipe(Effect.provide(truncatedStream([], cause)), Effect.flip)
expect(error.reason).toMatchObject({ _tag: "Transport", operation: "read" })
expect(error.reason.cause).toBe(cause)
expect(error.reason.http?.status).toBe(200)
expect(error.reason.body).toBeUndefined()
}),
)
it.effect("retains successful response headers on an incomplete stream", () =>
Effect.gen(function* () {
const error = yield* LLMClient.generate(request).pipe(Effect.provide(fixedResponse("", { headers })), Effect.flip)
expect(error.reason).toMatchObject({ _tag: "InvalidProviderOutput", classification: "incomplete-stream" })
expect(error.reason.http?.headers).toEqual(headers)
expect(error.reason.body).toBeUndefined()
}),
)
})
+184 -10
View File
@@ -4,14 +4,25 @@ import * as OpenAIChat from "../src/protocols/openai-chat.js"
import * as OpenAIResponses from "../src/protocols/openai-responses.js"
import {
AIError,
AIErrorReason,
AuthenticationError,
ContentPart,
InvalidRequestReason,
ContentPolicyError,
HttpContext,
InvalidProviderOutputError,
InvalidRequestError,
LLMEvent,
LLMRequest,
LanguageModel,
ModelID,
NoRouteError,
ProviderID,
TransportReason,
ProviderInternalError,
QuotaExceededError,
RateLimitError,
RouteID,
TransportError,
UnknownProviderError,
Usage,
} from "../src/schema/index.js"
import { ProviderShared } from "../src/protocols/shared.js"
@@ -144,19 +155,20 @@ describe("AI.Usage", () => {
test("AI errors expose the shared runtime tag", async () => {
const error = new AIError({
module: "test",
method: "call",
reason: new InvalidRequestReason({ message: "invalid" }),
reason: new InvalidRequestError({ message: "invalid" }),
})
expect(error._tag).toBe("AI.Error")
expect(error.message).toBe("invalid")
expect(error.cause).toBe(error.reason)
expect(error.reason.cause).toBeUndefined()
expect(
await Effect.runPromise(Effect.fail(error).pipe(Effect.catchTag("AI.Error", () => Effect.succeed("caught")))),
).toBe("caught")
})
test("transport errors serialize execution facts", () => {
const reason = new TransportReason({
message: "connection closed",
const reason = new TransportError({
message: "Connection closed",
transport: "websocket",
operation: "read",
phase: "receive",
@@ -164,14 +176,176 @@ test("transport errors serialize execution facts", () => {
recovery: "fail",
})
expect(Schema.encodeSync(TransportReason)(reason)).toEqual({
expect(Schema.encodeSync(TransportError)(reason)).toEqual({
_tag: "Transport",
message: "connection closed",
message: "Connection closed",
transport: "websocket",
operation: "read",
phase: "receive",
delivery: "ambiguous",
recovery: "fail",
})
expect(Schema.decodeUnknownSync(TransportReason)(Schema.encodeSync(TransportReason)(reason))).toEqual(reason)
expect(Schema.decodeUnknownSync(TransportError)(Schema.encodeSync(TransportError)(reason))).toEqual(reason)
})
test("AI errors serialize diagnostics only on their typed reason", () => {
const cause = new SyntaxError("Unexpected end of JSON input")
const error = new AIError({
reason: new InvalidRequestError({
message: "Invalid provider response",
body: '{"error":',
http: new HttpContext({
url: "https://provider.test/v1/messages",
status: 400,
headers: { "request-id": "req_123" },
}),
cause,
parameter: "messages",
classification: "context-overflow",
}),
})
const encoded = Schema.encodeSync(AIError)(error)
expect(encoded).toEqual({
_tag: "AI.Error",
reason: {
_tag: "InvalidRequest",
message: "Invalid provider response",
body: '{"error":',
http: {
url: "https://provider.test/v1/messages",
status: 400,
headers: { "request-id": "req_123" },
},
cause: { name: "SyntaxError", message: cause.message, stack: cause.stack },
parameter: "messages",
classification: "context-overflow",
},
})
const decoded = Schema.decodeUnknownSync(Schema.fromJsonString(AIError))(
Schema.encodeSync(Schema.fromJsonString(AIError))(error),
)
expect(error).not.toHaveProperty("body")
expect(error).not.toHaveProperty("http")
expect(error.cause).toBe(error.reason)
expect(error.reason.cause).toBe(cause)
expect(decoded).toBeInstanceOf(AIError)
expect(decoded.reason).toBeInstanceOf(InvalidRequestError)
expect(decoded.message).toBe("Invalid provider response")
expect(decoded.reason.message).toBe(decoded.message)
expect(decoded.reason.body).toBe('{"error":')
expect(decoded.reason.http).toEqual(error.reason.http)
expect(decoded.cause).toBe(decoded.reason)
expect(decoded.reason.cause).toBeInstanceOf(Error)
expect(decoded.reason.cause).toMatchObject({ name: "SyntaxError", message: cause.message, stack: cause.stack })
expect(decoded.reason).toMatchObject({ parameter: "messages", classification: "context-overflow" })
})
test("AI error reasons are tagged Errors with required messages", () => {
const reasons = [
new InvalidRequestError({ message: "Invalid request" }),
new NoRouteError({
message: "No route",
route: RouteID.make("missing"),
provider: model.provider,
model: model.id,
}),
new AuthenticationError({ message: "Missing credentials", kind: "missing" }),
new RateLimitError({ message: "Rate limited" }),
new QuotaExceededError({ message: "Quota exceeded" }),
new ContentPolicyError({ message: "Content blocked" }),
new ProviderInternalError({ message: "Provider failed" }),
new TransportError({ message: "Connection failed", transport: "http", operation: "request" }),
new InvalidProviderOutputError({ message: "Invalid output" }),
new UnknownProviderError({ message: "Unknown failure" }),
]
expect(reasons.map((reason) => reason._tag)).toEqual([
"InvalidRequest",
"NoRoute",
"Authentication",
"RateLimit",
"QuotaExceeded",
"ContentPolicy",
"ProviderInternal",
"Transport",
"InvalidProviderOutput",
"UnknownProvider",
])
reasons.forEach((reason) => {
expect(reason).toBeInstanceOf(Error)
const encoded = Schema.encodeSync(AIErrorReason)(reason)
const decoded = Schema.decodeUnknownSync(AIErrorReason)(encoded)
expect(decoded).toBeInstanceOf(reason.constructor)
expect(decoded.message).toBe(reason.message)
expect(Schema.decodeUnknownOption(AIErrorReason)({ ...encoded, message: undefined })._tag).toBe("None")
})
})
test("AI error reason enrichment preserves non-enumerable diagnostics", () => {
const cause = new Error("socket disconnected")
const reason = new TransportError({
message: "Connection closed",
body: "close frame detail",
http: new HttpContext({ url: "https://provider.test/responses", status: 101, headers: { upgrade: "websocket" } }),
cause,
transport: "websocket",
operation: "read",
phase: "close",
})
expect(Object.prototype.propertyIsEnumerable.call(reason, "message")).toBe(false)
expect(Object.prototype.propertyIsEnumerable.call(reason, "cause")).toBe(false)
const enriched = AIErrorReason.make({
// oxlint-disable-next-line typescript-eslint/no-misused-spread -- Copy fields rather than iterating the yieldable error.
...reason,
message: reason.message,
cause: reason.cause,
delivery: "ambiguous",
recovery: "retry-full",
})
const error = new AIError({ reason: enriched })
expect(enriched).toBeInstanceOf(TransportError)
expect(error.message).toBe(reason.message)
expect(error.cause).toBe(enriched)
expect(enriched.cause).toBe(cause)
expect(enriched.body).toBe(reason.body)
expect(enriched.http).toBe(reason.http)
expect(enriched).toMatchObject({ phase: "close", delivery: "ambiguous", recovery: "retry-full" })
})
test("AI errors support reason-specific handlers", async () => {
const limited = new AIError({ reason: new RateLimitError({ message: "Slow down", retryAfterMs: 2000 }) })
const invalid = new AIError({ reason: new InvalidRequestError({ message: "Invalid request", parameter: "model" }) })
expect(
await Effect.runPromise(
Effect.fail(limited).pipe(
Effect.catchReason("AI.Error", "RateLimit", (reason) => {
expect(reason).toBe(limited.reason)
expect(reason).toBeInstanceOf(RateLimitError)
return Effect.succeed(reason.retryAfterMs)
}),
),
),
).toBe(2000)
expect(
await Effect.runPromise(
Effect.forEach([limited, invalid], (error) =>
Effect.fail(error).pipe(
Effect.catchReasons("AI.Error", {
RateLimit: (reason) => Effect.succeed(reason.message),
InvalidRequest: (reason) => Effect.succeed(reason.parameter),
}),
),
),
),
).toEqual(["Slow down", "model"])
})
test("HTTP error context requires an observed response", () => {
const decode = Schema.decodeUnknownOption(HttpContext)
expect(decode({ status: 400, headers: {} })._tag).toBe("None")
expect(decode({ url: "https://provider.test", headers: {} })._tag).toBe("None")
expect(decode({ url: "https://provider.test", status: 400 })._tag).toBe("None")
expect(decode({ url: "https://provider.test", status: 0, headers: {} })._tag).toBe("None")
expect(decode({ url: "https://provider.test", status: Number.NaN, headers: {} })._tag).toBe("None")
})
+1 -1
View File
@@ -109,7 +109,7 @@ describe("ToolStream", () => {
const error = ToolStream.appendExisting(ADAPTER, ToolStream.empty<number>(), 0, "{}", "missing tool")
expect(error).toBeInstanceOf(AIError)
if (ToolStream.isError(error)) expect(error.reason.message).toBe("missing tool")
if (ToolStream.isError(error)) expect(error.message).toBe("missing tool")
}),
)
+2
View File
@@ -1,3 +1,5 @@
src/assets/theme.css
e2e/test-results
e2e/playwright-report
component-tests/test-results
component-tests/playwright-report
+28 -4
View File
@@ -31,7 +31,7 @@ Your app is ready to be deployed!
## E2E Testing
Playwright starts the Vite dev server automatically via `webServer`, and UI tests expect an opencode backend at `localhost:4096` by default.
Locally, Playwright starts the Vite dev server automatically via `webServer`, or reuses one already running at the configured address. The browser suite uses isolated API fixtures rather than a live opencode backend.
```bash
bunx playwright install chromium
@@ -39,11 +39,35 @@ bun run test:e2e:local
bun run test:e2e:local -- --grep "settings"
```
CI builds the app once and runs the same suite against Vite preview, serving production assets from `dist`. Managed built runs never reuse an existing server, so a running dev server cannot silently replace the production build. To run this mode locally:
```bash
bun run test:e2e:built
bun run test:e2e:built -- --grep "settings"
```
To test an already-running dev server without starting or building a server:
```bash
PLAYWRIGHT_BASE_URL=http://127.0.0.1:4444 bun run test:e2e
```
For an already-running production build, also set `PLAYWRIGHT_BUILD=1` so the fixture API uses the app's origin:
```bash
PLAYWRIGHT_BUILD=1 PLAYWRIGHT_BASE_URL=http://127.0.0.1:4444 bun run test:e2e
```
External targets must use HTTP because fixture URLs use HTTP. `PLAYWRIGHT_BASE_URL` skips server startup and building in either mode.
Compiled CLI startup and service lifecycle coverage runs separately in CI via `packages/cli/script/service-smoke.ts`.
Environment options:
- `PLAYWRIGHT_SERVER_HOST` / `PLAYWRIGHT_SERVER_PORT` (backend address, default: `localhost:4096`)
- `PLAYWRIGHT_PORT` (Vite dev server port, default: `3000`)
- `PLAYWRIGHT_BASE_URL` (override base URL, default: `http://localhost:<PLAYWRIGHT_PORT>`)
- `PLAYWRIGHT_BUILD=1` (build and preview locally; always enabled when `CI` is set)
- `PLAYWRIGHT_SERVER_HOST` / `PLAYWRIGHT_SERVER_PORT` (dev fixture API address, default: `127.0.0.1:4096`; built runs use the app's origin, matching production)
- `PLAYWRIGHT_PORT` (managed dev or preview server port, default: `3000`)
- `PLAYWRIGHT_BASE_URL` (use an externally managed app instead of starting a server; otherwise defaults to `http://127.0.0.1:<PLAYWRIGHT_PORT>`)
## Deployment
@@ -0,0 +1,26 @@
import { expect, story } from "../../storybook/playwright/story"
// Moved from packages/app/e2e/regression/prompt-thinking-level.spec.ts
story("shows the thinking level control while relevant", async ({ mount, page }) => {
const component = await mount("opencode-composer-flow--model-and-variant")
const composer = component.locator('[data-component="composer"]')
const input = composer.locator('[data-component="composer-editor"]')
const control = composer.getByRole("button", { name: "Choose model variant" })
await page.mouse.move(0, 0)
await page.evaluate(() => (document.activeElement as HTMLElement | null)?.blur())
await expect(control).toBeVisible()
await control.click()
const high = page.getByRole("menuitemradio", { name: "high" })
await expect(high).toBeVisible()
await page.mouse.move(0, 0)
await expect(control).toBeVisible()
await expect(high).toBeVisible()
await high.click()
await input.focus()
await expect(control).toBeVisible()
await input.blur()
await expect(control).toBeVisible()
})
+21
View File
@@ -44,6 +44,7 @@ The suite contains:
- single-session tab close timing through stable home restoration
- cached session repaint and mutation tracing
- streaming timeline throughput, RAF-gap, long-task, geometry, and remount diagnostics
- retained renderer heap with a large model catalog across repeated session navigation
All benchmarks import the shared `benchmark` fixture. Pages created through Playwright's `page` fixture automatically capture main-frame navigation history and emit a Chrome trace when `OPENCODE_PERFORMANCE_TRACE_DIR` is set. Benchmarks that need isolated browser contexts use `withBenchmarkPage`, which owns the context and the same diagnostics lifecycle.
@@ -78,6 +79,26 @@ Benchmarks do not assert machine-dependent performance budgets. Streaming proces
Committed smoke and regression tests continue to own correctness coverage for pagination, tab paint, context resize, collapse state, and composer spacing.
Tab-switch timing starts at `mousedown`, when mouse-selected tabs actually navigate, with a `click` fallback for keyboard activation. The probe excludes hidden/transparent content and intersects answers with their virtual-row clip and viewport. The tab workload requires the destination's final answer to be visible with Markdown ready. These results are not directly comparable to older click-start, geometry-only measurements. `stableObservedMs` includes confirmation across three correct samples; `firstCorrectObservedMs` is the first sample meeting all content and geometry checks. Neither is a compositor presentation timestamp.
Each tab scenario reports one sample, including its raw observations. Use Playwright's `--repeat-each=5` for repeated measurements. Cached scenarios warm the destination at the same panel width before leaving it; a separate resized scenario validates reuse after opening the review pane changes that width.
```sh
bunx playwright test --config e2e/performance/playwright.config.ts \
timeline/session-tab-switch-benchmark.spec.ts --repeat-each=5
```
## Retained renderer memory
Run the catalog workload against the production app bundle:
```sh
bunx playwright test --config e2e/performance/playwright.config.ts \
timeline/provider-memory-benchmark.spec.ts --repeat-each=3
```
`PROVIDER_MEMORY_MODELS` defaults to 1,200 and `PROVIDER_MEMORY_SWITCHES` defaults to 10. Each sample records Chromium's `Runtime.getHeapUsage` and `Memory.getDOMCounters` after an explicit garbage collection. This measures retained state, not allocation peaks or normal GC timing. It does not include worker heaps, the Electron main/GPU processes, or the OpenCode server, and must not be reported as total desktop RAM. Use identical model counts and navigation sequences for before/after comparisons.
## Chrome traces
Set `OPENCODE_PERFORMANCE_TRACE_DIR` to emit a standard Chrome DevTools trace for every benchmark page automatically:
@@ -1,7 +1,7 @@
import config from "../../playwright.config"
const port = Number(process.env.PLAYWRIGHT_PORT ?? 3000)
process.env.PLAYWRIGHT_SERVER_PORT = String(port)
process.env.PLAYWRIGHT_SERVER_PORT = new URL(process.env.PLAYWRIGHT_BASE_URL ?? `http://127.0.0.1:${port}`).port || "80"
process.env.OPENCODE_PERFORMANCE_RUN_ID ??= `${new Date().toISOString().replace(/[:.]/g, "-")}-${process.pid}`
export default {
@@ -12,9 +12,11 @@ export default {
fullyParallel: false,
workers: 1,
reporter: [["html", { outputFolder: "../playwright-report/performance", open: "never" }], ["line"]],
webServer: {
...config.webServer,
command: `bun run build && bun run serve -- --host 0.0.0.0 --port ${port} --strictPort`,
reuseExistingServer: false,
},
webServer: config.webServer
? {
...config.webServer,
command: `bun run build && bun run serve -- --host 0.0.0.0 --port ${port} --strictPort`,
reuseExistingServer: false,
}
: undefined,
}
@@ -126,13 +126,19 @@ test("keeps moving upward while drag-selecting above the timeline", async ({ pag
)
})
})
const textBox = await text.boundingBox()
const textBox = await text.evaluate((element) => {
const range = document.createRange()
range.selectNodeContents(element)
const rect = range.getClientRects()[0]
return rect ? { x: rect.x, y: rect.y, width: rect.width, height: rect.height } : null
})
const scrollBox = await scroller.boundingBox()
expect(textBox).not.toBeNull()
expect(scrollBox).not.toBeNull()
if (!textBox || !scrollBox) return
await page.mouse.move(textBox.x + textBox.width - 10, textBox.y + textBox.height / 2)
// Start on a text line, not the empty right edge or gap between wrapped lines.
await page.mouse.move(textBox.x + Math.min(20, textBox.width / 2), textBox.y + textBox.height / 2)
await page.mouse.down()
await page.mouse.move(textBox.x + 20, scrollBox.y - 120, { steps: 30 })
@@ -195,6 +201,45 @@ test("does not pull a keyboard-scrolled user during shell remeasurement", async
await reportVisualStability(testInfo, "keyboard-during-resize", trace, anchorPlan(regions))
})
test("keeps an older answer selected while scrolling within the interaction buffer", async ({ page }) => {
await setupTimeline(page, {
messages: history(80),
viewport: { width: 1400, height: 700 },
reducedMotion: true,
})
const scroller = page.locator(".scroll-view__viewport", { has: page.locator("[data-timeline-row]") })
const answer = page.getByText("History 78.", { exact: false })
await expect(answer).toBeVisible()
await expect
.poll(() =>
answer.evaluate((element) => element.closest('[data-component="markdown"]')?.hasAttribute("data-markdown-ready")),
)
.toBe(true)
const textBox = await answer.evaluate((element) => {
const range = document.createRange()
range.selectNodeContents(element)
const rect = range.getClientRects()[0]
return { x: rect.x, y: rect.y, width: rect.width, height: rect.height }
})
const scrollBox = await scroller.boundingBox()
expect(scrollBox).not.toBeNull()
if (!scrollBox) return
await page.mouse.move(textBox.x + Math.min(180, textBox.width - 2), textBox.y + textBox.height / 2)
await page.mouse.down()
await page.mouse.move(textBox.x + 2, textBox.y + textBox.height / 2, { steps: 30 })
await page.mouse.up()
await expect.poll(() => page.evaluate(() => window.getSelection()?.toString())).toContain("History 78.")
await page.mouse.move(scrollBox.x + scrollBox.width / 2, scrollBox.y + scrollBox.height / 2)
await page.mouse.wheel(0, -450)
await expect
.poll(() => scroller.evaluate((element) => element.scrollHeight - element.clientHeight - element.scrollTop))
.toBeGreaterThan(400)
await expect(answer).toHaveCount(1)
await expect.poll(() => page.evaluate(() => window.getSelection()?.toString())).toContain("History 78.")
await page.getByRole("heading", { name: "Timeline visual stability" }).click()
await expect.poll(() => page.evaluate(() => window.getSelection()?.isCollapsed)).toBe(true)
})
test("tracks keyboard scrolling from a focused timeline descendant", async ({ page }, testInfo) => {
const shellID = "prt_descendant_keyboard_01_shell"
const timeline = await setupTimeline(page, {
@@ -259,12 +304,16 @@ test("does not claim keyboard scrolling owned by a nested scrollable", async ({
const before = await scroller.evaluate((element) => element.scrollTop)
const nestedBefore = await nested.evaluate((element) => element.scrollTop)
await nested.press("PageUp")
await page.waitForTimeout(300)
await expect.poll(() => nested.evaluate((element) => element.scrollTop)).toBeLessThan(nestedBefore)
expect(await scroller.evaluate((element) => element.scrollTop)).toBe(before)
expect(await nested.evaluate((element) => element.scrollTop)).toBeLessThan(nestedBefore)
await nested.evaluate((element) => (element.scrollTop = 0))
await scroller.evaluate((element) => (element.scrollTop = Math.min(300, element.scrollHeight - element.clientHeight)))
await nested.evaluate((element) => element.scrollTo({ top: 0, behavior: "instant" }))
await expect.poll(() => nested.evaluate((element) => element.scrollTop)).toBe(0)
await scroller.evaluate((element) => {
element.dispatchEvent(new WheelEvent("wheel", { bubbles: true, cancelable: true, deltaY: -1 }))
element.scrollTo({ top: Math.min(300, element.scrollHeight - element.clientHeight), behavior: "instant" })
})
await expect.poll(() => scroller.evaluate((element) => element.scrollTop)).toBeLessThan(500)
const boundaryBefore = await scroller.evaluate((element) => element.scrollTop)
expect(boundaryBefore).toBeGreaterThan(0)
await nested.press("PageUp")
@@ -0,0 +1,98 @@
import { benchmark, expect } from "../benchmark"
import { mockOpenCodeServer } from "../../utils/mock-server"
import { expectSessionTitle } from "../../utils/waits"
import { fixture, pageMessages } from "./session-timeline-stress.fixture"
import { installStressSessionTabs, installTimelineSettings, stressSessionHref } from "./timeline-test-helpers"
import { waitForStableTimeline } from "./session-tab-switch-probe"
import type { CatalogUpdated } from "@opencode-ai/client/promise"
benchmark("measures retained renderer memory with a large model catalog", async ({ page, report }) => {
benchmark.setTimeout(120_000)
const count = Number(process.env.PROVIDER_MEMORY_MODELS ?? 1200)
const switches = Number(process.env.PROVIDER_MEMORY_SWITCHES ?? 10)
const provider = fixture.provider.all[0]
const selected = { ...provider.models["claude-opus-4-6"] }
await mockOpenCodeServer(page, {
directory: fixture.directory,
project: fixture.project,
sessions: fixture.sessions,
pageMessages,
provider: {
...fixture.provider,
all: [
{
...provider,
models: {
[selected.id]: selected,
...Object.fromEntries(
Array.from({ length: count - 1 }, (_, index) => {
const id = `catalog-model-${index}`
return [
id,
{
id,
name: `Catalog model ${index}`,
cost: { input: 1, output: 2 },
limit: { context: 200_000, output: 8192 },
variants: { high: { reasoningEffort: "high" } },
},
]
}),
),
},
},
],
},
})
await installTimelineSettings(page)
await installStressSessionTabs(page)
await page.goto(stressSessionHref(fixture.sourceID))
await expectSessionTitle(page, fixture.expected.sourceTitle)
await waitForStableTimeline(page, fixture.expected.sourceMessageIDs.at(-1)!)
await expect(page.locator('[data-action="composer-model"]')).toContainText("Claude Opus 4.6")
const cdp = await page.context().newCDPSession(page)
const samples = []
for (let index = 0; index <= switches; index++) {
if (index > 0) {
const target = index % 2 === 1
const id = target ? fixture.targetID : fixture.sourceID
await page.locator(`[data-slot="titlebar-tabs"] a[href="${stressSessionHref(id)}"]`).click()
await expectSessionTitle(page, target ? fixture.expected.targetTitle : fixture.expected.sourceTitle)
await waitForStableTimeline(
page,
(target ? fixture.expected.targetMessageIDs : fixture.expected.sourceMessageIDs).at(-1)!,
)
await expect(page.locator('[data-action="composer-model"]')).toContainText("Claude Opus 4.6")
}
// GC is an explicit retained-heap measurement, not an application optimization or readiness wait.
await cdp.send("HeapProfiler.collectGarbage")
samples.push({
switches: index,
heap: await cdp.send("Runtime.getHeapUsage"),
dom: await cdp.send("Memory.getDOMCounters"),
})
}
expect(samples).toHaveLength(switches + 1)
expect(samples.every((sample) => sample.heap.usedSize > 0)).toBe(true)
selected.name = "Updated catalog model"
await page.evaluate(
(event) => {
const host = window as Window & { __mockServerStream?: { push: (events: CatalogUpdated[]) => void } }
if (!host.__mockServerStream) throw new Error("Missing fixture event stream")
host.__mockServerStream.push([event])
},
{
id: "evt_catalog_refresh",
created: Date.now(),
type: "catalog.updated",
location: { directory: fixture.directory },
data: {},
} satisfies CatalogUpdated,
)
await expect(page.locator('[data-action="composer-model"]')).toContainText(selected.name)
report(
{ samples },
{ models: count, switches, gc: "explicit", scope: "renderer main isolate; not total desktop RAM" },
)
await cdp.detach()
})
@@ -11,115 +11,73 @@ import {
} from "./timeline-test-helpers"
import { measureSessionSwitch, waitForStableTimeline } from "./session-tab-switch-probe"
type Result = Awaited<ReturnType<typeof measureSessionSwitch>>
const scenarios = [
{ cached: false, review: false, resized: false },
{ cached: false, review: true, resized: false },
{ cached: true, review: false, resized: false },
{ cached: true, review: true, resized: false },
{ cached: true, review: true, resized: true },
]
benchmark(
"benchmarks session tab switching with and without the review pane",
async ({ browser, report }, testInfo) => {
benchmark.setTimeout(360_000)
const runs = Number(process.env.SESSION_TAB_SWITCH_RUNS ?? 5)
const results = {
closed: { cold: [] as Result[], hot: [] as Result[] },
open: { cold: [] as Result[], hot: [] as Result[] },
}
for (const reviewPane of ["closed", "open"] as const) {
for (const mode of ["cold", "hot"] as const) {
for (let run = 0; run < runs; run++) {
results[reviewPane][mode].push(
await withBenchmarkPage(
browser,
`session-tab-switch-${reviewPane}-${mode}-${run}`,
(page) => trial(page, mode, reviewPane),
testInfo,
),
)
scenarios.forEach((scenario) => {
const name = `tab switch: ${scenario.cached ? "cached" : "unmounted"}, review ${scenario.review ? "open" : "closed"}${scenario.resized ? ", resized" : ""}`
benchmark(name, async ({ browser, report }, testInfo) => {
const result = await withBenchmarkPage(
browser,
name,
async (page) => {
await mockStressTimeline(page, { vcsDiff: createReviewDiffs() })
await installTimelineSettings(page)
await installStressSessionTabs(page)
await page.goto(stressSessionHref(fixture.sourceID))
await expectSessionTitle(page, fixture.expected.sourceTitle)
await waitForStableTimeline(page, fixture.expected.sourceMessageIDs.at(-1)!)
if (scenario.review && !scenario.resized) await openReviewPane(page)
if (scenario.cached) {
await switchSession(page, fixture.targetID, fixture.expected.targetTitle)
const answer = page.locator(`[data-timeline-part-id="${fixture.expected.targetPartIDs.at(-1)}"]`)
await expect(answer.locator('[data-component="markdown"]')).toHaveAttribute("data-markdown-ready", "")
await expect
.poll(() =>
answer.evaluate((element) => element.checkVisibility({ checkOpacity: true, checkVisibilityCSS: true })),
)
.toBe(true)
await waitForStableTimeline(page, fixture.expected.targetMessageIDs.at(-1)!)
await switchSession(page, fixture.sourceID, fixture.expected.sourceTitle)
}
}
}
report({ results, summary: summarizeReviewPane(results) }, { runs, reviewDiffs: createReviewDiffs().length })
},
)
if (scenario.resized) await openReviewPane(page)
await waitForStableTimeline(page, fixture.expected.sourceMessageIDs.at(-1)!)
async function trial(page: Page, mode: "cold" | "hot", reviewPane: "closed" | "open") {
const reviewDiffs = createReviewDiffs()
await mockStressTimeline(page, { vcsDiff: reviewDiffs })
await installTimelineSettings(page)
await installStressSessionTabs(page)
if (mode === "hot") {
await page.goto(stressSessionHref(fixture.targetID))
await expectSessionTitle(page, fixture.expected.targetTitle)
await waitForStableTimeline(page, fixture.expected.targetMessageIDs.at(-1)!)
await switchSession(page, fixture.sourceID, fixture.expected.sourceTitle)
} else {
await page.goto(stressSessionHref(fixture.sourceID))
await expectSessionTitle(page, fixture.expected.sourceTitle)
}
await waitForStableTimeline(page, fixture.expected.sourceMessageIDs.at(-1)!)
if (reviewPane === "open") {
await openReviewPane(page)
await waitForStableTimeline(page, fixture.expected.sourceMessageIDs.at(-1)!)
}
const destinationIDs = fixture.messages[fixture.targetID].map((message) => message.id)
const sourceIDs = fixture.messages[fixture.sourceID].map((message) => message.id)
const lastID = fixture.expected.targetMessageIDs.at(-1)!
const href = stressSessionHref(fixture.targetID)
const result = await measureSessionSwitch(page, {
destinationIDs,
sourceIDs,
lastID,
href,
switch: () => switchSession(page, fixture.targetID, fixture.expected.targetTitle),
})
return result
}
function summarize(results: Record<"cold" | "hot", Result[]>) {
const stats = (values: (number | null)[]) => {
const sorted = values.filter((value): value is number => value !== null).sort((a, b) => a - b)
return {
min: sorted[0] ?? null,
median: sorted[Math.floor(sorted.length / 2)] ?? null,
max: sorted.at(-1) ?? null,
missing: values.length - sorted.length,
}
}
return Object.fromEntries(
Object.entries(results).map(([mode, values]) => [
mode,
{
firstDestinationObservedMs: stats(values.map((value) => value.firstDestinationObservedMs)),
firstCorrectObservedMs: stats(values.map((value) => value.firstCorrectObservedMs)),
stableObservedMs: stats(values.map((value) => value.stableObservedMs)),
return measureSessionSwitch(page, {
destinationIDs: fixture.messages[fixture.targetID].map((message) => message.id),
sourceIDs: fixture.messages[fixture.sourceID].map((message) => message.id),
lastID: fixture.expected.targetMessageIDs.at(-1)!,
requiredPartID: fixture.expected.targetPartIDs.at(-1),
href: stressSessionHref(fixture.targetID),
switch: () => switchSession(page, fixture.targetID, fixture.expected.targetTitle),
})
},
]),
)
}
function summarizeReviewPane(results: Record<"closed" | "open", Record<"cold" | "hot", Result[]>>) {
return Object.fromEntries(
Object.entries(results).map(([reviewPane, values]) => [
reviewPane,
summarize(values as Record<"cold" | "hot", Result[]>),
]),
)
}
testInfo,
)
expect(result.unknownSamples).toBe(0)
expect(result.wrongDestinationSamples).toBe(0)
if (scenario.cached) expect(result.blankSamples).toBe(0)
report(result, { ...scenario, inputEvent: "mousedown", requireReadyAnswer: true })
})
})
async function switchSession(page: Page, sessionID: string, title: string) {
const href = stressSessionHref(sessionID)
const tab = page.locator(`[data-slot="titlebar-tabs"] a[href="${href}"]`).first()
await expect(tab).toBeVisible()
const tab = page.locator(`[data-slot="titlebar-tabs"] a[href="${stressSessionHref(sessionID)}"]`)
await expect(tab).toHaveCount(1)
await tab.click()
await expectSessionTitle(page, title)
}
async function openReviewPane(page: Page) {
await page.getByRole("button", { name: "Toggle review" }).click()
const panel = page.locator("#review-panel")
await expect(panel).toBeVisible()
await expect(page.locator("#review-panel")).toBeVisible()
await page.waitForFunction(() => {
const panel = document.querySelector<HTMLElement>("#review-panel")
const text = panel?.textContent ?? ""
const text = document.querySelector("#review-panel")?.textContent ?? ""
return text.includes("generated-000.ts") && text.includes("+3")
})
}
@@ -20,9 +20,10 @@ export function classifySessionSwitch(samples: SessionSwitchSample[]) {
const firstCorrect = samples.findIndex(isCorrectDestination)
const stable = samples.findIndex((_, index) => isStableSessionSwitch(samples.slice(index, index + 3)))
return {
samples,
firstDestinationObservedMs: samples[firstDestination]?.observedAtMs ?? null,
firstCorrectObservedMs: samples[firstCorrect]?.observedAtMs ?? null,
stableObservedMs: samples[stable + 2]?.observedAtMs ?? null,
stableObservedMs: stable < 0 ? null : samples[stable + 2].observedAtMs,
wrongDestinationSamples: samples
.slice(firstDestination)
.filter((sample) => sample.destination.length > 0 && !sample.last).length,
@@ -0,0 +1,67 @@
import { benchmark, expect } from "../benchmark"
import { measureSessionSwitch } from "./session-tab-switch-probe"
import type { SessionSwitchSample } from "./session-tab-switch-metrics"
benchmark("starts at mousedown and excludes hidden or unfinished destination content", async ({ page, report }) => {
await page.setContent(`
<a href="/session/destination">Destination</a>
<div class="scroll-view__viewport" style="height:200px;overflow:auto">
<div data-timeline-row="message" data-timeline-key="row" data-message-id="source">
<div data-timeline-part-id="answer"><div data-component="markdown">Destination answer</div></div>
</div>
</div>
`)
await page.evaluate(() => {
document.querySelector("a")!.addEventListener("mousedown", () => {
const row = document.querySelector<HTMLElement>("[data-message-id]")!
row.dataset.messageId = "destination"
row.style.visibility = "hidden"
})
})
const result = await measureSessionSwitch(page, {
destinationIDs: ["destination"],
sourceIDs: ["source"],
lastID: "destination",
requiredPartID: "answer",
requireBottomAnchor: false,
href: "/session/destination",
switch: async () => {
// No click is dispatched: the probe must observe the event that activates tabs.
await page.getByRole("link", { name: "Destination" }).dispatchEvent("mousedown", { button: 0 })
await page.waitForFunction(() => {
const host = window as Window & { __sessionSwitchProbe?: { samples: SessionSwitchSample[] } }
return host.__sessionSwitchProbe?.samples.some((sample) => !sample.hasVisibleRows)
})
await page.locator("[data-message-id]").evaluate((row) => row.style.removeProperty("visibility"))
await page.waitForFunction(() => {
const host = window as Window & { __sessionSwitchProbe?: { samples: SessionSwitchSample[] } }
return host.__sessionSwitchProbe?.samples.some(
(sample) => sample.destination.length > 0 && sample.requiredPartVisible === false,
)
})
const beforeClip = await page.evaluate(() => {
const row = document.querySelector<HTMLElement>("[data-timeline-key]")!
row.style.cssText = "height:10px;position:relative;overflow:clip"
const answer = row.querySelector<HTMLElement>("[data-timeline-part-id]")!
answer.style.cssText = "position:absolute;top:30px;width:150px"
answer.querySelector('[data-component="markdown"]')!.setAttribute("data-markdown-ready", "")
return (
(window as Window & { __sessionSwitchProbe?: { samples: SessionSwitchSample[] } }).__sessionSwitchProbe
?.samples.length ?? 0
)
})
await page.waitForFunction((count) => {
const host = window as Window & { __sessionSwitchProbe?: { samples: SessionSwitchSample[] } }
return host.__sessionSwitchProbe?.samples.slice(count).some((sample) => sample.requiredPartVisible === false)
}, beforeClip)
await page.locator("[data-timeline-key]").evaluate((row) => {
row.style.height = "100px"
})
},
})
expect(result.blankSamples).toBeGreaterThan(0)
expect(result.firstCorrectObservedMs).not.toBeNull()
expect(result.stableObservedMs).not.toBeNull()
expect(result.firstCorrectObservedMs).toBeGreaterThan(result.firstDestinationObservedMs!)
report(result)
})
@@ -25,7 +25,7 @@ async function installSessionSwitchProbe(
let running = true
const reviewLevels: Record<string, string> = {
panel: "#review-panel",
tabs: '#review-panel [data-component="tabs"]',
tabs: '#review-panel [data-component="tabs"]',
body: '#review-panel [data-slot="session-review-v2-body"]',
review: '#review-panel [data-component="session-review-v2"]',
preview: '#review-panel [data-slot="session-review-v2-preview"]',
@@ -37,7 +37,6 @@ async function installSessionSwitchProbe(
if (!running || started === undefined) return
setTimeout(() => {
if (!running || started === undefined) return
const observedAtMs = performance.now() - started
const reviewPanel = document.querySelector<HTMLElement>("#review-panel")
const reviewFile = reviewPanel?.querySelector('[data-component="file"][data-mode="diff"]')
const initialReviewFile = initialReviewNodes.file
@@ -63,26 +62,30 @@ async function installSessionSwitchProbe(
)
if (root) {
const view = root.getBoundingClientRect()
const visible = [...root.querySelectorAll<HTMLElement>("[data-message-id]")]
.filter((element) => {
const rect = element.getBoundingClientRect()
return rect.bottom > view.top && rect.top < view.bottom
})
.map((element) => element.dataset.messageId!)
const hasVisibleRows = [...root.querySelectorAll<HTMLElement>("[data-timeline-key]")].some((element) => {
const inViewport = (element: HTMLElement) => {
if (!element.checkVisibility({ checkOpacity: true, checkVisibilityCSS: true })) return false
const rect = element.getBoundingClientRect()
return rect.bottom > view.top && rect.top < view.bottom
})
const clip = element.closest<HTMLElement>("[data-timeline-key]")?.getBoundingClientRect() ?? view
return (
Math.min(rect.bottom, clip.bottom, view.bottom) > Math.max(rect.top, clip.top, view.top) &&
Math.min(rect.right, clip.right, view.right) > Math.max(rect.left, clip.left, view.left)
)
}
const visible = [...root.querySelectorAll<HTMLElement>("[data-message-id]")]
.filter(inViewport)
.map((element) => element.dataset.messageId!)
const hasVisibleRows = [...root.querySelectorAll<HTMLElement>("[data-timeline-key]")].some(inViewport)
const requiredPartVisible = requiredPartID
? [...root.querySelectorAll<HTMLElement>("[data-timeline-part-id]")].some((element) => {
if (element.dataset.timelinePartId !== requiredPartID) return false
const rect = element.getBoundingClientRect()
return rect.width > 0 && rect.height > 0 && rect.bottom > view.top && rect.top < view.bottom
if (!element.textContent?.trim()) return false
if (element.querySelector('[data-component="markdown"]:not([data-markdown-ready])')) return false
return inViewport(element)
})
: undefined
const spacer = root.querySelector<HTMLElement>('[data-timeline-row="bottom-spacer"]')?.getBoundingClientRect()
samples.push({
observedAtMs,
observedAtMs: performance.now() - started,
destination: visible.filter((id) => destination.has(id)),
source: visible.filter((id) => source.has(id)),
hasVisibleRows,
@@ -94,7 +97,7 @@ async function installSessionSwitchProbe(
})
} else {
samples.push({
observedAtMs,
observedAtMs: performance.now() - started,
destination: [],
source: [],
hasVisibleRows: false,
@@ -107,23 +110,25 @@ async function installSessionSwitchProbe(
requestAnimationFrame(sample)
}, 0)
}
document.addEventListener(
"click",
(event) => {
const link = event.target instanceof Element ? event.target.closest("a") : undefined
if (link?.getAttribute("href") !== href) return
started = performance.now()
for (const [name, selector] of Object.entries(reviewLevels)) {
initialReviewNodes[name] = document.querySelector(selector)
}
requestAnimationFrame(sample)
},
{ capture: true, once: true },
)
const start = (event: MouseEvent) => {
if (started !== undefined || event.button !== 0) return
const link = event.target instanceof Element ? event.target.closest("a") : undefined
if (link?.getAttribute("href") !== href) return
started = performance.now()
for (const [name, selector] of Object.entries(reviewLevels)) {
initialReviewNodes[name] = document.querySelector(selector)
}
requestAnimationFrame(sample)
}
// Tabs activate on mousedown; click alone misses the synchronous navigation work.
document.addEventListener("mousedown", start, true)
document.addEventListener("click", start, true)
;(window as Window & { __sessionSwitchProbe?: SessionSwitchProbe }).__sessionSwitchProbe = {
samples,
stop: () => {
running = false
document.removeEventListener("mousedown", start, true)
document.removeEventListener("click", start, true)
},
}
}, input)
@@ -53,6 +53,15 @@ test("reports missing correctness without throwing", () => {
expect(result.stableObservedMs).toBeNull()
})
test("does not report stability for only two correct samples", () => {
const result = classifySessionSwitch([
{ observedAtMs: 16, destination: ["destination"], source: [], hasVisibleRows: true, last: true, bottomErrorPx: 0 },
{ observedAtMs: 32, destination: ["destination"], source: [], hasVisibleRows: true, last: true, bottomErrorPx: 0 },
])
expect(result.firstCorrectObservedMs).toBe(16)
expect(result.stableObservedMs).toBeNull()
})
test("requires an explicitly tracked part to be visible", () => {
const result = classifySessionSwitch([
{
@@ -0,0 +1,120 @@
import { expect, test } from "@playwright/test"
import { mockOpenCodeServer } from "../utils/mock-server"
import { expectAppVisible } from "../utils/waits"
const draftID = "draft_large_paste"
const directory = "/repo/large-paste"
const server = `http://${process.env.PLAYWRIGHT_SERVER_HOST ?? "127.0.0.1"}:${process.env.PLAYWRIGHT_SERVER_PORT ?? "4096"}`
test.use({ permissions: ["clipboard-read", "clipboard-write"] })
test.beforeEach(async ({ page }) => {
await mockOpenCodeServer(page, {
directory,
project: {
id: "proj_large_paste",
worktree: directory,
vcs: "git",
name: "large-paste",
time: { created: 1700000000000, updated: 1700000000000 },
sandboxes: [],
},
provider: { all: [], connected: [], default: {} },
sessions: [],
pageMessages: () => ({ items: [] }),
})
await page.addInitScript(
({ directory, draftID, server }) => {
localStorage.setItem("opencode-theme-id", "oc-2")
localStorage.setItem("opencode-color-scheme", "dark")
localStorage.setItem(
"opencode.global.dat:server",
JSON.stringify({
projects: { local: [{ worktree: directory, expanded: true }] },
lastProject: { local: directory },
}),
)
localStorage.setItem(
"opencode.window.browser.dat:tabs",
JSON.stringify([{ type: "draft", draftID, server, directory }]),
)
},
{ directory, draftID, server },
)
await page.goto(`/new-session?draftId=${draftID}`)
const input = page.locator('[data-component="composer-editor"]')
await expectAppVisible(input)
await expect(input).toBeEditable()
await expect
.poll(() => input.evaluate((element) => getComputedStyle(element, "::before").content))
.toBe(`"${String.fromCodePoint(0x200b)}"`)
await input.click()
})
for (const lines of [6000, 25000]) {
test(`keeps a ${lines}-line crash report editable in a new session`, async ({ page }) => {
const input = page.getByRole("textbox", { name: "Prompt", exact: true })
const text = "Thread 0 Crashed:\n" + "0 Example 0x0000000100000000 frame + 32\n".repeat(lines) + "End of report"
await page.evaluate((text) => navigator.clipboard.writeText(text), text)
const events = await input.evaluateHandle((element) => {
const events = { count: 0 }
element.addEventListener("input", () => events.count++)
return events
})
await page.keyboard.press("ControlOrMeta+V")
await expect.poll(async () => (await input.innerText()) === text).toBe(true)
expect(await events.evaluate((events) => events.count)).toBe(1)
await expect(input).toBeFocused()
await page.keyboard.type("!")
await expect.poll(async () => (await input.innerText()) === text + "!").toBe(true)
})
}
for (const text of [
"single line <b> &amp;",
"first\nsecond",
"\n\n indented\ttext \n\nlast\n\n",
'literal <b>bold</b> &amp; & < > "quotes"\n<script>not code</script>\n<img src="example">',
"first\r\nsecond\rthird",
]) {
test(`preserves text and native undo: ${JSON.stringify(text)}`, async ({ page }) => {
const input = page.getByRole("textbox", { name: "Prompt", exact: true })
await page.evaluate((text) => navigator.clipboard.writeText(text), text)
await page.keyboard.press("ControlOrMeta+V")
const expected = text.replace(/\r\n?/g, "\n")
await expect.poll(() => input.innerText()).toBe(expected)
await expect(input.locator("b, script, img")).toHaveCount(0)
await page.keyboard.press("ControlOrMeta+Z")
await expect(input).toBeEmpty()
await page.keyboard.press("ControlOrMeta+Shift+Z")
await expect.poll(() => input.innerText()).toBe(expected)
})
}
test("replaces only the selected text and leaves the caret after the paste", async ({ page }) => {
const input = page.getByRole("textbox", { name: "Prompt", exact: true })
await page.evaluate(() => navigator.clipboard.writeText("one\ntwo"))
await page.keyboard.type("before replace after")
await expect(input).toHaveText("before replace after")
await page.evaluate(() => document.fonts.ready)
const word = await input.evaluate((element) => {
const range = document.createRange()
range.setStart(element.firstChild!, 7)
range.setEnd(element.firstChild!, 14)
const rect = range.getBoundingClientRect()
return { x: rect.x, y: rect.y + rect.height / 2, width: rect.width }
})
await page.mouse.move(word.x, word.y)
await page.mouse.down()
await page.mouse.move(word.x + word.width, word.y, { steps: 5 })
await page.mouse.up()
await expect.poll(() => page.evaluate(() => window.getSelection()?.toString())).toBe("replace")
await page.keyboard.press("ControlOrMeta+V")
await expect.poll(() => input.innerText()).toBe("before one\ntwo after")
await page.keyboard.press("ControlOrMeta+Z")
await expect(input).toHaveText("before replace after")
await page.keyboard.press("ControlOrMeta+Shift+Z")
await expect.poll(() => input.innerText()).toBe("before one\ntwo after")
await page.keyboard.type("!")
await expect.poll(() => input.innerText()).toBe("before one\ntwo! after")
})
@@ -3,7 +3,7 @@ import { base64Encode } from "@opencode-ai/util/encode"
import { currentSession } from "../utils/mock-server"
import { installSseTransport } from "../utils/sse-transport"
const serverA = "http://127.0.0.1:4096"
const serverA = `http://${process.env.PLAYWRIGHT_SERVER_HOST ?? "127.0.0.1"}:${process.env.PLAYWRIGHT_SERVER_PORT ?? "4096"}`
const serverB = "http://127.0.0.1:4097"
const sessionA = session("ses_server_a", "C:/server-a", "Server A session")
const sessionB = session("ses_server_b", "/home/server-b", "Server B session")
@@ -12,17 +12,17 @@ test("closing the active server's last tab opens the remaining server tab", asyn
const requests: string[] = []
await mockServers(page, requests)
await page.addInitScript(
({ serverB, sessionA, sessionB }) => {
({ serverA, serverB, sessionA, sessionB }) => {
localStorage.setItem("opencode.global.dat:server", JSON.stringify({ list: [serverB] }))
localStorage.setItem(
"opencode.window.browser.dat:tabs",
JSON.stringify([
{ type: "session", server: "http://127.0.0.1:4096", sessionId: sessionA },
{ type: "session", server: serverA, sessionId: sessionA },
{ type: "session", server: serverB, sessionId: sessionB },
]),
)
},
{ serverB, sessionA: sessionA.id, sessionB: sessionB.id },
{ serverA, serverB, sessionA: sessionA.id, sessionB: sessionB.id },
)
const hrefA = `/server/${base64Encode(serverA)}/session/${sessionA.id}`
@@ -55,7 +55,7 @@ function session(id: string, directory: string, title: string) {
async function mockServers(page: Page, requests: string[]) {
await installSseTransport(page, { server: serverA })
await installSseTransport(page, { server: serverB })
await page.route("**/*", async (route) => {
await page.route("**/api/**", async (route) => {
const url = new URL(route.request().url())
if (url.origin !== serverA && url.origin !== serverB) return route.fallback()
requests.push(url.toString())
@@ -0,0 +1,178 @@
import { base64Encode } from "@opencode-ai/util/encode"
import { expect, test } from "@playwright/test"
import { mockOpenCodeServer } from "../utils/mock-server"
import { expectSessionTitle } from "../utils/waits"
const directory = "C:\\OpenCode\\main"
const workspace = "C:\\OpenCode\\worktree"
const projectID = "proj_mcp_workspace"
const sessionID = "ses_mcp_workspace"
const title = "Workspace MCP routing"
const server = `http://${process.env.PLAYWRIGHT_SERVER_HOST ?? "127.0.0.1"}:${process.env.PLAYWRIGHT_SERVER_PORT ?? "4096"}`
for (const shared of [true, false]) {
test(`toggles the workspace MCP when the default location ${shared ? "has" : "does not have"} the server`, async ({
page,
}, testInfo) => {
const connected = new Set<string>()
const requests: { path: string; directory: string }[] = []
await mockOpenCodeServer(page, {
directory,
project: {
id: projectID,
worktree: directory,
vcs: "git",
name: "mcp-workspace",
time: { created: 1700000000000, updated: 1700000000000 },
sandboxes: [workspace],
},
provider: { all: [], connected: [], default: {} },
sessions: [{ id: sessionID, projectID, directory: workspace, title }],
pageMessages: () => ({ items: [] }),
})
await page.route("**/api/mcp**", async (route) => {
if (route.request().method() === "OPTIONS") return route.fallback()
const url = new URL(route.request().url())
const target = url.searchParams.get("location[directory]") ?? directory
requests.push({ path: url.pathname, directory: target })
if (url.pathname === "/api/mcp/figma-desktop/connect") {
connected.add(target)
return route.fulfill({ status: 204 })
}
if (url.pathname === "/api/mcp/figma-desktop/disconnect") {
connected.delete(target)
return route.fulfill({ status: 204 })
}
return route.fulfill({
json: {
location: { directory: target },
data:
url.pathname === "/api/mcp/resource"
? { resources: [], templates: [] }
: !shared && target !== workspace
? []
: [{ name: "figma-desktop", status: { status: connected.has(target) ? "connected" : "disabled" } }],
},
})
})
await page.goto(`/server/${base64Encode(server)}/session/${sessionID}`)
await expectSessionTitle(page, title)
await expect(page.getByRole("textbox", { name: "Prompt", exact: true })).toBeEditable()
await page.keyboard.press("ControlOrMeta+;")
const dialog = page.getByRole("dialog", { name: "MCPs", exact: true })
await expect(dialog.getByText("figma-desktop", { exact: true })).toBeVisible()
const toggle = dialog.getByRole("switch")
await expect(toggle).not.toBeChecked()
await expect(toggle).toBeEnabled()
requests.length = 0
await dialog.locator('[data-slot="switch-control"]').click()
await expect(toggle).toBeChecked()
await expect(toggle).toBeEnabled()
expect(connected).toEqual(new Set([workspace]))
expect(requests).toContainEqual({ path: "/api/mcp/figma-desktop/connect", directory: workspace })
expect(requests).toContainEqual({ path: "/api/mcp/resource", directory: workspace })
expect(requests.every((request) => request.directory === workspace)).toBe(true)
await testInfo.attach("workspace-connected", { body: await page.screenshot(), contentType: "image/png" })
requests.length = 0
await dialog.getByText("figma-desktop", { exact: true }).click()
await expect(toggle).not.toBeChecked()
await expect(toggle).toBeEnabled()
expect(connected.size).toBe(0)
expect(requests).toContainEqual({ path: "/api/mcp/figma-desktop/disconnect", directory: workspace })
expect(requests.every((request) => request.directory === workspace)).toBe(true)
})
}
for (const surface of ["popover", "dialog"] as const) {
test(`shows connection failures from the MCP ${surface} and allows reconnecting`, async ({ page }, testInfo) => {
const error = "Streamable HTTP error: Error POSTing to endpoint: 404 Not Found"
const state = { fail: true, status: surface === "popover" ? "failed" : "disabled" }
const requests: { path: string; directory: string }[] = []
await page.addInitScript(() => {
localStorage.setItem("settings.v3", JSON.stringify({ general: { showStatus: true } }))
})
await mockOpenCodeServer(page, {
directory,
project: {
id: projectID,
worktree: directory,
vcs: "git",
name: "mcp-workspace",
time: { created: 1700000000000, updated: 1700000000000 },
sandboxes: [workspace],
},
provider: { all: [], connected: [], default: {} },
sessions: [{ id: sessionID, projectID, directory: workspace, title }],
pageMessages: () => ({ items: [] }),
})
await page.route("**/api/mcp**", async (route) => {
if (route.request().method() === "OPTIONS") return route.fallback()
const url = new URL(route.request().url())
const target = url.searchParams.get("location[directory]") ?? directory
requests.push({ path: url.pathname, directory: target })
if (url.pathname === "/api/mcp/figma-desktop/connect") {
state.status = state.fail ? "failed" : "connected"
// Connection failures are reported by the refreshed status, not the HTTP response.
return route.fulfill({ status: 204 })
}
return route.fulfill({
json: {
location: { directory: target },
data:
url.pathname === "/api/mcp/resource"
? { resources: [], templates: [] }
: [
{
name: "figma-desktop",
status: { status: target === workspace ? state.status : "connected", error },
},
],
},
})
})
await page.goto(`/server/${base64Encode(server)}/session/${sessionID}`)
await expectSessionTitle(page, title)
await expect(page.getByRole("textbox", { name: "Prompt", exact: true })).toBeEditable()
if (surface === "popover") await page.getByRole("button", { name: "Status", exact: true }).click()
if (surface === "dialog") await page.keyboard.press("ControlOrMeta+;")
const panel =
surface === "popover" ? page.getByRole("tabpanel") : page.getByRole("dialog", { name: "MCPs", exact: true })
const toggle = panel.getByRole("switch")
await expect(panel.getByText("figma-desktop", { exact: true })).toBeVisible()
await expect(toggle).not.toBeChecked()
await expect(toggle).toBeEnabled()
requests.length = 0
await panel.locator('[data-slot="switch-control"]').click()
const toast = page
.getByRole("listitem", { includeHidden: true })
.filter({ has: page.getByText("Request failed", { exact: true }) })
await expect(toast.getByText(`figma-desktop: ${error}`, { exact: true })).toBeVisible()
await expect(toggle).not.toBeChecked()
await expect(toggle).toBeEnabled()
expect(requests.filter((request) => request.path.endsWith("/connect"))).toEqual([
{ path: "/api/mcp/figma-desktop/connect", directory: workspace },
])
expect(requests.every((request) => request.directory === workspace)).toBe(true)
await expect(toast).toHaveCSS("opacity", "1")
await testInfo.attach("mcp-connection-error", { body: await page.screenshot(), contentType: "image/png" })
if (surface === "popover") await page.keyboard.press("Escape")
if (surface === "dialog") await panel.getByRole("button", { name: "Close", exact: true }).click()
await expect(panel).toBeHidden()
await toast.getByRole("button", { name: "Dismiss", exact: true }).click()
await expect(toast).toBeHidden()
state.fail = false
if (surface === "popover") await page.getByRole("button", { name: "Status", exact: true }).click()
if (surface === "dialog") await page.keyboard.press("ControlOrMeta+;")
await expect(toggle).toBeEnabled()
await panel.locator('[data-slot="switch-control"]').click()
await expect(toggle).toBeChecked()
await expect(toggle).toBeEnabled()
await expect(toast).toBeHidden()
})
}
@@ -0,0 +1,303 @@
import { expect, test, type Page } from "@playwright/test"
import { base64Encode } from "@opencode-ai/util/encode"
import { currentSession, mockOpenCodeServer } from "../utils/mock-server"
import { expectAppVisible } from "../utils/waits"
const directory = "C:/OpenCode/WorkspacePending"
const workspace = "C:/OpenCode/pending-workspace"
const projectID = "proj_workspace_pending"
const draftID = "draft_workspace_pending"
const otherID = "ses_workspace_pending_other"
const text = "Create the workspace, then explain the pending session."
const server = `http://${process.env.PLAYWRIGHT_SERVER_HOST ?? "127.0.0.1"}:${process.env.PLAYWRIGHT_SERVER_PORT ?? "4096"}`
const sessionPath = `/server/${base64Encode(server)}/session/`
const draftPath = `/new-session?draftId=${draftID}`
const headers = { "access-control-allow-origin": "*" }
test.use({ serviceWorkers: "block", viewport: { width: 1280, height: 900 } })
for (const viewport of [
{ name: "desktop", width: 1280, height: 900 },
{ name: "mobile", width: 390, height: 844 },
]) {
test(`shows a pending workspace session immediately on ${viewport.name}`, async ({ page }, testInfo) => {
await page.setViewportSize(viewport)
const mock = await openDraft(page)
const pending = await submitPending(page, mock)
await expect(pending.message).toBeInViewport()
await expect(pending.shimmer).toBeInViewport()
await testInfo.attach("creating-worktree", {
body: await page.screenshot({ path: testInfo.outputPath(`pending-${viewport.name}.png`) }),
contentType: "image/png",
})
if (viewport.name === "mobile") {
await page.locator("html").evaluate((element) => {
element.dir = "rtl"
})
await expect(page.locator("html")).toHaveAttribute("dir", "rtl")
await expect(pending.message).toBeInViewport()
await expect(pending.shimmer).toBeInViewport()
await expect(page.locator('[data-component="session-preparing"]')).toHaveCSS("direction", "rtl")
expect(
await page
.locator('[data-component="session-preparing"]')
.evaluate((element) => element.scrollWidth <= element.clientWidth),
).toBe(true)
}
if (viewport.name === "desktop") {
await page.locator(`[data-titlebar-tab-link][href="${sessionPath}${otherID}"]`).click()
await expect(page).toHaveURL(`${sessionPath}${otherID}`)
await expect(page.locator('[data-component="composer-editor"]')).toBeEditable()
await expect(pending.shimmer).toBeHidden()
await page.locator(`[data-titlebar-tab-link][href="${sessionPath}${pending.sessionID}"]`).click()
await expect(page).toHaveURL(pending.url)
await expect(pending.message).toHaveAttribute("data-timeline-part-id", `${pending.messageID}:text:0`)
await expect(pending.shimmer).toHaveAttribute("data-active", "true")
expect(mock.calls).toEqual(["worktree"])
await page.locator(`[data-titlebar-tab-link][href="${sessionPath}${otherID}"]`).click()
await expect(page).toHaveURL(`${sessionPath}${otherID}`)
await page.locator('[data-component="composer-editor"]').fill("Keep focus in this other session")
await expect(page.locator('[data-component="composer-editor"]')).toBeFocused()
}
expect(mock.calls).toEqual(["worktree"])
mock.worktree.resolve({ status: 200, json: { directory: workspace } })
await expect
.poll(() => mock.prompts)
.toEqual([{ sessionID: pending.sessionID, body: expect.objectContaining({ id: pending.messageID, text }) }])
expect(mock.creates).toEqual([
expect.objectContaining({ id: pending.sessionID, location: { directory: workspace } }),
])
expect(mock.calls).toEqual(["worktree", "session", "prompt"])
if (viewport.name === "desktop") {
await expect(page.locator(`[data-titlebar-tab-link][href="${sessionPath}${pending.sessionID}"]`)).toContainText(
"Created workspace session",
)
await expect(page).toHaveURL(`${sessionPath}${otherID}`)
await expect(page.locator('[data-component="composer-editor"]')).toHaveText("Keep focus in this other session")
await expect(page.locator('[data-component="composer-editor"]')).toBeFocused()
await page.locator(`[data-titlebar-tab-link][href="${sessionPath}${pending.sessionID}"]`).click()
}
await expect(page).toHaveURL(pending.url)
await expect(pending.shimmer).toHaveCount(0)
await expect(pending.message).toHaveCount(1)
await expect(pending.message.locator('[data-slot="user-message-text"]')).toHaveText(text)
await expect(pending.message).toHaveAttribute("data-timeline-part-id", `${pending.messageID}:text:0`)
})
}
test("restores the original draft when worktree creation fails", async ({ page }) => {
const mock = await openDraft(page)
const pending = await submitPending(page, mock)
mock.worktree.resolve({ status: 500, json: { message: "Worktree creation failed in the fixture" } })
await expect(page).toHaveURL(draftPath)
await expect(page.getByText("Failed to create worktree", { exact: true })).toBeVisible()
await expect(page.locator('[data-component="composer-editor"]')).toHaveText(text)
await expect(page.locator('[data-action="composer-submit"]')).toBeEnabled()
await expect(page.getByRole("button", { name: "New workspace", exact: true })).toBeVisible()
await expect(pending.shimmer).toHaveCount(0)
await expect(pending.message).toHaveCount(0)
await expect(page.locator(`[data-titlebar-tab-link][href="${sessionPath}${pending.sessionID}"]`)).toHaveCount(0)
expect(mock.calls).toEqual(["worktree"])
expect(mock.creates).toEqual([])
expect(mock.prompts).toEqual([])
})
test("retains the draft and reuses the created workspace after session creation fails", async ({ page }) => {
const mock = await openDraft(page, { failSessionCreate: true })
const pending = await submitPending(page, mock)
mock.worktree.resolve({ status: 200, json: { directory: workspace } })
await expect(page).toHaveURL(draftPath)
await expect(page.getByText("Failed to create session", { exact: true })).toBeVisible()
await expect(page.locator('[data-component="composer-editor"]')).toHaveText(text)
await expect(page.locator('[data-action="composer-submit"]')).toBeEnabled()
await expect(page.getByRole("button", { name: "pending-workspace", exact: true })).toBeVisible()
await expect(pending.shimmer).toHaveCount(0)
await expect(pending.message).toHaveCount(0)
expect(mock.creates).toEqual([expect.objectContaining({ id: pending.sessionID, location: { directory: workspace } })])
expect(mock.calls).toEqual(["worktree", "session"])
expect(mock.prompts).toEqual([])
await page.locator('[data-action="composer-submit"]').click()
await expect.poll(() => mock.prompts.length).toBe(1)
expect(mock.creates).toHaveLength(2)
expect(mock.creates[1]).toMatchObject({ location: { directory: workspace } })
expect(mock.prompts[0]).toMatchObject({ sessionID: mock.creates[1].id, body: { text } })
expect(mock.calls).toEqual(["worktree", "session", "session", "prompt"])
await expect(page).toHaveURL(`${sessionPath}${mock.creates[1].id}`)
await expect(page.locator('[data-component="user-message"] [data-slot="user-message-text"]')).toHaveText(text)
})
test("restores the draft after closing and revisiting a pending session that fails", async ({ page }) => {
const mock = await openDraft(page)
const pending = await submitPending(page, mock)
const tab = page.locator(`[data-titlebar-tab-link][href="${sessionPath}${pending.sessionID}"]`)
await page.locator("[data-titlebar-tab-slot]").filter({ has: tab }).locator('[data-slot="tab-close"] button').click()
await expect(page).toHaveURL(`${sessionPath}${otherID}`)
await expect(page.locator('[data-component="composer-editor"]')).toBeEditable()
await expect(tab).toHaveCount(0)
await expect(pending.shimmer).toHaveCount(0)
await page.goBack()
await expect(page).toHaveURL(pending.url)
await expect(tab).toHaveCount(1)
await expect(tab).toBeVisible()
await expect(pending.message).toHaveCount(1)
await expect(pending.message.locator('[data-slot="user-message-text"]')).toHaveText(text)
await expect(pending.message).toHaveAttribute("data-timeline-part-id", `${pending.messageID}:text:0`)
await expect(pending.shimmer).toBeVisible()
await expect(pending.shimmer).toContainText("Creating worktree")
await expect(pending.shimmer).toHaveAttribute("data-active", "true")
expect(mock.calls).toEqual(["worktree"])
mock.worktree.resolve({ status: 500, json: { message: "Worktree creation failed after revisiting the session" } })
await expect(page).toHaveURL(draftPath)
await expect(page.getByText("Failed to create worktree", { exact: true })).toBeVisible()
await expect(page.locator('[data-component="composer-editor"]')).toHaveText(text)
await expect(page.locator('[data-action="composer-submit"]')).toBeEnabled()
await expect(page.getByRole("button", { name: "New workspace", exact: true })).toBeVisible()
await expect(page.locator(`[data-titlebar-tab-link][href="${draftPath}"]`)).toHaveCount(1)
await expect(tab).toHaveCount(0)
await expect(pending.shimmer).toHaveCount(0)
await expect(pending.message).toHaveCount(0)
expect(mock.calls).toEqual(["worktree"])
expect(mock.creates).toEqual([])
expect(mock.prompts).toEqual([])
})
async function openDraft(page: Page, options?: { failSessionCreate?: boolean }) {
const worktree = Promise.withResolvers<{ status: number; json: { directory?: string; message?: string } }>()
const calls: string[] = []
const creates: Record<string, unknown>[] = []
const prompts: { sessionID: string; body: Record<string, unknown> }[] = []
const project = {
id: projectID,
worktree: directory,
vcs: "git",
name: "workspace-pending",
time: { created: 1700000000000, updated: 1700000000000 },
sandboxes: [] as string[],
}
const sessions = [currentSession({ id: otherID, projectID, title: "Other session" }, directory)]
await mockOpenCodeServer(page, {
directory,
project,
provider: {
all: [
{
id: "opencode",
name: "OpenCode",
models: { "pending-model": { id: "pending-model", name: "Pending Model", limit: { context: 200_000 } } },
},
],
connected: ["opencode"],
default: { providerID: "opencode", modelID: "pending-model" },
},
sessions,
pageMessages: () => ({ items: [] }),
onPrompt: (input) => prompts.push(input),
})
page.on("request", (request) => {
if (request.method() !== "POST") return
const path = new URL(request.url()).pathname
if (path === `/api/worktree/${projectID}`) calls.push("worktree")
if (path === "/api/session") calls.push("session")
if (/^\/api\/session\/[^/]+\/prompt$/.test(path)) calls.push("prompt")
})
await page.route(`**/api/worktree/${projectID}`, async (route) => {
if (route.request().method() !== "POST") return route.fallback()
// Keep the real HTTP response pending until the test has checked the preview.
const response = await worktree.promise
if (response.status === 200) project.sandboxes.push(workspace)
await route.fulfill({ ...response, headers })
})
await page.route("**/api/session", async (route) => {
if (route.request().method() !== "POST") return route.fallback()
const body: Record<string, unknown> = route.request().postDataJSON()
creates.push(body)
if (options?.failSessionCreate && creates.length === 1) {
return route.fulfill({ status: 500, json: { message: "Session creation failed in the fixture" }, headers })
}
if (typeof body.id !== "string") throw new Error("Session creation must use the client-reserved ID")
const session = currentSession({ ...body, id: body.id, projectID, title: "Created workspace session" }, workspace)
sessions.push(session)
return route.fulfill({ json: { data: session }, headers })
})
await page.route("**/api/location?**", (route) => {
if (route.request().method() !== "GET") return route.fallback()
return route.fulfill({
json: {
directory: new URL(route.request().url()).searchParams.get("location[directory]") ?? directory,
project: { id: projectID, directory, canonical: directory },
},
headers,
})
})
await page.addInitScript(
({ directory, draftID, otherID, server }) => {
localStorage.setItem(
"opencode.global.dat:server",
JSON.stringify({
projects: { local: [{ worktree: directory, expanded: true }] },
lastProject: { local: directory },
}),
)
localStorage.setItem(
"opencode.window.browser.dat:tabs",
JSON.stringify([
{ type: "draft", draftID, server, directory },
{ type: "session", sessionId: otherID, server },
]),
)
},
{ directory, draftID, otherID, server },
)
await page.goto(draftPath)
await expectAppVisible(page.locator('[data-component="composer-editor"]'))
await page.getByRole("button", { name: "Local", exact: true }).click()
await page.getByRole("menuitem", { name: "New workspace", exact: true }).click()
await expect(page.getByRole("button", { name: "New workspace", exact: true })).toBeVisible()
await expect(page.locator('[data-component="composer-editor"]')).toBeEditable()
return { worktree, calls, creates, prompts }
}
async function submitPending(page: Page, mock: Awaited<ReturnType<typeof openDraft>>) {
await page.locator('[data-component="composer-editor"]').fill(text)
await expect(page.locator('[data-action="composer-submit"]')).toBeEnabled()
await page.locator('[data-action="composer-submit"]').click()
await expect(page).toHaveURL((url) => url.pathname.startsWith(sessionPath) && /\/ses_[^/]+$/.test(url.pathname))
const url = page.url()
const sessionID = new URL(url).pathname.slice(sessionPath.length)
const preparing = page.locator('[data-component="session-preparing"]')
const message = page.locator('[data-component="user-message"]')
const shimmer = preparing.getByRole("status").locator('[data-component="text-shimmer"]')
await expect(preparing).toBeVisible()
await expect(preparing.locator('[data-component="user-message"]')).toHaveCount(1)
await expect(message).toHaveCount(1)
await expect(message.locator('[data-slot="user-message-text"]')).toHaveText(text)
await expect(message).toHaveAttribute("data-timeline-part-id", /^.+:text:0$/)
const messageID = (await message.getAttribute("data-timeline-part-id"))!.replace(/:text:0$/, "")
await expect(shimmer).toBeVisible()
await expect(shimmer).toContainText("Creating worktree")
await expect(shimmer).toHaveAttribute("data-active", "true")
await expect.poll(() => mock.calls).toEqual(["worktree"])
expect(mock.creates).toEqual([])
expect(mock.prompts).toEqual([])
return { url, sessionID, messageID, message, shimmer }
}
@@ -0,0 +1,103 @@
import { expect, test } from "@playwright/test"
import { base64Encode } from "@opencode-ai/util/encode"
import { mockOpenCodeServer } from "../utils/mock-server"
const directory = "C:/Projects/extensions-demo"
const server = `http://${process.env.PLAYWRIGHT_SERVER_HOST ?? "127.0.0.1"}:${process.env.PLAYWRIGHT_SERVER_PORT ?? "4096"}`
const session = {
id: "ses_project_extensions",
title: "Existing session",
directory,
projectID: "proj_extensions_demo",
time: { created: 1700000000000, updated: 1700000000000 },
}
test.use({ viewport: { width: 1440, height: 1000 }, colorScheme: "dark" })
test("project Extensions stays inside settings while plugins load", async ({ page }) => {
await mockOpenCodeServer(page, {
directory,
project: {
id: session.projectID,
canonical: directory,
name: "Extensions demo",
vcs: "git",
time: session.time,
sandboxes: [],
},
provider: { all: [], connected: [], default: {} },
sessions: [session],
pageMessages: () => ({ items: [] }),
})
await page.addInitScript(
({ server, sessionID, directory }) => {
localStorage.setItem(
"opencode.global.dat:server",
JSON.stringify({ projects: { local: [{ worktree: directory, expanded: true }] } }),
)
localStorage.setItem(
"opencode.window.browser.dat:tabs",
JSON.stringify([{ type: "session", server, sessionId: sessionID }]),
)
},
{ server, sessionID: session.id, directory },
)
const href = `/server/${base64Encode(server)}/session/${session.id}`
await page.goto(href)
await expect(page.getByRole("heading", { name: session.title, exact: true })).toBeVisible()
await page.keyboard.press("Control+,")
const settings = page.getByTestId("settings-screen")
await settings.getByRole("tab", { name: "Projects", exact: true }).click()
await settings.getByText("Extensions demo", { exact: true }).click()
const dialog = page.getByRole("dialog")
await expect(dialog.getByRole("textbox", { name: "Name", exact: true })).toBeFocused()
const globalPlugins = Promise.withResolvers<void>()
const projectPlugins = Promise.withResolvers<void>()
await page.route(
(url) => url.pathname === "/api/plugin",
async (route) => {
const project = new URL(route.request().url()).searchParams.get("location[directory]")
await (project ? projectPlugins : globalPlugins).promise
await route.fulfill({
json: {
location: project ? { directory: project } : {},
data: (project ? ["shared-plugin", "project-plugin"] : ["shared-plugin"]).map((id) => ({
id,
source: { type: "package", package: id },
status: "active",
tui: false,
})),
},
})
},
)
const requested = page.waitForRequest((request) => {
const url = new URL(request.url())
return url.pathname === "/api/plugin" && url.searchParams.get("location[directory]") === directory
})
await dialog.getByRole("tab", { name: "Extensions", exact: true }).click()
await requested
await expect(page).toHaveURL(href)
await expect(dialog.getByRole("heading", { name: "Extensions", exact: true })).toBeVisible()
await expect(settings).toBeVisible()
await expect(page.getByRole("heading", { name: session.title, exact: true, includeHidden: true })).toBeHidden()
await dialog.getByRole("tab", { name: "Plugins", exact: true }).click()
await expect(dialog.getByRole("tab", { name: "Plugins", exact: true })).toHaveAttribute("aria-selected", "true")
globalPlugins.resolve()
await dialog.getByRole("tab", { name: "Scripts", exact: true }).click()
await expect(dialog.getByRole("heading", { name: "Scripts", exact: true })).toBeVisible()
await dialog.getByRole("tab", { name: "Extensions", exact: true }).click()
projectPlugins.resolve()
await dialog.getByRole("tab", { name: "Plugins", exact: true }).click()
await expect(dialog.getByText("project-plugin", { exact: true })).toBeVisible()
await dialog.getByRole("button", { name: "Shared with all projects 1", exact: true }).click()
await expect(dialog.getByText("shared-plugin", { exact: true })).toBeVisible()
await expect(page).toHaveURL(href)
await page.keyboard.press("Escape")
await expect(dialog).toBeHidden()
await expect(settings.getByRole("tab", { name: "Projects", exact: true })).toHaveAttribute("aria-selected", "true")
await expect(page.getByRole("heading", { name: session.title, exact: true, includeHidden: true })).toBeHidden()
})
@@ -3,7 +3,7 @@ import { expect, test, type Page, type Route } from "@playwright/test"
import { installSseTransport } from "../utils/sse-transport"
import { currentSession } from "../utils/mock-server"
const serverA = `http://127.0.0.1:${process.env.PLAYWRIGHT_SERVER_PORT ?? "4096"}`
const serverA = `http://${process.env.PLAYWRIGHT_SERVER_HOST ?? "127.0.0.1"}:${process.env.PLAYWRIGHT_SERVER_PORT ?? "4096"}`
const serverB = "http://127.0.0.1:4097"
const directoryA = "C:/server-a"
const directoryB = "/home/server-b"
@@ -308,7 +308,7 @@ async function mockServers(
permissionResponses: PermissionResponse[] = [],
options: MockServerOptions = {},
) {
await page.route("**/*", async (route) => {
await page.route("**/api/**", async (route) => {
const url = new URL(route.request().url())
if (url.origin !== serverA && url.origin !== serverB) return route.fallback()
const remote = url.origin === serverB
@@ -2,7 +2,7 @@ import { expect, test, type Page, type Route } from "@playwright/test"
import { base64Encode } from "@opencode-ai/util/encode"
import { currentSession } from "../utils/mock-server"
const serverA = "http://127.0.0.1:4096"
const serverA = `http://${process.env.PLAYWRIGHT_SERVER_HOST ?? "127.0.0.1"}:${process.env.PLAYWRIGHT_SERVER_PORT ?? "4096"}`
const serverB = "http://127.0.0.1:4097"
const sessionA = session("ses_server_a", "C:/server-a", "Server A session")
const sessionB = session("ses_server_b", "/home/server-b", "Server B session")
@@ -51,7 +51,7 @@ function session(id: string, directory: string, title: string) {
}
async function mockServers(page: Page) {
await page.route("**/*", async (route) => {
await page.route("**/api/**", async (route) => {
const url = new URL(route.request().url())
if (url.origin !== serverA && url.origin !== serverB) return route.fallback()
const current = url.origin === serverA ? sessionA : sessionB
@@ -12,54 +12,6 @@ test.beforeEach(async ({ page }) => {
await openReview(page)
})
test("opens the comment editor when code is clicked", async ({ page }) => {
const review = page.locator('[data-component="session-review"]')
const line = review.getByText("export const value = 'after'", { exact: true })
await expectAppVisible(line)
await line.click()
await expect(review.getByRole("textbox")).toBeVisible()
await expect(review.locator('[data-slot="line-comment-editor-label"]')).toHaveText("Commenting on line 2")
})
test("opens the comment editor when a line number is clicked", async ({ page }) => {
const review = page.locator('[data-component="session-review"]')
const lineNumber = review.locator('[data-column-number="1"]').last()
await expectAppVisible(lineNumber)
await lineNumber.click()
await expect(review.getByRole("textbox")).toBeVisible()
await expect(review.locator('[data-slot="line-comment-editor-label"]')).toHaveText("Commenting on line 1")
})
test("opens the comment editor for a line number range", async ({ page }) => {
const review = page.locator('[data-component="session-review"]')
const start = review.locator('[data-column-number="1"]').last()
const end = review.locator('[data-column-number="3"]').last()
await expectAppVisible(start)
await expectAppVisible(end)
await start.dragTo(end)
await expect(review.getByRole("textbox")).toBeVisible()
await expect(review.locator('[data-slot="line-comment-editor-label"]')).toHaveText("Commenting on lines 1-3")
})
test("shows a comment button when a diff line is hovered", async ({ page }) => {
const review = page.locator('[data-component="session-review"]')
const line = review.getByText("export const first = 1", { exact: true })
await expectAppVisible(line)
const comment = review.getByRole("button", { name: "Comment", exact: true, includeHidden: true })
await expect(comment).toHaveCount(1)
await line.dispatchEvent("pointermove", { pointerType: "mouse", bubbles: true, composed: true })
await expect(comment).toBeVisible()
await expect(comment).toHaveCSS("pointer-events", "auto")
await comment.dispatchEvent("click")
await expect(review.getByRole("textbox")).toBeVisible()
await expect(review.locator('[data-slot="line-comment-editor-label"]')).toHaveText("Commenting on line 1")
})
test("stages a submitted line comment in the prompt context", async ({ page }) => {
page.on("request", (request) => {
expect.soft(request.method(), `unexpected ${request.method()} ${new URL(request.url()).pathname}`).toBe("GET")
@@ -0,0 +1,89 @@
import { expect, test } from "@playwright/test"
import { setupTimeline } from "../performance/timeline-stability/fixture"
for (const width of [1400, 390]) {
for (const profile of [
{ locale: "en", direction: "ltr" },
{ locale: "en", direction: "rtl" },
{ locale: "ar", direction: "rtl" },
]) {
test(`keeps notices on one line: ${profile.locale} ${profile.direction} ${width}`, async ({ page }, info) => {
const command =
"bun run inspect --target src/renderer/session-timeline.ts --output artifacts/inspection-report.json ".repeat(5)
const descriptions = [
`${command}--finished`,
`Instructions changed\n${command}--updated`,
`\u0645\u0631\u0627\u062c\u0639\u0629 ${command}--reviewed`,
]
await setupTimeline(page, {
locale: profile.locale,
viewport: { width, height: 900 },
sessionMessages: [
{
id: "msg_notice_user",
type: "user",
text: "Inspect the project and report completion.",
time: { created: 1 },
},
{
id: "msg_notice_shell",
type: "synthetic",
text: "Complete",
description: descriptions[0],
metadata: { source: "shell", state: "completed" },
time: { created: 2 },
},
{ id: "msg_notice_system", type: "system", text: descriptions[1], time: { created: 3 } },
{
id: "msg_notice_agent",
type: "synthetic",
text: "Complete",
description: descriptions[2],
metadata: { source: "subagent", state: "completed", agent: "general" },
time: { created: 4 },
},
],
})
await page
.locator("html")
.evaluate((element, direction) => element.setAttribute("dir", direction), profile.direction)
const notices = page.locator('[data-slot="session-timeline-notice"]')
await expect(notices).toHaveCount(3)
await expect(notices).toContainText(descriptions)
await page.locator("[data-timeline-virtual-content]").screenshot({ path: info.outputPath("notices.png") })
await expect
.poll(() =>
notices.evaluateAll((nodes) =>
nodes.map((node) => {
const style = getComputedStyle(node)
const element = node as HTMLElement
return {
direction: style.direction,
whiteSpace: style.whiteSpace,
textOverflow: style.textOverflow,
overflow: style.overflowX,
singleLine:
Math.abs(
element.clientHeight -
parseFloat(style.paddingTop) -
parseFloat(style.paddingBottom) -
parseFloat(style.lineHeight),
) <= 1,
clipped: element.scrollWidth > element.clientWidth,
}
}),
),
)
.toEqual(
Array.from({ length: 3 }, () => ({
direction: profile.direction,
whiteSpace: "nowrap",
textOverflow: "ellipsis",
overflow: "hidden",
singleLine: true,
clipped: true,
})),
)
})
}
}
@@ -1,5 +1,5 @@
import { expect, test, type Page } from "@playwright/test"
import type { OpenCodeEvent } from "@opencode-ai/client/promise"
import type { OpenCodeEvent, SessionMessageInfo } from "@opencode-ai/client/promise"
import { base64Encode } from "@opencode-ai/util/encode"
import { mockOpenCodeServer } from "../utils/mock-server"
import { expectAppVisible } from "../utils/waits"
@@ -18,7 +18,7 @@ type InboxRow = {
delivery: "steer" | "queue"
}
function createQueueMock(seed: string[]) {
function createQueueMock(seed: string[], messages: SessionMessageInfo[] = []) {
const rows: InboxRow[] = seed.map((text, index) => ({
id: `inb_seed_${index + 1}`,
sessionID,
@@ -32,13 +32,16 @@ function createQueueMock(seed: string[]) {
const changes: { inboxID: string; action: "cancel" | "steer" }[] = []
const log: string[] = []
let sequence = 0
const emit = (type: OpenCodeEvent["type"], data: OpenCodeEvent["data"]) => {
const emit = <Type extends OpenCodeEvent["type"]>(
type: Type,
data: Extract<OpenCodeEvent, { type: Type }>["data"],
) => {
sequence += 1
events.push({
id: `evt_queue_${sequence}`,
type,
created: Date.now(),
durable: { aggregateID: sessionID, seq: sequence, version: 1 },
durable: { aggregateID: sessionID, seq: sequence, version: type === "session.tool.success" ? 2 : 1 },
data,
} as OpenCodeEvent)
}
@@ -47,6 +50,8 @@ function createQueueMock(seed: string[]) {
prompts,
changes,
log,
messages,
emit,
events: () => events.splice(0),
onPrompt: (input: { sessionID: string; body: Record<string, unknown> }) => {
prompts.push(input.body)
@@ -126,10 +131,11 @@ async function openSession(page: Page, mock: ReturnType<typeof createQueueMock>,
directory,
title: "Session queue regression",
version: "dev",
model: { id: "queue-model", providerID: "opencode" },
time: { created: 1700000000000, updated: 1700000000000 },
},
],
pageMessages: () => ({ items: [] }),
pageMessages: () => ({ items: mock.messages }),
sessionStatus: () => ({ [sessionID]: { type: "running" } }),
inbox: () => mock.rows.map((row) => ({ ...row, payload: { ...row.payload } })),
onPrompt: mock.onPrompt,
@@ -201,12 +207,15 @@ test("editing restores the existing draft and replaces only the original queue p
await view.input.fill("my in-progress draft")
await original.click()
await expect(view.input).toHaveText("tighten the error copy")
await expect(view.input).toBeFocused()
await view.input.press("Escape")
await expect(view.input).toHaveText("my in-progress draft")
await original.click()
await expect(view.input).toHaveText("tighten the error copy")
await expect(view.input).toBeFocused()
await view.input.fill("tighten the error copy and add a retry hint")
await expect(view.input).toHaveText("tighten the error copy and add a retry hint")
await view.input.press("Enter")
await expect(view.rows.locator('[data-action="session-queue-edit"]')).toHaveText([
@@ -224,3 +233,126 @@ test("editing restores the existing draft and replaces only the original queue p
expect(mock.changes.map((change) => change.action)).toEqual(["cancel", "cancel", "cancel"])
expect(mock.log[0]).toBe("prompt:queue")
})
for (const delivery of ["steer", "queue"] as const) {
test(`keeps finished tools above a pending ${delivery === "queue" ? "queue-to-steer" : "steer"} follow-up`, async ({
page,
}, testInfo) => {
const model = { id: "queue-model", providerID: "opencode" }
const userID = "msg_queue_initial_user"
const assistantID = "msg_queue_continued_assistant"
const followUp = "U2: Also check the retry path."
const mock = createQueueMock(
[],
[
{ id: userID, type: "user", text: "U1: Inspect the queue ordering.", time: { created: 1700000000000 } },
{
id: "msg_queue_initial_assistant",
type: "assistant",
agent: "build",
model,
content: [{ type: "text", text: "A1: I will inspect the current implementation." }],
finish: "tool-calls",
time: { created: 1700000000001, completed: 1700000000002 },
},
],
)
const view = await openSession(page, mock, delivery)
const transcript = page.locator("[data-timeline-virtual-content]")
const thinking = transcript.locator('[data-timeline-row="Thinking"]')
await expect(transcript.getByText("A1: I will inspect the current implementation.", { exact: true })).toBeVisible()
await expect(thinking).toHaveCount(0)
await expect(view.input).toBeEditable()
await view.input.fill(followUp)
await view.input.press("Enter")
await expect.poll(() => mock.rows.map((row) => row.delivery)).toEqual([delivery])
await expect(view.input).toHaveText("")
const inboxID = mock.rows[0].id
const pending = transcript.locator(`[data-timeline-row="UserMessage"][data-message-id="${inboxID}"]`)
if (delivery === "queue") {
const queued = view.rows.filter({ hasText: followUp })
await expect(queued).toBeVisible()
await expect(pending).toHaveCount(0)
await expect(thinking).toHaveCount(0)
await queued.hover()
await queued.getByRole("button", { name: "Steer", exact: true }).click()
await expect.poll(() => mock.changes).toEqual([{ inboxID, action: "steer" }])
}
await expect(view.rows).toHaveCount(0)
await expect(pending).toContainText(followUp)
await expect(thinking).toHaveCount(0)
// The next assistant step still belongs to U1: U2 has been admitted, not delivered.
mock.emit("session.step.started", { sessionID, assistantMessageID: assistantID, agent: "build", model })
for (const tool of [
{ id: "tool_queue_read", name: "read", input: { path: "src/queue.ts" } },
{ id: "tool_queue_grep", name: "grep", input: { pattern: "retry", path: "src" } },
]) {
const ref = { sessionID, assistantMessageID: assistantID, id: tool.id }
mock.emit("session.tool.input.started", { ...ref, name: tool.name })
mock.emit("session.tool.input.ended", { ...ref, text: JSON.stringify(tool.input) })
mock.emit("session.tool.called", { ...ref, input: tool.input, executed: true })
mock.emit("session.tool.success", {
...ref,
content: [{ type: "text", text: "Inspection complete." }],
executed: true,
})
}
mock.emit("session.step.ended", {
sessionID,
assistantMessageID: assistantID,
finish: "tool-calls",
cost: 0,
tokens: { input: 100, output: 20, reasoning: 0, cache: { read: 0, write: 0 } },
})
const tools = page.locator('[data-timeline-part-ids="tool_queue_read,tool_queue_grep"]')
await expect(tools).toBeVisible()
await expect(tools).toContainText(/Used\s*Read, Grep/)
await expect(tools.locator('[data-component="tag"]')).toHaveText("2")
await expect(thinking).toHaveCount(0)
await expect(pending).toBeVisible()
expect(mock.rows.map((row) => ({ id: row.id, delivery: row.delivery }))).toEqual([
{ id: inboxID, delivery: "steer" },
])
await transcript.screenshot({ path: testInfo.outputPath("pending-steer.png") })
// Soft assertions let delivery run too, even when the pending ordering regresses.
await expect.soft(tools.or(pending)).toHaveText([/Used\s*Read, Grep/, /U2: Also check the retry path\./])
await expect
.soft(transcript.locator('[data-timeline-row="AssistantPart"]').filter({ has: tools }))
.toHaveAttribute("data-message-id", userID)
await expect
.configure({ soft: true })
.poll(async () => {
const boxes = await Promise.all([tools.boundingBox(), pending.boundingBox()])
return boxes.every((box) => box !== null) && boxes[0]!.y + boxes[0]!.height <= boxes[1]!.y
})
.toBe(true)
mock.rows.splice(0, 1)
mock.emit("session.inbox.delivered", { sessionID, inboxID })
await expect(thinking).toHaveCount(0)
await expect(pending).toHaveCount(1)
await expect(transcript.locator('[data-timeline-row="UserMessage"]')).toHaveCount(2)
await expect(transcript.locator('[data-timeline-row="AssistantPart"]').filter({ has: tools })).toHaveAttribute(
"data-message-id",
userID,
)
const later = { sessionID, assistantMessageID: "msg_queue_follow_up_assistant" }
mock.emit("session.step.started", { ...later, agent: "build", model })
mock.emit("session.text.started", { ...later, ordinal: 0 })
mock.emit("session.text.ended", { ...later, ordinal: 0, text: "A3: Now checking the retry path for U2." })
const response = transcript
.locator('[data-timeline-row="AssistantPart"]')
.filter({ hasText: "A3: Now checking the retry path for U2." })
await expect(response).toHaveAttribute("data-message-id", inboxID)
await expect(thinking).toHaveCount(0)
await expect(tools.or(pending).or(response)).toHaveText([
/Used\s*Read, Grep/,
/U2: Also check the retry path\./,
/A3: Now checking the retry path for U2\./,
])
})
}
@@ -0,0 +1,191 @@
import { expect, test } from "@playwright/test"
import type { SessionMessageAssistant, ShellInfo } from "@opencode-ai/client/promise"
import { directory, sessionID, setupTimeline } from "../performance/timeline-stability/fixture"
const shell = {
id: "sh_background",
status: "running",
command: "bun run check",
cwd: directory,
shell: "bash",
file: "/tmp/check.out",
metadata: { sessionID },
time: { started: 2 },
} satisfies ShellInfo
for (const grouped of [false, true]) {
for (const status of ["exited", "killed", "timeout"] as const) {
test(`stops ${grouped ? "grouped" : "standalone"} background shell shimmer when ${status}`, async ({
page,
}, info) => {
const message: SessionMessageAssistant = {
id: "msg_background",
type: "assistant",
agent: "build",
model: { id: "model", providerID: "provider" },
content: [shell.id, "sh_other"].map((id) => ({
type: "tool",
id: `call_${id}`,
name: "shell",
state: {
status: "completed",
input: { command: shell.command },
content: [{ type: "text", text: "Command moved to the background." }],
metadata: { shellID: id, status: "running" },
},
time: { created: 2, completed: 3 },
})),
time: { created: 2, completed: 3 },
}
if (grouped)
message.content.unshift({
type: "tool",
id: "call_read",
name: "read",
state: {
status: "completed",
input: { path: "package.json" },
content: [{ type: "text", text: "{}" }],
metadata: {},
},
time: { created: 1, completed: 2 },
})
const timeline = await setupTimeline(page, {
viewport: { width: grouped ? 390 : 1400, height: 900 },
settings: { shellToolPartsExpanded: !grouped },
sessionStatus: { [sessionID]: { type: "busy" } },
sessionMessages: [
{ id: "msg_user", type: "user", text: "Run two independent checks.", time: { created: 1 } },
message,
],
})
const state = { finished: false, requests: 0 }
await page.route("**/api/shell?*", (route) =>
route.fulfill({
json: { location: { directory }, data: [...(state.finished ? [] : [shell]), { ...shell, id: "sh_other" }] },
}),
)
await page.route("**/api/shell/*/output?*", (route) => {
const url = new URL(route.request().url())
const target = url.pathname.includes(`/${shell.id}/`)
if (target) state.requests++
const output = target && state.finished ? "Checking project\nCheck finished\n" : "Checking project\n"
const cursor = Number(url.searchParams.get("cursor") ?? 0)
const end = Math.min(output.length, cursor + 17)
return route.fulfill({
json: {
location: { directory },
data: {
output: output.slice(cursor, end),
cursor: end,
size: output.length,
truncated: false,
},
},
})
})
await page.clock.install()
await page.reload()
await timeline.transport.waitForConnection()
const group = page.locator('[data-component="collapsed-tool-group"]')
const groupTrigger = group.locator(':scope > [data-component="collapsible"] > [data-slot="collapsible-trigger"]')
if (grouped) {
await expect(group).toHaveAttribute("data-timeline-part-ids", "call_read,call_sh_background,call_sh_other")
await expect(groupTrigger).toHaveAttribute("aria-expanded", "false")
await groupTrigger.click()
}
const card = page.locator(`[data-timeline-part-id="call_${shell.id}"]`)
const shimmer = card.locator('[data-component="text-shimmer"]')
const other = page.locator('[data-timeline-part-id="call_sh_other"] [data-component="text-shimmer"]')
await expect(shimmer).toHaveAttribute("data-active", "true")
await expect(other).toHaveAttribute("data-active", "true")
if (grouped) await card.locator('[data-slot="collapsible-trigger"]').click()
await expect(card.locator('[data-slot="bash-result"]')).toHaveText("Checking project")
state.finished = true
await timeline.transport.send({
id: "evt_shell_exited",
created: 4,
type: "shell.exited",
location: { directory },
data: { id: shell.id, status, exit: status === "exited" ? 0 : 1 },
})
await expect(shimmer).toHaveAttribute("data-active", "false")
await expect(other).toHaveAttribute("data-active", "true")
await expect(card.locator('[data-slot="bash-result"]')).toHaveText("Checking project\nCheck finished")
await expect(card.locator('[data-slot="collapsible-trigger"]')).toHaveAttribute("aria-expanded", "true")
await page.locator("[data-timeline-virtual-content]").screenshot({ path: info.outputPath("shell-finished.png") })
const requests = state.requests
await page.clock.fastForward(5_000)
expect(state.requests).toBe(requests)
await page.reload()
if (grouped) await groupTrigger.click()
await expect(shimmer).toHaveAttribute("data-active", "false")
await expect(other).toHaveAttribute("data-active", "true")
})
}
}
test("shows the authoritative foreground result after streaming shell output", async ({ page }) => {
const timeline = await setupTimeline(page, {
settings: { shellToolPartsExpanded: true },
sessionMessages: [
{ id: "msg_user", type: "user", text: "Run the check.", time: { created: 1 } },
{
id: "msg_foreground",
type: "assistant",
agent: "build",
model: { id: "model", providerID: "provider" },
content: [
{
type: "tool",
id: "call_foreground",
name: "shell",
state: { status: "running", input: { command: shell.command }, metadata: { shellID: shell.id } },
time: { created: 2 },
},
],
time: { created: 2 },
},
],
})
await page.route("**/api/shell/*/output?*", (route) =>
route.fulfill({
json: {
location: { directory },
data: {
output: Number(new URL(route.request().url()).searchParams.get("cursor")) === 0 ? "Checking project\n" : "",
cursor: 17,
size: 17,
truncated: false,
},
},
}),
)
await page.reload()
await timeline.transport.waitForConnection()
const card = page.locator('[data-timeline-part-id="call_foreground"]')
const shimmer = card.locator('[data-component="text-shimmer"]')
await expect(shimmer).toHaveAttribute("data-active", "true")
await expect(card.locator('[data-slot="bash-result"]')).toHaveText("Checking project")
await timeline.transport.send({
id: "evt_foreground_complete",
created: 3,
type: "session.tool.success",
durable: { aggregateID: sessionID, seq: 0, version: 2 },
data: {
sessionID,
assistantMessageID: "msg_foreground",
id: "call_foreground",
executed: true,
content: [{ type: "text", text: "Checking project\nCheck finished\nCommand exited with code 0." }],
metadata: { status: "completed", exit: 0 },
},
})
await expect(shimmer).toHaveAttribute("data-active", "false")
await expect(card.locator('[data-slot="bash-result"]')).toHaveText(
"Checking project\nCheck finished\nCommand exited with code 0.",
)
})
@@ -1,16 +1,43 @@
import { expect, test } from "@playwright/test"
import { assistantMessage, setupTimeline, shell, userMessage } from "../performance/timeline-stability/fixture"
import {
assistantMessage,
setupTimeline,
shell,
textPart,
userMessage,
} from "../performance/timeline-stability/fixture"
test("space activates a focused timeline button instead of scrolling", async ({ page }) => {
const shellID = "prt_space_button_shell"
await setupTimeline(page, {
messages: [userMessage(), assistantMessage([shell(shellID, "completed", lines(5))])],
messages: [
userMessage(),
assistantMessage([
shell(shellID, "completed", lines(5)),
textPart(
"prt_space_following",
"Following content leaves room to focus the command away from the bottom. ".repeat(40),
),
]),
],
settings: { shellToolPartsExpanded: false },
reducedMotion: true,
seedHistory: true,
})
const scroller = page.locator(".scroll-view__viewport", { has: page.locator("[data-timeline-row]") })
const trigger = page.getByRole("button", { name: "Used Shell" })
await expect
.poll(() => scroller.evaluate((element) => element.scrollHeight - element.clientHeight))
.toBeGreaterThan(300)
await trigger.scrollIntoViewIfNeeded()
await scroller.hover()
await page.mouse.wheel(0, -100)
await expect
.poll(() => scroller.evaluate((element) => element.scrollHeight - element.clientHeight - element.scrollTop))
.toBeGreaterThan(50)
await expect(trigger).toBeInViewport()
await trigger.focus()
await expect(trigger).toBeFocused()
const before = await scroller.evaluate((element) => element.scrollTop)
await trigger.press("Space")
await expect(trigger).toHaveAttribute("aria-expanded", "true")
@@ -84,6 +84,39 @@ const assistantMessage = {
} satisfies SessionMessageInfo
test.describe("regression: session timeline local row state", () => {
test("preserves a patch file choice as new calls join its Used group", async ({ page }) => {
const events: EventPayload[] = []
const part = { ...editPart, tool: "patch" }
await mockServer(page, events, [userMessage, { ...assistantMessage, content: [toolContent(part)] }])
await configurePage(page, false)
await page.goto(sessionHref())
await expectSessionTitle(page, title)
const group = page.locator('[data-component="collapsed-tool-group"]')
const summary = group.getByRole("button", { name: "Used Patch", exact: true })
await summary.click()
await group.locator(`[data-timeline-part-id="${editPartID}"]`).evaluate((element) => {
element.setAttribute("data-disclosure-probe", "existing")
})
const wrapper = group.locator('[data-disclosure-probe="existing"]')
const trigger = wrapper.locator('[data-scope="apply-patch"] button')
await expect(trigger).toHaveAttribute("aria-expanded", "false")
await trigger.click()
await expect(trigger).toHaveAttribute("aria-expanded", "true")
const original = await wrapper.elementHandle()
for (const count of [2, 3]) {
if (count === 3) await trigger.click()
const id = `prt_patch_${count}`
events.push(...toolEvents({ ...part, id, callID: id }))
await expect(group.locator('[data-component="tag"]')).toHaveText(String(count))
await expect(group).toHaveAttribute("data-timeline-part-ids", new RegExp(`${id}$`))
await expect(trigger).toHaveAttribute("aria-expanded", String(count === 2))
await expect(summary).toHaveAttribute("aria-expanded", "true")
expect(await original!.evaluate((node) => node.isConnected)).toBe(true)
}
})
test("keeps a manually collapsed tool collapsed when later assistant content streams", async ({ page }) => {
const events: EventPayload[] = []
await mockServer(page, events)
@@ -208,19 +241,19 @@ test.describe("regression: session timeline local row state", () => {
})
})
async function configurePage(page: Page) {
await page.addInitScript(() => {
async function configurePage(page: Page, expanded = true) {
await page.addInitScript((expanded) => {
localStorage.setItem(
"settings.v3",
JSON.stringify({
general: {
editToolPartsExpanded: true,
shellToolPartsExpanded: true,
editToolPartsExpanded: expanded,
shellToolPartsExpanded: expanded,
showReasoningSummaries: true,
},
}),
)
})
}, expanded)
}
async function expectExpanded(locator: Locator, expected: boolean) {
@@ -8,21 +8,6 @@ import {
userText,
} from "../performance/timeline-stability/fixture"
test("renders completed write content", async ({ page }) => {
const id = "prt_file_projection_write"
await setupTimeline(page, {
messages: [
userMessage(),
assistantMessage([
toolPart(id, "write", "completed", { path: "src/write.ts", content: "export const written = true\n" }),
]),
],
settings: { editToolPartsExpanded: true },
})
await expect(page.locator(`[data-timeline-part-id="${id}"] [data-component="write-content"]`)).toBeVisible()
})
test("renders a completed single-file patch", async ({ page }) => {
const id = "prt_file_projection_single_patch"
await setupTimeline(page, {
@@ -1,53 +0,0 @@
import { expect, test } from "@playwright/test"
import { assistantMessage, setupTimeline, toolPart, userMessage } from "../performance/timeline-stability/fixture"
import { createTwoFilesPatch } from "diff"
test("keeps patch file disclosures independent", async ({ page }) => {
const patchID = "prt_nested_patch"
const files = [patchFile("src/a.ts", "modified"), patchFile("src/b.ts", "added"), patchFile("src/old.ts", "deleted")]
await setupTimeline(page, {
messages: [
userMessage(),
assistantMessage([
toolPart(
patchID,
"patch",
"completed",
{ patchText: "Update three files" },
{ metadata: { files } },
),
]),
],
settings: { editToolPartsExpanded: true },
})
const wrapper = page.locator(`[data-timeline-part-id="${patchID}"]`)
const modified = wrapper.locator('[data-scope="apply-patch"] [data-type="update"]')
const deleted = wrapper.locator('[data-scope="apply-patch"] [data-type="delete"]')
await expect(wrapper.locator('[data-scope="apply-patch"] [aria-expanded="false"]')).toHaveCount(3)
await deleted.getByRole("button").click()
await expect(deleted.getByRole("button")).toHaveAttribute("aria-expanded", "true")
await expect(modified.getByRole("button")).toHaveAttribute("aria-expanded", "false")
await modified.getByRole("button").click()
await expect(modified.getByRole("button")).toHaveAttribute("aria-expanded", "true")
await deleted.getByRole("button").click()
await expect(deleted.getByRole("button")).toHaveAttribute("aria-expanded", "false")
await expect(modified.getByRole("button")).toHaveAttribute("aria-expanded", "true")
})
function patchFile(file: string, status: "added" | "modified" | "deleted") {
const before = status === "added" ? "" : source(false)
const after = status === "deleted" ? "" : source(true)
return {
file,
status,
patch: createTwoFilesPatch(`a/${file}`, `b/${file}`, before, after),
additions: status === "deleted" ? 0 : 4,
deletions: status === "added" ? 0 : 3,
}
}
function source(changed: boolean) {
return Array.from({ length: 12 }, (_, index) => `export const value${index} = ${changed ? index + 1 : index}\n`).join(
"",
)
}
@@ -109,29 +109,85 @@ test("shimmers and expands a running shell command", async ({ page }) => {
await expect(tool.locator('[data-slot="bash-pre"]')).toContainText("still running")
})
test("transitions thinking and hidden reasoning through busy to idle", async ({ page }) => {
const reasoningID = "prt_reasoning_hidden"
const assistant = assistantMessage([reasoningPart(reasoningID, "## Inspecting stability")], { completed: false })
const timeline = await setupTimeline(page, {
messages: [userMessage(), assistant],
settings: { showReasoningSummaries: false },
cpuRate: 4,
for (const open of [false, true]) {
test(`keeps ${open ? "expanded" : "collapsed"} reasoning intent from Thinking through standalone shell into Used`, async ({
page,
}) => {
const reasoningID = `prt_reasoning_hidden_${open}`
const shellID = `prt_reasoning_shell_${open}`
const assistant = assistantMessage([reasoningPart(reasoningID, "## Inspecting stability")], { completed: false })
const timeline = await setupTimeline(page, {
messages: [userMessage(), assistant],
settings: { showReasoningSummaries: false },
cpuRate: 4,
})
const reasoning = page.locator(`[data-timeline-part-id="${renderedPartID(reasoningID)}"]`)
await expect(page.locator('[data-timeline-row="Thinking"]')).toBeVisible()
await expect(page.getByText("Inspecting stability", { exact: true })).toBeVisible()
const thought = reasoning.locator('[data-slot="collapsible-trigger"]')
await expect(thought).toHaveAttribute("aria-expanded", "false")
await thought.click()
await expect(thought).toHaveAttribute("aria-expanded", "true")
if (!open) await thought.click()
await expect(thought).toHaveAttribute("aria-expanded", String(open))
await timeline.send(partUpdated(shell(shellID, "running")))
const group = page.locator('[data-component="collapsed-tool-group"]')
await expect(page.locator(`[data-timeline-part-id="${shellID}"]`)).toBeVisible()
await expect(group).toHaveCount(0)
await expect(page.locator('[data-timeline-row="Thinking"]')).toHaveCount(0)
await expect(thought).toContainText("Thought")
await expect(thought).not.toContainText("Inspecting stability")
await expect(thought).toHaveAttribute("aria-expanded", String(open))
await timeline.send(partUpdated(shell(shellID, "completed", "done")))
await timeline.send(messageUpdated(completedAssistantInfo(assistant)))
await timeline.send(status("idle"))
const used = group.getByRole("button", { name: "Used Shell", exact: true })
await expect(used).toHaveAttribute("aria-expanded", "false")
await used.click()
await expect(used).toHaveAttribute("aria-expanded", "true")
await expect(group.locator(`[data-timeline-part-id="${shellID}"]`)).toBeVisible()
await expect(group.getByRole("button", { name: "Thought", exact: true })).toHaveAttribute(
"aria-expanded",
String(open),
)
await expect(group.locator('[data-component="tag"]')).toHaveText("1")
await expect(page.locator('[data-timeline-row="Thinking"]')).toHaveCount(0)
await expect(used).toHaveAttribute("aria-expanded", "true")
if (!open) await thought.click()
await expect(reasoning.getByRole("heading", { name: "Inspecting stability", exact: true })).toBeVisible()
await used.click()
await expect(used).toHaveAttribute("aria-expanded", "false")
await used.click()
await expect(reasoning.getByRole("button", { name: "Thought", exact: true })).toHaveAttribute(
"aria-expanded",
"true",
)
await expect(reasoning.getByRole("heading", { name: "Inspecting stability", exact: true })).toBeVisible()
})
await timeline.send(status("busy"), 150)
}
await expect(page.locator('[data-timeline-row="Thinking"]')).toBeVisible()
await expect(page.getByText("Inspecting stability", { exact: true })).toBeVisible()
await expect(page.locator(`[data-timeline-part-id="${reasoningID}"]`)).toHaveCount(0)
await timeline.send(partUpdated(shell("prt_reasoning_shell", "running")), 160)
await expect(page.locator('[data-timeline-row="Thinking"]')).toBeVisible()
await timeline.send(partUpdated(shell("prt_reasoning_shell", "completed", "done")), 180)
await timeline.send(messageUpdated(completedAssistantInfo(assistant)), 100)
await timeline.send(status("idle"), 300)
await expect(page.locator('[data-timeline-row="Thinking"]')).toHaveCount(0)
await expect(page.locator(`[data-timeline-part-id="${reasoningID}"]`)).toHaveCount(0)
})
for (const transition of ["reasoning-end", "idle", "retry"] as const) {
test(`stops active Thinking on ${transition} without a following tool`, async ({ page }) => {
const id = `prt_reasoning_stop_${transition}`
const text = "## Inspecting stability\n\nThe timeline is ready for the next step."
const timeline = await setupTimeline(page, {
messages: [userMessage(), assistantMessage([reasoningPart(id, text)], { completed: false })],
})
const part = page.locator(`[data-timeline-part-id="${renderedPartID(id)}"]`)
const trigger = part.locator('[data-slot="collapsible-trigger"]')
await expect(page.locator('[data-timeline-row="Thinking"]')).toBeVisible()
await expect(trigger).toHaveAttribute("aria-expanded", "false")
await timeline.send(transition === "reasoning-end" ? partUpdated(reasoningPart(id, text)) : status(transition))
await expect(trigger).toContainText("Thought")
await expect(page.locator('[data-timeline-row="Thinking"]')).toHaveCount(0)
await expect(page.locator('[data-timeline-row="Retry"]')).toHaveCount(transition === "retry" ? 1 : 0)
await trigger.click()
await expect(trigger).toHaveAttribute("aria-expanded", "true")
await expect(part.getByText("The timeline is ready for the next step.", { exact: true })).toBeVisible()
})
}
test("moves busy through retry and recovery to final idle content", async ({ page }) => {
test("does not infer Thinking from busy, retry, or recovery without reasoning", async ({ page }) => {
const assistant = assistantMessage([], { completed: false })
const timeline = await setupTimeline(page, {
messages: [
@@ -151,18 +207,17 @@ test("moves busy through retry and recovery to final idle content", async ({ pag
assistant,
],
})
await timeline.send(status("busy"), 140)
await expect(page.locator('[data-timeline-row="Thinking"]')).toBeVisible()
await expect(page.locator('[data-timeline-row="Thinking"]')).toHaveCount(0)
await expect(page.locator('[data-timeline-row="DiffSummary"]')).toHaveCount(0)
await timeline.send(status("retry"), 180)
await timeline.send(status("retry"))
await expect(page.locator('[data-timeline-row="Retry"]')).toBeVisible()
await expect(page.locator('[data-timeline-row="Thinking"]')).toHaveCount(0)
await timeline.send(stepStarted(assistant), 180)
await timeline.send(stepStarted(assistant))
await expect(page.locator('[data-timeline-row="Retry"]')).toHaveCount(0)
await expect(page.locator('[data-timeline-row="Thinking"]')).toBeVisible()
await timeline.send(partUpdated(textPart("prt_recovered", "Recovered response")), 140)
await timeline.send(messageUpdated(completedAssistantInfo(assistant)), 100)
await timeline.send(status("idle"), 350)
await expect(page.locator('[data-timeline-row="Thinking"]')).toHaveCount(0)
await timeline.send(partUpdated(textPart("prt_recovered", "Recovered response")))
await timeline.send(messageUpdated(completedAssistantInfo(assistant)))
await timeline.send(status("idle"))
await expect(page.locator('[data-timeline-row="Thinking"]')).toHaveCount(0)
await expect(page.locator(`[data-timeline-part-id="${renderedPartID("prt_recovered")}"]`)).toContainText(
"Recovered response",
@@ -5,6 +5,7 @@ import {
compactionEnded,
compactionFailed,
compactionStarted,
directory,
event,
session,
sessionID,
@@ -122,6 +123,7 @@ test("updates running compactions to failed and cancelled boundaries", async ({
await timeline.send(compactionStarted({ sessionID, reason: "auto", recent: "" }))
await timeline.send(compactionDelta({ sessionID, text: "Partial summary that should be discarded." }))
await expect(page.getByText("Partial summary that should be discarded.", { exact: true })).toBeVisible()
await timeline.send(
compactionFailed({
sessionID,
@@ -140,6 +142,9 @@ test("updates running compactions to failed and cancelled boundaries", async ({
await expect(failed).not.toContainText("Partial summary that should be discarded.")
await timeline.send(compactionStarted({ sessionID, reason: "manual", recent: "" }))
await expect(compactions).toHaveCount(2)
await timeline.send(compactionDelta({ sessionID, text: "Summary before cancellation." }))
await expect(page.getByText("Summary before cancellation.", { exact: true })).toBeVisible()
await timeline.send(
compactionFailed({
sessionID,
@@ -152,88 +157,7 @@ test("updates running compactions to failed and cancelled boundaries", async ({
const cancelled = compactions.filter({ hasNotText: "The provider rejected the summary." })
await expect(cancelled.getByText("Session compacted", { exact: true })).toBeVisible()
await expect(cancelled).not.toContainText("Cancellation detail should stay hidden.")
})
test("shows a delegating row while subagent input streams", async ({ page }) => {
await setupTimeline(page, {
sessionMessages: [
user,
{
...assistant(false),
content: [
{
type: "tool",
id: "call_subagent",
name: "subagent",
state: { status: "streaming", input: "" },
time: { created: 2 },
},
],
},
],
})
const delegating = page.locator('[data-component="task-tool-delegating"]')
await expect(delegating).toBeVisible()
const shimmer = delegating.locator('[data-component="text-shimmer"]')
await expect(shimmer).toHaveAttribute("aria-label", "Delegating agent...")
await expect(shimmer).toHaveCSS("line-height", "16px")
const icon = delegating.locator('[data-slot="icon-svg"]')
await expect(icon.locator('use[href="#opencode-v2-icon-subagent"]')).toBeVisible()
await expect(icon).toHaveCSS("color", "rgb(174, 174, 174)")
await expect(page.locator('[data-component="task-tool-card"]')).toHaveCount(0)
await expect(page.locator('[data-timeline-row="Thinking"]')).toHaveCount(0)
})
test("renders the moved location notice in its compact timeline style", async ({ page }) => {
const directory = `/Users/usrnk1/Developer/opencode/${"nested-directory/".repeat(24)}session`
await page.setViewportSize({ width: 480, height: 720 })
await setupTimeline(page, {
sessionMessages: [
user,
{
id: "msg_location",
type: "location-switched",
location: { directory },
time: { created: 2 },
},
],
})
const notice = page.locator('[data-slot="session-timeline-notice"][data-type="location-switched"]')
const label = notice.locator('[data-slot="session-timeline-notice-label"]')
const value = notice.locator('[data-slot="session-timeline-notice-value"]')
const tooltipTrigger = notice.locator('[data-component="tooltip-v2-trigger"]')
await expect(label).toHaveText("Moved to")
await expect(value).toHaveText(directory)
await expect(notice).not.toContainText("·")
await expect(notice.locator("svg")).toHaveCount(0)
await expect(notice).toHaveCSS("height", "28px")
await expect(notice).toHaveCSS("gap", "8px")
await expect(notice).toHaveCSS("padding-top", "4px")
await expect(notice).toHaveCSS("padding-bottom", "4px")
await expect(label).toHaveCSS("font-size", "13px")
await expect(label).toHaveCSS("font-weight", "530")
await expect(label).toHaveCSS("line-height", "16px")
await expect(label).toHaveCSS("color", "rgb(128, 128, 128)")
await expect(value).toHaveCSS("font-size", "13px")
await expect(value).toHaveCSS("font-weight", "440")
await expect(value).toHaveCSS("line-height", "16px")
await expect(value).toHaveCSS("color", "rgb(128, 128, 128)")
await expect(value).toHaveCSS("text-overflow", "ellipsis")
await expect(value).toHaveCSS("white-space", "nowrap")
await expect(value).toHaveAttribute("dir", "ltr")
await expect.poll(() => value.evaluate((element) => element.scrollWidth > element.clientWidth)).toBe(true)
const tooltip = page.getByText("Session working directory changed", { exact: true })
await label.hover()
await expect(tooltip).toBeVisible()
await page.mouse.move(0, 0)
await expect(tooltip).toBeHidden()
await tooltipTrigger.focus()
await expect(tooltipTrigger).toBeFocused()
await expect(tooltip).toBeVisible()
await expect(cancelled).not.toContainText("Summary before cancellation.")
})
test("moves blocking work to the background with Ctrl+B", async ({ page }) => {
@@ -271,11 +195,6 @@ test("moves blocking work to the background with Ctrl+B", async ({ page }) => {
await request
})
test("waits for completion before labeling requested background work", async ({ page }) => {
await setupTimeline(page, { sessionMessages: [user, assistant(false, true, undefined, true)] })
await expect(page.locator('[data-component="task-tool-card"]')).not.toContainText("(background)")
})
test("navigates from a running subagent card and hides background controls in the child", async ({ page }) => {
const childID = "ses_running_child"
await setupTimeline(page, {
@@ -290,6 +209,56 @@ test("navigates from a running subagent card and hides background controls in th
await expect(page.getByText(/move running work to the background/i)).toHaveCount(0)
})
for (const name of ["shell", "subagent"] as const) {
test(`keeps the background shortcut available for a grouped running ${name}`, async ({ page }) => {
const message = assistant(false, true)
await setupTimeline(page, {
sessionMessages: [
user,
{
...message,
content: [
{
type: "tool",
id: "call_read",
name: "read",
state: {
status: "completed",
input: { path: "src/example.ts" },
content: [{ type: "text", text: "export const example = true" }],
metadata: {},
},
time: { created: 1, completed: 2 },
},
{
type: "tool",
id: "call_running",
name,
state: {
status: "running",
input:
name === "shell" ? { command: "echo checking" } : { agent: "general", description: "Inspect code" },
metadata: {},
},
time: { created: 3 },
},
],
},
],
})
const group = page.locator('[data-timeline-part-ids="call_read,call_running"]')
await expect(group).toBeVisible()
await expect(group.locator('[data-slot="collapsible-trigger"]')).toHaveAttribute("aria-expanded", "false")
await expect(page.locator('[data-component="session-background-hint"]')).toBeVisible()
const request = page.waitForRequest(
(request) =>
request.method() === "POST" && new URL(request.url()).pathname === `/api/session/${sessionID}/background`,
)
await page.keyboard.press("Control+b")
await request
})
}
test("shows a badge for active background work", async ({ page }) => {
const childID = "ses_background_child"
await setupTimeline(page, {
@@ -380,6 +349,24 @@ test("separates blocking and already-backgrounded work into two rows", async ({
},
})
await timeline.transport.send({
id: "evt_background_shell_created",
created: 3,
type: "shell.created",
location: { directory },
data: {
info: {
id: "shell_backgrounded",
status: "running",
command: "sleep 120",
cwd: directory,
shell: "bash",
file: "/tmp/background.out",
metadata: { sessionID },
time: { started: 2 },
},
},
})
const backgroundCard = page.locator('[data-timeline-part-id="call_backgrounded"]')
await expect(page.getByText(/move running work to the background/i)).toBeVisible()
await page.getByRole("button", { name: "Session details" }).click()
@@ -7,86 +7,9 @@ import {
toolPart,
userMessage,
userText,
type PartSeed,
} from "../performance/timeline-stability/fixture"
test.describe("session timeline projection", () => {
test("renders every admitted tool family and hides timeline-only exclusions", async ({ page }) => {
const parts = [
toolPart("prt_01_read", "read", "completed", { path: "src/a.ts" }),
toolPart("prt_02_glob", "glob", "completed", { path: ".", pattern: "**/*.ts" }),
toolPart("prt_03_grep", "grep", "completed", { path: ".", pattern: "value" }),
toolPart("prt_04_list", "list", "completed", { path: "src" }),
toolPart("prt_webfetch", "webfetch", "completed", { url: "https://example.com" }),
toolPart(
"prt_websearch",
"websearch",
"completed",
{ query: "timeline stability" },
{ output: "https://example.com/result" },
),
toolPart("prt_task", "subagent", "completed", {
description: "Inspect timeline",
agent: "explore",
prompt: "Inspect the timeline implementation.",
}),
toolPart(
"prt_bash",
"shell",
"completed",
{ command: "printf stable" },
{ output: "stable", title: "printf stable" },
),
editPart("prt_edit"),
toolPart("prt_write", "write", "completed", { path: "src/new.ts", content: "export const stable = true\n" }),
patchPart("prt_patch"),
toolPart("prt_todo", "todowrite", "completed", { todos: [{ content: "Hidden", status: "pending" }] }),
toolPart(
"prt_question",
"question",
"completed",
{ questions: [{ question: "Keep stable?", header: "Stability", options: [] }] },
{ metadata: { answers: [["Yes"]] } },
),
toolPart("prt_skill", "skill", "completed", { name: "stability" }),
toolPart("prt_custom", "custom_mcp_tool", "completed", { target: "timeline", count: 2 }),
]
await setupTimeline(page, { messages: [userMessage(), assistantMessage(parts)] })
const first = page.locator(
'[data-timeline-part-ids="prt_01_read,prt_02_glob,prt_03_grep,prt_04_list,prt_webfetch,prt_websearch,prt_task,prt_bash,prt_edit,prt_write,prt_patch"]',
)
const second = page.locator('[data-timeline-part-ids="prt_skill,prt_custom"]')
await expect(first).toBeVisible()
await expect(second).toBeVisible()
await first.getByRole("button").click()
await second.getByRole("button").click()
for (const id of [
"prt_webfetch",
"prt_websearch",
"prt_task",
"prt_bash",
"prt_edit",
"prt_write",
"prt_patch",
"prt_question",
"prt_skill",
"prt_custom",
]) {
await expect(page.locator(`[data-timeline-part-id="${id}"]`).first(), id).toBeVisible()
}
const patch = page.locator('[data-timeline-part-id="prt_patch"]')
await expect(patch.getByText("1 file", { exact: true })).toBeVisible()
await expect(patch.getByRole("button", { name: "Patch 1 file", exact: true })).toHaveCount(0)
await expect(patch.getByRole("button")).toHaveCount(1)
await expect(patch.locator('[data-scope="apply-patch"] button[aria-expanded="false"]')).toHaveCount(1)
await expect(patch.locator('[data-slot="message-part-title-filename"]')).toHaveCount(0)
await expect(patch.locator('[data-slot="message-part-actions"]')).toHaveCount(0)
const edit = page.locator('[data-timeline-part-id="prt_edit"]')
await expect(edit).toContainText("Edit")
await expect(page.locator('[data-timeline-part-id="prt_todo"]')).toHaveCount(0)
})
test("combines adjacent patch calls and repeated files into one group", async ({ page }) => {
const first = "prt_patch_first"
const second = "prt_patch_second"
@@ -158,43 +81,6 @@ test.describe("session timeline projection", () => {
await expect(page.locator(`[data-timeline-part-id="${first}"], [data-timeline-part-id="${second}"]`)).toHaveCount(0)
})
test("combines adjacent edit calls and repeated files into one group", async ({ page }) => {
const first = "prt_edit_first"
const second = "prt_edit_second"
await setupTimeline(page, {
messages: [
userMessage(),
assistantMessage([
toolPart(
first,
"edit",
"completed",
{ path: "src/first.ts", oldString: "one", newString: "two" },
{
metadata: { files: [patchFile("src/first.ts", "modified")] },
},
),
toolPart(
second,
"edit",
"completed",
{ path: "src/first.ts", oldString: "two", newString: "three" },
{
metadata: { files: [patchFile("src/first.ts", "modified")] },
},
),
]),
],
settings: { editToolPartsExpanded: true },
})
const group = page.locator(`[data-timeline-part-ids="${first},${second}"]`)
await expect(group.locator('[data-slot="basic-tool-tool-title"]')).toContainText("Edit")
await expect(group.getByText("1 file", { exact: true })).toBeVisible()
await expect(group.locator('[data-slot="apply-patch-filename"]')).toHaveText(["first.ts"])
await expect(group.locator('[data-scope="apply-patch"] button')).toHaveAttribute("aria-expanded", "true")
})
test("projects gaps, dividers, assistant parts, and errors together", async ({ page }) => {
const firstUser = userMessage(
[
@@ -236,25 +122,6 @@ test.describe("session timeline projection", () => {
await expect(page.locator('[data-timeline-row="TurnGap"]')).toBeVisible()
})
test("renders interruption independently when the turn is not compacted", async ({ page }) => {
const user = userMessage()
const before = assistantMessage([{ id: "prt_before", type: "text", text: "Before" }], {
id: "msg_1001_before",
error: { type: "MessageAbortedError", message: "Stopped" },
})
const after = assistantMessage([{ id: "prt_after", type: "text", text: "After" }], {
id: "msg_1002_after",
created: 1700000003000,
})
await setupTimeline(page, { messages: [user, before, after] })
await expect(page.getByText("Interrupted", { exact: true })).toBeVisible()
const rows = await page
.locator('[data-timeline-row="AssistantPart"], [data-timeline-row="TurnDivider"]')
.evaluateAll((elements) => elements.map((element) => element.getAttribute("data-timeline-row")))
expect(rows).toEqual(["AssistantPart", "TurnDivider", "AssistantPart"])
})
test("renders aliased and long custom model notices", async ({ page }) => {
const shortName = "GPT-5.4 nano"
const longName = "Company Gateway Extra Long Context Model for Narrow Timeline Layouts"
@@ -291,77 +158,8 @@ test.describe("session timeline projection", () => {
await expect(longNotice.locator("[title]")).toHaveAttribute("title", `Switched to ${longName}`)
await expect.poll(() => longNotice.evaluate((element) => element.scrollWidth <= element.clientWidth)).toBe(true)
})
test("renders user image, file attachment, file reference, and agent reference", async ({ page }) => {
const text = "Use @explore with @src/a.ts and inspect the attachments"
const parts: PartSeed<"user">[] = [
userText(text, { id: "prt_user_rich" }),
{
id: "prt_user_image",
type: "file",
mime: "image/png",
filename: "pixel.png",
url: "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg==",
},
{
id: "prt_user_attachment",
type: "file",
mime: "application/json",
filename: "tsconfig.json",
url: "data:application/json;base64,e30=",
},
{
id: "prt_user_reference",
type: "file",
mime: "text/plain",
filename: "a.ts",
url: "src/a.ts",
source: { type: "file", path: "src/a.ts", text: { value: "@src/a.ts", start: 18, end: 27 } },
},
{
id: "prt_user_agent",
type: "agent",
name: "explore",
source: { value: "@explore", start: 4, end: 12 },
},
]
await setupTimeline(page, { messages: [userMessage(parts), assistantMessage()] })
await expect(page.getByAltText("pixel.png")).toBeVisible()
await expect(page.getByText("tsconfig.json")).toBeVisible()
await expect(page.getByText("@src/a.ts", { exact: true })).toBeVisible()
await expect(page.getByText("@explore", { exact: true })).toBeVisible()
})
})
function editPart(id: string) {
return toolPart(
id,
"edit",
"completed",
{ path: "src/a.ts", oldString: "export const value = 1", newString: "export const value = 2" },
{
metadata: {
files: [patchFile("src/a.ts", "modified")],
},
},
)
}
function patchPart(id: string) {
return toolPart(
id,
"patch",
"completed",
{ patchText: "Update the projected files" },
{
metadata: {
files: [patchFile("src/a.ts", "modified")],
},
},
)
}
function patchFile(file: string, status: "added" | "modified" | "deleted") {
return {
file,
@@ -4,89 +4,144 @@ import {
assistantMessage,
reasoningPart,
setupTimeline,
status,
textPart,
toolPart,
userMessage,
} from "../performance/timeline-stability/fixture"
const profiles = [
{ name: "summaries off no reasoning", summaries: false, reasoning: "", other: false, thinking: true, body: false },
{
name: "summaries off reasoning heading",
summaries: false,
reasoning: "## Inspecting stability",
other: false,
thinking: true,
body: false,
},
{
name: "summaries off with visible tool",
summaries: false,
reasoning: "## Inspecting stability",
other: true,
thinking: true,
body: false,
},
{ name: "summaries on no content", summaries: true, reasoning: "", other: false, thinking: true, body: false },
{
name: "summaries on blank reasoning",
summaries: true,
reasoning: " ",
other: false,
thinking: true,
body: false,
},
{
name: "summaries on visible reasoning",
summaries: true,
reasoning: "## Inspecting stability",
other: false,
thinking: false,
body: true,
},
{
name: "summaries on visible tool no reasoning",
summaries: true,
reasoning: "",
other: true,
thinking: false,
body: false,
},
] as const
for (const profile of profiles) {
test(`projects busy reasoning profile ${profile.name}`, async ({ page }) => {
const reasoningID = `prt_reasoning_matrix_${profiles.indexOf(profile)}`
const parts = [
...(profile.reasoning ? [reasoningPart(reasoningID, profile.reasoning)] : []),
...(profile.other
? [toolPart(`prt_reasoning_tool_${profiles.indexOf(profile)}`, "skill", "running", { name: "inspect" })]
: []),
]
const timeline = await setupTimeline(page, {
messages: [userMessage(), assistantMessage(parts, { completed: false })],
settings: { showReasoningSummaries: profile.summaries },
})
await timeline.send(status("busy"), 150)
await expect(page.locator('[data-timeline-row="Thinking"]')).toHaveCount(profile.thinking ? 1 : 0)
await expect(page.locator(`[data-timeline-part-id="${assistantID}:reasoning:0"]`)).toHaveCount(profile.body ? 1 : 0)
if (!profile.summaries && profile.reasoning.trim()) {
await expect(page.getByText("Inspecting stability", { exact: true })).toBeVisible()
}
test("changes live reasoning through Settings and persists Hidden, Compact, and Full", async ({ page }) => {
await setupTimeline(page, {
messages: [
userMessage(),
assistantMessage(
[
reasoningPart(
"prt_reasoning_settings",
"## Inspecting stability\n\nThe selected mode controls these details.",
),
],
{ completed: false },
),
],
})
const part = page.locator(`[data-timeline-part-id="${assistantID}:reasoning:0"]`)
await expect(part.getByRole("button")).toHaveAttribute("aria-expanded", "false")
const settings = page.getByTestId("settings-screen")
const select = settings.locator('[data-action="settings-reasoning-mode"] [data-component="select-v2"]')
for (const label of ["Full", "Hidden", "Compact"] as const) {
await page.keyboard.press("Control+,")
await expect(settings.getByText("Model reasoning", { exact: true })).toBeVisible()
await expect(select).toHaveAttribute("aria-expanded", "false")
await select.click()
await expect(page.getByRole("listbox").getByRole("option")).toHaveText(["Hidden", "Compact", "Full"])
await page.getByRole("option", { name: label, exact: true }).click()
await expect(select).toHaveText(label)
await expect(select).toHaveAttribute("aria-expanded", "false")
await expect
.poll(() => page.evaluate(() => JSON.parse(localStorage.getItem("settings.v3") ?? "{}").general?.reasoningMode))
.toBe(label.toLowerCase())
await settings.getByRole("button", { name: "Back to app", exact: true }).click()
await expect(settings).toBeHidden()
await expect(page.locator('[data-timeline-row="Thinking"]')).toHaveCount(label === "Hidden" ? 0 : 1)
await expect(part).toHaveCount(label === "Hidden" ? 0 : 1)
if (label === "Hidden") {
await expect(page.getByText("The selected mode controls these details.", { exact: true })).toBeHidden()
continue
}
await expect(part.getByRole("button")).toHaveAttribute("aria-expanded", String(label === "Full"))
if (label === "Full")
await expect(part.getByText("The selected mode controls these details.", { exact: true })).toBeVisible()
if (label === "Compact") {
await expect(part.getByRole("button")).toContainText("Inspecting stability")
await expect(part.getByText("The selected mode controls these details.", { exact: true })).toBeHidden()
}
}
await page.keyboard.press("Control+,")
await expect(select).toHaveText("Compact")
})
// The persisted boolean migrates to compact (false) or full (true).
for (const summaries of [false, true]) {
for (const profile of ["none", "blank", "heading", "tool", "text"] as const) {
test(`projects legacy ${summaries ? "full" : "compact"} reasoning with ${profile}`, async ({ page }) => {
await setupTimeline(page, {
messages: [
userMessage(),
assistantMessage(
[
...(profile === "none"
? []
: [
reasoningPart(
`prt_reasoning_${summaries}_${profile}`,
profile === "blank"
? " "
: "## Inspecting stability\n\nI will inspect the timeline before changing its state.",
),
]),
...(profile === "tool"
? [toolPart(`prt_reasoning_tool_${summaries}`, "skill", "running", { name: "inspect" })]
: []),
...(profile === "text" ? [textPart(`prt_reasoning_text_${summaries}`, "The timeline is stable.")] : []),
],
{ completed: false },
),
],
settings: { showReasoningSummaries: summaries },
})
const part = page.locator(`[data-timeline-part-id="${assistantID}:reasoning:0"]`)
await expect(page.locator('[data-timeline-row="Thinking"]')).toHaveCount(
profile === "blank" || profile === "heading" ? 1 : 0,
)
if (profile === "none") {
await expect(part).toHaveCount(0)
return
}
if (profile === "blank") {
await expect(part).toContainText("Thinking")
await expect(part.getByRole("heading")).toHaveCount(0)
return
}
if (profile === "tool") {
const group = page.locator('[data-component="collapsed-tool-group"]')
const used = group.locator(':scope > [data-component="collapsible"] > [data-slot="collapsible-trigger"]')
await expect(used).toContainText("UsedSkill")
await expect(used).toHaveAttribute("aria-expanded", "false")
await expect(page.getByText("Inspecting stability", { exact: true })).toBeHidden()
await expect(group.locator('[data-component="tag"]')).toHaveText("1")
await used.click()
await expect(used).toHaveAttribute("aria-expanded", "true")
await expect(group.locator(`[data-timeline-part-id="prt_reasoning_tool_${summaries}"]`)).toBeVisible()
await expect(group.locator('[data-component="reasoning-part"]')).toHaveCount(1)
}
if (profile === "text") await expect(page.getByText("The timeline is stable.", { exact: true })).toBeVisible()
const trigger = part.locator('[data-slot="collapsible-trigger"]')
const body = part.getByText("I will inspect the timeline before changing its state.", { exact: true })
await expect(trigger).toContainText(profile === "heading" ? "Thinking" : "Thought")
await expect(trigger).toHaveAttribute("aria-expanded", String(summaries))
if (!summaries) {
await expect(body).toBeHidden()
if (profile === "heading") await expect(trigger).toContainText("Inspecting stability")
await trigger.click()
await expect(trigger).toHaveAttribute("aria-expanded", "true")
}
await expect(body).toBeVisible()
await trigger.click()
await expect(trigger).toHaveAttribute("aria-expanded", "false")
await expect(body).toBeHidden()
if (profile !== "heading") await expect(trigger).not.toContainText("Inspecting stability")
})
}
}
test("does not infer reasoning visibility from provider identity", async ({ page }) => {
const timeline = await setupTimeline(page, {
await setupTimeline(page, {
messages: [
userMessage(),
assistantMessage([textPart("prt_provider_text", "No reasoning payload")], { completed: false }),
],
settings: { showReasoningSummaries: true },
})
await timeline.send(status("busy"), 150)
await expect(page.locator('[data-timeline-row="Thinking"]')).toHaveCount(0)
await expect(page.locator('[data-timeline-part-id*="reasoning"]')).toHaveCount(0)
@@ -1,4 +1,5 @@
import { expect, test } from "@playwright/test"
import { createTwoFilesPatch } from "diff"
import {
assistantMessage,
completedAssistantInfo,
@@ -7,33 +8,25 @@ import {
renderedPartID,
setupTimeline,
shell,
toolPart,
status,
textPart,
toolPart,
userMessage,
} from "../performance/timeline-stability/fixture"
test("groups every collapsed tool until visible text separates the stack", async ({ page }) => {
const parts = [
toolPart("prt_boundary_01_read", "read", "completed", { path: "src/a.ts" }),
textPart("prt_boundary_02_text", "Boundary text"),
toolPart("prt_boundary_03_glob", "glob", "completed", { path: ".", pattern: "**/*.ts" }),
toolPart("prt_boundary_04_grep", "grep", "completed", { path: ".", pattern: "stable" }),
shell("prt_boundary_05_shell", "completed", "done"),
toolPart("prt_boundary_06_list", "list", "completed", { path: "src" }),
]
await setupTimeline(page, { messages: [userMessage(), assistantMessage(parts)] })
test("reducer-hardening: converges when idle arrives before final part and message completion", async ({ page }) => {
const textID = "prt_event_order_text"
const assistant = assistantMessage([textPart(textID, "Partial")], { completed: false })
const timeline = await setupTimeline(page, { messages: [userMessage(), assistant] })
await timeline.send(status("busy"), 100)
await timeline.send(status("idle"), 100)
await timeline.send(partUpdated(textPart(textID, "Final after early idle")), 120)
await timeline.send(messageUpdated(completedAssistantInfo(assistant)), 250)
await expect(page.locator('[data-timeline-part-ids="prt_boundary_01_read"]')).toBeVisible()
const group = page.locator(
'[data-timeline-part-ids="prt_boundary_03_glob,prt_boundary_04_grep,prt_boundary_05_shell,prt_boundary_06_list"]',
await expect(page.locator('[data-timeline-row="Thinking"]')).toHaveCount(0)
await expect(page.locator(`[data-timeline-part-id="${renderedPartID(textID)}"]`)).toContainText(
"Final after early idle",
)
await expect(group).toBeVisible()
await expect(group.getByRole("button")).toHaveAccessibleName("Used Glob, Grep, Shell, List")
await expect(group.locator('[data-component="tag"]')).toHaveText("4")
await expect(page.locator('[data-timeline-row="AssistantPart"]')).toHaveCount(3)
await expect(page.locator('[data-timeline-spacing="content"]')).toHaveCount(2)
await expect(page.locator('[data-timeline-spacing="content"]').nth(0)).toHaveCSS("padding-top", "16px")
})
test("expands a mixed collapsed tool stack without expanding its individual calls", async ({ page }) => {
@@ -52,7 +45,7 @@ test("expands a mixed collapsed tool stack without expanding its individual call
const group = page.locator(
'[data-timeline-part-ids="prt_stack_shell_1,prt_stack_explore,prt_stack_patch,prt_stack_shell_2"]',
)
const summary = group.getByRole("button", { name: "Used Shell, Explore, Patch" })
const summary = group.getByRole("button", { name: "Used Shell, Agent, Patch" })
await expect(summary).toHaveAttribute("aria-expanded", "false")
await expect(summary).toHaveCSS("height", "28px")
await expect(group.locator('[data-component="tag"]')).toHaveText("4")
@@ -86,6 +79,61 @@ test("leaves tools expanded by settings outside the collapsed stack", async ({ p
await expect(page.locator('[data-timeline-spacing="tool"]')).toHaveCSS("padding-top", "8px")
})
test("combines follow-up patches into one three-file stack inside Used", async ({ page }) => {
const file = (path: string, before: number, after: number) => ({
file: path,
status: "modified",
additions: 1,
deletions: 1,
patch: createTwoFilesPatch(
path,
path,
`export const value = ${before}\n`,
`export const value = ${after}\n`,
"",
"",
{ context: Infinity },
),
})
const timeline = await setupTimeline(page, {
messages: [
userMessage(),
assistantMessage([
shell("patch_shell", "completed"),
toolPart(
"patch_first",
"patch",
"completed",
{},
{
metadata: { files: [file("src/a.ts", 0, 1), file("src/b.ts", 0, 1)] },
},
),
]),
],
})
const group = page.locator('[data-component="collapsed-tool-group"]')
await group.getByRole("button", { name: "Used Shell, Patch", exact: true }).click()
await expect(group.getByText("2 files", { exact: true })).toBeVisible()
await timeline.send(
partUpdated(
toolPart(
"patch_next",
"patch",
"completed",
{},
{
metadata: { files: [file("src/a.ts", 1, 2), file("src/c.ts", 0, 1)] },
},
),
),
)
await expect(group.locator('[data-component="tag"]')).toHaveText("3")
await expect(group.locator('[data-component="apply-patch-tool"]')).toHaveCount(1)
await expect(group.getByText("3 files", { exact: true })).toBeVisible()
await expect(group.locator('[data-slot="apply-patch-filename"]')).toHaveText(["a.ts", "b.ts", "c.ts"])
})
test("keeps failed search calls and their error cards inside the collapsed stack", async ({ page }) => {
const parts = [
toolPart(
@@ -132,18 +180,3 @@ test("keeps failed search calls and their error cards inside the collapsed stack
"Search timed out after 30 seconds",
)
})
test("reducer-hardening: converges when idle arrives before final part and message completion", async ({ page }) => {
const textID = "prt_event_order_text"
const assistant = assistantMessage([textPart(textID, "Partial")], { completed: false })
const timeline = await setupTimeline(page, { messages: [userMessage(), assistant] })
await timeline.send(status("busy"), 100)
await timeline.send(status("idle"), 100)
await timeline.send(partUpdated(textPart(textID, "Final after early idle")), 120)
await timeline.send(messageUpdated(completedAssistantInfo(assistant)), 250)
await expect(page.locator('[data-timeline-row="Thinking"]')).toHaveCount(0)
await expect(page.locator(`[data-timeline-part-id="${renderedPartID(textID)}"]`)).toContainText(
"Final after early idle",
)
})
@@ -7,31 +7,6 @@ import {
userMessage,
} from "../performance/timeline-stability/fixture"
test("renders every tool error outcome without leaking hidden tools", async ({ page }) => {
const ordinary = ["shell", "edit", "write", "patch", "webfetch", "websearch", "subagent", "skill", "mcp_probe"]
const parts = ordinary.map((tool, index) =>
toolPart(`prt_error_${index}`, tool, "error", errorInput(tool), { error: `${tool} failed visibly` }),
)
parts.push(
toolPart("prt_question_dismissed", "question", "error", questionInput(), {
error: "The user dismissed this question",
}),
toolPart("prt_question_error", "question", "error", questionInput(), { error: "Question transport failed" }),
toolPart("prt_todo_error", "todowrite", "error", { todos: [] }, { error: "Hidden todo failure" }),
)
await setupTimeline(page, { messages: [userMessage(), assistantMessage(parts)] })
const group = page.locator(`[data-timeline-part-ids="${ordinary.map((_, index) => `prt_error_${index}`).join(",")}"]`)
await expect(group.locator('[data-component="tag"]')).toHaveText(String(ordinary.length))
await group.getByRole("button").click()
await expect(page.locator('[data-kind="tool-error-card"]')).toHaveCount(ordinary.length + 1)
await expect(page.getByText(/dismissed/i)).toBeVisible()
await expect(page.locator('[data-timeline-part-id="prt_todo_error"]')).toHaveCount(0)
for (let index = 0; index < ordinary.length; index++) {
await expect(page.locator(`[data-timeline-part-id="prt_error_${index}"]`)).toBeVisible()
}
})
test("transitions shell and question through running error outcomes", async ({ page }) => {
const shellID = "prt_transition_error_shell"
const questionID = "prt_transition_error_question"
@@ -138,62 +113,6 @@ test("preserves surviving grouped patch state when its first patch fails", async
.toBeGreaterThanOrEqual(-0.5)
})
test("labels all web search provider variants", async ({ page }) => {
const parts = [
toolPart(
"prt_search_parallel",
"websearch",
"completed",
{ query: "parallel" },
{ metadata: { provider: "parallel" } },
),
toolPart("prt_search_exa", "websearch", "completed", { query: "exa" }, { metadata: { provider: "exa" } }),
toolPart("prt_search_generic", "websearch", "completed", { query: "generic" }),
]
await setupTimeline(page, { messages: [userMessage(), assistantMessage(parts)] })
await page.getByRole("button", { name: "Used Parallel Web Search, Exa Web Search, Web Search" }).click()
const tools = page.locator('[data-component="context-tool-group-list"]')
await expect(tools.getByRole("button", { name: /Parallel Web Search/ })).toBeVisible()
await expect(tools.getByRole("button", { name: /Exa Web Search/ })).toBeVisible()
await expect(tools.getByRole("button", { name: /^Web Search/ })).toBeVisible()
})
test("labels completed searches with result counts", async ({ page }) => {
const glob = "prt_glob_count"
const grep = "prt_grep_count"
await setupTimeline(page, {
messages: [
userMessage(),
assistantMessage([
toolPart(glob, "glob", "completed", { path: ".", pattern: "**/*.ts" }, { metadata: { count: 1 } }),
toolPart(grep, "grep", "completed", { path: ".", pattern: "value" }, { metadata: { matches: 12 } }),
]),
],
})
const group = page.locator(`[data-timeline-part-ids="${glob},${grep}"]`)
await group.locator('[data-slot="collapsible-trigger"]').click()
const rows = group.locator('[data-component="context-tool-group-list"] [data-component="tool-trigger"]')
await expect(rows.filter({ hasText: "Glob" })).toContainText("(1 match)")
await expect(rows.filter({ hasText: "Grep" })).toContainText("(12 matches)")
})
test("labels read tools from their path input", async ({ page }) => {
const id = "prt_read_path"
await setupTimeline(page, {
messages: [userMessage(), assistantMessage([toolPart(id, "read", "completed", { path: "src/a.ts" })])],
})
const group = page.locator(`[data-timeline-part-ids="${id}"]`)
await group.locator('[data-slot="collapsible-trigger"]').click()
await expect(
group
.locator('[data-component="context-tool-group-list"] [data-component="tool-trigger"]')
.filter({ hasText: "Read" }),
).toContainText("a.ts")
})
test("groups instruction files loaded by the same read", async ({ page }) => {
const id = "prt_read_instructions"
await setupTimeline(page, {
@@ -221,36 +140,6 @@ test("groups instruction files loaded by the same read", async ({ page }) => {
await expect(loaded.locator('[data-slot="tool-loaded-kind"]')).toHaveCount(0)
})
test("labels skill tools from IDs and result metadata", async ({ page }) => {
const pending = "prt_skill_id"
const completed = "prt_skill_name"
await setupTimeline(page, {
messages: [
userMessage(),
assistantMessage([
toolPart(pending, "skill", "running", { id: "frontend-design" }),
toolPart(completed, "skill", "completed", { id: "opencode" }, { metadata: { name: "OpenCode" } }),
]),
],
})
const group = page.locator(`[data-timeline-part-ids="${pending},${completed}"]`)
await expect(group.getByRole("button")).toHaveAccessibleName("Used Skill")
await expect(group.locator('[data-component="tag"]')).toHaveText("2")
await group.getByRole("button").click()
const loaded = group.locator('[data-component="tool-loaded-item"]')
await expect(loaded).toHaveCount(1)
await expect(loaded).toHaveAttribute("aria-label", "Loaded frontend-design, OpenCode skills")
await expect(loaded).toHaveCSS("line-height", "16px")
await expect(loaded.locator('[data-slot="tool-loaded-label"]')).toHaveText("Loaded")
await expect(loaded.locator('[data-slot="tool-loaded-kind"]')).toHaveText("skills")
const names = loaded.locator('[data-component="text-shimmer"]')
await expect(names).toHaveCount(2)
await expect(names.nth(0)).toHaveAttribute("aria-label", "frontend-design")
await expect(names.nth(1)).toHaveAttribute("aria-label", "OpenCode")
})
test("groups only consecutive successful skill tools", async ({ page }) => {
const parts = [
toolPart("prt_skill_first", "skill", "completed", { id: "ocpr" }),
@@ -273,14 +162,3 @@ test("groups only consecutive successful skill tools", async ({ page }) => {
function questionInput() {
return { questions: [{ header: "Stability", question: "Keep it stable?", options: [] }] }
}
function errorInput(tool: string) {
if (tool === "shell") return { command: "exit 1" }
if (["edit", "write"].includes(tool)) return { path: "src/error.ts", content: "" }
if (tool === "patch") return { patchText: "Update src/error.ts" }
if (tool === "webfetch") return { url: "https://example.com" }
if (tool === "websearch") return { query: "failure" }
if (tool === "subagent") return { description: "Fail subagent", agent: "explore", prompt: "Inspect the failure." }
if (tool === "skill") return { name: "failure" }
return { target: "failure" }
}
@@ -0,0 +1,121 @@
import { expect, test } from "@playwright/test"
import { mockOpenCodeServer } from "../utils/mock-server"
const directory = "C:/Projects/settings-demo"
const sandboxes = Array.from({ length: 12 }, (_, index) => `${directory}/workspace-${index + 1}`)
test.use({ viewport: { width: 1440, height: 1000 }, colorScheme: "dark" })
test.beforeEach(async ({ page }) => {
await mockOpenCodeServer(page, {
directory,
project: {
id: "proj_settings_demo",
canonical: directory,
name: "Settings demo",
vcs: "git",
time: { created: 1700000000000, updated: 1700000000000 },
sandboxes,
},
provider: { all: [], connected: [], default: {} },
sessions: sandboxes.map((directory, index) => ({
id: `ses_settings_${index + 1}`,
title: `Workspace ${index + 1} session`,
directory,
projectID: "proj_settings_demo",
time: { created: 1700000000000, updated: 1700000000000 },
})),
pageMessages: () => ({ items: [] }),
})
await page.goto("/")
await page.getByRole("button", { name: "Settings", exact: true }).click()
await expect(page.getByTestId("settings-screen").getByRole("tab", { name: "Preferences" })).toBeVisible()
})
test("workspaces opens without waiting for inventory or sessions", async ({ page }) => {
const inventory = Promise.withResolvers<void>()
const sessions = Promise.withResolvers<void>()
await page.route("**/api/worktree/*", async (route) => {
await inventory.promise
await route.fallback()
})
await page.route("**/api/session?*", async (route) => {
if (new URL(route.request().url()).searchParams.has("directory")) await sessions.promise
await route.fallback()
})
const settings = page.getByTestId("settings-screen")
const requested = page.waitForRequest((request) => new URL(request.url()).pathname.startsWith("/api/worktree/"))
await settings.getByRole("tab", { name: "Workspaces", exact: true }).click()
await requested
await expect(settings.getByRole("heading", { name: "Workspaces", exact: true })).toBeVisible()
await expect(settings.getByRole("button", { name: "Back to app" })).toBeVisible()
await expect(settings.getByText("No workspaces", { exact: true })).toHaveCount(0)
inventory.resolve()
await expect(settings.getByText(sandboxes[0], { exact: true })).toBeVisible()
await expect(settings.getByText("12 workspaces", { exact: true })).toBeVisible()
sessions.resolve()
await expect(settings.getByText("Workspace 1 session", { exact: true })).toBeVisible()
const refresh = Promise.withResolvers<void>()
await page.route("**/api/worktree/*", async (route) => {
await refresh.promise
await route.fallback()
})
await settings.getByRole("tab", { name: "Preferences", exact: true }).click()
await settings.getByRole("tab", { name: "Workspaces", exact: true }).click()
await expect(settings.getByText("Workspace 1 session", { exact: true })).toBeVisible()
refresh.resolve()
})
test("extensions opens without waiting for MCPs or plugins", async ({ page }) => {
const mcps = Promise.withResolvers<void>()
const plugins = Promise.withResolvers<void>()
await page.route("**/api/mcp", async (route) => {
await mcps.promise
await route.fulfill({
json: { location: { directory }, data: [{ name: "demo-mcp", status: { status: "connected" } }] },
})
})
await page.route("**/api/plugin", async (route) => {
await plugins.promise
await route.fulfill({
json: {
location: { directory },
data: [
{ id: "demo-plugin", source: { type: "package", package: "demo-plugin" }, status: "active", tui: false },
],
},
})
})
const settings = page.getByTestId("settings-screen")
const requested = page.waitForRequest((request) => new URL(request.url()).pathname === "/api/mcp")
await settings.getByRole("tab", { name: "Extensions", exact: true }).click()
await requested
await expect(settings.getByRole("heading", { name: "Extensions", exact: true })).toBeVisible()
await expect(settings.getByRole("button", { name: "Back to app" })).toBeVisible()
await settings.getByRole("tab", { name: "Plugins", exact: true }).click()
await expect(settings.getByRole("tab", { name: "Plugins", exact: true })).toHaveAttribute("aria-selected", "true")
plugins.resolve()
await expect(settings.getByText("demo-plugin", { exact: true })).toBeVisible()
mcps.resolve()
await settings.getByRole("tab", { name: "MCPs", exact: true }).click()
await expect(settings.getByRole("switch", { name: "demo-mcp" })).toBeChecked()
})
test("workspace inventory uses the settings panel scroll area", async ({ page }) => {
const settings = page.getByTestId("settings-screen")
await settings.getByRole("tab", { name: "Workspaces", exact: true }).click()
await expect(settings.getByText("Workspace 1 session", { exact: true })).toBeVisible()
const list = settings.locator('[data-component="settings-list"]')
await expect(list).toHaveCSS("max-height", "none")
await expect(list).toHaveCSS("overflow-y", "visible")
await settings.getByText("Workspace 12 session", { exact: true }).scrollIntoViewIfNeeded()
await expect(settings.getByText("Workspace 12 session", { exact: true })).toBeInViewport()
await expect(settings.getByRole("button", { name: "Back to app" })).toBeInViewport()
await page.setViewportSize({ width: 390, height: 844 })
await expect(list).toHaveCSS("max-height", "none")
await expect(list).toHaveCSS("overflow-y", "visible")
await settings.getByText("Workspace 12 session", { exact: true }).scrollIntoViewIfNeeded()
await expect(settings.getByText("Workspace 12 session", { exact: true })).toBeInViewport()
})
@@ -52,7 +52,7 @@ test("shows parent lineage while the child timeline loads", async ({ page }) =>
await page.goto(sessionHref(parentID))
await expectSessionTitle(page, parentTitle)
await page.getByRole("button", { name: "Used Explore" }).click()
await page.getByRole("button", { name: "Used Agent" }).click()
await page.locator(`a[href="${sessionHref(childID)}"]`).click()
await Promise.all([requested.promise, expect(page).toHaveURL(sessionHref(childID))])
await Promise.all([
@@ -77,7 +77,7 @@ test("keeps the parent visible while the child session resolves", async ({ page
await page.goto(sessionHref(parentID))
await expectSessionTitle(page, parentTitle)
await page.getByRole("button", { name: "Used Explore" }).click()
await page.getByRole("button", { name: "Used Agent" }).click()
await page.locator(`a[href="${sessionHref(childID)}"]`).click()
await requested.promise
await Promise.all([expect(page).toHaveURL(sessionHref(parentID)), expectSessionTitle(page, parentTitle)]).finally(
@@ -195,7 +195,7 @@ async function setup(page: Page, events?: () => OpenCodeEvent[]) {
async function openChildFromParent(page: Page) {
await page.goto(sessionHref(parentID))
await expectSessionTitle(page, parentTitle)
await page.getByRole("button", { name: "Used Explore" }).click()
await page.getByRole("button", { name: "Used Agent" }).click()
const card = page.locator(`a[href="${sessionHref(childID)}"]`)
await expect(card).toBeVisible()

Some files were not shown because too many files have changed in this diff Show More