Compare commits

..
Author SHA1 Message Date
Shoubhit Dash 48aedea579 refactor(ai): simplify response fragment guards 2026-08-31 19:30:04 +05:30
Shoubhit Dash 96c0f0d8c3 fix(ai): enforce exclusive checkpoint representations 2026-08-31 19:29:23 +05:30
Shoubhit Dash 4b7d49c9a4 fix(ai): preserve compacted image detail on replay 2026-08-31 19:27:34 +05:30
Shoubhit Dash 0e1081b0a5 fix(ai): apply cache policy to bedrock messages 2026-08-31 19:26:21 +05:30
Shoubhit Dash ce96bbe8a7 refactor(ai): share request preparation across operations 2026-08-31 19:25:28 +05:30
Shoubhit Dash 027f0e7cc9 fix(ai): reject order-unsafe checkpoint recovery 2026-08-31 19:23:58 +05:30
Shoubhit Dash 0a442fa2f8 chore: merge v2 into provider-compaction 2026-08-31 19:08:10 +05:30
Brendan Allan 5a4914c670 feat(app): refine mobile shell home and settings (#46388) 2026-08-31 21:20:17 +08:00
Kit Langton 3a797bf6e4 fix(ai): validate canonical tool results (#46062) 2026-08-31 09:14:31 -04:00
Shoubhit Dash 59d1ab783c refactor(ai): make compaction conversion synchronous 2026-08-31 17:29:38 +05:30
Shoubhit Dash 6b4426bbd3 docs(ai): describe typed compaction history 2026-08-31 17:19:57 +05:30
Shoubhit Dash 1b9f762ea5 refactor(ai): model compaction as typed conversation parts 2026-08-31 17:19:45 +05:30
opencode-agent[bot]andvimtor 33536da231 fix(core): commit undo before compaction (#46383)
Co-authored-by: vimtor <36263538+vimtor@users.noreply.github.com>
2026-08-31 17:03:51 +05:30
Shoubhit Dash c8b4963d98 refactor(ai): simplify compaction control flow 2026-08-31 16:24:01 +05:30
Shoubhit Dash 14a90331bc docs(ai): explain provider compaction and replay ownership 2026-08-31 16:11:38 +05:30
Shoubhit Dash cd2880075a test(ai): cover compaction across http and websocket flows 2026-08-31 16:11:17 +05:30
Shoubhit Dash af22c16249 fix(ai): validate compaction boundaries and incomplete blocks 2026-08-31 16:09:00 +05:30
Shoubhit Dash bc4db825a4 fix(core): reject unsupported compaction replay in ai sdk routes 2026-08-31 16:08:32 +05:30
Shoubhit Dash 4f0ba4f06e feat(ai): add bedrock messages route for claude compaction 2026-08-31 16:02:30 +05:30
Shoubhit Dash 8dd92e2d95 feat(ai): add explicit responses compaction calls 2026-08-31 15:58:29 +05:30
Shoubhit Dash cb5a6c7db4 feat(ai): support anthropic compaction and iteration usage 2026-08-31 15:53:57 +05:30
Shoubhit Dash eaa3dfe04e feat(ai): support automatic responses compaction 2026-08-31 15:51:01 +05:30
Shoubhit Dash 5fd800d169 feat(ai): preserve provider compaction in messages and events 2026-08-31 15:48:42 +05:30
Shoubhit Dash e56ceed32b Revert "fix(tui): surface subagent permissions and questions" (#46376) 2026-08-31 15:41:46 +05:30
Shoubhit Dash b2c7246134 fix(core): refresh git references on daily activity (#45575) 2026-08-31 15:36:37 +05:30
Shoubhit Dash 9c39e75ce2 fix(tui): surface subagent permissions and questions (#44976) 2026-08-31 14:38:05 +05:30
opencode-agent[bot]andBrendonovich 0dad76e618 fix(ui): prevent menu items from shrinking (#46353)
Co-authored-by: Brendonovich <14191578+Brendonovich@users.noreply.github.com>
2026-08-31 16:00:55 +08:00
Luke Parker 90fb6562ce fix(shell): bound post-exit pipe draining on all platforms (#46085) 2026-08-31 16:53:35 +10:00
opencode-agent[bot]andBrendonovich 174d263890 fix(app): save session titles on blur and add tab context menu (#46113)
Co-authored-by: Brendonovich <14191578+Brendonovich@users.noreply.github.com>
2026-08-31 13:07:32 +08:00
opencode-agent[bot]andBrendonovich 5ec29e7a87 refactor(desktop): use password-only server authentication (#45958)
Co-authored-by: Brendonovich <14191578+Brendonovich@users.noreply.github.com>
2026-08-31 13:06:35 +08:00
Luke Parker 3c6b85acf7 fix(app): reveal pasted composer content with custom scrollbar (#46339) 2026-08-31 14:38:28 +10:00
Luke Parker 50e77f66fd fix(app): keep composer select all scoped to the editor (#46338) 2026-08-31 14:38:00 +10:00
Aiden Cline d484f070d1 fix(core): recover reads with non-breaking spaces (#45807) 2026-08-30 23:13:26 -05:00
Luke Parker 8890294bf0 fix(desktop): preserve Windows editing shortcuts (#46336) 2026-08-31 04:07:42 +00:00
Luke Parker a1925de0c1 fix(core): flush trailing stream chunks while providers pause (#46326) 2026-08-31 14:03:17 +10:00
Aiden Cline 24e826d06b fix(ai): validate Bedrock media data (#46333) 2026-08-30 22:58:58 -05:00
Aiden Cline 19625400c1 fix(ai): normalize tool result history (#46309) 2026-08-30 22:19:48 -05:00
opencode-agent[bot] 52cbe7b3ee chore: update nix node_modules hashes 2026-08-31 02:04:13 +00:00
Dax 6a2c3e91c7 feat(plugin): add typed rpc and custom events (#46105) 2026-08-30 21:28:19 -04:00
Aiden Cline 4afd8e81be fix(ai): deduplicate request tools (#46306) 2026-08-30 16:24:46 -05:00
Aiden Cline 485bdc9c4e fix(ai): omit empty Responses user messages (#46297) 2026-08-30 16:03:41 -05:00
Aiden Cline 1a3aee39de fix(ai): sanitize empty Bedrock tool input keys (#46296) 2026-08-30 16:02:31 -05:00
Aiden Cline 2ddf257c6f fix(ai): filter empty Anthropic messages (#46291) 2026-08-30 14:39:46 -05:00
Aiden Cline ef50e0b6d8 fix(ai): omit empty Bedrock system blocks (#46294) 2026-08-30 14:38:51 -05:00
Aiden Cline 583a1a2b6f fix(ai): omit empty Anthropic system blocks (#46289) 2026-08-30 14:21:05 -05:00
Aiden Cline 68c1207b52 fix(ai): close Gemini text before tools (#46286) 2026-08-30 14:08:25 -05:00
Aiden Cline f77647ad12 fix(ai): accumulate bedrock redacted content (#46283) 2026-08-30 13:45:39 -05:00
Aiden Cline c746ea3210 fix(ai): use unique Gemini block ids (#46279) 2026-08-30 13:21:35 -05:00
Aiden Cline d323b34826 feat(ai): add native Mistral provider (#46278) 2026-08-30 12:56:01 -05:00
Aiden Cline bf50194f99 fix(ai): reject truncated bedrock frames (#46281) 2026-08-30 12:47:33 -05:00
Kit Langton 4a977b2b31 refactor(core): bind standalone skill activation to Session (#46077)
Move standalone skill activation into ID-bound Session handles and delegate from the public service. Preserve current-placement lookup, raw skill content, ambient publication context, validation order, and host-scoped detached resume behavior. Cover ownership and lifecycle contracts with focused regressions.
2026-08-30 10:22:03 -04:00
Aiden Cline b1e3a7b222 fix(ai): preserve forced reasoning signature (#46218) 2026-08-29 23:09:03 -05:00
opencode-agent[bot]andBrendonovich b12d43698e fix(app): recover sessions with unavailable locations (#46215)
Co-authored-by: Brendonovich <14191578+Brendonovich@users.noreply.github.com>
2026-08-30 11:51:48 +08:00
opencode-agent[bot] 5cd2c22b27 chore: update nix node_modules hashes 2026-08-30 03:43:36 +00:00
Aiden Cline c2ef80a287 fix(ai): fail malformed converse output (#46193) 2026-08-29 22:29:53 -05:00
opencode-agent[bot]andBrendonovich 4df3029536 fix(desktop): scope library validation exception to CLI (#46212)
Co-authored-by: Brendonovich <14191578+Brendonovich@users.noreply.github.com>
2026-08-30 03:29:31 +00:00
Aiden Cline e70d667a9f fix(ai): preserve Anthropic finish across usage deltas (#46171) 2026-08-29 15:45:27 -05:00
Kit Langton 8ba434b597 refactor(core): move projected Session reads into Store (#46075)
Move existing Session list and message queries into SessionStore. Preserve public response wrapping, Session existence checks, pagination, ordering, and typed message decoding errors.
2026-08-29 09:31:04 -04:00
Kit Langton 171947787c test(tui): wait for diff base search focus (#46083)
Wait for the diff-base search input to receive focus before typing. Preserve existing assertions and timeouts while removing the render-versus-focus test race.
2026-08-29 08:37:32 -04:00
Luke Parker 106629aa11 feat(infra): deploy beta web app with SST (#46086) 2026-08-29 14:51:22 +10:00
Luke Parker 3ee2e482ce fix(app): preserve Windows panel top outlines (#46090) 2026-08-29 04:40:55 +00:00
Kit Langton 849824efd2 refactor(core): merge defaults for selected MCP servers (#46072) 2026-08-28 23:48:29 -04:00
Kit Langton cf2c3a536d refactor(core): reuse catalog response digest (#46071) 2026-08-28 23:41:49 -04:00
opencode-agent[bot] 7852cecd72 chore: update nix node_modules hashes 2026-08-29 03:33:15 +00:00
Kit Langton 6cfffeb031 refactor(core): avoid encoding rejected image candidates (#46073) 2026-08-28 23:32:58 -04:00
Kit Langton 6e954f75ee refactor(core): isolate Session admission and controls (#46019)
Separate ID-bound Session policy from host routing. Bind Inbox and Location preparation dependencies at construction, preserve admission and execution semantics, and cover the extracted ownership contracts directly.
2026-08-28 23:22:25 -04:00
Kit Langton 0116a98371 refactor(tui): share app lifecycle test fixture
Reuse a file-local fixture for renderer, HTTP server, event stream, app startup, and teardown across lifecycle tests. Preserve scenario-specific handlers, configuration, deferred responses, and assertions while removing 187 lines of repeated setup.
2026-08-28 23:19:49 -04:00
Kit Langton a38cbd42aa refactor(core): isolate shell tool preparation
Name the tool-owned pre-spawn preparation boundary while preserving hook edits, permission ordering, directory validation, and effective timeout reporting. Strengthen the existing regression assertions.
2026-08-28 23:17:48 -04:00
Luke Parker 4ab31867c4 fix(app): reduce session-switch latency (#46044) 2026-08-29 13:16:53 +10:00
Luke Parker 51a082cea3 fix(core): release exited shell execution state (#46058) 2026-08-29 03:08:50 +00:00
Kit Langton e3bda5e2d0 feat(tui): add branch review scopes 2026-08-28 22:46:23 -04:00
Kit Langton ad9117b107 fix(session): run user shells immediately in the background
Run user shell commands concurrently with model execution, retain their output in one shell entry, and admit non-waking completion messages. Share result capture and notification formatting across shell callers.
2026-08-28 22:46:20 -04:00
Kit Langton 6b7a1d419c fix(core): preserve Responses tool identities (#45658) 2026-08-28 22:14:13 -04:00
Brendan Allan c0d0f5f4bc feat(cli): configure server CORS origins (#45544) 2026-08-29 09:41:52 +08:00
opencode-agent[bot]andHona 095ed63ea0 fix(app): keep workspace submit buttons neutral (#46052)
Co-authored-by: Hona <10430890+Hona@users.noreply.github.com>
2026-08-29 11:20:41 +10:00
Kit Langton fa5ccac707 refactor(core): share subagent completion delivery (#46054) 2026-08-29 01:20:26 +00:00
Kit Langton 6809be2d0a fix(core): reuse permission policy for pending approvals (#46050) 2026-08-28 21:03:52 -04:00
Aiden Cline a5f8869b35 fix(ai): reject unknown chat finishes (#46036) 2026-08-28 18:22:47 -05:00
Aiden Cline 1c9c5305a6 fix(ai): classify retryable server failures (#46038) 2026-08-28 18:11:52 -05:00
Aiden Cline cf014bf2c1 fix(ai): avoid truncated tool calls (#46040) 2026-08-28 18:11:40 -05:00
Aiden Cline 8e7190f795 fix(ai): respect completed response item text (#45854) 2026-08-28 17:32:19 -05:00
Kit Langton 82b6e0e316 chore: enforce mapError simplifications (#46028) 2026-08-28 17:59:18 -04:00
Kit Langton 5b39f5184f test(core): share selector sentinels (#45748) 2026-08-28 17:59:12 -04:00
Kit Langton 1da25727b2 refactor(core): remove unused insert state (#45738) 2026-08-28 17:59:07 -04:00
Kit Langton 51d53f45c1 test(ci): register omitted V2 unit suites (#45970) 2026-08-28 21:58:54 +00:00
Kit Langton f779b2748a test(core): align runner service style (#45753) 2026-08-28 17:47:29 -04:00
Kit Langton f2fb191f53 test(core): inject vertex auth transport (#45751) 2026-08-28 17:47:24 -04:00
Kit Langton 0b32bdf1e5 test(core): use disposable session fixtures (#45745) 2026-08-28 17:47:18 -04:00
Kit Langton 2751813454 test(core): simplify provider test fixtures (#45743) 2026-08-28 17:47:13 -04:00
Aiden Cline f61858e683 fix(ai): avoid filtered tool calls (#46029) 2026-08-28 16:44:05 -05:00
Kit Langton 87525e00b9 test(core): synchronize retry tests on scheduled events (#46027) 2026-08-28 21:37:04 +00:00
Kit Langton 803b7718b8 refactor(core): import canonical schema contracts (#45746) 2026-08-28 17:29:07 -04:00
Kit Langton 8a24a01bff test(core): repair runner fixture claims (#45737) 2026-08-28 17:29:03 -04:00
Kit Langton 9e39a4fbdf test(core): simplify skill test setup (#45721) 2026-08-28 17:29:00 -04:00
Kit Langton d0baff184b fix(core): make provider publication cancellation-safe (#46026) 2026-08-28 17:24:27 -04:00
Kit Langton d82a0b28a9 docs: clarify release workflow (#46025) 2026-08-28 17:18:47 -04:00
Kit Langton bd379e13cb refactor(core): remove unused instruction import (#45747) 2026-08-28 17:18:44 -04:00
Kit Langton 6e1f783aec refactor(core): expose pty layer directly (#45733) 2026-08-28 17:18:41 -04:00
Kit Langton edef6a4b15 refactor(core): flatten git runner arguments (#45731) 2026-08-28 17:18:38 -04:00
Kit Langton 5990679ebd refactor: use typed error mapping operators (#45727) 2026-08-28 17:18:35 -04:00
Kit Langton b0c8a8c827 fix(util): refresh flock heartbeat time (#45720) 2026-08-28 17:18:32 -04:00
Aiden Cline d837ffe70f feat(plugin): add session retry hook (#45999) 2026-08-28 16:16:19 -05:00
Kit Langton b1d7dd82fc refactor(core): use nonempty array guards (#45718) 2026-08-28 17:12:03 -04:00
Kit Langton e409567428 refactor(core): express operation policies as transforms (#45713) 2026-08-28 17:11:59 -04:00
Kit Langton 9538c2171f refactor(core): derive config recognition fields (#45701) 2026-08-28 17:11:55 -04:00
Kit Langton 0a718be0d9 refactor(plugin): normalize web search API type (#45717) 2026-08-28 16:55:32 -04:00
Kit Langton 67845091ba fix(util): share streamed stderr consumption (#45716) 2026-08-28 16:55:28 -04:00
Kit Langton fe788b7842 fix(core): use runtime update columns (#45700) 2026-08-28 16:55:24 -04:00
Kit Langton ee42eb3ca3 fix(core): render built-in command arguments (#45697) 2026-08-28 16:55:21 -04:00
Kit Langton 80323a4deb test(core): name acquired test services (#45696) 2026-08-28 16:55:17 -04:00
Kit Langton a35f96f427 fix(core): preserve late opaque reasoning (#45694) 2026-08-28 16:55:13 -04:00
Kit Langton d354c3d640 fix(server): avoid loading locations for pending reads (#45994) 2026-08-28 20:44:48 +00:00
Kit Langton ce005ce002 refactor(core): remove unused result plumbing (#45695) 2026-08-28 16:44:27 -04:00
Kit Langton 964245bc2a test(core): isolate project adoption fixture (#46009) 2026-08-28 16:18:36 -04:00
Kit Langton aea3e7c1d2 fix(core): refresh plugins in background (#45993) 2026-08-28 19:43:12 +00:00
Kit Langton 3625942952 feat(core): pass session context to MCP tools (#46008) 2026-08-28 15:43:06 -04:00
Kit Langton 0593a6b8eb fix(core): isolate models seed replay (#45686) 2026-08-28 15:26:22 -04:00
Kit Langton 426e5c6389 fix(tui): open recent picker before server reads (#45977)
Open the recent-session and project picker synchronously with selectable cached rows and independent refreshes. Reconcile committed moves and deletions without restoring stale rows, preserve dismissal and selection through delayed reads, and keep filtered selections visible after asynchronous results arrive.
2026-08-28 15:18:24 -04:00
Kit Langton ebdfcf4866 test(tui): remove brittle animation sampling (#46003) 2026-08-28 19:16:47 +00:00
Kit Langton 000d0882c3 docs(core): record adapter fork boundaries (#45691) 2026-08-28 15:14:46 -04:00
Kit Langton 6062e30cb9 test(core): assert typed failures directly (#45690) 2026-08-28 15:14:42 -04:00
Kit Langton 3badee1a3c test(core): simplify config test setup (#45689) 2026-08-28 15:14:39 -04:00
Kit Langton 4a0256d374 test(core): use effect search harness (#45682) 2026-08-28 15:14:36 -04:00
Kit Langton 52ec62bef0 docs: correct config discovery boundary (#45681) 2026-08-28 15:14:32 -04:00
Kit Langton 31af9858fd test(core): assert pending prompt identity (#45680) 2026-08-28 15:14:28 -04:00
Kit Langton 3151660fbb refactor(core): name shell records as commands (#45693) 2026-08-28 15:00:37 -04:00
Kit Langton 0362ef48ff test(core): isolate transport metrics (#45688) 2026-08-28 15:00:33 -04:00
Kit Langton facd7ff452 refactor(core): simplify V1 migration effects (#45685) 2026-08-28 15:00:29 -04:00
Kit Langton 134cdda333 fix(core): normalize SDK file data (#45679) 2026-08-28 14:56:46 -04:00
Kit Langton 5634ef1bb6 refactor(core): simplify manual compaction (#45678) 2026-08-28 14:56:40 -04:00
Kit Langton 2379ab3d51 fix(core): defer memory filesystem observation (#45675) 2026-08-28 14:56:36 -04:00
Kit Langton 5c908ebba5 refactor(core): simplify reference config precedence (#45673) 2026-08-28 14:56:31 -04:00
Kit Langton ba0755d933 refactor(core): reuse platform contract types (#45666) 2026-08-28 14:56:27 -04:00
Kit Langton f7d6b00c1e test(core): use collected arrays directly (#45665) 2026-08-28 14:56:21 -04:00
Kit Langton 0bb6cf37be fix(tui): animate automatic session renames (#45957) 2026-08-28 18:43:53 +00:00
Kit Langton da57b27277 refactor(core): flatten durable commit validation (#45662) 2026-08-28 14:39:43 -04:00
Kit Langton 42a3fec594 docs(core): clarify runtime ownership (#45671) 2026-08-28 14:39:28 -04:00
Kit Langton d15034264b test(core): reuse session projection fixtures (#45661) 2026-08-28 14:37:33 -04:00
Kit Langton 1ce3c7e580 fix(tui): stop flashing when jumping to session start
Buffer bulk history privately and publish once with a bounded head window. Preserve pending anchor compensation while ensuring newer navigation cancels obsolete Home work. Add atomic-loading and real-App navigation regressions.
2026-08-28 14:35:31 -04:00
Kit Langton 07f27c4eca refactor(core): simplify migration narrowing (#45677) 2026-08-28 14:25:34 -04:00
Kit Langton 196893cfeb refactor(core): remove unused config editor (#45991) 2026-08-28 14:21:21 -04:00
opencode-agent[bot]andrekram1-node f6992059be fix(core): isolate shell jobs by shell ID (#45985)
Co-authored-by: rekram1-node <rekram1-node@users.noreply.github.com>
2026-08-28 13:13:16 -05:00
Kit Langton 732f949a65 fix(tui): admit compaction requests optimistically (#45973)
Render queued compaction feedback before model setup, coalesce repeated gestures, and reconcile canonical admissions without restoring consumed rows. Serialize prompt and compaction preparation through the existing per-session admission chain.
2026-08-28 14:00:09 -04:00
James Long cd3b12c579 feat(tui): resize terminal panes (#45979) 2026-08-28 13:47:50 -04:00
Kit Langton 0d6232ffef test(core): share git test fixtures (#45640) 2026-08-28 13:37:42 -04:00
Kit Langton 6da20f0efe chore(protocol): refresh committed OpenAPI baseline (#45969) 2026-08-28 13:37:11 -04:00
Aiden Cline 8e25e83e5a fix(core): continue retryable failures after durable output (#45861) 2026-08-28 12:35:00 -05:00
Kit Langton fe188f8722 fix(core): normalize directory page headings (#45641) 2026-08-28 13:29:41 -04:00
Kit Langton e4bc8b765b refactor(core): reuse MCP prompt helpers (#45660) 2026-08-28 13:26:51 -04:00
Kit Langton d28b6e9ac2 test(core): construct valid tool contexts (#45657) 2026-08-28 13:26:41 -04:00
Kit Langton ac3cd1b183 test(core): simplify provider smoke tests (#45656) 2026-08-28 13:26:27 -04:00
Kit Langton c601d3b021 refactor(core): share provider factory loading (#45654) 2026-08-28 13:26:15 -04:00
Kit Langton 9bc2165e5c refactor(core): reuse lazy shell parser initialization (#45651) 2026-08-28 13:26:04 -04:00
Kit Langton 374d317412 test(core): simplify provider test setup (#45635) 2026-08-28 13:25:53 -04:00
Kit Langton e50c89834e refactor(core): reuse plugin response wrapper (#45648) 2026-08-28 13:25:40 -04:00
Kit Langton f367c202d9 refactor(core): remove unused command dependency (#45623) 2026-08-28 13:25:27 -04:00
Kit Langton 9d33d83bb4 refactor(core): reuse native option projection (#45647) 2026-08-28 13:23:33 -04:00
Kit Langton 8f1eff50aa refactor(core): deepen snapshot comparison helper (#45653) 2026-08-28 13:22:33 -04:00
Kit Langton e7918e25fd refactor(core): remove ineffective typed recovery (#45631) 2026-08-28 13:16:13 -04:00
Aiden Cline 1eff84615c fix(ai): keep chat reasoning in one lifecycle (#45864) 2026-08-28 12:14:58 -05:00
Kit Langton f1fd6ba3c0 test(core): await registry readiness barriers (#45630) 2026-08-28 13:14:41 -04:00
Kit Langton 39daec9bec refactor(core): narrow session internals (#45638) 2026-08-28 13:05:16 -04:00
Kit Langton 484f5faf8d docs(plugin): update current API examples (#45627) 2026-08-28 13:04:59 -04:00
Kit Langton 9a227d186f fix(core): remove unreachable MCP authorization path (#45633) 2026-08-28 13:03:17 -04:00
Kit Langton fc27061838 refactor(core): remove tool context aliases (#45639) 2026-08-28 13:03:01 -04:00
Kit Langton d71cc3be77 test(core): guarantee lock fixture cleanup (#45636) 2026-08-28 13:02:50 -04:00
Kit Langton 503e680672 test(core): preserve session layer inference (#45629) 2026-08-28 13:02:21 -04:00
Kit Langton 1f77f4a4ed refactor(core): inline auxiliary model operations (#45974) 2026-08-28 12:34:01 -04:00
Kit Langton e8fa7daed5 fix(client): preserve live activity during hydration (#45975)
Merge session status changes received during activity hydration and ignore superseded reconnect responses. Cover terminal events, starts, deletion, and overlapping connections so idle sessions do not regain stale TUI spinners.
2026-08-28 12:30:21 -04:00
Kit Langton 936c73b54d refactor(core): simplify tool settlement (#45967) 2026-08-28 12:04:27 -04:00
Kit Langton b9cb4fc36a chore(core): retire prompt-cache diagnostics (#45965) 2026-08-28 11:55:55 -04:00
Kit Langton a808a02f05 refactor(core): separate config file editing (#45818)
Separate source-file editing from Config with explicit targets, fresh raw JSON reads, owned synchronous mutation, and comment-preserving verified writeback. Preserve own JSON source keys and leave configuration discovery, precedence, and watching unchanged.
2026-08-28 11:23:48 -04:00
James Long e142a783f7 refactor(tui): extract reusable pane resize logic (#45939) 2026-08-28 11:20:23 -04:00
Kit Langton 201536f265 test: repair stale V2 baseline coverage (#45826) 2026-08-28 10:40:04 -04:00
Kit Langton 0d42e76006 feat(ai): add first-class TestLLM controls (#45828) 2026-08-28 09:49:47 -04:00
James Long ca47949475 feat(plugin): expose experimental terminal reads (#45792) 2026-08-28 08:03:35 -04:00
Filip 1c8e557eb4 Revert "fix(cli): prevent repeated updates and npm cache growth" (#45865) 2026-08-28 07:23:43 +00:00
Aiden Cline 11ca603490 fix(ai): finish chat streams at done sentinel (#45850) 2026-08-28 00:53:53 -05:00
Aiden Cline 85d8b07f09 fix(ai): ignore late converse tool deltas (#45847) 2026-08-28 00:38:01 -05:00
Aiden Cline d2ee536c16 fix(core): classify AISDK network failures as transport errors (#45840) 2026-08-28 00:21:17 -05:00
Aiden Cline e12e04f482 fix(core): strengthen background shell guidance (#45843) 2026-08-28 00:19:15 -05:00
Kit Langton 5743537945 test(core): use effectful temp fixtures (#45637) 2026-08-28 01:01:04 -04:00
Aiden Cline 18e22cd82e refactor(ai): simplify response stream state (#45835) 2026-08-27 23:56:42 -05:00
Aiden Cline bdf019a9ac fix(ai): detect DashScope input length overflow errors (#45834) 2026-08-27 23:43:14 -05:00
Aiden Cline 5cbafc57c0 refactor(ai): default unrecognized provider failures to retry (#45825) 2026-08-27 23:40:52 -05:00
Aiden Cline 55674b858b fix(ai): make final snapshots authoritative for text and reasoning (#45831) 2026-08-27 23:38:19 -05:00
Luke Parker bb390f435c fix(session-ui): enable word diffs in unified view (#45833) 2026-08-28 04:31:18 +00:00
Kit Langton 92b9eebab2 feat(tui): streamline diff review workflow (#45817) 2026-08-28 00:11:29 -04:00
Aiden Cline 074413a96d fix(ai): normalize response item boundaries (#45789) 2026-08-27 23:08:50 -05:00
opencode-agent[bot] 4685ba8d3e chore: update nix node_modules hashes 2026-08-28 04:03:16 +00:00
Kit Langton d2c307ab30 test(core): remove unused test imports (#45674) 2026-08-27 23:58:41 -04:00
Kit Langton a5193634d0 test(core): use direct test imports (#45683) 2026-08-27 23:58:34 -04:00
Kit Langton 95725d121a chore: remove unused changesets (#45827) 2026-08-27 23:55:57 -04:00
Aiden Cline 56f2559798 fix(codemode): report original tool errors (#45820) 2026-08-27 22:54:36 -05:00
Kit Langton 38bffc9db1 chore(deps): upgrade Effect to rc.112 (#45819) 2026-08-27 23:43:37 -04:00
Kit Langton 1d3308d904 test(core): refresh test vocabulary (#45714) 2026-08-27 23:26:45 -04:00
opencode-agent[bot] 3c9e5adefa chore: update nix node_modules hashes 2026-08-28 03:23:31 +00:00
Kit Langton a065ad4ba7 feat(core): vanilla instance discovery option (#45752) 2026-08-28 03:16:29 +00:00
Kit Langton 048aec32e7 refactor(core): share local provider settings fold (#45645) 2026-08-27 23:13:13 -04:00
Kit Langton a258b17fac test(core): use partial service mocks (#45703) 2026-08-27 23:13:04 -04:00
Kit Langton 07facd2086 refactor(util): share process stop policy (#45725) 2026-08-27 23:12:57 -04:00
Kit Langton 6bf342a861 refactor(core): consolidate HTML inline handling (#45649) 2026-08-27 23:12:50 -04:00
Kit Langton 2cca1ae860 refactor(core): use nonthrowing URL parsing (#45726) 2026-08-27 23:12:43 -04:00
Kit Langton becf5bf482 refactor(core): retain changed worktrees directly (#45709) 2026-08-27 23:12:36 -04:00
Kit Langton afe9e579d6 refactor(core): simplify instruction preview blobs (#45702) 2026-08-27 23:12:28 -04:00
Kit Langton 0b82fe60ea refactor(core): consolidate model error mapping (#45722) 2026-08-27 23:12:19 -04:00
Luke Parker 6fad330efc chore(deps): upgrade Solid and Router (#45808) 2026-08-28 03:03:08 +00:00
Kit Langton 87f21a301d fix(core): forward plugin interrupt options (#45632) 2026-08-27 23:01:57 -04:00
Kit Langton 0123fed65d fix(core): preserve Responses tool strictness (#45663) 2026-08-27 23:01:43 -04:00
Kit Langton 64ef85159d fix(core): preserve assistant text metadata (#45650) 2026-08-27 23:01:33 -04:00
Kit Langton e63779d9e1 fix(core): keep vertex settings serializable (#45704) 2026-08-27 23:01:20 -04:00
Kit Langton ac55c05c30 fix(core): correct compaction history direction (#45698) 2026-08-27 23:01:07 -04:00
Dax Raad 8fa672ed71 docs: document diagnostic profile signals 2026-08-27 22:53:28 -04:00
Kit Langton d84d1421ae test(core): bind acquired test services (#45711) 2026-08-27 22:43:09 -04:00
Kit Langton dd2674e9c4 refactor(core): reuse Mercurial diff preparation (#45652) 2026-08-27 22:42:47 -04:00
Kit Langton ade8175ad8 refactor(core): use environment driver namespace (#45707) 2026-08-27 22:42:36 -04:00
Kit Langton b0db52da6f refactor(tui): remove inert diff viewer setup (#45687) 2026-08-27 22:42:29 -04:00
Kit Langton 9d673d7fb3 docs(core): align built-in prompt guidance (#45642) 2026-08-27 22:42:19 -04:00
Kit Langton 9cab8b45b2 refactor(core): remove unreachable transport state (#45628) 2026-08-27 22:42:11 -04:00
Kit Langton 17aa942541 refactor(core): reuse tool name normalization (#45644) 2026-08-27 22:42:00 -04:00
Kit Langton cbc3e63869 refactor(core): simplify skill discovery validation (#45729) 2026-08-27 22:41:51 -04:00
Kit Langton 2ce126e2df test(core): simplify runner scenario fixtures (#45440) 2026-08-27 22:37:07 -04:00
Aiden Cline 1faba03cfe fix(core): normalize Windows tool paths (#45806) 2026-08-27 21:36:22 -05:00
Kit Langton 09b0edd662 refactor(core): compare plugin generations directly (#45728) 2026-08-27 22:31:42 -04:00
Kit Langton 0c75b7b3c7 feat(core): per-instance plugin input (#45732) 2026-08-28 02:31:36 +00:00
Kit Langton fd77a721ac test(core): align session test vocabulary (#45670) 2026-08-27 22:31:33 -04:00
Kit Langton 11aab207f0 test(core): remove dead test setup (#45669) 2026-08-27 22:31:26 -04:00
Kit Langton 04c9e01dad refactor: normalize filesystem boundary imports (#45730) 2026-08-27 22:31:15 -04:00
Kit Langton 762291b2a8 feat(core): durable session metadata at creation (#45805) 2026-08-28 02:30:55 +00:00
Kit Langton e0bad10065 test(core): exercise fsutil passthroughs (#45667) 2026-08-27 22:30:43 -04:00
Kit Langton e6d656dec1 refactor(core): clarify PTY binary validation (#45715) 2026-08-27 22:30:35 -04:00
Kit Langton d34dd6c14a refactor(core): type native PTY import explicitly (#45710) 2026-08-27 22:30:15 -04:00
Kit Langton 43c8690c14 docs(core): correct newtype decoding guidance (#45736) 2026-08-27 22:30:07 -04:00
Kit Langton 35d681afab test(core): make test sentinels effective (#45706) 2026-08-27 22:29:59 -04:00
Kit Langton 8ab08f7fc9 refactor(core): remove infallible layer conversion (#45672) 2026-08-27 22:29:44 -04:00
Luke Parker c5113ee74a chore(desktop): trim external dependency archives (#45804) 2026-08-28 02:24:46 +00:00
Kit Langton 88988b8c12 fix(core): preserve typed config test-layer errors (#45778) 2026-08-27 22:13:34 -04:00
opencode-agent[bot] 2389111bf6 chore: update nix node_modules hashes 2026-08-28 02:08:36 +00:00
Luke Parker b92b84f33d chore(deps): refresh markdown dependencies (#45793) 2026-08-28 01:45:46 +00:00
opencode-agent[bot] 078522e534 chore: update nix node_modules hashes 2026-08-28 01:28:15 +00:00
Aiden Cline 7111e71528 fix(core): treat project paths as internal (#45799) 2026-08-27 20:26:25 -05:00
Luke Parker 0ce3214844 chore(deps): upgrade Sentry and verify reporting (#45796) 2026-08-28 01:13:17 +00:00
Luke Parker 132ec56fda chore(ui): refresh Kobalte and Solid primitives (#45794) 2026-08-28 01:08:44 +00:00
opencode-agent[bot] 216af25ecc chore: update nix node_modules hashes 2026-08-28 01:00:46 +00:00
Luke Parker 6881cffc41 chore(deps): refresh Vite and Electron Builder tooling (#45790) 2026-08-28 00:46:32 +00:00
opencode-agent[bot] aecea54e95 chore: update nix node_modules hashes 2026-08-28 00:37:45 +00:00
Luke Parker c0fa9aff72 fix(app): render local Markdown images through the server (#45780) 2026-08-28 00:30:00 +00:00
opencode-agent[bot]andrekram1-node d5be4cf888 fix(core): share native model defaults with v1 config migration (#45760)
Co-authored-by: rekram1-node <rekram1-node@users.noreply.github.com>
2026-08-27 19:25:56 -05:00
Aiden Cline 7b6b0f2717 fix(core): cap provider retry-after delays at fifteen minutes (#45787) 2026-08-27 19:23:06 -05:00
James Long a03a1dd854 chore: upgrade opentui to 0.5.9 (#45785) 2026-08-27 20:20:59 -04:00
Luke Parker 8f350df392 chore(deps): remove unused frontend dependencies (#45779) 2026-08-28 00:19:57 +00:00
Luke Parker 4cecfe8e79 chore(desktop): remove stale native packaging (#45786) 2026-08-28 10:18:53 +10:00
Kit Langton 89ca9fc641 fix(core): complete supervisor flush when plugin activation fails (#45783) 2026-08-28 00:16:25 +00:00
Kit Langton a2dcf2dff0 fix(core): wake sessions for recovered shell outcomes (#45781) 2026-08-27 20:11:56 -04:00
Luke Parker 7477e21b02 fix(session-ui): upgrade DOMPurify and restore Mermaid rendering (#45773) 2026-08-28 00:00:21 +00:00
Kit Langton 59e7b32c92 refactor(core): extract instance module from location services (#45705) 2026-08-27 23:57:52 +00:00
Luke Parker 4681c734b2 fix(desktop): bundle selected main-process dependencies (#45774) 2026-08-27 23:53:51 +00:00
Luke Parker 84fa689686 fix(desktop): preserve lazy Sentry loading (#45776) 2026-08-27 23:41:09 +00:00
Kit Langton 62d9aa9838 refactor(core): delegate credential activation (#45692) 2026-08-27 19:39:25 -04:00
Kit Langton 02550b36c7 refactor(util): narrow layer visitor options (#45734) 2026-08-27 19:39:18 -04:00
Kit Langton 35231b408e refactor(core): use keyed MCP credential reads (#45708) 2026-08-27 19:39:12 -04:00
Kit Langton 74f5317489 docs(tui): complete session tabs story guidance (#45712) 2026-08-27 19:39:05 -04:00
Kit Langton 81e099a441 fix(core): preserve benchmark directory argument (#45634) 2026-08-27 19:38:58 -04:00
Luke Parker 498436f225 chore(desktop): upgrade Electron to 42.10.1 (#45775) 2026-08-27 23:36:39 +00:00
opencode-agent[bot]andneriousy b20748f821 fix(cli): prevent repeated updates and npm cache growth (#45091)
Co-authored-by: neriousy <34747899+neriousy@users.noreply.github.com>
2026-08-28 01:30:34 +02:00
Kit Langton f0d5b5e073 refactor(core): remove unused response stream state (#45742) 2026-08-27 19:27:21 -04:00
Kit Langton 4f6c4ddf1c test(core): guarantee execution scope cleanup (#45740) 2026-08-27 19:27:05 -04:00
Kit Langton 792f84b59d refactor(core): reuse execute visibility predicate (#45699) 2026-08-27 19:26:55 -04:00
Kit Langton b4bd14864a refactor(core): remove unused responses state (#45659) 2026-08-27 19:26:48 -04:00
Kit Langton 6bbf2e34e7 refactor(core): parse MCP prompt arguments once (#45646) 2026-08-27 19:26:40 -04:00
Luke Parker d432823515 fix(deps): upgrade diff to 8.0.4 (#45772) 2026-08-27 23:24:33 +00:00
opencode-agent[bot]andneriousy bdf2e84812 feat(core): support Azure CLI authentication (#45086)
Co-authored-by: neriousy <34747899+neriousy@users.noreply.github.com>
2026-08-27 23:22:22 +00:00
Kit Langton a438d34fcd chore: lint identity pipe callbacks (#45771) 2026-08-27 19:11:03 -04:00
Kit Langton fa7a1ca39e chore: remove duplicate oxlint options (#45719) 2026-08-27 19:10:55 -04:00
Kit Langton 38e4c97aa9 refactor(core): import filesystem search schemas directly (#45684) 2026-08-27 19:10:50 -04:00
Kit Langton 569a39c4ba refactor(core): preserve Copilot handler inference (#45668) 2026-08-27 19:10:44 -04:00
Kit Langton bfb00cbb75 refactor(core): remove unused chat state (#45664) 2026-08-27 19:10:36 -04:00
Kit Langton a067c45cb6 test(schema): target config color contract (#45626) 2026-08-27 19:10:29 -04:00
Kit Langton 0a7baaaed5 refactor(core): reuse shared record guard (#45643) 2026-08-27 19:06:52 -04:00
Luke Parker 454ed81556 fix(desktop): minify production bundles (#45770) 2026-08-27 23:06:28 +00:00
Kit Langton 7fe8bf5b24 refactor(core): remove duplicate config failure (#45625) 2026-08-27 18:52:36 -04:00
Kit Langton 991a0d05b5 refactor(core): remove identity schema pipes (#45624) 2026-08-27 18:52:20 -04:00
opencode-agent[bot]andneriousy 3288a12528 fix(ai): safely replay unsigned Bedrock reasoning (#45765)
Co-authored-by: neriousy <34747899+neriousy@users.noreply.github.com>
2026-08-28 00:51:07 +02:00
Kit Langton 091396742e refactor(core): isolate tool test fixture state (#45452) 2026-08-27 18:48:44 -04:00
Kit Langton ba85531931 refactor(core): remove redundant coordinator test scopes (#45451) 2026-08-27 18:48:20 -04:00
Kit Langton 0440bd96be refactor(core): simplify Effect test setup (#45446) 2026-08-27 18:47:56 -04:00
Kit Langton ace4249711 refactor(core): name session runner test services (#45445) 2026-08-27 18:47:44 -04:00
Kit Langton 7d007670a0 refactor(core): simplify duplicate test layers (#45444) 2026-08-27 18:47:20 -04:00
Kit Langton 98e3fa9d0d test(core): budget real startup in session revert test (#45762) 2026-08-27 18:43:54 -04:00
opencode-agent[bot]andjlongster 5d4ce14dc4 fix(core): skip tracing streamed LLM deltas (#45761)
Co-authored-by: jlongster <17031+jlongster@users.noreply.github.com>
2026-08-27 17:59:43 -04:00
Simon Klee 8381153418 fix(tui): respect selection copy modes (#45758) 2026-08-27 23:53:07 +02:00
Kit Langton 96d84626f8 fix(core): keep project labels stable across clones (#45735)
Keep shared project labels stable across clones while explicitly using each selected checkout for worktree creation and setup. Preserve real directory rename handling and cover the TUI Home and app workspace creation paths.
2026-08-27 17:35:18 -04:00
Kit Langton f607ca4c72 fix(tui): unify attention indicators with unread accent (#45741)
Use the unread accent color for question and permission status indicators by default, preserving semantic tokens and explicit theme overrides.
2026-08-27 17:15:44 -04:00
Kit Langton 39416a0d95 test(ai): align test runners with API boundaries (#45469) 2026-08-27 17:07:02 -04:00
opencode-agent[bot]andiamdavidhill 84a012a0e9 fix(app): show grouped tool counts inline (#45603)
Co-authored-by: iamdavidhill <1879069+iamdavidhill@users.noreply.github.com>
2026-08-27 21:01:38 +00:00
opencode-agent[bot]andiamdavidhill 16a0996bd4 fix(ui): replace error icon with updated svg (#45604)
Co-authored-by: iamdavidhill <1879069+iamdavidhill@users.noreply.github.com>
2026-08-28 06:50:56 +10:00
Kit Langton 60d5f83ffd test(ai): reuse executor HTTP fixtures (#45468) 2026-08-27 16:12:48 -04:00
Kit Langton 1455995ac7 refactor(core): use word casing for MCP namespaces (#45618) 2026-08-27 16:11:50 -04:00
Kit Langton 52c04508a2 test(core): share controlled websearch test layer (#45465) 2026-08-27 16:09:06 -04:00
Kit Langton 5a67fcc17e test(httpapi-codegen): share emitted-module fixtures (#45463) 2026-08-27 15:53:46 -04:00
Kit Langton 73b575468e test(server): simplify scoped endpoint fixtures (#45466) 2026-08-27 15:53:30 -04:00
Kit Langton 0c77f6ed5b refactor(core): remove unreachable permission branch (#45617)
Remove the impossible pre-hook deny aggregation branch while preserving configured denials and post-hook denial handling.
2026-08-27 14:48:11 -04:00
Kit Langton 284b222489 refactor(core): reuse read newline locator (#45616)
Reuse the reader newline locator for the terminal tree leaf while preserving accumulated offsets and the whole-tree fallback. Add chunk-boundary coverage.
2026-08-27 14:47:58 -04:00
Kit Langton 705606face refactor(core): reuse formatter executable helper (#45615)
Reuse the existing executable formatter helper for gofmt, mix, zig, and ktlint while preserving arguments, extensions, ordering, and lazy lookup.
2026-08-27 14:47:10 -04:00
Kit Langton b738ef970d test(core): stabilize Windows live I/O fixtures (#45595) 2026-08-27 14:45:07 -04:00
Kit Langton 4112698e72 refactor(core): simplify session runner control flow (#45614) 2026-08-27 14:42:12 -04:00
Aiden Cline a609174969 feat(core): expand tildes in tool path resolution (#45605) 2026-08-27 13:41:39 -05:00
Kit Langton d3694a5383 refactor(core): share read media types (#45597)
Reuse the reader-owned media MIME set in the read tool leaf while preserving both ingestion and unsupported-base64 validation boundaries.
2026-08-27 14:24:41 -04:00
Kit Langton fafcea42e6 refactor(core): share patch write path (#45588)
Share the identical file write, error mapping, and result recording path used by additions and non-moving updates. Keep deletion and moving-update behavior explicit.
2026-08-27 12:51:58 -04:00
opencode-agent[bot]andrekram1-node 95c3c3f962 fix(mcp): retry initial 404 without injected codemode (#45563)
Co-authored-by: rekram1-node <rekram1-node@users.noreply.github.com>
2026-08-27 11:51:34 -05:00
Aiden Cline 497a24c17d refactor(ai): consolidate provider error diagnostics (#45381) 2026-08-27 11:28:57 -05:00
opencode-agent[bot] f1521000ec chore: update nix node_modules hashes 2026-08-27 16:15:47 +00:00
Kit Langton 8a3e9890d2 chore(core): narrow websocket platform import (#45585) 2026-08-27 16:02:37 +00:00
Kit Langton 2142de87dd test(core): make websocket overflow fixture deterministic (#45586) 2026-08-27 16:00:39 +00:00
Kit Langton 367cf59612 refactor(core): simplify WebFetch error narrowing (#45582)
Use the typed HTTP error reason instead of manual object probing and an unchecked response cast. Preserve the single challenge-only retry and cover ordinary 403 failures without retry.
2026-08-27 11:56:34 -04:00
James Long 2b8a3e7ee0 fix(pty): own daemon lifetime and hand off restarts (#45551) 2026-08-27 11:52:48 -04:00
Kit Langton 6dbb5f5008 fix(core): reuse ignore checks during index refresh (#45459) 2026-08-27 11:44:44 -04:00
Kit Langton 8d1d1ee982 refactor(core): align model content helper names (#45458) 2026-08-27 11:44:37 -04:00
Kit Langton c33dc8808b refactor(core): rename model package predicate (#45457) 2026-08-27 11:44:30 -04:00
Kit Langton 5253603583 fix(core): preserve downloaded skill directory IDs (#45455) 2026-08-27 11:40:55 -04:00
Kit Langton 5fb3040505 test(core): scope skill discovery fixtures (#45464) 2026-08-27 11:25:17 -04:00
Kit Langton df7dae3706 refactor(plugin): reuse tool executor conversion (#45572)
Reuse the private executor/progress conversion for Promise tool list, get, and update while preserving captured executors, identifiers, and explicit optional-field clearing. Add focused adapter regression coverage.
2026-08-27 11:25:12 -04:00
Kit Langton 83ff4253f5 test: share session message expectations (#45454) 2026-08-27 10:58:12 -04:00
Kit Langton 89cb5146d4 test(cli): scope temporary directories to test lifetimes (#45467) 2026-08-27 10:57:06 -04:00
James Long b0b5bf6749 feat(tui): jump to background tool starts (#45554) 2026-08-27 10:56:51 -04:00
Kit Langton ef9b9fb6bf test(client): scope service subprocess fixtures per test (#45471) 2026-08-27 10:55:20 -04:00
James Long c05d481ec6 fix(pty): use server shell and session cwd for terminals (#45552) 2026-08-27 10:39:27 -04:00
Shoubhit Dash 8e726dc7d8 fix(core): preserve imported session parents (#45566) 2026-08-27 19:57:59 +05:30
Shoubhit Dash 23f0ee53cf feat(plugin): intercept user prompts before admission (#45550) 2026-08-27 18:58:00 +05:30
Shoubhit Dash 46aa25f236 feat(plugin): expose model generation options to session hooks (#45268) 2026-08-27 17:30:37 +05:30
Brendan Allan 803ead32e7 feat(app): use channel-specific favicon and PWA icons (#45543) 2026-08-27 11:17:11 +00:00
Luke Parker 42422a1e03 fix(app): show sessions while worktrees are created (#45529) 2026-08-27 19:44:23 +10:00
Brendan Allan b1374978c9 fix(app): prevent Safari shell content from collapsing 2026-08-27 16:59:43 +08:00
Luke Parker ed95fdaa27 fix(app): show MCP connection failures as toasts (#45522) 2026-08-27 08:46:54 +00:00
Luke Parker 53a4829672 fix(app): restore fullscreen settings version footer (#45419) 2026-08-27 17:57:04 +10:00
Luke Parker 7036294543 fix(app): align thinking states and reasoning settings (#45515) 2026-08-27 17:50:39 +10:00
opencode-agent[bot]andHona 23506b5fb4 fix(app): scope MCP picker toggles to workspace (#45509)
Co-authored-by: Hona <10430890+Hona@users.noreply.github.com>
2026-08-27 17:20:38 +10:00
Luke Parker 2af02d0ad7 fix(app): use worktree inventory for session accents (#45506) 2026-08-27 07:03:16 +00:00
opencode-agent[bot]andBrendonovich 26ee104829 fix(app): prevent renderer OOM on multiline paste (#45497)
Co-authored-by: Brendonovich <14191578+Brendonovich@users.noreply.github.com>
2026-08-27 14:31:28 +08:00
opencode-agent[bot]andBrendonovich 8252897a33 fix(app): precache complete builds before serving cached HTML (#45495)
Co-authored-by: Brendonovich <14191578+Brendonovich@users.noreply.github.com>
2026-08-27 14:23:31 +08:00
Luke Parker 7000607fd0 fix(app): stop shimmer when background shells finish (#45494) 2026-08-27 16:15:34 +10:00
opencode-agent[bot]andBrendonovich fa1ab5f8e1 fix(app): restore worktree message accent for further trial (#45486)
Co-authored-by: Brendonovich <14191578+Brendonovich@users.noreply.github.com>
2026-08-27 05:18:54 +00:00
Aiden Cline e288e0fc4d feat(plugin): allow tool call repair before lookup (#45453) 2026-08-27 00:10:43 -05:00
Luke Parker ff5b5d00f9 fix(app): preserve tool disclosures when groups update (#45474) 2026-08-27 15:02:39 +10:00
Luke Parker 990f7ee000 fix(app): keep timeline notices on one line (#45479) 2026-08-27 04:47:50 +00:00
Luke Parker e2d6c4bd4a fix(app): merge adjacent patches inside used groups (#45477) 2026-08-27 04:36:27 +00:00
1019 changed files with 60959 additions and 23348 deletions
-5
View File
@@ -1,5 +0,0 @@
---
"@opencode-ai/core": patch
---
Preserve prompt cache prefixes when sessions move between locations with unchanged instructions.
-5
View File
@@ -1,5 +0,0 @@
---
"@opencode-ai/core": patch
---
Fix OpenCode Console device authorization URLs when the server returns an origin-rooted verification path.
+5
View File
@@ -0,0 +1,5 @@
---
"@opencode-ai/core": patch
---
Correct directory page headings when the read offset is zero.
-5
View File
@@ -1,5 +0,0 @@
---
"@opencode-ai/core": patch
---
Prompt and synthetic inbox ID reuse is now idempotent: reusing an ID within the same Session succeeds and returns the first admission, ignoring the retried payload, metadata, and delivery mode. Previously reuse with a differing payload failed with a conflict. Cross-Session and cross-type reuse still fail, and control items keep their operation-specific conflict behavior.
@@ -1,5 +0,0 @@
---
"@opencode-ai/core": patch
---
Nested AGENTS.md instructions are re-injected after compaction. Previously the in-memory dedup claim outlived the synthetic message that compaction dropped from model-visible history, so nested instructions were silently lost for the rest of the process lifetime. The claim now only guards in-flight loads; the synthetic message metadata in durable history is the sole lasting ledger, so any history truncation (compaction, revert) self-heals on the next read in that subtree.
-5
View File
@@ -1,5 +0,0 @@
---
"@opencode-ai/plugin": patch
---
Derive Promise plugin API request and response conversion from the canonical protocol schemas.
@@ -1,8 +0,0 @@
---
"@opencode-ai/core": minor
"@opencode-ai/schema": minor
"@opencode-ai/protocol": minor
"@opencode-ai/client": minor
---
Remove the unused question request API and use session forms for question tool interactions.
-5
View File
@@ -1,5 +0,0 @@
---
"@opencode-ai/core": patch
---
Include both paths of renamed files in new snapshot change lists so undo restores the original file instead of only deleting the renamed file.
@@ -1,5 +0,0 @@
---
"@opencode-ai/core": patch
---
Apply shared Session model-request preparation to transient generation.
-9
View File
@@ -1,9 +0,0 @@
---
"@opencode-ai/core": minor
"@opencode-ai/schema": minor
"@opencode-ai/protocol": minor
"@opencode-ai/client": minor
---
Add an opt-in portable shell permission scanner. Opaque commands use normal shell authorization without inferring
external directories, while the default tree-sitter path remains unchanged.
-10
View File
@@ -1,10 +0,0 @@
---
"@opencode-ai/core": patch
---
Make the experimental portable shell scanner authoritative, with no Tree-sitter
fallback. Scan common Bash and PowerShell control flow, heredocs, functions,
expressions, quoting, and substitutions natively. Preserve existing redirect and
declaration permission matching, and make PowerShell saved approvals cover the
original command spelling. Parser failures remain visible without changing the
permission engine. The default Tree-sitter path is unchanged.
-5
View File
@@ -1,5 +0,0 @@
---
"@opencode-ai/core": patch
---
Simplify interrupt continuation: the steer-scoped resume decision now lives in SessionExecution as a post-cleanup inbox check, and the run coordinator drops its continuation state machine. Wakes arriving during cancellation cleanup now restart a normal full drain, and interrupting an idle session with continue now resumes pending steering input. Recovery-applied moves now end with the same full wake as inbox-admitted moves, retrying any stranded inbox work at the new location. Interrupting with continue now also resumes a next-in-line control item: between-turn manual compaction and moves run under any drain scope, while queued prompts remain parked.
-6
View File
@@ -1,6 +0,0 @@
---
"@opencode-ai/client": patch
"@opencode-ai/plugin": patch
---
Add form reply and cancellation operations that reconcile terminal forms in the local TUI projection.
-5
View File
@@ -1,5 +0,0 @@
---
"@opencode-ai/core": patch
---
Title generation and compaction summaries now build their model requests through the shared session request boundary, gaining unsupported-media filtering and image bounds while explicitly opting out of session context hooks: plugins that shape the agent conversation do not observe title or compaction requests. Title requests gain the fork-aware session prompt cache key, and compaction summaries in forked sessions reuse the fork root's prompt cache key instead of the fork's own.
-6
View File
@@ -1,6 +0,0 @@
---
"@opencode-ai/plugin": patch
"@opencode-ai/core": patch
---
Add transport-neutral Session model request hooks and provider-scoped hook registration so eligible OpenAI Responses requests can prefer WebSocket without bypassing HTTP-only middleware.
+1
View File
@@ -2,3 +2,4 @@ packages/core/migration/**/snapshot.json linguist-generated
packages/core/src/database/migration.gen.ts linguist-generated
packages/core/src/models-dev/snapshot.txt linguist-generated
packages/core/src/**/*.txt text eol=lf
packages/httpapi-codegen/test/generated/*.ts text eol=lf
+3 -1
View File
@@ -5,6 +5,7 @@ on:
branches:
- dev
- production
- beta
workflow_dispatch:
concurrency: ${{ github.workflow }}-${{ github.ref }}
@@ -15,7 +16,7 @@ permissions:
jobs:
deploy:
if: github.repository == 'anomalyco/opencode' && (github.ref_name == 'dev' || github.ref_name == 'production')
if: github.repository == 'anomalyco/opencode' && (github.ref_name == 'dev' || github.ref_name == 'production' || github.ref_name == 'beta')
runs-on: ubuntu-latest
environment: ${{ github.ref_name }}
steps:
@@ -28,6 +29,7 @@ jobs:
node-version: "24"
- uses: aws-actions/configure-aws-credentials@7474bc4690e29a8392af63c5b98e7449536d5c3a # v4.3.1
if: github.ref_name != 'beta'
with:
role-to-assume: ${{ vars.AWS_DEPLOY_ROLE_ARN }}
role-session-name: opencode-${{ github.run_id }}
+6
View File
@@ -244,6 +244,12 @@ jobs:
CI: true
timeout-minutes: 30
- name: Verify service worker precaching and upgrades
if: env.E2E_ENABLED == 'true'
working-directory: packages/app
run: bunx playwright test --config e2e/service-worker/playwright.config.ts
timeout-minutes: 5
- name: Upload Playwright artifacts
if: always() && env.E2E_ENABLED == 'true'
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
-6
View File
@@ -41,11 +41,5 @@
// Warn when spreading non-plain objects (Headers, class instances, etc.)
"typescript/no-misused-spread": "warn"
},
"options": {
"typeAware": true
},
"options": {
"typeAware": true
},
"ignorePatterns": ["**/node_modules", "**/dist", "**/.build", "**/.sst", "**/*.d.ts", "**/sdk.gen.ts"]
}
+2
View File
@@ -1,6 +1,7 @@
- After changing the public Protocol or Server `HttpApi`, run `bun run generate` from `packages/client`. Do not edit generated client files directly.
- Keep runtime dependencies directed from Schema to Core and Protocol, then from Core and Protocol to Server. Client runtime code may depend on Schema and Protocol but never Core or Server; `sdk` composes Client, Core, and Server.
- Current implementation changes belong in `packages/core`, `packages/cli`, `packages/server`, `packages/protocol`, `packages/schema`, and related generated client surfaces when required.
- This repository does not use Changesets. Do not add `.changeset` files; follow the existing release workflow instead.
- The default branch in this repo is `v2`.
- Base all new branches and worktrees on `v2`, or `origin/v2` when the local `v2` ref is unavailable. Do not base them on `dev`.
- Local `main` ref may not exist; use `v2` or `origin/v2` for diffs.
@@ -45,6 +46,7 @@ Examples: `fix(tui): simplify thinking toggle styling`, `docs: update contributi
### General Principles
- Keep things in one function unless composable or reusable
- Validate unknown values once at the boundary that owns them. Pass typed values inward instead of repeating `typeof value === "object"` and property-existence checks. Do not defensively revalidate values already guaranteed by a schema, constructor, or internal type.
- Do not extract single-use helpers preemptively. Inline the logic at the call site unless the helper is reused, hides a genuinely complex boundary, or has a clear independent name that improves the caller.
- Before adding complexity for a speculative or vanishingly unlikely race or security edge case, explain the concrete failure mode, likelihood, and complexity cost to the user and get their buy-in. Do not silently expand scope for theoretical robustness.
- Avoid `try`/`catch` where possible
+224 -360
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -2,7 +2,7 @@
exact = true
# Only install newly resolved package versions published at least 3 days ago.
minimumReleaseAge = 259200
minimumReleaseAgeExcludes = ["@ai-sdk/amazon-bedrock", "@ai-sdk/anthropic", "@brendonovich/vite-plugin-opencode", "@opencode-ai/sdk", "@opencode-ai/pty", "@opencode-ai/pty-darwin-arm64", "@opencode-ai/pty-darwin-x64", "@opencode-ai/pty-linux-arm64-gnu", "@opencode-ai/pty-linux-arm64-musl", "@opencode-ai/pty-linux-x64-gnu", "@opencode-ai/pty-linux-x64-musl", "@opentui/core", "@opentui/core-darwin-arm64", "@opentui/core-darwin-x64", "@opentui/core-linux-arm64", "@opentui/core-linux-arm64-musl", "@opentui/core-linux-x64", "@opentui/core-linux-x64-musl", "@opentui/core-win32-arm64", "@opentui/core-win32-x64", "@opentui/keymap", "@opentui/solid", "opentui-spinner", "gitlab-ai-provider", "opencode-gitlab-auth", "@ff-labs/fff-node", "@ff-labs/fff-bun", "@ff-labs/fff-bin-darwin-arm64", "@ff-labs/fff-bin-darwin-x64", "@ff-labs/fff-bin-linux-arm64-gnu", "@ff-labs/fff-bin-linux-arm64-musl", "@ff-labs/fff-bin-linux-x64-gnu", "@ff-labs/fff-bin-linux-x64-musl", "@ff-labs/fff-bin-win32-arm64", "@ff-labs/fff-bin-win32-x64", "@pierre/diffs", "@pierre/theming", "app-builder-lib", "dmg-builder", "electron-builder", "electron-publish", "blume"]
minimumReleaseAgeExcludes = ["@ai-sdk/amazon-bedrock", "@ai-sdk/anthropic", "@brendonovich/vite-plugin-opencode", "@opencode-ai/sdk", "@opencode-ai/pty", "@opencode-ai/pty-darwin-arm64", "@opencode-ai/pty-darwin-x64", "@opencode-ai/pty-linux-arm64-gnu", "@opencode-ai/pty-linux-arm64-musl", "@opencode-ai/pty-linux-x64-gnu", "@opencode-ai/pty-linux-x64-musl", "@opentui/core", "@opentui/core-darwin-arm64", "@opentui/core-darwin-x64", "@opentui/core-linux-arm64", "@opentui/core-linux-arm64-musl", "@opentui/core-linux-x64", "@opentui/core-linux-x64-musl", "@opentui/core-win32-arm64", "@opentui/core-win32-x64", "@opentui/keymap", "@opentui/solid", "opentui-spinner", "gitlab-ai-provider", "opencode-gitlab-auth", "@ff-labs/fff-node", "@ff-labs/fff-bun", "@ff-labs/fff-bin-darwin-arm64", "@ff-labs/fff-bin-darwin-x64", "@ff-labs/fff-bin-linux-arm64-gnu", "@ff-labs/fff-bin-linux-arm64-musl", "@ff-labs/fff-bin-linux-x64-gnu", "@ff-labs/fff-bin-linux-x64-musl", "@ff-labs/fff-bin-win32-arm64", "@ff-labs/fff-bin-win32-x64", "@pierre/diffs", "@pierre/theming", "app-builder-lib", "dmg-builder", "electron", "electron-builder", "electron-publish", "blume", "mermaid"]
[test]
root = "./do-not-run-tests-from-root"
+2 -8
View File
@@ -1,4 +1,5 @@
import { domain } from "./stage"
import { createWebApp } from "./webapp"
const GITHUB_APP_ID = new sst.Secret("GITHUB_APP_ID")
const GITHUB_APP_PRIVATE_KEY = new sst.Secret("GITHUB_APP_PRIVATE_KEY")
@@ -59,11 +60,4 @@ new sst.cloudflare.x.Astro("Web", {
},
})
new sst.cloudflare.StaticSite("WebApp", {
domain: "app." + domain,
path: "packages/app",
build: {
command: "bun turbo build",
output: "./dist",
},
})
createWebApp("app." + domain)
+18
View File
@@ -0,0 +1,18 @@
export function createWebApp(domain: string) {
return new sst.cloudflare.StaticSite("WebApp", {
domain,
path: "packages/app",
environment:
$app.stage === "beta"
? {
OPENCODE_CHANNEL: "beta",
VITE_SENTRY_ENVIRONMENT: "beta",
}
: undefined,
build: {
// Preserve Sentry credentials and run source-map uploads on every deployment.
command: "bun run build",
output: "./dist",
},
})
}
+7 -9
View File
@@ -4,7 +4,7 @@
bun,
nodejs,
darwin,
electron_41,
callPackage,
makeWrapper,
writableTmpDirAsHomeHook,
autoPatchelfHook,
@@ -13,7 +13,7 @@
opencode,
}:
let
electron = electron_41;
electron = callPackage ./electron.nix { };
in
stdenv.mkDerivation (finalAttrs: {
pname = "opencode-desktop";
@@ -48,7 +48,7 @@ stdenv.mkDerivation (finalAttrs: {
desktopName = "OpenCode";
exec = "opencode-desktop %U";
icon = "ai.opencode.desktop";
# Electron 41 derives X11 WM_CLASS from app.name.
# Electron derives X11 WM_CLASS from app.name.
startupWMClass = "OpenCode";
categories = [ "Development" ];
});
@@ -67,12 +67,10 @@ stdenv.mkDerivation (finalAttrs: {
# https://github.com/electron/electron/issues/31121
# mac builds use a .app bundle which doesnt have this issue
+ lib.optionalString stdenv.isLinux ''
BASE_PATH=packages/desktop
FILES=(src/main/windows.ts)
for file in "''${FILES[@]}"; do
substituteInPlace $BASE_PATH/$file \
--replace-fail "process.resourcesPath" "'$out/opt/opencode-desktop/resources'"
done
substituteInPlace \
packages/desktop/src/main/windows/appearance.ts \
packages/desktop/src/main/service/desktop-cli.ts \
--replace-fail "process.resourcesPath" "'$out/opt/opencode-desktop/resources'"
'';
preBuild = ''
+13
View File
@@ -0,0 +1,13 @@
{ callPackage, path }:
let
version = (builtins.fromJSON (builtins.readFile ../packages/desktop/package.json)).devDependencies.electron;
in
(callPackage (path + "/pkgs/development/tools/electron/binary/generic.nix") { }) version {
# Electron 42.10.1 SHASUMS256.txt; update with the desktop package version.
aarch64-linux = "20e68d6c4e47f3ebf59de7c6b1f8b8bec6a6ebda6a451132f9b465f3f13ce467";
x86_64-linux = "2452b27112d92387471fa2488aafac85d79ea3f2ee1216c0abd5150d6c12362b";
aarch64-darwin = "ac7194a3dfd81930ba35355c01620262c1254752859b42dcb8f4b9e4d174a871";
x86_64-darwin = "4489aba55477a0082266cb690db1c829503ba3338048599d8fd243953df37dab";
# fetchzip hashes the unpacked headers, not the release tarball.
headers = "sha256-4eUy3BZVvxTl7KUOsxio7769lL6ag/ecbeK+qLURWMI=";
}
+4 -4
View File
@@ -1,8 +1,8 @@
{
"nodeModules": {
"x86_64-linux": "sha256-iYdVrLtyKmjlyypisF9SqzgyriWT90kSCh3crxw9AKU=",
"aarch64-linux": "sha256-BV2t4w5ujArbtSC/Qfm3gLzevQW9A6hMgOyPVp94g/o=",
"aarch64-darwin": "sha256-EwMq7zaxzzcsmH0Pjqu4ftGdcM8Lna8mvHgKzRcVI8g=",
"x86_64-darwin": "sha256-PokzxlkQy6JvHADF2ZMIIDI1u9ZjSNNedpmR9gvHS5c="
"x86_64-linux": "sha256-fG6VYtNC0pce4VM9po7vVucPuJul42yuuijTjNSr7rk=",
"aarch64-linux": "sha256-3TznrmNqdt25cOxia6vcdi/5qKaeyLPIsNXGYBSJNrs=",
"aarch64-darwin": "sha256-8Kmagb5tfECSWZNsIJgrRP1d3X5tuEoWLEWkV3UENZo=",
"x86_64-darwin": "sha256-mIV+mDwIGD02BNYZVi37sY4ls1T01N6z76eBtH0sKiA="
}
}
+27 -23
View File
@@ -39,24 +39,24 @@
"packages/stats/*"
],
"catalog": {
"@effect/opentelemetry": "4.0.0-rc.111",
"@effect/platform-node": "4.0.0-rc.111",
"@effect/platform-node-shared": "4.0.0-rc.111",
"@effect/sql-sqlite-bun": "4.0.0-rc.111",
"@effect/opentelemetry": "4.0.0-rc.112",
"@effect/platform-node": "4.0.0-rc.112",
"@effect/platform-node-shared": "4.0.0-rc.112",
"@effect/sql-sqlite-bun": "4.0.0-rc.112",
"@npmcli/arborist": "9.4.0",
"@types/bun": "1.3.13",
"@types/cross-spawn": "6.0.6",
"@octokit/rest": "22.0.0",
"@hono/standard-validator": "0.2.0",
"@hono/zod-validator": "0.4.2",
"@opentui/core": "0.5.8",
"@opentui/keymap": "0.5.8",
"@opentui/solid": "0.5.8",
"@opentui/core": "0.5.9",
"@opentui/keymap": "0.5.9",
"@opentui/solid": "0.5.9",
"@tanstack/solid-virtual": "3.13.37",
"@shikijs/stream": "4.2.0",
"@shikijs/stream": "4.4.3",
"@standard-schema/spec": "1.1.0",
"ulid": "3.0.1",
"@kobalte/core": "0.13.11",
"@kobalte/core": "0.13.13",
"@corvu/drawer": "0.2.4",
"@types/luxon": "3.7.1",
"@types/node": "24.12.2",
@@ -67,13 +67,16 @@
"@openauthjs/openauth": "0.0.0-20250322224806",
"@pierre/diffs": "1.2.10",
"opentui-spinner": "0.0.7",
"@solid-primitives/storage": "4.3.3",
"@solid-primitives/event-listener": "2.4.6",
"@solid-primitives/media": "2.3.6",
"@solid-primitives/resize-observer": "2.2.0",
"@solid-primitives/storage": "4.4.0",
"@tailwindcss/vite": "4.1.11",
"diff": "8.0.2",
"dompurify": "3.3.1",
"diff": "8.0.4",
"dompurify": "3.4.14",
"drizzle-kit": "1.0.0-rc.5-ab785fc",
"drizzle-orm": "1.0.0-rc.5-169397b",
"effect": "4.0.0-rc.111",
"effect": "4.0.0-rc.112",
"ai": "6.0.168",
"cross-spawn": "7.0.6",
"hono": "4.10.7",
@@ -81,9 +84,9 @@
"fuzzysort": "3.1.0",
"get-east-asian-width": "1.6.0",
"luxon": "3.6.1",
"marked": "18.0.7",
"marked": "18.0.11",
"marked-shiki": "1.2.1",
"remend": "1.3.0",
"remend": "1.3.1",
"@playwright/test": "1.59.1",
"semver": "7.7.4",
"typescript": "5.8.2",
@@ -92,17 +95,17 @@
"remeda": "2.26.0",
"resolve.exports": "2.0.3",
"sst": "4.13.1",
"shiki": "4.2.0",
"shiki": "4.4.3",
"solid-list": "0.3.0",
"string-width": "7.2.0",
"tailwindcss": "4.1.11",
"vite": "7.1.4",
"vite": "7.3.6",
"@solidjs/meta": "0.29.4",
"@solidjs/router": "0.15.4",
"@solidjs/router": "1.0.0",
"@solidjs/start": "https://pkg.pr.new/@solidjs/start@dfb2020",
"@sentry/solid": "10.36.0",
"@sentry/vite-plugin": "4.6.0",
"solid-js": "1.9.10",
"@sentry/solid": "10.71.0",
"@sentry/vite-plugin": "5.4.0",
"solid-js": "1.9.15",
"solid-sonner": "0.3.1",
"vite-plugin-solid": "2.11.10",
"@lydell/node-pty": "1.2.0-beta.12"
@@ -158,7 +161,8 @@
"@effect/platform-node-shared": "catalog:",
"@types/bun": "catalog:",
"@types/node": "catalog:",
"effect": "catalog:"
"effect": "catalog:",
"solid-js": "catalog:"
},
"patchedDependencies": {
"@ai-sdk/openai-compatible@2.0.41": "patches/@ai-sdk%2Fopenai-compatible@2.0.41.patch",
@@ -166,7 +170,7 @@
"@npmcli/agent@4.0.2": "patches/@npmcli%2Fagent@4.0.2.patch",
"@silvia-odwyer/photon-node@0.3.4": "patches/@silvia-odwyer%2Fphoton-node@0.3.4.patch",
"@standard-community/standard-openapi@0.2.9": "patches/@standard-community%2Fstandard-openapi@0.2.9.patch",
"solid-js@1.9.10": "patches/solid-js@1.9.10.patch",
"solid-js@1.9.15": "patches/solid-js@1.9.15.patch",
"@ai-sdk/mistral@3.0.51": "patches/@ai-sdk%2Fmistral@3.0.51.patch",
"gcp-metadata@8.1.2": "patches/gcp-metadata@8.1.2.patch",
"pacote@21.5.0": "patches/pacote@21.5.0.patch",
+8
View File
@@ -19,6 +19,14 @@ Per-type constructors live on the type, not as top-level re-exports. Use `Messag
- Use `testEffect(...)` from `test/lib/effect.ts` for tests requiring Effect layers.
- Keep provider tests fixture-first. Live provider calls must stay behind `RECORD=true` and required API-key checks.
## Errors
- `AIError` wraps a union of tagged reason errors. It stores only `reason`, derives `message` from the reason, and exposes the reason as its `cause`.
- Each reason owns its readable `message`, category-specific fields, and optional `body`, `http`, and underlying exception in `cause`.
- `reason.body` is the sole original-response or triggering-event payload field. Preserve original text before schema decoding removes fields; do not replace the complete event with only its nested error.
- `reason.http` describes an observed HTTP response with required `url`, `status`, and response `headers`. Do not invent status codes or derive a separate request ID from headers.
- Reclassification and transport recovery must preserve the reason's body, HTTP context, and underlying cause. Error `message` and `cause` are non-enumerable: copy them explicitly when constructing an enriched reason with its constructor or `AIErrorReason.make`.
## Architecture
This package is an Effect Schema-first LLM core. The Schema classes in `src/schema/` are the canonical runtime data model. Convenience functions in `src/llm.ts` are thin constructors that return those same Schema class instances; they should improve callsites without creating a second model.
+130 -11
View File
@@ -214,22 +214,141 @@ the requests sent by code under test:
import { Effect } from "effect"
import { TestLLM } from "@opencode-ai/ai/testing"
const testLLM = TestLLM.layer({
fallback: TestLLM.text("Hello from the test model", "text-1"),
})
// TestLLM.clientLayer provides LLMClient.Service and consumes TestLLM.Service.
const programWithTestClient = Effect.gen(function* () {
const test = yield* TestLLM.Test
yield* test.push(TestLLM.text("Hello from the test model", "text-1"))
const result = yield* program
const test = yield* TestLLM.Service
console.log(test.requests)
console.log(yield* test.requests())
return result
}).pipe(Effect.provide(TestLLM.clientLayer), Effect.provide(testLLM))
}).pipe(Effect.provide(TestLLM.testLayer()))
```
`TestLLM.push(...)` scripts one-shot responses, `TestLLM.always(...)` changes the fallback, and
`TestLLM.wait(...)` lets concurrent tests wait until a request has arrived. Every received canonical request is
available on the yielded `TestLLM.Service`.
`testLayer()` provides the same object under `LLMClient.Service` and `TestLLM.Test`. Production consumes the
normal client; tests use the additional controls. Each layer build has fresh state.
- `test.push(...)` queues one-shot responses in execution order. Each argument is one response.
- `test.always(response)` installs a repeatable fallback. The layer's `fallback` option sets its initial value.
- `test.serve(request => response)` installs a request-dependent fallback. `always` and `serve` replace each
other without changing queued replies; queued replies take precedence.
- `test.requests()` returns an array snapshot. `transformRequest` changes only the recorded observation;
`serve` receives the original canonical request.
- `test.wait(count)` waits for request arrivals, not output or completion, and supports concurrent waiters.
- `test.gate()` returns a scoped gate with countable `started` notifications and a `release` Effect. Release
unblocks all requests captured by that gate; closing its scope also releases it. Effect-aware test runners
already provide Scope.
Constructing `stream()` or `generate()` does not record a request, invoke a responder, or consume a script.
Each execution does. An exhausted queue without a fallback defects immediately rather than waiting for a
future reply.
Responses remain canonical event arrays or arbitrary `Stream<LLMEvent, AIError>` values. The client consumes
supplied streams directly, preserving failure identity, finalizers, incomplete output, and post-finish tails;
it does not repair or truncate them.
The published legacy `Service`, `layer`, `clientLayer`, and module-level controls remain available as adapters
over the same implementation, including the legacy live `requests` array. New tests should use `Test` and
`testLayer`.
## Provider compaction
Compaction is opt-in. The package supports automatic compaction in OpenAI/Azure Responses and Anthropic Messages (including Claude on Vertex and Bedrock Messages), and explicit compaction calls in OpenAI/Azure/xAI Responses. Model and deployment support still depends on the provider.
This is different from prompt caching, server-side history storage, or truncation. Compaction returns provider-owned context that must be replayed to continue the conversation.
### Automatic compaction
Inside an `Effect.gen`, enable OpenAI compaction with typed provider options:
```ts
import { LLM, LLMClient, LLMRequest, Message } from "@opencode-ai/ai"
import { OpenAI } from "@opencode-ai/ai/providers"
const request = LLM.request({
model: OpenAI.configure({ apiKey }).responses("gpt-5.3-codex"),
messages,
providerOptions: {
contextManagement: [{ type: "compaction", compactThreshold: 200_000 }],
},
})
const response = yield * LLMClient.generate(request)
const next = LLMRequest.update(request, {
messages: [...request.messages, response.message, Message.user("Continue")],
})
```
`store: false` remains the default. Keep the entire `response.message`, not just `response.text`. Compaction events become ordered `CompactionPart`s alongside text and reasoning. The conversation contains everything needed to continue; there is no separate replay object or hidden provider transcript.
A compaction part has `provider` and exactly one representation: `encrypted` for Responses, or `text` for Anthropic. Responses also preserves the optional checkpoint `id`. These fields survive message serialization without becoming visible assistant text. Sending a checkpoint to another provider or an incompatible API fails rather than silently losing context.
```ts
import { CompactionPart, ProviderID } from "@opencode-ai/ai"
CompactionPart.make({ provider: ProviderID.make("openai"), id: "cmp_123", encrypted: "..." })
CompactionPart.make({ provider: ProviderID.make("anthropic"), text: "Summary of the conversation..." })
```
For Anthropic, use:
```ts
providerOptions: {
contextManagement: {
edits: [{
type: "compact_20260112",
trigger: { type: "input_tokens", value: 150_000 },
pauseAfterCompaction: true,
instructions: "Summarize the task and decisions. Do not call tools while summarizing.",
}],
},
}
```
- The trigger is optional (provider default: 150,000 tokens), with a minimum of 50,000.
- Custom instructions replace Anthropic's default summarization instructions.
- The route adds `compact-2026-01-12` to existing beta headers, including when replaying a checkpoint without enabling new compactions.
- A pause is exposed as `response.finishReason.raw === "compaction"`. The caller explicitly issues the next request; the package never automatically resumes.
- Anthropic can return a compaction block with `content: null` when summarization fails. This becomes a compaction part with `text: null`, which is **not** a successful replacement for prior history. The package never prunes history automatically.
- `Usage` totals include all reported Anthropic `usage.iterations`, including compaction. `contextTokens` separately reports the final message iteration's inclusive input size, when available. A compaction-only pause does not report a post-compaction context size. Raw iteration usage remains in `providerMetadata`.
Bedrock's Converse API does not support this feature. Select the native Claude Messages route explicitly; the default `.model(...)` remains Converse:
```ts
import { AmazonBedrock } from "@opencode-ai/ai/providers"
const model = AmazonBedrock.configure({ region: "us-east-1", credentials }).messages("us.anthropic.claude-opus-4-6-v1")
```
The corresponding package entrypoint is `@opencode-ai/ai/providers/amazon-bedrock/messages`. It uses InvokeModelWithResponseStream, AWS event-stream framing, bearer or SigV4 auth, and `anthropic_beta` in the request body.
### Explicit compaction
`LLMClient.compact(request)` performs exactly one HTTP call to `/responses/compact`, using the selected route's endpoint, credentials, query, and HTTP middleware. It returns a `CompactionResponse` containing replacement `messages` and usage, not a normal generation response.
```ts
const compacted = yield * LLMClient.compact(request)
const next = LLMRequest.update(request, {
messages: [...compacted.messages, Message.user("Continue")],
})
const response = yield * LLMClient.generate(next)
```
Replace the prior window with `compacted.messages`. Do not append it to the original transcript or extract only the encrypted item: the provider may retain additional messages in its output. Retained user and assistant messages remain ordinary messages with typed text, media, or reasoning parts, in their original order. Provider-specific message IDs, status, and phase use `providerMetadata`, not a raw output array hidden in an assistant message. Unsupported returned item types fail explicitly. Generation-only body overlays such as `stream` and `store` are not sent to the compact endpoint.
The input must still fit the model's context window. Explicit compaction is not an overflow-recovery operation. xAI supports this explicit path, not the automatic OpenAI option. Unsupported routes, including Bedrock Mantle, do not inherit an explicit compact endpoint simply because they use a Responses protocol.
### Ownership and verification
The AI package transports options and typed conversation parts. It does not schedule compaction, persist Session checkpoints, select history, switch providers, or replace Core's existing local compaction policy. Native compaction is not enabled for OpenCode Sessions by this feature; Session integration must persist these parts before enabling it. The AI SDK bridge rejects native compaction parts rather than dropping them. Provider-executed tool APIs and persistence changes are a separate follow-up.
Tests cover serialized round trips, real local HTTP plus a tool loop, AWS binary frames and signing, provider errors, malformed blocks, and usage accounting. Live provider tests are gated by `RECORD=true` and the relevant API keys:
```sh
# Run from packages/ai. Only records the selected new cassette group.
RECORD=true RECORDED_PREFIX=openai-compaction bun test test/provider/compaction.recorded.test.ts
RECORD=true RECORDED_PREFIX=xai-compaction bun test test/provider/compaction.recorded.test.ts
RECORD=true RECORDED_PREFIX=anthropic-compaction bun test test/provider/compaction.recorded.test.ts
```
Provider references: [OpenAI](https://developers.openai.com/api/docs/guides/compaction), [Azure](https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/responses#server-side-compaction), [Anthropic](https://platform.claude.com/docs/en/build-with-claude/compaction), [Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/claude-messages-compaction.html), [xAI](https://docs.x.ai/developers/advanced-api-usage/context-compaction).
## Caching
+1
View File
@@ -40,6 +40,7 @@ const RESPECTS_INLINE_HINTS = new Set([
"anthropic-messages",
"google-vertex-messages",
"bedrock-converse",
"bedrock-messages",
"openrouter",
])
+5 -4
View File
@@ -1,7 +1,7 @@
import { Effect, Schema } from "effect"
import {
HttpOptions,
InvalidRequestReason,
InvalidRequestError,
AIError,
ModelID,
ProviderID,
@@ -158,9 +158,10 @@ export function generate(input: ImageRequest | ImageRequestInput) {
try: () => (input instanceof ImageRequest ? input : request(input)),
catch: (error) =>
new AIError({
module: "Image",
method: "generate",
reason: new InvalidRequestReason({ message: error instanceof Error ? error.message : String(error) }),
reason: new InvalidRequestError({
message: error instanceof Error ? error.message : String(error),
cause: error,
}),
}),
}).pipe(Effect.flatMap((request) => ImageClient.generate(request as unknown as ImageRequestFor<ImageOptions>)))
}
+4 -7
View File
@@ -3,7 +3,7 @@ import { LLMClient, Service } from "./route/client.js"
import {
GenerationOptions,
HttpOptions,
InvalidProviderOutputReason,
InvalidProviderOutputError,
AIError,
LLMEvent,
LLMRequest,
@@ -116,9 +116,7 @@ const runGenerateObject = Effect.fn("LLM.generateObject")(function* (
)
if (!call || !LLMEvent.is.toolCall(call))
return yield* new AIError({
module: "LLM",
method: "generateObject",
reason: new InvalidProviderOutputReason({
reason: new InvalidProviderOutputError({
message: `generateObject: model did not call the forced \`${GENERATE_OBJECT_TOOL_NAME}\` tool`,
}),
})
@@ -126,10 +124,9 @@ const runGenerateObject = Effect.fn("LLM.generateObject")(function* (
Effect.mapError(
(error) =>
new AIError({
module: "LLM",
method: "generateObject",
reason: new InvalidProviderOutputReason({
reason: new InvalidProviderOutputError({
message: `generateObject: tool input failed schema decode: ${error.message}`,
cause: error,
}),
}),
),
+221 -47
View File
@@ -6,8 +6,12 @@ import { Auth } from "../route/auth.js"
import { Endpoint } from "../route/endpoint.js"
import { Framing } from "../route/framing.js"
import { Protocol } from "../route/protocol.js"
import { Headers } from "effect/unstable/http"
import { HttpTransport } from "../route/transport/index.js"
import {
AIError,
HttpOptions,
LLMRequest,
LLMEvent,
mergeJsonRecords,
Usage,
@@ -15,7 +19,6 @@ import {
type FinishReasonDetails,
type FinishReason,
type JsonSchema,
type LLMRequest,
type MediaPart,
type ProviderMetadata,
type ToolCallPart,
@@ -61,6 +64,7 @@ export type ThinkingInput =
))
export interface OptionsInput {
readonly contextManagement?: ContextManagement
readonly [key: string]: unknown
readonly thinking?: ThinkingInput
readonly effort?: string
@@ -89,6 +93,23 @@ export interface OptionsInput {
export type ProviderOptionsInput = OptionsInput
export const ContextManagement = Schema.Struct({
edits: Schema.Array(
Schema.Struct({
type: Schema.Literal("compact_20260112"),
trigger: Schema.optional(
Schema.Struct({
type: Schema.Literal("input_tokens"),
value: Schema.Int.check(Schema.isGreaterThanOrEqualTo(50000)),
}),
),
pauseAfterCompaction: Schema.optional(Schema.Boolean),
instructions: Schema.optional(Schema.String),
}),
),
})
export type ContextManagement = typeof ContextManagement.Type
// =============================================================================
// Request Body Schema
// =============================================================================
@@ -236,7 +257,12 @@ const AnthropicUserBlock = Schema.Union([
AnthropicToolResultBlock,
])
type AnthropicUserBlock = Schema.Schema.Type<typeof AnthropicUserBlock>
const AnthropicCompactionBlock = Schema.Struct({
type: Schema.Literal("compaction"),
content: Schema.NullOr(Schema.String),
})
const AnthropicAssistantBlock = Schema.Union([
AnthropicCompactionBlock,
AnthropicTextBlock,
AnthropicThinkingBlock,
AnthropicRedactedThinkingBlock,
@@ -312,6 +338,18 @@ const AnthropicContainer = Schema.Union([
])
const AnthropicBodyFields = {
context_management: Schema.optional(
Schema.Struct({
edits: Schema.Array(
Schema.Struct({
type: Schema.Literal("compact_20260112"),
trigger: ContextManagement.fields.edits.value.fields.trigger,
pause_after_compaction: Schema.optional(Schema.Boolean),
instructions: Schema.optional(Schema.String),
}),
),
}),
),
model: Schema.String,
system: optionalArray(AnthropicTextBlock),
messages: Schema.Array(AnthropicMessage),
@@ -335,7 +373,7 @@ const AnthropicBodyFields = {
export const AnthropicMessagesBody = Schema.Struct(AnthropicBodyFields)
export type AnthropicMessagesBody = Schema.Schema.Type<typeof AnthropicMessagesBody>
const AnthropicUsage = Schema.StructWithRest(
const AnthropicIterationUsage = Schema.StructWithRest(
Schema.Struct({
input_tokens: optionalNull(Schema.Number),
output_tokens: Schema.optional(Schema.Number),
@@ -354,6 +392,13 @@ const AnthropicUsage = Schema.StructWithRest(
}),
[Schema.Record(Schema.String, Schema.Unknown)],
)
const AnthropicUsage = Schema.StructWithRest(
Schema.Struct({
...AnthropicIterationUsage.schema.fields,
iterations: Schema.optional(Schema.Array(AnthropicIterationUsage)),
}),
[JsonObject],
)
type AnthropicUsage = Schema.Schema.Type<typeof AnthropicUsage>
const AnthropicStreamBlock = Schema.Struct({
@@ -377,6 +422,7 @@ type AnthropicStreamBlock = Schema.Schema.Type<typeof AnthropicStreamBlock>
const decodeAnthropicStreamBlock = Schema.decodeUnknownOption(AnthropicStreamBlock)
const AnthropicStreamDelta = Schema.Struct({
content: optionalNull(Schema.String),
type: Schema.optional(Schema.String),
text: Schema.optional(Schema.String),
thinking: Schema.optional(Schema.String),
@@ -406,6 +452,8 @@ const AnthropicEvent = Schema.Struct({
type AnthropicEvent = Schema.Schema.Type<typeof AnthropicEvent>
interface ParserState {
readonly provider: LLMRequest["model"]["provider"]
readonly compactions: Readonly<Record<number, string | null>>
readonly providerMetadataKey: string
readonly tools: ToolStream.State<number>
readonly reasoningSignatures: Readonly<Record<number, string>>
@@ -831,6 +879,7 @@ const lowerMessages = Effect.fn("AnthropicMessages.lowerMessages")(function* (
const content: AnthropicUserBlock[] = []
for (const part of message.content) {
if (part.type === "text") {
if (part.text.trim().length === 0) continue
content.push({ type: "text", text: part.text, cache_control: cacheControl(breakpoints, part.cache) })
continue
}
@@ -840,14 +889,21 @@ const lowerMessages = Effect.fn("AnthropicMessages.lowerMessages")(function* (
}
return yield* ProviderShared.unsupportedContent("Anthropic Messages", "user", ["text", "media"])
}
messages.push({ role: "user", content })
if (content.length > 0) messages.push({ role: "user", content })
continue
}
if (message.role === "assistant") {
const content: AnthropicAssistantBlock[] = []
for (const part of message.content) {
if (part.type === "compaction") {
if (part.provider !== request.model.provider || part.text === undefined)
return yield* invalid("Compaction state must be replayed to its originating provider and API")
content.push({ type: "compaction", content: part.text })
continue
}
if (part.type === "text") {
if (part.text.trim().length === 0) continue
content.push({ type: "text", text: part.text, cache_control: cacheControl(breakpoints, part.cache) })
continue
}
@@ -891,7 +947,7 @@ const lowerMessages = Effect.fn("AnthropicMessages.lowerMessages")(function* (
`Anthropic Messages assistant messages only support text, reasoning, and tool-call content for now`,
)
}
messages.push({ role: "assistant", content })
if (content.length > 0) messages.push({ role: "assistant", content })
continue
}
@@ -1001,6 +1057,9 @@ const resolveThinking = Effect.fn("AnthropicMessages.resolveThinking")(function*
})
const fromRequest = Effect.fn("AnthropicMessages.fromRequest")(function* (request: LLMRequest) {
const management = yield* ProviderShared.validateWith(
Schema.decodeUnknownEffect(Schema.UndefinedOr(ContextManagement)),
)(request.providerOptions?.contextManagement)
const generation = request.generation
const toolSchemaCompatibility = request.model.compatibility?.toolSchema
// Allocate the 4-breakpoint budget in invalidation order: tools → system →
@@ -1019,10 +1078,11 @@ const fromRequest = Effect.fn("AnthropicMessages.fromRequest")(function* (reques
)
// Anthropic rejects tool_choice when tools are absent; "none" is only meaningful with tools present.
const toolChoice = tools === undefined || !request.toolChoice ? undefined : yield* lowerToolChoice(request.toolChoice)
const systemParts = request.system.filter((part) => part.text.length > 0)
const system =
request.system.length === 0
systemParts.length === 0
? undefined
: request.system.map((part) => ({
: systemParts.map((part) => ({
type: "text" as const,
text: part.text,
cache_control: cacheControl(breakpoints, part.cache),
@@ -1034,7 +1094,7 @@ const fromRequest = Effect.fn("AnthropicMessages.fromRequest")(function* (reques
)
}
const options = yield* resolveOptions(request)
return {
const body = {
model: request.model.id,
system,
messages,
@@ -1055,6 +1115,18 @@ const fromRequest = Effect.fn("AnthropicMessages.fromRequest")(function* (reques
metadata: options.metadata,
service_tier: options.service_tier,
}
if (!management) return body
return {
...body,
context_management: {
edits: management.edits.map((edit) => ({
type: edit.type,
trigger: edit.trigger,
pause_after_compaction: edit.pauseAfterCompaction,
instructions: edit.instructions,
})),
},
}
})
// =============================================================================
@@ -1076,18 +1148,31 @@ const mapFinishReason = (reason: string | null | undefined): FinishReason => {
// expose that subset through `output_tokens_details.thinking_tokens`.
const mapUsage = (usage: AnthropicUsage | undefined, providerMetadataKey: string): Usage | undefined => {
if (!usage) return undefined
const nonCached = usage.input_tokens ?? undefined
const cacheRead = usage.cache_read_input_tokens ?? undefined
const cacheWrite = usage.cache_creation_input_tokens ?? undefined
const iterations = usage.iterations?.length ? usage.iterations : [usage]
const last = usage.iterations?.at(-1)
const nonCached = ProviderShared.sumTokens(...iterations.map((item) => item.input_tokens ?? undefined))
const cacheRead = ProviderShared.sumTokens(...iterations.map((item) => item.cache_read_input_tokens ?? undefined))
const cacheWrite = ProviderShared.sumTokens(
...iterations.map((item) => item.cache_creation_input_tokens ?? undefined),
)
const inputTokens = ProviderShared.sumTokens(nonCached, cacheRead, cacheWrite)
const outputTokens = ProviderShared.sumTokens(...iterations.map((item) => item.output_tokens))
return new Usage({
inputTokens,
outputTokens: usage.output_tokens,
outputTokens,
contextTokens:
last?.type === "message"
? ProviderShared.sumTokens(
last.input_tokens ?? undefined,
last.cache_read_input_tokens ?? undefined,
last.cache_creation_input_tokens ?? undefined,
)
: undefined,
nonCachedInputTokens: nonCached,
cacheReadInputTokens: cacheRead,
cacheWriteInputTokens: cacheWrite,
reasoningTokens: usage.output_tokens_details?.thinking_tokens,
totalTokens: ProviderShared.totalTokens(inputTokens, usage.output_tokens, undefined),
reasoningTokens: ProviderShared.sumTokens(...iterations.map((item) => item.output_tokens_details?.thinking_tokens)),
totalTokens: ProviderShared.totalTokens(inputTokens, outputTokens, undefined),
providerMetadata: { [providerMetadataKey]: usage },
})
}
@@ -1109,6 +1194,7 @@ const mergeUsage = (left: Usage | undefined, right: Usage | undefined, providerM
return new Usage({
inputTokens,
outputTokens,
contextTokens: right.contextTokens ?? left.contextTokens,
nonCachedInputTokens,
cacheReadInputTokens,
cacheWriteInputTokens,
@@ -1167,7 +1253,6 @@ const onContentBlockStart = (
event: AnthropicEvent & { readonly content_block: AnthropicStreamBlock },
): StepResult => {
const block = event.content_block
if (!block) return [state, NO_EVENTS]
if (block.type === "tool_use" || block.type === "server_tool_use") {
if (event.index === undefined || !block.id) return [state, NO_EVENTS]
@@ -1262,7 +1347,16 @@ const onContentBlockDelta = Effect.fn("AnthropicMessages.onContentBlockDelta")(f
) {
const delta = event.delta
if (delta?.type === "text_delta" && delta.text) {
if (delta.type === "compaction_delta") {
if (event.index === undefined || !(event.index in state.compactions) || delta.content === undefined)
return yield* ProviderShared.eventError(ADAPTER, "Compaction delta is missing its block or content")
return [
{ ...state, compactions: { ...state.compactions, [event.index]: delta.content } },
NO_EVENTS,
] satisfies StepResult
}
if (delta.type === "text_delta" && delta.text) {
if (!state.lifecycle.text.has(`text-${event.index ?? 0}`)) return [state, NO_EVENTS] satisfies StepResult
const events: LLMEvent[] = []
return [
@@ -1271,7 +1365,7 @@ const onContentBlockDelta = Effect.fn("AnthropicMessages.onContentBlockDelta")(f
] satisfies StepResult
}
if (delta?.type === "thinking_delta" && delta.thinking) {
if (delta.type === "thinking_delta" && delta.thinking) {
if (!state.lifecycle.reasoning.has(`reasoning-${event.index ?? 0}`)) return [state, NO_EVENTS] satisfies StepResult
const events: LLMEvent[] = []
return [
@@ -1283,7 +1377,7 @@ const onContentBlockDelta = Effect.fn("AnthropicMessages.onContentBlockDelta")(f
] satisfies StepResult
}
if (delta?.type === "signature_delta" && delta.signature) {
if (delta.type === "signature_delta" && delta.signature) {
const index = event.index ?? 0
if (!state.lifecycle.reasoning.has(`reasoning-${index}`)) return [state, NO_EVENTS] satisfies StepResult
return [
@@ -1295,7 +1389,7 @@ const onContentBlockDelta = Effect.fn("AnthropicMessages.onContentBlockDelta")(f
] satisfies StepResult
}
if (delta?.type === "input_json_delta" && event.index !== undefined) {
if (delta.type === "input_json_delta" && event.index !== undefined) {
if (!delta.partial_json) return [state, NO_EVENTS] satisfies StepResult
if (!state.tools[event.index]) return [state, NO_EVENTS] satisfies StepResult
const result = ToolStream.appendExisting(
@@ -1320,6 +1414,18 @@ const onContentBlockStop = Effect.fn("AnthropicMessages.onContentBlockStop")(fun
event: AnthropicEvent,
) {
if (event.index === undefined) return [state, NO_EVENTS] satisfies StepResult
if (event.index in state.compactions) {
const { [event.index]: content, ...compactions } = state.compactions
const events: LLMEvent[] = []
const lifecycle = Lifecycle.stepStart(state.lifecycle, events)
events.push(
LLMEvent.compaction({
provider: state.provider,
text: content,
}),
)
return [{ ...state, compactions, lifecycle }, events] satisfies StepResult
}
const result = yield* ToolStream.finish(ADAPTER, state.tools, event.index)
const events: LLMEvent[] = []
const resultEvents = result.events ?? []
@@ -1343,31 +1449,51 @@ const onMessageDelta = (
event: AnthropicEvent & { readonly delta?: AnthropicStreamDelta },
): StepResult => {
const usage = mergeUsage(state.usage, mapUsage(event.usage, state.providerMetadataKey), state.providerMetadataKey)
const pendingFinish = (() => {
const stopReason = event.delta?.stop_reason
if (stopReason === null || stopReason === undefined) return state.pendingFinish
const stopSequence = event.delta?.stop_sequence
const finishMetadata =
stopSequence === null || stopSequence === undefined
? state.pendingFinish?.providerMetadata
: providerMetadata(state.providerMetadataKey, { stopSequence })
return {
reason: {
normalized: mapFinishReason(stopReason),
raw: stopReason,
},
providerMetadata: finishMetadata,
}
})()
return [
{
...state,
usage,
pendingFinish: {
reason: {
normalized: mapFinishReason(event.delta?.stop_reason),
raw: event.delta?.stop_reason ?? undefined,
},
providerMetadata:
event.delta?.stop_sequence === null || event.delta?.stop_sequence === undefined
? undefined
: providerMetadata(state.providerMetadataKey, { stopSequence: event.delta.stop_sequence }),
},
pendingFinish,
},
NO_EVENTS,
]
}
const onMessageStop = Effect.fn("AnthropicMessages.onMessageStop")(function* (state: ParserState) {
if (Object.keys(state.compactions).length)
return yield* ProviderShared.eventError(ADAPTER, "Response ended with an incomplete compaction block")
const result = yield* ToolStream.finishAll(ADAPTER, state.tools)
const events: LLMEvent[] = []
const lifecycle = result.events.length ? Lifecycle.stepStart(state.lifecycle, events) : state.lifecycle
events.push(...result.events)
const finished = Lifecycle.finish(lifecycle, events, {
const closed = Object.entries(state.reasoningSignatures).reduce(
(current, [index, signature]) =>
Lifecycle.reasoningEnd(
current,
events,
`reasoning-${index}`,
providerMetadata(state.providerMetadataKey, { signature }),
),
lifecycle,
)
const finished = Lifecycle.finish(closed, events, {
reason: state.pendingFinish?.reason ?? {
normalized: "unknown",
raw: undefined,
@@ -1387,25 +1513,31 @@ const providerErrorMessage = (event: AnthropicEvent): string => {
return message || type || "Anthropic Messages stream error"
}
const onError = (event: AnthropicEvent) =>
Effect.fail(
const onError = (event: AnthropicEvent) => {
const message = providerErrorMessage(event)
const body = ProviderShared.encodeJson(event)
return Effect.fail(
new AIError({
module: ADAPTER,
method: "stream",
reason: classifyProviderFailure({ message: providerErrorMessage(event), code: event.error?.type }),
reason: classifyProviderFailure({ message, rawBody: body }),
}),
)
}
const isKnownStreamBlockType = (type: string) =>
type === "text" ||
type === "thinking" ||
type === "redacted_thinking" ||
type === "tool_use" ||
type === "server_tool_use" ||
isServerToolResultType(type)
const isKnownStreamDeltaType = (type: string) =>
type === "text_delta" || type === "thinking_delta" || type === "signature_delta" || type === "input_json_delta"
const STREAM_BLOCK_TYPES = new Set([
"compaction",
"text",
"thinking",
"redacted_thinking",
"tool_use",
"server_tool_use",
])
const STREAM_DELTA_TYPES = new Set([
"compaction_delta",
"text_delta",
"thinking_delta",
"signature_delta",
"input_json_delta",
])
const invalidStreamEvent = (event: AnthropicEvent) =>
Effect.fail(
@@ -1434,7 +1566,16 @@ const step = (state: ParserState, event: AnthropicEvent) => {
if (event.type === "content_block_start") {
if (!ProviderShared.isRecord(event.content_block) || typeof event.content_block.type !== "string")
return invalidStreamEvent(event)
if (!isKnownStreamBlockType(event.content_block.type)) return Effect.succeed<StepResult>([state, NO_EVENTS])
if (event.content_block.type === "compaction") {
const decoded = Schema.decodeUnknownOption(AnthropicCompactionBlock)(event.content_block)
if (event.index === undefined || Option.isNone(decoded)) return invalidStreamEvent(event)
return Effect.succeed<StepResult>([
{ ...state, compactions: { ...state.compactions, [event.index]: decoded.value.content } },
NO_EVENTS,
])
}
if (!STREAM_BLOCK_TYPES.has(event.content_block.type) && !isServerToolResultType(event.content_block.type))
return Effect.succeed<StepResult>([state, NO_EVENTS])
const decoded = decodeAnthropicStreamBlock(event.content_block)
if (Option.isNone(decoded)) return invalidStreamEvent(event)
const block = decoded.value
@@ -1448,7 +1589,7 @@ const step = (state: ParserState, event: AnthropicEvent) => {
}
if (event.type === "content_block_delta") {
if (!ProviderShared.isRecord(event.delta)) return invalidStreamEvent(event)
if (typeof event.delta.type === "string" && !isKnownStreamDeltaType(event.delta.type))
if (typeof event.delta.type === "string" && !STREAM_DELTA_TYPES.has(event.delta.type))
return Effect.succeed<StepResult>([state, NO_EVENTS])
const decoded = decodeAnthropicStreamDelta(event.delta)
if (Option.isNone(decoded)) return invalidStreamEvent(event)
@@ -1482,6 +1623,8 @@ export const protocol = Protocol.make({
stream: {
event: Protocol.jsonEvent(AnthropicEvent),
initial: (request) => ({
provider: request.model.provider,
compactions: {},
providerMetadataKey: request.model.route.providerMetadataKey ?? String(request.model.provider),
tools: ToolStream.empty<number>(),
reasoningSignatures: {},
@@ -1491,6 +1634,37 @@ export const protocol = Protocol.make({
},
})
export const transport = <Body extends Pick<AnthropicMessagesBody, "messages" | "context_management">>() => {
const http = HttpTransport.httpJson<Body, string>({ framing })
return {
...http,
prepare: (input: Parameters<typeof http.prepare>[0]) => {
if (
!input.body.context_management?.edits.length &&
!input.body.messages.some((message) => message.content.some((block) => block.type === "compaction"))
)
return http.prepare(input)
const headers = Headers.fromInput(input.request.http?.headers)
const betas = new Set(
(headers["anthropic-beta"] ?? "")
.split(",")
.map((item) => item.trim())
.filter(Boolean),
)
betas.add("compact-2026-01-12")
return http.prepare({
...input,
request: LLMRequest.update(input.request, {
http: new HttpOptions({
...input.request.http,
headers: { ...headers, "anthropic-beta": [...betas].join(",") },
}),
}),
})
},
}
}
export const route = Route.make({
id: ADAPTER,
provider: "anthropic",
@@ -1500,7 +1674,7 @@ export const route = Route.make({
baseURL: DEFAULT_BASE_URL,
}),
auth: Auth.none,
framing,
transport: transport<AnthropicMessagesBody>(),
headers: () => ({ "anthropic-version": "2023-06-01" }),
})
+154 -81
View File
@@ -1,4 +1,4 @@
import { Effect, Schema } from "effect"
import { Effect, Encoding, Schema } from "effect"
import { Route } from "../route/client.js"
import { Endpoint } from "../route/endpoint.js"
import { Protocol } from "../route/protocol.js"
@@ -262,24 +262,28 @@ const providerMetadata = (key: string, metadata: Record<string, unknown>): Provi
const reasoningSignature = (part: ReasoningPart, providerMetadataKey: string) => {
const metadata = part.providerMetadata?.[providerMetadataKey]
return (
part.encrypted ??
(ProviderShared.isRecord(metadata) && typeof metadata.signature === "string" ? metadata.signature : undefined)
)
if (part.encrypted !== undefined) return part.encrypted
if (ProviderShared.isRecord(metadata) && typeof metadata.signature === "string") return metadata.signature
}
const reasoningRedactedData = (part: ReasoningPart, providerMetadataKey: string) => {
const metadata = part.providerMetadata?.[providerMetadataKey]
return ProviderShared.isRecord(metadata) && typeof metadata.redactedData === "string"
? metadata.redactedData
: undefined
if (ProviderShared.isRecord(metadata) && typeof metadata.redactedData === "string") return metadata.redactedData
}
const removeEmptyToolInputKeys = (input: unknown): unknown => {
if (Array.isArray(input)) return input.map(removeEmptyToolInputKeys)
if (!ProviderShared.isRecord(input)) return input
return Object.fromEntries(
Object.entries(input).flatMap(([key, value]) => (key === "" ? [] : [[key, removeEmptyToolInputKeys(value)]])),
)
}
const lowerToolCall = (part: ToolCallPart): BedrockToolUseBlock => ({
toolUse: {
toolUseId: part.id,
name: part.name,
input: part.input,
input: removeEmptyToolInputKeys(part.input),
},
})
@@ -374,6 +378,12 @@ const lowerMessages = Effect.fn("BedrockConverse.lowerMessages")(function* (
content.push({ reasoningContent: { redactedContent: redactedData } })
continue
}
if (signature === undefined || signature.trim().length === 0) {
// Interrupted streams and model switches can leave unsigned reasoning.
// Preserve readable history as text rather than replay invalid reasoningContent.
if (part.text.trim().length > 0) content.push(...textWithCache(breakpoints, part.text, part.cache))
continue
}
content.push({ reasoningContent: { reasoningText: { text: part.text, signature } } })
continue
}
@@ -382,7 +392,7 @@ const lowerMessages = Effect.fn("BedrockConverse.lowerMessages")(function* (
continue
}
}
messages.push({ role: "assistant", content })
if (content.length > 0) messages.push({ role: "assistant", content })
continue
}
@@ -408,7 +418,12 @@ const lowerMessages = Effect.fn("BedrockConverse.lowerMessages")(function* (
const lowerSystem = (
breakpoints: BedrockCache.Breakpoints,
system: ReadonlyArray<LLMRequest["system"][number]>,
): BedrockSystemBlock[] => system.flatMap((part) => textWithCache(breakpoints, part.text, part.cache))
) => {
const content = system
.filter((part) => part.text.length > 0)
.flatMap((part) => textWithCache(breakpoints, part.text, part.cache))
return content.length === 0 ? undefined : content
}
const fromRequest = Effect.fn("BedrockConverse.fromRequest")(function* (request: LLMRequest) {
const toolChoice = request.toolChoice ? yield* lowerToolChoice(request.toolChoice) : undefined
@@ -416,38 +431,42 @@ const fromRequest = Effect.fn("BedrockConverse.fromRequest")(function* (request:
// Bedrock-Claude shares Anthropic's 4-breakpoint cap. Spend the budget in
// tools → system → messages order to favour the highest-impact prefixes.
const breakpoints = BedrockCache.breakpoints()
const toolConfig =
request.tools.length > 0
? {
tools: lowerTools(request.model.compatibility?.toolSchema, breakpoints, request.tools),
// Converse has no native "none". Keep definitions stable for prompt
// caching and omit only the unsupported choice.
toolChoice,
}
: undefined
const system = request.system.length === 0 ? undefined : lowerSystem(breakpoints, request.system)
const toolConfig = (() => {
if (request.tools.length === 0) return undefined
return {
tools: lowerTools(request.model.compatibility?.toolSchema, breakpoints, request.tools),
// Converse has no native "none". Keep definitions stable for prompt
// caching and omit only the unsupported choice.
toolChoice,
}
})()
const system = lowerSystem(breakpoints, request.system)
const messages = yield* lowerMessages(request, breakpoints)
if (breakpoints.dropped > 0) {
yield* Effect.logWarning(
`Bedrock Converse: dropped ${breakpoints.dropped} cache breakpoint(s); the API allows at most ${BedrockCache.BEDROCK_BREAKPOINT_CAP} per request.`,
)
}
return {
modelId: request.model.id,
messages,
system,
inferenceConfig:
const inferenceConfig = (() => {
if (
generation?.maxTokens === undefined &&
generation?.temperature === undefined &&
generation?.topP === undefined &&
(generation?.stop === undefined || generation.stop.length === 0)
? undefined
: {
maxTokens: generation?.maxTokens,
temperature: generation?.temperature,
topP: generation?.topP,
stopSequences: generation?.stop,
},
)
return undefined
return {
maxTokens: generation?.maxTokens,
temperature: generation?.temperature,
topP: generation?.topP,
stopSequences: generation?.stop,
}
})()
return {
modelId: request.model.id,
messages,
system,
inferenceConfig,
toolConfig,
// Converse's base inferenceConfig has no topK; Anthropic/Nova accept it
// as a model-specific field, so it goes through additionalModelRequestFields.
@@ -463,7 +482,6 @@ const mapFinishReason = (reason: string): FinishReason => {
if (reason === "max_tokens" || reason === "model_context_window_exceeded") return "length"
if (reason === "tool_use") return "tool-calls"
if (reason === "content_filtered" || reason === "guardrail_intervened") return "content-filter"
if (reason === "malformed_model_output" || reason === "malformed_tool_use") return "error"
return "unknown"
}
@@ -490,6 +508,7 @@ const mapUsage = (usage: BedrockUsageSchema | undefined, providerMetadataKey: st
interface ParserState {
readonly providerMetadataKey: string
readonly tools: ToolStream.State<number>
readonly finishedTools: ReadonlySet<number>
// Bedrock splits the finish into `messageStop` (carries `stopReason`) and
// `metadata` (carries usage). Hold the terminal event in state so `onHalt`
// can emit exactly one finish after both chunks have had a chance to arrive.
@@ -497,6 +516,16 @@ interface ParserState {
readonly hasToolCalls: boolean
readonly lifecycle: Lifecycle.State
readonly reasoningSignatures: Readonly<Record<number, string>>
readonly reasoningRedactedContent: Readonly<Record<number, ReadonlyArray<Uint8Array>>>
}
const encodeRedactedContent = (chunks: ReadonlyArray<Uint8Array>) => {
const bytes = new Uint8Array(chunks.reduce((total, chunk) => total + chunk.length, 0))
chunks.reduce((offset, chunk) => {
bytes.set(chunk, offset)
return offset + chunk.length
}, 0)
return Encoding.encodeBase64(bytes)
}
const step = (state: ParserState, event: BedrockEvent) =>
@@ -544,23 +573,46 @@ const step = (state: ParserState, event: BedrockEvent) =>
const index = event.contentBlockDelta.contentBlockIndex
const reasoning = event.contentBlockDelta.delta.reasoningContent
const events: LLMEvent[] = []
const redactedData = reasoning.redactedContent ?? reasoning.data
const metadata = reasoning.signature
? providerMetadata(state.providerMetadataKey, { signature: reasoning.signature })
: redactedData !== undefined
? providerMetadata(state.providerMetadataKey, { redactedData })
: undefined
const lifecycle =
reasoning.text !== undefined || metadata !== undefined
? Lifecycle.reasoningDelta(state.lifecycle, events, `reasoning-${index}`, reasoning.text ?? "", metadata)
: state.lifecycle
const redactedChunks = yield* (() => {
if (reasoning.redactedContent === undefined) return Effect.succeed(undefined)
return Effect.fromResult(Encoding.decodeBase64(reasoning.redactedContent)).pipe(
Effect.map((chunk) => [...(state.reasoningRedactedContent[index] ?? []), chunk]),
Effect.mapError((cause) =>
ProviderShared.eventError(
ADAPTER,
"Bedrock Converse reasoningContent.redactedContent contains invalid base64 data",
undefined,
cause,
),
),
)
})()
const redactedData = redactedChunks === undefined ? reasoning.data : encodeRedactedContent(redactedChunks)
const metadata = (() => {
if (reasoning.signature) return providerMetadata(state.providerMetadataKey, { signature: reasoning.signature })
if (redactedData !== undefined) return providerMetadata(state.providerMetadataKey, { redactedData })
})()
const lifecycle = (() => {
if (reasoning.text === undefined && metadata === undefined) return state.lifecycle
return Lifecycle.reasoningDelta(state.lifecycle, events, `reasoning-${index}`, reasoning.text ?? "", metadata)
})()
const reasoningRedactedContent = (() => {
if (redactedChunks !== undefined) return { ...state.reasoningRedactedContent, [index]: redactedChunks }
if (reasoning.data === undefined) return state.reasoningRedactedContent
return Object.fromEntries(
Object.entries(state.reasoningRedactedContent).filter(([key]) => key !== String(index)),
)
})()
const reasoningSignatures = (() => {
if (!reasoning.signature) return state.reasoningSignatures
return { ...state.reasoningSignatures, [index]: reasoning.signature }
})()
return [
{
...state,
lifecycle,
reasoningSignatures: reasoning.signature
? { ...state.reasoningSignatures, [index]: reasoning.signature }
: state.reasoningSignatures,
reasoningSignatures,
reasoningRedactedContent,
},
events,
] as const
@@ -568,6 +620,7 @@ const step = (state: ParserState, event: BedrockEvent) =>
if (event.contentBlockDelta?.delta?.toolUse) {
const index = event.contentBlockDelta.contentBlockIndex
if (state.finishedTools.has(index)) return [state, []] as const
const result = ToolStream.appendExisting(
ADAPTER,
state.tools,
@@ -587,16 +640,24 @@ const step = (state: ParserState, event: BedrockEvent) =>
const result = yield* ToolStream.finish(ADAPTER, state.tools, index)
const events: LLMEvent[] = []
const resultEvents = result.events ?? []
const lifecycle = resultEvents.length
? Lifecycle.stepStart(state.lifecycle, events)
: Lifecycle.reasoningEnd(
Lifecycle.textEnd(state.lifecycle, events, `text-${index}`),
events,
`reasoning-${index}`,
state.reasoningSignatures[index]
? providerMetadata(state.providerMetadataKey, { signature: state.reasoningSignatures[index] })
: undefined,
)
const lifecycle = (() => {
if (resultEvents.length) return Lifecycle.stepStart(state.lifecycle, events)
const metadata = (() => {
const signature = state.reasoningSignatures[index]
if (signature) return providerMetadata(state.providerMetadataKey, { signature })
const redactedContent = state.reasoningRedactedContent[index]
if (redactedContent)
return providerMetadata(state.providerMetadataKey, {
redactedData: encodeRedactedContent(redactedContent),
})
})()
return Lifecycle.reasoningEnd(
Lifecycle.textEnd(state.lifecycle, events, `text-${index}`),
events,
`reasoning-${index}`,
metadata,
)
})()
events.push(...resultEvents)
return [
{
@@ -606,15 +667,28 @@ const step = (state: ParserState, event: BedrockEvent) =>
state.hasToolCalls,
lifecycle,
tools: result.tools,
finishedTools: resultEvents.length > 0 ? new Set([...state.finishedTools, index]) : state.finishedTools,
reasoningSignatures: Object.fromEntries(
Object.entries(state.reasoningSignatures).filter(([key]) => key !== String(index)),
),
reasoningRedactedContent: Object.fromEntries(
Object.entries(state.reasoningRedactedContent).filter(([key]) => key !== String(index)),
),
},
events,
] as const
}
if (event.messageStop) {
if (
event.messageStop.stopReason === "malformed_model_output" ||
event.messageStop.stopReason === "malformed_tool_use"
)
return yield* ProviderShared.eventError(
ADAPTER,
`Bedrock Converse stopped with ${event.messageStop.stopReason}`,
ProviderShared.encodeJson(event),
)
return [
{
...state,
@@ -645,15 +719,13 @@ const step = (state: ParserState, event: BedrockEvent) =>
}
if (event.exception) {
const message =
event.exception.details.message ?? event.exception.details.originalMessage ?? "Bedrock Converse stream error"
const body = ProviderShared.encodeJson(event)
return yield* new AIError({
module: ADAPTER,
method: "stream",
reason: classifyProviderFailure({
message:
event.exception.details.message ??
event.exception.details.originalMessage ??
"Bedrock Converse stream error",
code: event.exception.type,
message,
rawBody: body,
}),
})
}
@@ -663,23 +735,22 @@ const step = (state: ParserState, event: BedrockEvent) =>
const framing = BedrockEventStream.framing(ADAPTER)
const onHalt = (state: ParserState): ReadonlyArray<LLMEvent> =>
state.pendingFinish
? (() => {
const events: LLMEvent[] = []
Lifecycle.finish(state.lifecycle, events, {
reason: {
...state.pendingFinish.reason,
normalized:
state.pendingFinish.reason.normalized === "stop" && state.hasToolCalls
? "tool-calls"
: state.pendingFinish.reason.normalized,
},
usage: state.pendingFinish.usage,
})
return events
})()
: []
const onHalt = (state: ParserState): ReadonlyArray<LLMEvent> => {
if (!state.pendingFinish) return []
const normalized = (() => {
if (state.pendingFinish.reason.normalized === "stop" && state.hasToolCalls) return "tool-calls"
return state.pendingFinish.reason.normalized
})()
const events: LLMEvent[] = []
Lifecycle.finish(state.lifecycle, events, {
reason: {
...state.pendingFinish.reason,
normalized,
},
usage: state.pendingFinish.usage,
})
return events
}
// =============================================================================
// Protocol And Bedrock Route
@@ -699,10 +770,12 @@ export const protocol = Protocol.make({
initial: (request) => ({
providerMetadataKey: request.model.route.providerMetadataKey ?? String(request.model.provider),
tools: ToolStream.empty<number>(),
finishedTools: new Set<number>(),
pendingFinish: undefined,
hasToolCalls: false,
lifecycle: Lifecycle.initial(),
reasoningSignatures: {},
reasoningRedactedContent: {},
}),
step,
onHalt: (state) => Effect.succeed(onHalt(state)),
@@ -1,6 +1,7 @@
import { EventStreamCodec } from "@smithy/eventstream-codec"
import { fromUtf8, toUtf8 } from "@smithy/util-utf8"
import { Effect, Stream } from "effect"
import { Effect, Encoding, Stream } from "effect"
import { AIError, AIErrorReason, InvalidProviderOutputError } from "../schema/index.js"
import { Framing } from "../route/framing.js"
import { ProviderShared } from "./shared.js"
@@ -21,6 +22,10 @@ interface FrameBufferState {
const initialFrameBuffer: FrameBufferState = { buffer: new Uint8Array(0), offset: 0 }
type FrameInput = { readonly _tag: "Chunk"; readonly bytes: Uint8Array } | { readonly _tag: "End" }
const endOfStream: FrameInput = { _tag: "End" }
const appendChunk = (state: FrameBufferState, chunk: Uint8Array): FrameBufferState => {
const remaining = state.buffer.length - state.offset
// Compact: drop the consumed prefix and append the new chunk in one alloc.
@@ -32,9 +37,23 @@ const appendChunk = (state: FrameBufferState, chunk: Uint8Array): FrameBufferSta
return { buffer: next, offset: 0 }
}
const consumeFrames = (route: string) => (state: FrameBufferState, chunk: Uint8Array) =>
const consumeFrames = (route: string) => (state: FrameBufferState, input: FrameInput) =>
Effect.gen(function* () {
let cursor = appendChunk(state, chunk)
if (input._tag === "End") {
const remaining = state.buffer.subarray(state.offset)
if (remaining.length > 0)
return yield* new AIError({
reason: new InvalidProviderOutputError({
route,
classification: "incomplete-stream",
message: `Incomplete Bedrock Converse event-stream frame: ${remaining.length} buffered bytes remain at end of stream`,
body: Encoding.encodeBase64(remaining),
}),
})
return [state, []] as const
}
let cursor = appendChunk(state, input.bytes)
const out: object[] = []
while (cursor.buffer.length - cursor.offset >= 4) {
const view = cursor.buffer.subarray(cursor.offset)
@@ -49,10 +68,14 @@ const consumeFrames = (route: string) => (state: FrameBufferState, chunk: Uint8A
`Failed to decode Bedrock Converse event-stream frame: ${
error instanceof Error ? error.message : String(error)
}`,
Encoding.encodeBase64(view.subarray(0, totalLength)),
error,
),
})
cursor = { buffer: cursor.buffer, offset: cursor.offset + totalLength }
const payload = utf8.decode(decoded.body)
const body = ProviderShared.encodeJson({ headers: decoded.headers, body: payload })
const messageType = decoded.headers[":message-type"]?.value
if (messageType === "error") {
const code = decoded.headers[":error-code"]?.value
@@ -61,6 +84,7 @@ const consumeFrames = (route: string) => (state: FrameBufferState, chunk: Uint8A
route,
[code, message].filter((value): value is string => typeof value === "string").join(": ") ||
"Bedrock Converse event-stream error",
body,
)
}
const eventType =
@@ -70,7 +94,6 @@ const consumeFrames = (route: string) => (state: FrameBufferState, chunk: Uint8A
? decoded.headers[":exception-type"]?.value
: undefined
if (typeof eventType !== "string") continue
const payload = utf8.decode(decoded.body)
if (!payload) continue
// The AWS event stream pads short payloads with a `p` field. Drop it
// before handing the object to the chunk schema. JSON decode goes
@@ -80,11 +103,21 @@ const consumeFrames = (route: string) => (state: FrameBufferState, chunk: Uint8A
route,
payload,
"Failed to parse Bedrock Converse event-stream payload",
).pipe(
Effect.mapError(
(error) =>
new AIError({
reason: AIErrorReason.make({ ...error.reason, message: error.message, cause: error.reason.cause, body }),
}),
),
)) as Record<string, unknown>
delete parsed.p
out.push(
messageType === "exception" ? { exception: { type: eventType, details: parsed } } : { [eventType]: parsed },
)
out.push({
...(messageType === "exception"
? { exception: { type: eventType, details: parsed } }
: { [eventType]: parsed }),
rawBody: body,
})
}
return [cursor, out] as const
})
@@ -97,7 +130,13 @@ const consumeFrames = (route: string) => (state: FrameBufferState, chunk: Uint8A
*/
export const framing = (route: string): Framing.Definition<object> => ({
id: "aws-event-stream",
frame: (bytes) => bytes.pipe(Stream.mapAccumEffect(() => initialFrameBuffer, consumeFrames(route))),
body: (frame) => ("rawBody" in frame && typeof frame.rawBody === "string" ? frame.rawBody : undefined),
frame: (bytes) =>
bytes.pipe(
Stream.map((bytes): FrameInput => ({ _tag: "Chunk", bytes })),
Stream.concat(Stream.succeed(endOfStream)),
Stream.mapAccumEffect(() => initialFrameBuffer, consumeFrames(route)),
),
})
export * as BedrockEventStream from "./bedrock-event-stream.js"
@@ -0,0 +1,96 @@
import { Effect, Encoding, Schema, Struct } from "effect"
import { Headers } from "effect/unstable/http"
import { AIError } from "../schema/index.js"
import { Route } from "../route/client.js"
import { Endpoint } from "../route/endpoint.js"
import { Protocol } from "../route/protocol.js"
import { classifyProviderFailure } from "../provider-error.js"
import { AnthropicMessages } from "./anthropic-messages.js"
import { BedrockEventStream } from "./bedrock-event-stream.js"
import { BedrockAuth } from "./utils/bedrock-auth.js"
import { JsonObject, ProviderShared } from "./shared.js"
const ID = "bedrock-messages"
const VERSION = "bedrock-2023-05-31"
const Body = Schema.Struct({
...Struct.omit(AnthropicMessages.AnthropicMessagesBody.fields, ["model", "stream"]),
anthropic_version: Schema.Literal(VERSION),
anthropic_beta: Schema.optional(Schema.Array(Schema.String)),
})
const Event = Schema.Struct({
chunk: Schema.optional(Schema.Struct({ bytes: Schema.String })),
exception: Schema.optional(
Schema.Struct({
type: Schema.String,
details: Schema.StructWithRest(
Schema.Struct({ message: Schema.optional(Schema.String), originalMessage: Schema.optional(Schema.String) }),
[JsonObject],
),
}),
),
})
export const protocol = Protocol.make({
id: ID,
body: {
schema: Body,
from: Effect.fn("BedrockMessages.fromRequest")(function* (request) {
const body = yield* AnthropicMessages.protocol.body.from(request)
const headers = Headers.fromInput(request.http?.headers)
const betas = new Set(
(headers["anthropic-beta"] ?? "")
.split(",")
.map((value) => value.trim())
.filter(Boolean),
)
if (
body.context_management?.edits.length ||
body.messages.some((message) => message.content.some((block) => block.type === "compaction"))
)
betas.add("compact-2026-01-12")
return {
...Struct.omit(body, ["model", "stream"]),
anthropic_version: VERSION,
anthropic_beta: betas.size ? [...betas] : undefined,
} satisfies typeof Body.Type
}),
},
stream: {
event: Event,
initial: AnthropicMessages.protocol.stream.initial,
step: Effect.fn("BedrockMessages.step")(function* (state, event) {
if (event.exception)
return yield* new AIError({
reason: classifyProviderFailure({
message: event.exception.details.message ?? event.exception.details.originalMessage ?? event.exception.type,
rawBody: ProviderShared.encodeJson(event),
}),
})
if (!event.chunk) return yield* ProviderShared.eventError(ID, "Bedrock Messages event is missing its chunk")
const text = yield* Effect.fromResult(Encoding.decodeBase64String(event.chunk.bytes)).pipe(
Effect.mapError((cause) =>
ProviderShared.eventError(ID, "Invalid Bedrock Messages chunk encoding", undefined, cause),
),
)
const decoded = yield* Schema.decodeUnknownEffect(AnthropicMessages.protocol.stream.event)(text).pipe(
Effect.mapError((cause) => ProviderShared.eventError(ID, "Invalid Bedrock Messages event", undefined, cause)),
)
return yield* AnthropicMessages.protocol.stream.step(state, decoded)
}),
},
})
export const route = Route.make({
id: ID,
provider: "amazon-bedrock",
providerMetadataKey: "anthropic",
protocol,
endpoint: Endpoint.path(
({ request }) => `/model/${encodeURIComponent(request.model.id)}/invoke-with-response-stream`,
{ baseURL: "https://bedrock-runtime.us-east-1.amazonaws.com" },
),
auth: BedrockAuth.auth,
framing: BedrockEventStream.framing(ID),
})
export * as BedrockMessages from "./bedrock-messages.js"
+95 -24
View File
@@ -6,6 +6,7 @@ import { Endpoint } from "../route/endpoint.js"
import { Framing } from "../route/framing.js"
import { Protocol } from "../route/protocol.js"
import {
AIError,
LLMEvent,
Usage,
type FinishReason,
@@ -17,6 +18,7 @@ import {
type ToolCallPart,
type ToolDefinition,
} from "../schema/index.js"
import { classifyProviderFailure } from "../provider-error.js"
import { JsonObject, optionalArray, optionalNull, ProviderShared } from "./shared.js"
import { GeminiToolSchema } from "./utils/gemini-tool-schema.js"
import { Lifecycle } from "./utils/lifecycle.js"
@@ -221,6 +223,7 @@ const GeminiPromptFeedback = Schema.StructWithRest(
type GeminiPromptFeedback = Schema.Schema.Type<typeof GeminiPromptFeedback>
const GeminiEvent = Schema.Struct({
error: Schema.optional(Schema.Unknown),
candidates: optionalNull(Schema.Array(GeminiCandidate)),
promptFeedback: optionalNull(GeminiPromptFeedback),
usageMetadata: optionalNull(GeminiUsage),
@@ -237,6 +240,10 @@ interface ParserState {
readonly lifecycle: Lifecycle.State
readonly reasoningSignature?: string
readonly textSignature?: string
readonly reasoningId?: string
readonly textId?: string
readonly nextReasoningId: number
readonly nextTextId: number
readonly seenCallIds?: ReadonlySet<string>
}
@@ -568,19 +575,23 @@ const finish = (state: ParserState): ReadonlyArray<LLMEvent> => {
const events: LLMEvent[] = []
let lifecycle = state.lifecycle
if (state.reasoningSignature !== undefined)
if (state.reasoningId !== undefined)
lifecycle = Lifecycle.reasoningEnd(
lifecycle,
events,
"reasoning-0",
providerMetadata(state.providerMetadataKey, { thoughtSignature: state.reasoningSignature }),
state.reasoningId,
state.reasoningSignature === undefined
? undefined
: providerMetadata(state.providerMetadataKey, { thoughtSignature: state.reasoningSignature }),
)
if (state.textSignature !== undefined)
if (state.textId !== undefined)
lifecycle = Lifecycle.textEnd(
lifecycle,
events,
"text-0",
providerMetadata(state.providerMetadataKey, { thoughtSignature: state.textSignature }),
state.textId,
state.textSignature === undefined
? undefined
: providerMetadata(state.providerMetadataKey, { thoughtSignature: state.textSignature }),
)
Lifecycle.finish(lifecycle, events, {
reason: {
@@ -598,6 +609,18 @@ const finish = (state: ParserState): ReadonlyArray<LLMEvent> => {
}
const step = (state: ParserState, event: GeminiEvent) => {
if (ProviderShared.isRecord(event.error) && typeof event.error.message === "string") {
const body = ProviderShared.encodeJson(event)
return Effect.fail(
new AIError({
reason: classifyProviderFailure({
message: event.error.message,
status: typeof event.error.code === "number" ? event.error.code : undefined,
rawBody: body,
}),
}),
)
}
const nextState = {
...state,
promptFeedback: event.promptFeedback ?? state.promptFeedback,
@@ -617,6 +640,10 @@ const step = (state: ParserState, event: GeminiEvent) => {
let lifecycle = nextState.lifecycle
let reasoningSignature = nextState.reasoningSignature
let textSignature = nextState.textSignature
let reasoningId = nextState.reasoningId
let textId = nextState.textId
let nextReasoningId = nextState.nextReasoningId
let nextTextId = nextState.nextTextId
// Supplier ids must be tracked across chunks of the same response, not just within one event's parts.
const seenCallIds = new Set(nextState.seenCallIds)
@@ -642,27 +669,51 @@ const step = (state: ParserState, event: GeminiEvent) => {
else if (signature !== undefined && "text" in part) textSignature = signature
if ("text" in part && part.text.length > 0) {
if (part.thought) {
if (textId !== undefined) {
lifecycle = Lifecycle.textEnd(
lifecycle,
events,
textId,
textSignature
? providerMetadata(state.providerMetadataKey, { thoughtSignature: textSignature })
: undefined,
)
textId = undefined
textSignature = undefined
}
if (reasoningId === undefined) {
reasoningId = `reasoning-${nextReasoningId}`
nextReasoningId += 1
}
lifecycle = Lifecycle.reasoningDelta(
lifecycle,
events,
"reasoning-0",
reasoningId,
part.text,
signature ? providerMetadata(state.providerMetadataKey, { thoughtSignature: signature }) : undefined,
)
continue
}
lifecycle = Lifecycle.reasoningEnd(
lifecycle,
events,
"reasoning-0",
reasoningSignature
? providerMetadata(state.providerMetadataKey, { thoughtSignature: reasoningSignature })
: undefined,
)
if (reasoningId !== undefined) {
lifecycle = Lifecycle.reasoningEnd(
lifecycle,
events,
reasoningId,
reasoningSignature
? providerMetadata(state.providerMetadataKey, { thoughtSignature: reasoningSignature })
: undefined,
)
reasoningId = undefined
reasoningSignature = undefined
}
if (textId === undefined) {
textId = `text-${nextTextId}`
nextTextId += 1
}
lifecycle = Lifecycle.textDelta(
lifecycle,
events,
"text-0",
textId,
part.text,
textSignature ? providerMetadata(state.providerMetadataKey, { thoughtSignature: textSignature }) : undefined,
)
@@ -680,14 +731,28 @@ const step = (state: ParserState, event: GeminiEvent) => {
const duplicate = supplied !== undefined && seenCallIds.has(supplied)
if (supplied !== undefined) seenCallIds.add(supplied)
const id = supplied !== undefined && !duplicate ? supplied : `tool_${crypto.randomUUID().replaceAll("-", "")}`
lifecycle = Lifecycle.reasoningEnd(
lifecycle,
events,
"reasoning-0",
reasoningSignature
? providerMetadata(state.providerMetadataKey, { thoughtSignature: reasoningSignature })
: undefined,
)
if (reasoningId !== undefined) {
lifecycle = Lifecycle.reasoningEnd(
lifecycle,
events,
reasoningId,
reasoningSignature
? providerMetadata(state.providerMetadataKey, { thoughtSignature: reasoningSignature })
: undefined,
)
reasoningId = undefined
reasoningSignature = undefined
}
if (textId !== undefined) {
lifecycle = Lifecycle.textEnd(
lifecycle,
events,
textId,
textSignature ? providerMetadata(state.providerMetadataKey, { thoughtSignature: textSignature }) : undefined,
)
textId = undefined
textSignature = undefined
}
lifecycle = Lifecycle.stepStart(lifecycle, events)
events.push(
LLMEvent.toolCall({
@@ -710,6 +775,10 @@ const step = (state: ParserState, event: GeminiEvent) => {
lifecycle,
reasoningSignature,
textSignature,
reasoningId,
textId,
nextReasoningId,
nextTextId,
seenCallIds,
finishReason: candidate.finishReason ?? nextState.finishReason,
},
@@ -737,6 +806,8 @@ export const protocol = Protocol.make({
providerMetadataKey: request.model.route.providerMetadataKey ?? String(request.model.provider),
hasToolCalls: false,
lifecycle: Lifecycle.initial(),
nextReasoningId: 0,
nextTextId: 0,
}),
step,
onHalt: (state) => Effect.succeed(finish(state)),
+11 -34
View File
@@ -9,15 +9,7 @@ import {
type ImageRoute,
} from "../image.js"
import { Auth, type Definition as AuthDefinition } from "../route/auth.js"
import {
InvalidProviderOutputReason,
AIError,
Usage,
mergeHttpOptions,
mergeJsonRecords,
type HttpOptions,
type ProviderMetadata,
} from "../schema/index.js"
import { AIError, Usage, mergeHttpOptions, mergeJsonRecords, type HttpOptions } from "../schema/index.js"
import { ProviderShared } from "./shared.js"
import { ImageInputs } from "./utils/image-input.js"
@@ -124,13 +116,6 @@ const nativeOptions = (options: GoogleImageOptions | undefined) => {
)
}
const invalidOutput = (message: string, providerMetadata?: ProviderMetadata) =>
new AIError({
module: ADAPTER,
method: "generate",
reason: new InvalidProviderOutputReason({ message, route: ADAPTER, providerMetadata }),
})
const applyQuery = (url: string, query: Record<string, string> | undefined) => {
if (!query) return url
const next = new URL(url)
@@ -169,11 +154,9 @@ export const model = (input: ModelInput) => {
HttpClientRequest.bodyText(text, "application/json"),
),
)
const payload = yield* response.json.pipe(
Effect.mapError(() => invalidOutput("Failed to read the Google Images response")),
)
const decoded = yield* Schema.decodeUnknownEffect(GoogleImageResponse)(payload).pipe(
Effect.mapError(() => invalidOutput("Google Images returned an invalid response")),
const output = yield* ProviderShared.imageResponse(ADAPTER, "Google Images", response)
const decoded = yield* Schema.decodeUnknownEffect(Schema.fromJsonString(GoogleImageResponse))(output.body).pipe(
Effect.mapError((cause) => output.invalid("Google Images returned an invalid response", cause)),
)
const candidates = decoded.candidates ?? []
const candidateMetadata = candidates.map((candidate, candidateIndex) => ({
@@ -208,9 +191,10 @@ export const model = (input: ModelInput) => {
)
const images = yield* Effect.forEach(encoded, (item) =>
Effect.fromResult(Encoding.decodeBase64(item.inlineData.data)).pipe(
Effect.mapError(() =>
invalidOutput(
Effect.mapError((cause) =>
output.invalid(
`Google Images candidate ${item.candidateIndex} part ${item.partIndex} contains invalid base64 data`,
cause,
),
),
Effect.map(
@@ -237,16 +221,10 @@ export const model = (input: ModelInput) => {
const finishReasons = candidates.flatMap((candidate) =>
candidate.finishReason === undefined ? [] : [candidate.finishReason],
)
return yield* invalidOutput(
return yield* output.invalid(
`Google Images returned no final images${
finishReasons.length === 0 ? "" : ` (finish reasons: ${finishReasons.join(", ")})`
}; inspect reason.providerMetadata.google for prompt feedback and candidate details`,
{
google: {
promptFeedback: decoded.promptFeedback,
candidates: candidateMetadata,
},
},
}; inspect body for prompt feedback and candidate details`,
)
}
const usage = decoded.usageMetadata
@@ -290,12 +268,11 @@ const googleImagePart = (image: ImageInput): Effect.Effect<Record<string, unknow
return Effect.succeed({ inlineData: { mimeType: image.mediaType, data: Encoding.encodeBase64(image.data) } })
if (image.type === "file-uri") return Effect.succeed({ fileData: { mimeType: image.mediaType, fileUri: image.uri } })
if (image.type === "url")
return ImageInputs.decodeDataUrl(image.url, ADAPTER).pipe(
return ImageInputs.decodeDataUrl(image.url).pipe(
Effect.flatMap((decoded) => {
if (decoded === undefined)
return Effect.fail(
ImageInputs.invalid(
ADAPTER,
"Google generateContent does not fetch public image URLs; use bytes, a data URL, or a Gemini file URI",
),
)
@@ -305,7 +282,7 @@ const googleImagePart = (image: ImageInput): Effect.Effect<Record<string, unknow
}),
)
return Effect.fail(
ImageInputs.invalid(ADAPTER, "Google generateContent requires Gemini file URIs rather than provider file IDs"),
ImageInputs.invalid("Google generateContent requires Gemini file URIs rather than provider file IDs"),
)
}
+2
View File
@@ -1,6 +1,8 @@
export * as AnthropicMessages from "./anthropic-messages.js"
export * as BedrockConverse from "./bedrock-converse.js"
export { BedrockMessages } from "./bedrock-messages.js"
export * as Gemini from "./gemini.js"
export * as MistralChat from "./mistral-chat.js"
export * as OpenAIChat from "./openai-chat.js"
export * as OpenAIImages from "./openai-images.js"
export * as OpenAICompatibleChat from "./openai-compatible-chat.js"
+780
View File
@@ -0,0 +1,780 @@
import { Effect, Schema } from "effect"
import { Auth } from "../route/auth.js"
import { Route } from "../route/client.js"
import { Endpoint } from "../route/endpoint.js"
import { Framing } from "../route/framing.js"
import { Protocol } from "../route/protocol.js"
import { HttpTransport } from "../route/transport/index.js"
import {
AIError,
InvalidProviderOutputError,
LLMEvent,
Usage,
type FinishReasonDetails,
type LLMRequest,
type MediaPart,
type ToolCallPart,
type ToolDefinition,
} from "../schema/index.js"
import { classifyProviderFailure } from "../provider-error.js"
import { JsonObject, optionalArray, optionalNull, ProviderShared } from "./shared.js"
import { Lifecycle } from "./utils/lifecycle.js"
import { ToolStream } from "./utils/tool-stream.js"
const ADAPTER = "mistral-chat"
const DONE = "[DONE]" as const
const TOOL_ID = /^[A-Za-z0-9]{9}$/
export const DEFAULT_BASE_URL = "https://api.mistral.ai/v1"
export const PATH = "/chat/completions"
const MistralTextContent = Schema.Struct({
type: Schema.Literal("text"),
text: Schema.String,
})
const MistralThinkingUnit = Schema.StructWithRest(
Schema.Struct({
type: Schema.optional(Schema.String),
text: Schema.optional(Schema.String),
}),
[Schema.Record(Schema.String, Schema.Unknown)],
)
type MistralThinkingUnit = Schema.Schema.Type<typeof MistralThinkingUnit>
const MistralThinkingContent = Schema.StructWithRest(
Schema.Struct({
type: Schema.Literal("thinking"),
thinking: Schema.Array(MistralThinkingUnit),
}),
[Schema.Record(Schema.String, Schema.Unknown)],
)
type MistralThinkingContent = Schema.Schema.Type<typeof MistralThinkingContent>
const isMistralThinkingContent = Schema.is(MistralThinkingContent)
const MistralUserContent = Schema.Union([
MistralTextContent,
Schema.Struct({ type: Schema.Literal("image_url"), image_url: Schema.String }),
Schema.Struct({ type: Schema.Literal("document_url"), document_url: Schema.String }),
])
type MistralUserContent = Schema.Schema.Type<typeof MistralUserContent>
const MistralAssistantToolCall = Schema.Struct({
id: Schema.String,
type: Schema.Literal("function"),
function: Schema.Struct({ name: Schema.String, arguments: Schema.String }),
})
type MistralAssistantToolCall = Schema.Schema.Type<typeof MistralAssistantToolCall>
const MistralMessage = Schema.Union([
Schema.Struct({ role: Schema.Literal("system"), content: Schema.String }),
Schema.Struct({
role: Schema.Literal("user"),
content: Schema.Union([Schema.String, Schema.Array(MistralUserContent)]),
}),
Schema.Struct({
role: Schema.Literal("assistant"),
content: Schema.Union([Schema.String, Schema.Array(Schema.Union([MistralTextContent, MistralThinkingContent]))]),
tool_calls: optionalArray(MistralAssistantToolCall),
prefix: Schema.optional(Schema.Literal(true)),
}),
Schema.Struct({
role: Schema.Literal("tool"),
tool_call_id: Schema.String,
name: Schema.String,
content: Schema.Union([Schema.String, Schema.Array(MistralUserContent)]),
}),
]).pipe(Schema.toTaggedUnion("role"))
type MistralMessage = Schema.Schema.Type<typeof MistralMessage>
const MistralTool = Schema.Struct({
type: Schema.Literal("function"),
function: Schema.Struct({
name: Schema.String,
description: Schema.String,
parameters: JsonObject,
strict: Schema.Literal(false),
}),
})
type MistralTool = Schema.Schema.Type<typeof MistralTool>
const MistralOptions = Schema.Struct({
safePrompt: Schema.optional(Schema.Boolean),
documentImageLimit: Schema.optional(Schema.Number),
documentPageLimit: Schema.optional(Schema.Number),
parallelToolCalls: Schema.optional(Schema.Boolean),
reasoningEffort: Schema.optional(Schema.String),
promptMode: Schema.optional(Schema.Literal("reasoning")),
promptCacheKey: Schema.optional(Schema.String),
})
export type ReasoningEffort = "none" | "minimal" | "low" | "medium" | "high" | "xhigh" | (string & {})
export type ProviderOptionsInput = {
readonly safePrompt?: boolean
readonly documentImageLimit?: number
readonly documentPageLimit?: number
readonly parallelToolCalls?: boolean
readonly reasoningEffort?: ReasoningEffort
readonly promptMode?: "reasoning"
readonly promptCacheKey?: string
readonly [key: string]: unknown
}
const MistralBody = Schema.Struct({
model: Schema.String,
messages: Schema.Array(MistralMessage),
tools: optionalArray(MistralTool),
tool_choice: Schema.optional(
Schema.Union([
Schema.Literals(["auto", "none", "any"]),
Schema.Struct({ type: Schema.Literal("function"), function: Schema.Struct({ name: Schema.String }) }),
]),
),
stream: Schema.Literal(true),
max_tokens: Schema.optional(Schema.Number),
random_seed: Schema.optional(Schema.Number),
temperature: Schema.optional(Schema.Number),
top_p: Schema.optional(Schema.Number),
frequency_penalty: Schema.optional(Schema.Number),
presence_penalty: Schema.optional(Schema.Number),
stop: optionalArray(Schema.String),
prompt_cache_key: Schema.optional(Schema.String),
safe_prompt: Schema.optional(Schema.Boolean),
document_image_limit: Schema.optional(Schema.Number),
document_page_limit: Schema.optional(Schema.Number),
parallel_tool_calls: Schema.optional(Schema.Boolean),
reasoning_effort: Schema.optional(Schema.String),
prompt_mode: Schema.optional(Schema.Literal("reasoning")),
})
export type MistralBody = Schema.Schema.Type<typeof MistralBody>
const MistralUsageDetails = Schema.StructWithRest(Schema.Struct({ cached_tokens: optionalNull(Schema.Number) }), [
Schema.Record(Schema.String, Schema.Unknown),
])
const MistralUsage = Schema.StructWithRest(
Schema.Struct({
prompt_tokens: optionalNull(Schema.Number),
completion_tokens: optionalNull(Schema.Number),
total_tokens: optionalNull(Schema.Number),
num_cached_tokens: optionalNull(Schema.Number),
prompt_token_details: optionalNull(MistralUsageDetails),
prompt_tokens_details: optionalNull(MistralUsageDetails),
}),
[Schema.Record(Schema.String, Schema.Unknown)],
)
const MistralOutputContent = Schema.StructWithRest(
Schema.Struct({
type: Schema.String,
text: optionalNull(Schema.String),
thinking: optionalNull(Schema.Unknown),
}),
[Schema.Record(Schema.String, Schema.Unknown)],
)
type MistralOutputContent = Schema.Schema.Type<typeof MistralOutputContent>
const MistralToolDelta = Schema.Struct({
index: optionalNull(Schema.Number),
id: optionalNull(Schema.String),
function: optionalNull(
Schema.Struct({
name: optionalNull(Schema.String),
arguments: optionalNull(Schema.Union([Schema.String, JsonObject])),
}),
),
})
type MistralToolDelta = Schema.Schema.Type<typeof MistralToolDelta>
const MistralChoice = Schema.StructWithRest(
Schema.Struct({
delta: optionalNull(
Schema.StructWithRest(
Schema.Struct({
content: optionalNull(Schema.Union([Schema.String, Schema.Array(MistralOutputContent)])),
tool_calls: optionalNull(Schema.Array(MistralToolDelta)),
}),
[Schema.Record(Schema.String, Schema.Unknown)],
),
),
finish_reason: optionalNull(Schema.String),
}),
[Schema.Record(Schema.String, Schema.Unknown)],
)
const MistralError = Schema.StructWithRest(
Schema.Struct({
message: Schema.String,
code: optionalNull(Schema.Union([Schema.String, Schema.Number])),
}),
[Schema.Record(Schema.String, Schema.Unknown)],
)
const MistralEvent = Schema.StructWithRest(
Schema.Struct({
choices: optionalNull(Schema.Array(MistralChoice)),
usage: optionalNull(MistralUsage),
error: optionalNull(MistralError),
}),
[Schema.Record(Schema.String, Schema.Unknown)],
)
type MistralEvent = Schema.Schema.Type<typeof MistralEvent>
const MistralStreamEvent = Schema.Union([Schema.Literal(DONE), Protocol.jsonEvent(MistralEvent)])
const hashID = (value: string) => {
const hash = (seed: number) => {
let result = seed
for (const char of value) result = Math.imul(result ^ char.charCodeAt(0), 16777619)
return (result >>> 0).toString(36)
}
return `${hash(2166136261).padStart(7, "0")}${hash(2246822519).padStart(7, "0")}`.slice(-9)
}
const toolIDNormalizer = (request: LLMRequest) => {
const ids = request.messages.flatMap((message) =>
message.content.flatMap((part) => (part.type === "tool-call" || part.type === "tool-result" ? [part.id] : [])),
)
const used = new Set(ids.filter((id) => TOOL_ID.test(id)))
const normalized = new Map<string, string>()
return (id: string) => {
if (TOOL_ID.test(id)) return id
const previous = normalized.get(id)
if (previous) return previous
let attempt = 0
let candidate = hashID(id)
while (used.has(candidate)) candidate = hashID(`${id}:${++attempt}`)
used.add(candidate)
normalized.set(id, candidate)
return candidate
}
}
const lowerMedia = Effect.fn("MistralChat.lowerMedia")(function* (part: MediaPart) {
const media = ProviderShared.normalizeMedia(part)
const url = typeof part.data === "string" && /^(?:https?:|data:)/.test(part.data) ? part.data : media.dataUrl
if (media.mime.startsWith("image/")) return { type: "image_url" as const, image_url: url }
if (media.mime === "application/pdf") return { type: "document_url" as const, document_url: url }
return yield* ProviderShared.invalidRequest(`Mistral Chat does not support media type ${part.mediaType}`)
})
const lowerUser = Effect.fn("MistralChat.lowerUser")(function* (message: LLMRequest["messages"][number]) {
const content: MistralUserContent[] = []
for (const part of message.content) {
if (part.type === "text") {
content.push({ type: "text", text: part.text })
continue
}
if (part.type === "media") {
content.push(yield* lowerMedia(part))
continue
}
return yield* ProviderShared.unsupportedContent("Mistral Chat", "user", ["text", "media"])
}
if (content.every((part) => part.type === "text"))
return { role: "user" as const, content: content.map((part) => part.text).join("") }
return { role: "user" as const, content }
})
const lowerToolCall = (part: ToolCallPart, normalizeID: (id: string) => string): MistralAssistantToolCall => ({
id: normalizeID(part.id),
type: "function",
function: { name: part.name, arguments: ProviderShared.encodeJson(part.input) },
})
const lowerAssistant = Effect.fn("MistralChat.lowerAssistant")(function* (
message: LLMRequest["messages"][number],
normalizeID: (id: string) => string,
prefix: boolean,
) {
const structured = message.content.some(
(part) => part.type === "reasoning" && isMistralThinkingContent(part.providerMetadata?.mistral?.thinking),
)
const content: Array<Schema.Schema.Type<typeof MistralTextContent> | MistralThinkingContent> = []
const text: string[] = []
const toolCalls: MistralAssistantToolCall[] = []
for (const part of message.content) {
if (part.type === "text") {
if (structured) content.push({ type: "text", text: part.text })
else text.push(part.text)
continue
}
if (part.type === "reasoning") {
const native = part.providerMetadata?.mistral?.thinking
if (structured && isMistralThinkingContent(native)) content.push(native)
else if (structured) content.push({ type: "text", text: part.text })
else text.push(part.text)
continue
}
if (part.type === "tool-call") {
toolCalls.push(lowerToolCall(part, normalizeID))
continue
}
return yield* ProviderShared.unsupportedContent("Mistral Chat", "assistant", ["text", "reasoning", "tool-call"])
}
return {
role: "assistant" as const,
content: structured ? content : text.join(""),
...(toolCalls.length > 0 ? { tool_calls: toolCalls } : {}),
...(prefix ? { prefix: true as const } : {}),
}
})
const lowerToolResults = Effect.fn("MistralChat.lowerToolResults")(function* (
message: LLMRequest["messages"][number],
normalizeID: (id: string) => string,
) {
const output: MistralMessage[] = []
for (const part of message.content) {
if (part.type !== "tool-result")
return yield* ProviderShared.unsupportedContent("Mistral Chat", "tool", ["tool-result"])
if (part.result.type !== "content") {
output.push({
role: "tool",
tool_call_id: normalizeID(part.id),
name: part.name,
content: ProviderShared.toolResultText(part),
})
continue
}
const content: MistralUserContent[] = []
for (const item of part.result.value) {
if (item.type === "text") {
content.push({ type: "text", text: item.text })
continue
}
content.push(yield* lowerMedia({ type: "media", mediaType: item.mime, data: item.uri, filename: item.name }))
}
output.push({
role: "tool",
tool_call_id: normalizeID(part.id),
name: part.name,
content: content.some((item) => item.type !== "text")
? content
: content.map((item) => (item.type === "text" ? item.text : "")).join(""),
})
}
return output
})
const lowerMessages = Effect.fn("MistralChat.lowerMessages")(function* (request: LLMRequest) {
const normalizeID = toolIDNormalizer(request)
const messages: MistralMessage[] =
request.system.length === 0 ? [] : [{ role: "system", content: ProviderShared.joinText(request.system) }]
for (const message of request.messages) {
if (message.role === "system") {
const update = yield* ProviderShared.wrappedSystemUpdate("Mistral Chat", message)
messages.push({
role: "user",
content: update.text,
})
continue
}
if (message.role === "user") {
messages.push(yield* lowerUser(message))
continue
}
if (message.role === "assistant") {
const hasToolCalls = message.content.some((part) => part.type === "tool-call")
const hasNativeThinking = message.content.some(
(part) => part.type === "reasoning" && isMistralThinkingContent(part.providerMetadata?.mistral?.thinking),
)
const text = message.content
.flatMap((part) => (part.type === "text" || part.type === "reasoning" ? [part.text] : []))
.join("")
if (!hasToolCalls && !hasNativeThinking && text.trim() === "") continue
messages.push(yield* lowerAssistant(message, normalizeID, !hasToolCalls && message === request.messages.at(-1)))
continue
}
messages.push(...(yield* lowerToolResults(message, normalizeID)))
}
return messages
})
const lowerTool = (tool: ToolDefinition): MistralTool => ({
type: "function",
function: { name: tool.name, description: tool.description, parameters: tool.inputSchema, strict: false },
})
export const fromRequest = Effect.fn("MistralChat.fromRequest")(function* (request: LLMRequest) {
const options = yield* ProviderShared.validateWith(Schema.decodeUnknownEffect(MistralOptions))(
request.providerOptions ?? {},
)
const selected = request.toolChoice?.type === "tool" ? request.toolChoice.name : undefined
if (request.toolChoice?.type === "tool" && !selected)
return yield* ProviderShared.invalidRequest("Mistral Chat tool choice requires a tool name")
if (options.reasoningEffort !== undefined && options.promptMode !== undefined)
return yield* ProviderShared.invalidRequest(
"Mistral Chat reasoningEffort and promptMode provider options are mutually exclusive",
)
const toolChoice = request.toolChoice
? yield* ProviderShared.matchToolChoice("Mistral Chat", request.toolChoice, {
auto: () => "auto" as const,
none: () => "none" as const,
required: () => "any" as const,
tool: (name) => ({ type: "function" as const, function: { name } }),
})
: undefined
return {
model: request.model.id,
messages: yield* lowerMessages(request),
tools: request.tools.length > 0 ? request.tools.map(lowerTool) : undefined,
tool_choice: toolChoice,
stream: true as const,
max_tokens: request.generation?.maxTokens,
random_seed: request.generation?.seed,
temperature: request.generation?.temperature,
top_p: request.generation?.topP,
frequency_penalty: request.generation?.frequencyPenalty,
presence_penalty: request.generation?.presencePenalty,
stop: request.generation?.stop,
prompt_cache_key: request.cache === "none" ? undefined : (options.promptCacheKey ?? request.promptCacheKey),
safe_prompt: options.safePrompt,
document_image_limit: options.documentImageLimit,
document_page_limit: options.documentPageLimit,
parallel_tool_calls:
options.parallelToolCalls ?? (request.toolChoice?.disableParallelToolUse === true ? false : undefined),
reasoning_effort: options.reasoningEffort,
prompt_mode: options.promptMode,
}
})
type ToolKey = string | number
interface PendingTool {
readonly id: string
readonly name?: string
readonly input: string
}
interface ActiveContent {
readonly type: "text" | "reasoning"
readonly id: string
readonly thinking?: MistralThinkingContent
}
export interface ParserState {
readonly tools: ToolStream.State<ToolKey>
readonly pendingTools: Partial<Record<ToolKey, PendingTool>>
readonly toolIDs: ReadonlyMap<string, string>
readonly usedToolIDs: ReadonlySet<string>
readonly completedTools: ReadonlyArray<LLMEvent>
readonly latestToolKey?: ToolKey
readonly generatedTools: number
readonly lifecycle: Lifecycle.State
readonly active?: ActiveContent
readonly nextContent: number
readonly usage?: Usage
readonly finishReason?: FinishReasonDetails
}
const mapUsage = (usage: MistralEvent["usage"]): Usage | undefined => {
if (!usage) return undefined
const input = usage.prompt_tokens ?? undefined
const reported =
usage.num_cached_tokens ??
usage.prompt_tokens_details?.cached_tokens ??
usage.prompt_token_details?.cached_tokens ??
undefined
const cached = input === undefined || reported === undefined ? undefined : Math.max(0, Math.min(input, reported))
const output = usage.completion_tokens ?? undefined
return new Usage({
inputTokens: input,
outputTokens: output,
nonCachedInputTokens: ProviderShared.subtractTokens(input, cached),
cacheReadInputTokens: cached,
totalTokens: ProviderShared.totalTokens(input, output, usage.total_tokens ?? undefined),
providerMetadata: { mistral: usage },
})
}
const mapFinishReason = (reason: string) => {
switch (reason) {
case "stop":
return "stop" as const
case "length":
case "model_length":
return "length" as const
case "tool_calls":
return "tool-calls" as const
case "content_filter":
return "content-filter" as const
case "error":
case "network_error":
return "error" as const
default:
return "unknown" as const
}
}
const thinkingUnits = (value: unknown): ReadonlyArray<MistralThinkingUnit> => {
if (typeof value === "string") return [{ type: "text", text: value }]
if (!Array.isArray(value)) return []
return value.filter(Schema.is(MistralThinkingUnit))
}
const thinkingText = (thinking: ReadonlyArray<MistralThinkingUnit>) =>
thinking.flatMap((unit) => (typeof unit.text === "string" ? [unit.text] : [])).join("")
const thinkingMetadata = (thinking: MistralThinkingContent) => ({ mistral: { thinking } })
const closeActive = (state: ParserState, events: LLMEvent[]) => {
if (!state.active) return state
const lifecycle =
state.active.type === "text"
? Lifecycle.textEnd(state.lifecycle, events, state.active.id)
: Lifecycle.reasoningEnd(
state.lifecycle,
events,
state.active.id,
thinkingMetadata(state.active.thinking ?? { type: "thinking", thinking: [] }),
thinkingText(state.active.thinking?.thinking ?? []),
)
return { ...state, lifecycle, active: undefined }
}
const appendText = (state: ParserState, events: LLMEvent[], text: string) => {
if (text.length === 0) return state
const current = state.active?.type === "text" ? state : closeActive(state, events)
const active = current.active ?? { type: "text" as const, id: `text-${current.nextContent}` }
return {
...current,
lifecycle: Lifecycle.textDelta(current.lifecycle, events, active.id, text),
active,
nextContent: current.active ? current.nextContent : current.nextContent + 1,
}
}
const appendThinking = (state: ParserState, events: LLMEvent[], part: MistralOutputContent) => {
const current = state.active?.type === "reasoning" ? state : closeActive(state, events)
const units = thinkingUnits(part.thinking)
const active = current.active ?? { type: "reasoning" as const, id: `reasoning-${current.nextContent}` }
const thinking = {
...active.thinking,
...part,
type: "thinking" as const,
thinking: [...(active.thinking?.thinking ?? []), ...units],
}
const text = thinkingText(units)
return {
...current,
lifecycle:
text.length > 0
? Lifecycle.reasoningDelta(current.lifecycle, events, active.id, text, thinkingMetadata(thinking))
: Lifecycle.reasoningStart(current.lifecycle, events, active.id, thinkingMetadata(thinking)),
active: { ...active, thinking },
nextContent: current.active ? current.nextContent : current.nextContent + 1,
}
}
const appendContent = (
state: ParserState,
events: LLMEvent[],
content: string | ReadonlyArray<MistralOutputContent>,
) => {
if (typeof content === "string") return appendText(state, events, content)
return content.reduce((current, part) => {
if (part.type === "text") return appendText(current, events, part.text ?? "")
if (part.type === "thinking") return appendThinking(current, events, part)
return closeActive(current, events)
}, state)
}
const normalizeStreamToolID = (state: ParserState, source: string) => {
if (TOOL_ID.test(source))
return { id: source, state: { ...state, usedToolIDs: new Set([...state.usedToolIDs, source]) } }
const previous = state.toolIDs.get(source)
if (previous) return { id: previous, state }
let attempt = 0
let id = hashID(source)
while (state.usedToolIDs.has(id)) id = hashID(`${source}:${++attempt}`)
return {
id,
state: {
...state,
toolIDs: new Map([...state.toolIDs, [source, id]]),
usedToolIDs: new Set([...state.usedToolIDs, id]),
},
}
}
const toolText = (tool: MistralToolDelta) => {
const value = tool.function?.arguments
if (typeof value === "string") return value
return value === null || value === undefined ? "" : ProviderShared.encodeJson(value)
}
const appendTools = Effect.fn("MistralChat.appendTools")(function* (
initial: ParserState,
events: LLMEvent[],
deltas: ReadonlyArray<MistralToolDelta>,
) {
if (deltas.length === 0) return initial
let state = closeActive(initial, events)
for (const [position, delta] of deltas.entries()) {
const wireID = delta.id?.trim() || undefined
const providedID = wireID === "null" ? undefined : wireID
const key =
delta.index ??
(providedID
? `id:${providedID}`
: deltas.length > 1
? `position:${position}`
: (state.latestToolKey ?? `missing:${state.generatedTools}`))
const existing = state.tools[key]
const pending = state.pendingTools[key]
const source = providedID ?? `generated:${String(key)}`
const normalized =
existing || pending ? { id: existing?.id ?? pending?.id ?? "", state } : normalizeStreamToolID(state, source)
state = normalized.state
const name = existing?.name ?? pending?.name ?? (delta.function?.name?.trim() || undefined)
const text = `${pending?.input ?? ""}${toolText(delta)}`
if (!name) {
state = {
...state,
pendingTools: { ...state.pendingTools, [key]: { id: normalized.id, input: text } },
latestToolKey: key,
generatedTools: state.generatedTools + (!providedID && !pending ? 1 : 0),
}
continue
}
const result = ToolStream.appendOrStart(
ADAPTER,
state.tools,
key,
{ id: normalized.id, name, text },
"Mistral Chat tool call delta is missing a name",
)
if (ToolStream.isError(result)) return yield* result
if (result.events.length > 0) state = { ...state, lifecycle: Lifecycle.stepStart(state.lifecycle, events) }
events.push(...result.events)
const pendingTools = { ...state.pendingTools }
delete pendingTools[key]
state = {
...state,
tools: result.tools,
pendingTools,
latestToolKey: key,
generatedTools: state.generatedTools + (!providedID && !existing && !pending ? 1 : 0),
}
}
return state
})
const hasLateContent = (event: MistralEvent) => {
const delta = event.choices?.[0]?.delta
if (typeof delta?.content === "string" && delta.content.length > 0) return true
if (Array.isArray(delta?.content) && delta.content.length > 0) return true
return (delta?.tool_calls ?? []).some(
(tool) => Boolean(tool.id) || Boolean(tool.function?.name) || tool.function?.arguments !== undefined,
)
}
const step = Effect.fn("MistralChat.step")(function* (state: ParserState, event: MistralEvent) {
if (event.error) {
const body = ProviderShared.encodeJson(event)
return yield* new AIError({
reason: classifyProviderFailure({
message: event.error.message,
status: typeof event.error.code === "number" ? event.error.code : undefined,
rawBody: body,
}),
})
}
const events: LLMEvent[] = []
const usage = mapUsage(event.usage) ?? state.usage
if (state.finishReason) {
if (hasLateContent(event))
return yield* ProviderShared.eventError(
ADAPTER,
"Mistral Chat received content after the finish reason",
ProviderShared.encodeJson(event),
)
return [{ ...state, usage }, events] as const
}
const choice = event.choices?.[0]
const withContent = choice?.delta?.content == null ? state : appendContent(state, events, choice.delta.content)
const withTools = yield* appendTools(withContent, events, choice?.delta?.tool_calls ?? [])
if (!choice?.finish_reason) return [{ ...withTools, usage }, events] as const
const finishReason = {
normalized: mapFinishReason(choice.finish_reason),
raw: choice.finish_reason,
}
const incomplete = finishReason.normalized === "length" || finishReason.normalized === "content-filter"
if (!incomplete && Object.keys(withTools.pendingTools).length > 0)
return yield* ProviderShared.eventError(
ADAPTER,
"Mistral Chat tool call delta is missing a name",
ProviderShared.encodeJson(event),
)
const finished =
!incomplete && Object.keys(withTools.tools).length > 0
? yield* ToolStream.finishAll(ADAPTER, withTools.tools)
: undefined
return [
{
...withTools,
tools: finished?.tools ?? withTools.tools,
completedTools: finished?.events ?? withTools.completedTools,
usage,
finishReason,
},
events,
] as const
})
const finishEvents = Effect.fn("MistralChat.finishEvents")(function* (state: ParserState) {
if (!state.finishReason)
return yield* new AIError({
reason: new InvalidProviderOutputError({
message: "Mistral Chat stream ended without finish_reason",
classification: "incomplete-stream",
route: ADAPTER,
}),
})
const events: LLMEvent[] = []
const closed = closeActive(state, events)
const lifecycle = closed.completedTools.length > 0 ? Lifecycle.stepStart(closed.lifecycle, events) : closed.lifecycle
events.push(...closed.completedTools)
const reason =
state.finishReason.normalized === "stop" && closed.completedTools.some(LLMEvent.is.toolCall)
? { ...state.finishReason, normalized: "tool-calls" as const }
: state.finishReason
Lifecycle.finish(lifecycle, events, { reason, usage: closed.usage })
return events
})
export const protocol = Protocol.make({
id: ADAPTER,
body: { schema: MistralBody, from: fromRequest },
stream: {
event: MistralStreamEvent,
initial: (): ParserState => ({
tools: ToolStream.empty<ToolKey>(),
pendingTools: {},
toolIDs: new Map(),
usedToolIDs: new Set(),
completedTools: [],
generatedTools: 0,
lifecycle: Lifecycle.initial(),
nextContent: 0,
}),
step: (state: ParserState, event) => (event === DONE ? Effect.succeed([state, []] as const) : step(state, event)),
terminal: (event) => event === DONE,
onHalt: finishEvents,
},
})
export const framing = Framing.sseWithDone
export const httpTransport = HttpTransport.sseJson.with<MistralBody>().with({ framing })
export const route = Route.make({
id: ADAPTER,
provider: "mistral",
providerMetadataKey: "mistral",
protocol,
endpoint: Endpoint.path(PATH, { baseURL: DEFAULT_BASE_URL }),
auth: Auth.none,
transport: httpTransport,
})
export * as MistralChat from "./mistral-chat.js"
@@ -1,6 +1,7 @@
import { Effect, Schema, Stream } from "effect"
import { Headers } from "effect/unstable/http"
import { Framing } from "../route/framing.js"
import type { HttpContext } from "../schema/index.js"
import {
HttpTransport,
WebSocketTransport,
@@ -60,8 +61,8 @@ const driver = (options: Options, body: string): WebSocketChannelDriver => {
observe: (_create, frame) =>
Effect.gen(function* () {
const event = yield* decodeEvent(frame).pipe(
Effect.mapError(() =>
ProviderShared.eventError(options.id, `Invalid ${options.name} WebSocket event`, frame),
Effect.mapError((cause) =>
ProviderShared.eventError(options.id, `Invalid ${options.name} WebSocket event`, frame, cause),
),
)
if (terminal)
@@ -73,13 +74,13 @@ const driver = (options: Options, body: string): WebSocketChannelDriver => {
if (event.type === "error") {
terminal = true
yield* OpenResponses.decodeKnownErrorEvent(event).pipe(
Effect.mapError(() =>
ProviderShared.eventError(options.id, `${options.name} returned a malformed error event`, frame),
Effect.mapError((cause) =>
ProviderShared.eventError(options.id, `${options.name} returned a malformed error event`, frame, cause),
),
)
return {
type: "provider-failure",
error: OpenResponses.providerFailure(options.id, event, `${options.name} stream error`),
error: OpenResponses.providerFailure(event, `${options.name} stream error`, frame),
}
}
if (event.type === "response.failed") {
@@ -92,7 +93,7 @@ const driver = (options: Options, body: string): WebSocketChannelDriver => {
)
return {
type: "provider-failure",
error: OpenResponses.providerFailure(options.id, event, `${options.name} response failed`),
error: OpenResponses.providerFailure(event, `${options.name} response failed`, frame),
}
}
if (event.type === "response.created") {
@@ -175,23 +176,37 @@ export const transport = <Body>(options: Options): Transport<Body, Prepared, str
channel,
}
}),
execute: (prepared, request, runtime, executeOptions) => {
if (!executeOptions?.webSocket || !prepared.channel) return http.execute(prepared.http, request, runtime)
const exchange: WebSocketChannelExchange = {
id: request.id ?? "request",
connect: {
url: prepared.channel.url,
headers: prepared.channel.headers,
rotateAfterMs: prepared.channel.rotateAfterMs,
},
fallback: () =>
Stream.unwrap(
http.execute(prepared.http, request, runtime).pipe(Effect.map((execution) => execution.frames)),
),
driver: prepared.channel.driver,
}
return executeOptions.webSocket.execute(exchange)
},
execute: (prepared, request, runtime, executeOptions) =>
Effect.gen(function* () {
if (!executeOptions?.webSocket || !prepared.channel) return yield* http.execute(prepared.http, request, runtime)
let fallbackHttp: HttpContext | undefined
const exchange: WebSocketChannelExchange = {
id: request.id ?? "request",
connect: {
url: prepared.channel.url,
headers: prepared.channel.headers,
rotateAfterMs: prepared.channel.rotateAfterMs,
},
fallback: () =>
Stream.unwrap(
http.execute(prepared.http, request, runtime).pipe(
Effect.map((execution) => {
fallbackHttp = execution.http
return execution.frames
}),
),
),
driver: prepared.channel.driver,
}
const execution = yield* executeOptions.webSocket.execute(exchange)
return {
frames: execution.frames,
complete: execution.complete,
get http() {
return fallbackHttp ?? execution.http
},
}
}),
}
}
@@ -1,4 +1,4 @@
import { AIError, TransportReason } from "../schema/index.js"
import { AIError, TransportError } from "../schema/index.js"
import type { ChannelCheckpoint, ChannelObservation, WebSocketChannelDriver } from "../route/transport/index.js"
import { Effect, Option, Schema } from "effect"
import * as ProviderShared from "./shared.js"
@@ -99,17 +99,17 @@ const incremental = (
const code = (event: OpenResponses.Event) => event.code || event.error?.code || event.response?.error?.code || undefined
const rejected = (
input: DriverInput,
observation: Extract<ChannelObservation, { readonly type: "provider-failure" }>,
recovery: "retry-full" | "rotate-and-retry-full",
): ChannelObservation => ({
type: "rejected",
recovery,
error: new AIError({
module: input.id,
method: "stream",
reason: new TransportReason({
reason: new TransportError({
message: observation.error.message,
body: observation.error.reason.body,
http: observation.error.reason.http,
cause: observation.error.reason.cause,
transport: "websocket",
operation: "read",
phase: "receive",
@@ -137,15 +137,16 @@ export const driver = (input: DriverInput): WebSocketChannelDriver => {
observe: (create, frame) =>
Effect.gen(function* () {
const event = yield* decodeEvent(frame).pipe(
Effect.mapError(() => ProviderShared.eventError(input.id, `Invalid ${input.name} WebSocket event`, frame)),
Effect.mapError((cause) =>
ProviderShared.eventError(input.id, `Invalid ${input.name} WebSocket event`, frame, cause),
),
)
const observation = yield* input.base.observe(create, frame)
if (event.type === "response.output_item.done" && event.item) output.push(event.item)
if (observation.type === "provider-failure") {
const rejection = code(event)
if (rejection === "previous_response_not_found") return rejected(input, observation, "retry-full")
if (rejection === "websocket_connection_limit_reached")
return rejected(input, observation, "rotate-and-retry-full")
if (rejection === "previous_response_not_found") return rejected(observation, "retry-full")
if (rejection === "websocket_connection_limit_reached") return rejected(observation, "rotate-and-retry-full")
}
if (observation.type !== "completed") return observation
const responseID = event.response?.id
+348 -168
View File
@@ -1,11 +1,11 @@
import { Effect, Schema } from "effect"
import { Effect, Option, Schema } from "effect"
import type { Content } from "@opencode-ai/schema/tool"
import { HttpTransport } from "../route/transport/index.js"
import { Protocol } from "../route/protocol.js"
import {
AIError,
LLMEvent,
ProviderInternalReason,
ProviderInternalError,
Usage,
type FinishReason,
type JsonSchema,
@@ -32,13 +32,14 @@ export const PATH = "/responses"
// =============================================================================
// Request Body Schema
// =============================================================================
const OpenResponsesInputText = Schema.Struct({
export const OpenResponsesInputText = Schema.Struct({
type: Schema.tag("input_text"),
text: Schema.String,
})
const OpenResponsesInputImage = Schema.Struct({
export const OpenResponsesInputImage = Schema.Struct({
type: Schema.tag("input_image"),
image_url: Schema.String,
detail: Schema.optional(Schema.String),
})
const OpenResponsesInputFile = Schema.Struct({
type: Schema.tag("input_file"),
@@ -54,7 +55,7 @@ const MediaInput = Schema.Union([OpenResponsesInputImage, OpenResponsesInputFile
export type MediaInput = Schema.Schema.Type<typeof MediaInput>
const OpenResponsesInputContent = Schema.Union([OpenResponsesInputText, MediaInput])
const OpenResponsesOutputText = Schema.Struct({
export const OpenResponsesOutputText = Schema.Struct({
type: Schema.tag("output_text"),
text: Schema.String,
})
@@ -62,6 +63,13 @@ const OpenResponsesOutputText = Schema.Struct({
export const MessagePhase = Schema.NullOr(Schema.Literals(["commentary", "final_answer"]))
type MessagePhase = Schema.Schema.Type<typeof MessagePhase>
export const MessageMetadata = Schema.Struct({
itemId: Schema.optional(Schema.String),
type: Schema.optional(Schema.Literal("message")),
status: Schema.optional(Schema.String),
phase: Schema.optional(MessagePhase),
})
const messagePhase = (value: unknown): MessagePhase | undefined => {
if (value === null || value === "commentary" || value === "final_answer") return value
return undefined
@@ -72,7 +80,7 @@ const OpenResponsesReasoningSummaryText = Schema.Struct({
text: Schema.String,
})
const OpenResponsesReasoningItem = Schema.Struct({
export const OpenResponsesReasoningItem = Schema.Struct({
type: Schema.tag("reasoning"),
id: Schema.optionalKey(Schema.String),
summary: Schema.Array(OpenResponsesReasoningSummaryText),
@@ -149,16 +157,30 @@ const OpenResponsesFunctionCallOutput = Schema.Union([
Schema.Array(OpenResponsesFunctionCallOutputContent),
])
export const CompactionItem = Schema.Struct({
type: Schema.Literal("compaction"),
id: optionalNull(Schema.String),
encrypted_content: Schema.String,
})
export const InputItem = Schema.Union([
CompactionItem,
Schema.Struct({ role: Schema.tag("system"), content: Schema.String }),
Schema.Struct({ role: Schema.tag("developer"), content: Schema.String }),
Schema.Struct({ role: Schema.tag("user"), content: Schema.Array(OpenResponsesInputContent) }),
Schema.Struct({
role: Schema.tag("user"),
content: Schema.Array(OpenResponsesInputContent),
type: Schema.optional(Schema.Literal("message")),
id: Schema.optional(Schema.String),
status: Schema.optional(Schema.String),
}),
Schema.Struct({
type: Schema.tag("message"),
id: Schema.optionalKey(Schema.String),
role: Schema.tag("assistant"),
content: Schema.Array(OpenResponsesOutputText),
phase: Schema.optionalKey(MessagePhase),
status: Schema.optional(Schema.String),
}),
OpenResponsesReasoningItem,
Schema.Struct({
@@ -267,7 +289,7 @@ const OpenResponsesBody = Schema.Struct({
})
export type OpenResponsesBody = Schema.Schema.Type<typeof OpenResponsesBody>
const OpenResponsesUsage = Schema.Struct({
export const OpenResponsesUsage = Schema.Struct({
input_tokens: Schema.optional(Schema.Number),
input_tokens_details: optionalNull(
Schema.Struct({
@@ -387,21 +409,25 @@ export interface Extension {
const BASE: Extension = { id: ADAPTER, name: NAME }
export interface ParserState {
readonly provider: LLMRequest["model"]["provider"]
readonly completedCompactions: ReadonlySet<string>
readonly id: string
readonly name: string
readonly providerMetadataKey: string
readonly tools: ToolStream.State<string>
// Call ids stay independent of item ids, which may be omitted or reused.
readonly completedTools: ReadonlySet<string>
readonly hasFunctionCall: boolean
readonly lifecycle: Lifecycle.State
readonly outputItems: Readonly<Record<number, string>>
readonly messageItems: ReadonlySet<string>
readonly messagePhases: Readonly<Record<string, MessagePhase | null>>
readonly message: { readonly id: string; readonly phase: MessagePhase | null | undefined } | undefined
readonly reasoningItems: Readonly<Record<string, ReasoningStreamItem>>
}
type ReasoningSummaryStatus = "active" | "can-conclude" | "concluded"
interface ReasoningStreamItem {
readonly open: boolean
readonly encryptedContent: string | null | undefined
// Keyed by the wire protocol's numeric `summary_index`. JS object keys coerce to
// strings, but typing the map as `Record<number, ...>` documents intent
@@ -499,7 +525,13 @@ const lowerMedia = Effect.fn("OpenResponses.lowerMedia")(function* (
...(url ? { file_url: url } : { file_data: media.dataUrl }),
}
}
return { type: "input_image" as const, image_url: url ?? media.dataUrl }
return {
type: "input_image" as const,
image_url: url ?? media.dataUrl,
detail: yield* ProviderShared.validateWith(Schema.decodeUnknownEffect(OpenResponsesInputImage.fields.detail))(
part.providerMetadata?.[request.model.route.providerMetadataKey ?? "openresponses"]?.detail,
),
}
})
const lowerUserContent = Effect.fnUntraced(function* (
@@ -566,6 +598,9 @@ const lowerMessages = Effect.fn("OpenResponses.lowerMessages")(function* (reques
const providerMetadataKey = request.model.route.providerMetadataKey ?? "openresponses"
for (const message of request.messages) {
const metadata = yield* ProviderShared.validateWith(
Schema.decodeUnknownEffect(Schema.UndefinedOr(MessageMetadata)),
)(message.providerMetadata?.[providerMetadataKey])
if (message.role === "system") {
input.push({
role: "developer",
@@ -575,10 +610,9 @@ const lowerMessages = Effect.fn("OpenResponses.lowerMessages")(function* (reques
}
if (message.role === "user") {
input.push({
role: "user",
content: yield* Effect.forEach(message.content, (part) => lowerUserContent(part, request, extension)),
})
const content = yield* Effect.forEach(message.content, (part) => lowerUserContent(part, request, extension))
if (content.length > 0)
input.push({ role: "user", content, type: metadata?.type, id: metadata?.itemId, status: metadata?.status })
continue
}
@@ -591,9 +625,10 @@ const lowerMessages = Effect.fn("OpenResponses.lowerMessages")(function* (reques
const groups = content.reduce<
Array<{ id: string | undefined; phase: MessagePhase | null | undefined; parts: TextPart[] }>
>((groups, part) => {
const metadata = part.providerMetadata?.[providerMetadataKey]
const id = itemID(part.providerMetadata, providerMetadataKey)
const phase = ProviderShared.isRecord(metadata) ? messagePhase(metadata.phase) : undefined
const partMetadata = part.providerMetadata?.[providerMetadataKey]
const id = itemID(part.providerMetadata, providerMetadataKey) ?? metadata?.itemId
const partPhase = messagePhase(partMetadata?.phase)
const phase = partPhase === undefined ? metadata?.phase : partPhase
const group = groups.at(-1)
if (group && group.id === id && group.phase === phase) group.parts.push(part)
else groups.push({ id, phase, parts: [part] })
@@ -604,6 +639,7 @@ const lowerMessages = Effect.fn("OpenResponses.lowerMessages")(function* (reques
type: "message" as const,
...(group.id === undefined ? {} : { id: group.id }),
role: "assistant" as const,
status: metadata?.status,
content: group.parts.map((part) => ({ type: "output_text" as const, text: part.text })),
...(group.phase === undefined ? {} : { phase: group.phase }),
})),
@@ -611,6 +647,15 @@ const lowerMessages = Effect.fn("OpenResponses.lowerMessages")(function* (reques
content.splice(0, content.length)
}
for (const part of message.content) {
if (part.type === "compaction") {
flushText()
if (part.provider !== request.model.provider || part.encrypted === undefined)
return yield* ProviderShared.invalidRequest(
"Compaction state must be replayed to its originating provider and API",
)
input.push({ type: "compaction", id: part.id, encrypted_content: part.encrypted })
continue
}
if (part.type === "text") {
content.push(part)
continue
@@ -778,7 +823,7 @@ export const fromRequest = Effect.fn("OpenResponses.fromRequest")(function* (req
// cached-read and cache-write subsets, and `output_tokens` (inclusive total)
// with a `reasoning_tokens` subset. Pass the totals through and derive the
// non-cached breakdown.
const mapUsage = (usage: OpenResponsesUsage | null | undefined, providerMetadataKey: string) => {
export const mapUsage = (usage: OpenResponsesUsage | null | undefined, providerMetadataKey: string) => {
if (!usage) return undefined
const cached = usage.input_tokens_details?.cached_tokens
const cacheWrite = usage.input_tokens_details?.cache_write_tokens
@@ -826,16 +871,16 @@ const TERMINAL_TYPES = new Set(["error", "response.completed", "response.incompl
export const terminal = (event: Event) => TERMINAL_TYPES.has(event.type)
const onOutputTextDelta = (state: ParserState, event: Event, id: string): StepResult => {
if (!event.delta || !state.messageItems.has(id)) return [state, NO_EVENTS]
if (!event.delta || state.message?.id !== id) return [state, NO_EVENTS]
const events: LLMEvent[] = []
const phase = state.messagePhases[id]
const phase = state.message.phase
const metadata = providerMetadata(state, { itemId: id, ...(phase === undefined ? {} : { phase }) })
const lifecycle = Lifecycle.textStart(state.lifecycle, events, id, metadata)
return [{ ...state, lifecycle: Lifecycle.textDelta(lifecycle, events, id, event.delta) }, events]
}
const onOutputTextDone = (state: ParserState, event: Event, id: string): StepResult => {
if (state.messageItems.has(id)) {
if (state.message?.id === id) {
if (state.lifecycle.text.has(id) || event.text === undefined) return [state, NO_EVENTS]
return onOutputTextDelta(state, { ...event, delta: event.text }, id)
}
@@ -843,21 +888,80 @@ const onOutputTextDone = (state: ParserState, event: Event, id: string): StepRes
return [{ ...state, lifecycle: Lifecycle.textEnd(state.lifecycle, events, id) }, events]
}
const decodeMessagePart = Schema.decodeUnknownOption(
Schema.Union([OpenResponsesOutputText, Schema.Struct({ type: Schema.tag("refusal"), refusal: Schema.String })]),
)
const decodeSummaryPart = Schema.decodeUnknownOption(OpenResponsesReasoningSummaryText)
const decodeReasoningPart = Schema.decodeUnknownOption(
Schema.Struct({ type: Schema.tag("reasoning_text"), text: Schema.String }),
)
const joinReasoningText = (parts: ReadonlyArray<string | undefined>) => {
if (!parts.some((part) => part !== undefined && part.length > 0)) return undefined
return parts.filter((part) => part !== undefined).join("\n\n")
}
export const outputItemID = (state: ParserState, event: Event) =>
event.output_index === undefined ? event.item_id : (state.outputItems[event.output_index] ?? event.item_id)
export const onReasoningDelta = (state: ParserState, event: Event, itemID: string): StepResult => {
const startReasoningSummaryPart = (state: ParserState, itemID: string, index: number): StepResult => {
const item = state.reasoningItems[itemID]
if (!event.delta || !item) return [state, NO_EVENTS]
const index = event.summary_index ?? 0
if (!item?.open || index === 0 || item.summaryParts[index] !== undefined) return [state, NO_EVENTS]
const events: LLMEvent[] = []
const lifecycle = Object.entries(item.summaryParts)
.filter((entry) => entry[1] !== "concluded")
.reduce(
(lifecycle, entry) =>
Lifecycle.reasoningEnd(lifecycle, events, `${itemID}:${entry[0]}`, providerMetadata(state, { itemId: itemID })),
state.lifecycle,
)
return [
{
...state,
lifecycle: Lifecycle.reasoningDelta(state.lifecycle, events, `${itemID}:${index}`, event.delta),
lifecycle: Lifecycle.reasoningStart(
lifecycle,
events,
`${itemID}:${index}`,
providerMetadata(state, { itemId: itemID, reasoningEncryptedContent: item.encryptedContent ?? null }),
),
reasoningItems: {
...state.reasoningItems,
[itemID]: { ...item, deltaIndexes: new Set([...item.deltaIndexes, index]) },
[itemID]: {
...item,
summaryParts: {
...Object.fromEntries(
Object.entries(item.summaryParts).map((entry) =>
entry[1] === "concluded" ? entry : [entry[0], "concluded" as const],
),
),
[index]: "active",
},
},
},
},
events,
]
}
export const onReasoningDelta = (state: ParserState, event: Event, itemID: string): StepResult => {
const item = state.reasoningItems[itemID]
if (!event.delta || !item?.open) return [state, NO_EVENTS]
const index = event.summary_index ?? 0
if (item.summaryParts[index] === "concluded") return [state, NO_EVENTS]
const [started, emitted] = startReasoningSummaryPart(state, itemID, index)
const current = started.reasoningItems[itemID]
if (!current) return [started, emitted]
const events: LLMEvent[] = [...emitted]
return [
{
...started,
lifecycle: Lifecycle.reasoningDelta(started.lifecycle, events, `${itemID}:${index}`, event.delta),
reasoningItems: {
...started.reasoningItems,
[itemID]: { ...current, deltaIndexes: new Set([...current.deltaIndexes, index]) },
},
},
events,
@@ -869,7 +973,7 @@ export const onReasoningDelta = (state: ParserState, event: Event, itemID: strin
// as a single delta unless that summary index already streamed one.
export const onReasoningDone = (state: ParserState, event: Event, itemID: string): StepResult => {
const item = state.reasoningItems[itemID]
if (!item || typeof event.text !== "string") return [state, NO_EVENTS]
if (!item?.open || typeof event.text !== "string") return [state, NO_EVENTS]
const index = event.summary_index ?? 0
if (item.deltaIndexes.has(index)) return [state, NO_EVENTS]
return onReasoningDelta(state, { ...event, delta: event.text }, itemID)
@@ -878,32 +982,48 @@ export const onReasoningDone = (state: ParserState, event: Event, itemID: string
const reasoningMetadata = (state: ParserState, item: StreamItem & { id: string }) =>
providerMetadata(state, { itemId: item.id, reasoningEncryptedContent: item.encrypted_content ?? null })
// Responses APIs stream reasoning items in a stable order:
// Responses APIs normally stream reasoning items in this order:
// `output_item.added` (reasoning) →
// `reasoning_summary_part.added` (index=0) →
// `reasoning_summary_text.delta` →
// `reasoning_summary_part.done` (index=0) →
// (repeat for index>0) →
// `output_item.done` (reasoning).
// The handlers below rely on this ordering: `onOutputItemAdded` seeds the
// per-item entry, `onReasoningSummaryPartAdded` for `summary_index === 0`
// short-circuits when the entry already exists, and higher-index handlers
// fold against the same entry. Behaviour for out-of-order events is
// best-effort, not guaranteed.
// `onOutputItemAdded` seeds the per-item entry, while each later part start is
// also an implicit boundary for the previous part. This keeps the common event
// lifecycle ordered when a compatible provider omits or delays a part-done event.
const onOutputItemAdded = (state: ParserState, event: Event): StepResult => {
const item = event.item
if (item?.type === "message" && item.id !== undefined) {
const itemID = item.id
const phase = messagePhase(item.phase)
// A new message closes earlier messages, including ones that never streamed.
const events: LLMEvent[] = []
const lifecycle = [...state.lifecycle.text]
.filter((id) => id !== itemID)
.reduce((lifecycle, id) => {
const openPhase = state.message?.id === id ? state.message.phase : undefined
return Lifecycle.textEnd(
lifecycle,
events,
id,
providerMetadata(state, { itemId: id, ...(openPhase === undefined ? {} : { phase: openPhase }) }),
)
}, state.lifecycle)
return [
{
...state,
messageItems: new Set([...state.messageItems, item.id]),
messagePhases: phase === undefined ? state.messagePhases : { ...state.messagePhases, [item.id]: phase },
lifecycle,
message: {
id: itemID,
phase: phase === undefined && state.message?.id === itemID ? state.message.phase : phase,
},
},
NO_EVENTS,
events,
]
}
if (item && isReasoningItem(item)) {
if (state.reasoningItems[item.id] !== undefined) return [state, NO_EVENTS]
const events: LLMEvent[] = []
return [
{
@@ -912,6 +1032,7 @@ const onOutputItemAdded = (state: ParserState, event: Event): StepResult => {
reasoningItems: {
...state.reasoningItems,
[item.id]: {
open: true,
encryptedContent: item.encrypted_content,
summaryParts: { 0: "active" },
deltaIndexes: new Set(),
@@ -923,6 +1044,8 @@ const onOutputItemAdded = (state: ParserState, event: Event): StepResult => {
}
if (item?.type !== "function_call" || !item.call_id) return [state, NO_EVENTS]
const id = item.id ?? item.call_id
if (Object.values(state.tools).some((tool) => tool?.id === item.call_id) || state.completedTools.has(item.call_id))
return [state, NO_EVENTS]
const metadata = item.id !== undefined ? providerMetadata(state, { itemId: item.id }) : undefined
const events: LLMEvent[] = []
const lifecycle = Lifecycle.stepStart(state.lifecycle, events)
@@ -943,55 +1066,14 @@ const onOutputItemAdded = (state: ParserState, event: Event): StepResult => {
const onReasoningSummaryPartAdded = (state: ParserState, event: Event): StepResult => {
if (event.item_id === undefined || event.summary_index === undefined) return [state, NO_EVENTS]
const item = state.reasoningItems[event.item_id]
if (!item) return [state, NO_EVENTS]
if (event.summary_index === 0) return [state, NO_EVENTS]
const events: LLMEvent[] = []
const closed = Object.entries(item.summaryParts)
.filter((entry) => entry[1] === "can-conclude")
.reduce(
(lifecycle, entry) =>
Lifecycle.reasoningEnd(
lifecycle,
events,
`${event.item_id}:${entry[0]}`,
providerMetadata(state, { itemId: event.item_id }),
),
state.lifecycle,
)
return [
{
...state,
lifecycle: Lifecycle.reasoningStart(
closed,
events,
`${event.item_id}:${event.summary_index}`,
providerMetadata(state, { itemId: event.item_id, reasoningEncryptedContent: item.encryptedContent ?? null }),
),
reasoningItems: {
...state.reasoningItems,
[event.item_id]: {
...item,
summaryParts: {
...Object.fromEntries(
Object.entries(item.summaryParts).map((entry) =>
entry[1] === "can-conclude" ? [entry[0], "concluded" as const] : entry,
),
),
[event.summary_index]: "active",
},
},
},
},
events,
]
return startReasoningSummaryPart(state, event.item_id, event.summary_index)
}
const onReasoningSummaryPartDone = (state: ParserState, event: Event): StepResult => {
if (event.item_id === undefined || event.summary_index === undefined) return [state, NO_EVENTS]
const item = state.reasoningItems[event.item_id]
if (!item) return [state, NO_EVENTS]
if (!item?.open) return [state, NO_EVENTS]
if (item.summaryParts[event.summary_index] !== "active") return [state, NO_EVENTS]
return [
{
...state,
@@ -1041,28 +1123,52 @@ const onFunctionCallArgumentsDelta = Effect.fn("OpenResponses.onFunctionCallArgu
return [{ ...state, lifecycle, tools: result.tools }, events] satisfies StepResult
})
const onOutputItemDone = Effect.fn("OpenResponses.onOutputItemDone")(function* (state: ParserState, event: Event) {
const item = event.item
const onOutputItemDone = Effect.fn("OpenResponses.onOutputItemDone")(function* (
state: ParserState,
item: Event["item"],
) {
if (!item) return [state, NO_EVENTS] satisfies StepResult
if (item.type === "message" && item.id !== undefined) {
const itemPhase = messagePhase(item.phase)
const phase = itemPhase === undefined ? state.messagePhases[item.id] : itemPhase
if (item.type === "compaction") {
if (!item.id || typeof item.encrypted_content !== "string")
return yield* ProviderShared.eventError(state.id, "Compaction output is missing its id or encrypted content")
if (state.completedCompactions.has(item.id)) return [state, NO_EVENTS] satisfies StepResult
const events: LLMEvent[] = []
const messageItems = new Set(state.messageItems)
messageItems.delete(item.id)
const { [item.id]: _phase, ...messagePhases } = state.messagePhases
const lifecycle = Lifecycle.stepStart(state.lifecycle, events)
events.push(
LLMEvent.compaction({
provider: state.provider,
id: item.id,
encrypted: item.encrypted_content,
}),
)
return [
{ ...state, lifecycle, completedCompactions: new Set([...state.completedCompactions, item.id]) },
events,
] satisfies StepResult
}
if (item.type === "message" && item.id !== undefined) {
const message = state.message?.id === item.id ? state.message : undefined
const itemPhase = messagePhase(item.phase)
const phase = itemPhase === undefined ? message?.phase : itemPhase
const parts: ReadonlyArray<unknown> = Array.isArray(item.content) ? item.content : []
const content: string[] = []
for (const part of parts) {
const decoded = Option.getOrUndefined(decodeMessagePart(part))
if (!decoded) continue
content.push(decoded.type === "output_text" ? decoded.text : decoded.refusal)
}
const text = content.length > 0 ? content.join("") : undefined
const metadata = providerMetadata(state, { itemId: item.id, ...(phase === undefined ? {} : { phase }) })
const events: LLMEvent[] = []
const lifecycle =
message && text ? Lifecycle.textStart(state.lifecycle, events, item.id, metadata) : state.lifecycle
return [
{
...state,
lifecycle: Lifecycle.textEnd(
state.lifecycle,
events,
item.id,
providerMetadata(state, { itemId: item.id, ...(phase === undefined ? {} : { phase }) }),
),
messageItems,
messagePhases,
lifecycle: Lifecycle.textEnd(lifecycle, events, item.id, metadata, text),
message: message ? undefined : state.message,
},
events,
] satisfies StepResult
@@ -1070,20 +1176,36 @@ const onOutputItemDone = Effect.fn("OpenResponses.onOutputItemDone")(function* (
if (item.type === "function_call") {
if (!item.call_id || !item.name) return [state, NO_EVENTS] satisfies StepResult
const id = item.id ?? item.call_id
const tools = state.tools[id]
? state.tools
: ToolStream.start(state.tools, id, {
id: item.call_id,
name: item.name,
providerMetadata: item.id !== undefined ? providerMetadata(state, { itemId: item.id }) : undefined,
})
const callID = item.call_id
if (state.completedTools.has(callID)) return [state, NO_EVENTS] satisfies StepResult
const metadata = item.id !== undefined ? providerMetadata(state, { itemId: item.id }) : undefined
const fallback = item.id ?? callID
// Match the pending tool by call id so item events that disagree on
// whether `item.id` is present still resolve the same call.
const registered =
state.tools[fallback] !== undefined
? fallback
: Object.keys(state.tools).find((key) => state.tools[key]?.id === callID)
const id = registered ?? fallback
const tools =
registered !== undefined
? state.tools
: ToolStream.start(state.tools, id, {
id: callID,
name: item.name,
providerMetadata: metadata,
})
const result =
item.arguments === undefined
? yield* ToolStream.finish(state.id, tools, id)
: yield* ToolStream.finishWithInput(state.id, tools, id, item.arguments)
const events: LLMEvent[] = []
const resultEvents = result.events ?? []
const finished = result.events ?? []
// A done-only call never streamed a start event, so open its lifecycle here.
const resultEvents =
registered !== undefined || finished.length === 0
? finished
: [LLMEvent.toolInputStart({ id: callID, name: item.name, providerMetadata: metadata }), ...finished]
const lifecycle = resultEvents.length ? Lifecycle.stepStart(state.lifecycle, events) : state.lifecycle
events.push(...resultEvents)
return [
@@ -1094,30 +1216,82 @@ const onOutputItemDone = Effect.fn("OpenResponses.onOutputItemDone")(function* (
resultEvents.some((event) => LLMEvent.is.toolCall(event) || LLMEvent.is.toolInputError(event)) ||
state.hasFunctionCall,
tools: result.tools,
completedTools: new Set([...state.completedTools, callID]),
},
events,
] satisfies StepResult
}
if (isReasoningItem(item)) {
const events: LLMEvent[] = []
if (state.reasoningItems[item.id]?.open === false) return [state, NO_EVENTS] satisfies StepResult
const metadata = reasoningMetadata(state, item)
const summaryParts: ReadonlyArray<unknown> = Array.isArray(item.summary) ? item.summary : []
const summary: Array<string | undefined> = []
for (const part of summaryParts) {
const decoded = Option.getOrUndefined(decodeSummaryPart(part))
// Keep missing entries so the array still matches the provider's summary indexes.
summary.push(decoded?.text)
}
const reasoningParts: ReadonlyArray<unknown> = Array.isArray(item.content) ? item.content : []
const content: string[] = []
for (const part of reasoningParts) {
const decoded = Option.getOrUndefined(decodeReasoningPart(part))
if (decoded) content.push(decoded.text)
}
const itemText = joinReasoningText(summary) ?? joinReasoningText(content)
const events: LLMEvent[] = []
const reasoningItem = state.reasoningItems[item.id]
if (reasoningItem) {
const lifecycle = Object.entries(reasoningItem.summaryParts)
.filter((entry) => entry[1] === "active" || entry[1] === "can-conclude")
.reduce(
(lifecycle, entry) => Lifecycle.reasoningEnd(lifecycle, events, `${item.id}:${entry[0]}`, metadata),
state.lifecycle,
)
const { [item.id]: _removed, ...reasoningItems } = state.reasoningItems
return [{ ...state, lifecycle, reasoningItems }, events] satisfies StepResult
const fragments = Object.entries(reasoningItem.summaryParts)
let lifecycle = state.lifecycle
for (const [index, status] of fragments) {
if (status === "concluded") continue
// Do not repeat earlier summaries that were already emitted as separate fragments.
const finalText = fragments.length === 1 ? itemText : summary[Number(index)]
lifecycle = Lifecycle.reasoningEnd(lifecycle, events, `${item.id}:${index}`, metadata, finalText || undefined)
}
return [
{
...state,
lifecycle,
reasoningItems: {
...state.reasoningItems,
[item.id]: {
...reasoningItem,
open: false,
encryptedContent: item.encrypted_content ?? reasoningItem.encryptedContent,
},
},
},
events,
] satisfies StepResult
}
if (!state.lifecycle.reasoning.has(item.id)) {
const lifecycle = Lifecycle.stepStart(state.lifecycle, events)
events.push(LLMEvent.reasoningStart({ id: item.id, providerMetadata: metadata }))
events.push(LLMEvent.reasoningEnd({ id: item.id, providerMetadata: metadata }))
return [{ ...state, lifecycle }, events] satisfies StepResult
events.push(
LLMEvent.reasoningEnd({
id: item.id,
providerMetadata: metadata,
text: itemText,
}),
)
return [
{
...state,
lifecycle,
reasoningItems: {
...state.reasoningItems,
[item.id]: {
open: false,
encryptedContent: item.encrypted_content,
summaryParts: { 0: "concluded" },
deltaIndexes: new Set(),
},
},
},
events,
] satisfies StepResult
}
return [
{ ...state, lifecycle: Lifecycle.reasoningEnd(state.lifecycle, events, item.id, metadata) },
@@ -1129,38 +1303,38 @@ const onOutputItemDone = Effect.fn("OpenResponses.onOutputItemDone")(function* (
})
const onResponseFinish = Effect.fn("OpenResponses.onResponseFinish")(function* (state: ParserState, event: Event) {
const reconciled =
event.type === "response.completed"
? yield* Effect.reduce(
event.response?.output ?? [],
() => [state, NO_EVENTS] satisfies StepResult,
([current, events], item) => {
const id = item.id ?? (item.type === "function_call" ? item.call_id : undefined)
if (
id === undefined ||
((item.type !== "function_call" || !current.tools[id]) &&
(item.type !== "reasoning" || !current.reasoningItems[id]))
)
return Effect.succeed([current, events] satisfies StepResult)
return onOutputItemDone(current, { type: "response.output_item.done", item }).pipe(
Effect.map(([next, emitted]) => [next, [...events, ...emitted]] satisfies StepResult),
)
},
)
: ([state, NO_EVENTS] satisfies StepResult)
const current = reconciled[0]
// Some compatible providers omit output_item.done even after completing the response.
const pending =
event.type === "response.completed"
? yield* ToolStream.finishAll(current.id, current.tools)
: { tools: current.tools, events: NO_EVENTS }
const events: LLMEvent[] = [...reconciled[1], ...pending.events]
const hasFunctionCall =
pending.events.some((event) => LLMEvent.is.toolCall(event) || LLMEvent.is.toolInputError(event)) ||
current.hasFunctionCall
let current = state
const events: LLMEvent[] = []
if (event.type === "response.completed") {
for (const item of event.response?.output ?? []) {
if (item.type !== "compaction" && item.type !== "function_call") continue
if (item.type === "compaction") {
// Terminal recovery cannot insert a checkpoint before already-emitted content.
if (state.lifecycle.stepStarted && !state.completedCompactions.has(item.id ?? ""))
return yield* ProviderShared.eventError(
state.id,
"Cannot recover a compaction checkpoint after output has been emitted",
)
}
if (item.type === "function_call" && !current.tools[item.id ?? item.call_id ?? ""]) continue
const [next, emitted] = yield* onOutputItemDone(current, item)
current = next
events.push(...emitted)
}
// Some compatible providers omit output_item.done even after completing the response.
const pending = yield* ToolStream.finishAll(current.id, current.tools)
current = {
...current,
tools: pending.tools,
hasFunctionCall:
current.hasFunctionCall ||
pending.events.some((event) => LLMEvent.is.toolCall(event) || LLMEvent.is.toolInputError(event)),
}
events.push(...pending.events)
}
const lifecycle = Lifecycle.finish(current.lifecycle, events, {
reason: {
normalized: mapFinishReason(event, hasFunctionCall),
normalized: mapFinishReason(event, current.hasFunctionCall),
raw: event.response?.incomplete_details?.reason,
},
usage: mapUsage(event.response?.usage, current.providerMetadataKey),
@@ -1172,7 +1346,7 @@ const onResponseFinish = Effect.fn("OpenResponses.onResponseFinish")(function* (
})
: undefined,
})
return [{ ...current, lifecycle, hasFunctionCall, tools: pending.tools }, events] satisfies StepResult
return [{ ...current, lifecycle }, events] satisfies StepResult
})
// Build the prettiest summary available from whatever the provider supplied.
@@ -1188,11 +1362,8 @@ const providerErrorMessage = (event: Event, nested: OpenResponsesErrorPayload |
return message || code
}
export const providerFailure = (id: string, event: Event, fallback: string) => {
export const providerFailure = (event: Event, fallback: string, body = ProviderShared.encodeJson(event)) => {
const nested = event.error ?? event.response?.error ?? undefined
const code = event.code || nested?.code || undefined
// Keep the full raw payload on the error even when the message is a summary.
const body = JSON.stringify(nested ?? event) ?? ""
const summary = providerErrorMessage(event, nested)
const message = summary ?? (body === "{}" ? fallback : body)
const status =
@@ -1207,18 +1378,11 @@ export const providerFailure = (id: string, event: Event, fallback: string) => {
event.response === undefined &&
summary === undefined &&
status === undefined
? new ProviderInternalReason({ message })
: classifyProviderFailure({ message, code, status, rawBody: body })
return new AIError({
module: id,
method: "stream",
body,
reason,
})
? new ProviderInternalError({ message, body })
: classifyProviderFailure({ message, status, rawBody: body })
return new AIError({ reason })
}
const providerError = (state: ParserState, event: Event, fallback: string) => providerFailure(state.id, event, fallback)
export const step = (state: ParserState, input: Event) => {
// The OpenAPI requires string IDs but imposes no minLength; empty is not missing.
const event =
@@ -1266,6 +1430,13 @@ export const step = (state: ParserState, input: Event) => {
if (event.type === "response.output_item.added") {
if (event.item?.type === "message" && event.item.id === undefined)
return ProviderShared.eventError(state.id, `${event.type} message is missing id`)
if (
event.item &&
isReasoningItem(event.item) &&
state.reasoningItems[event.item.id] === undefined &&
state.lifecycle.reasoning.size > 0
)
return ProviderShared.eventError(state.id, `${event.type} started reasoning before the previous item ended`)
const id = event.item?.id ?? (event.item?.type === "function_call" ? event.item.call_id : undefined)
return Effect.succeed(
onOutputItemAdded(
@@ -1283,14 +1454,21 @@ export const step = (state: ParserState, input: Event) => {
if (event.type === "response.output_item.done") {
if (event.item?.type === "message" && event.item.id === undefined)
return ProviderShared.eventError(state.id, `${event.type} message is missing id`)
return onOutputItemDone(state, event)
return onOutputItemDone(state, event.item)
}
if (event.type === "response.completed" || event.type === "response.incomplete") return onResponseFinish(state, event)
if (event.type === "response.failed") return providerError(state, event, `${state.name} response failed`)
if (event.type === "response.failed") return providerFailure(event, `${state.name} response failed`)
if (event.type === "error")
return decodeKnownErrorEvent(event).pipe(
Effect.mapError(() => ProviderShared.eventError(state.id, `${state.name} returned a malformed error event`)),
Effect.flatMap(() => providerError(state, event, `${state.name} stream error`)),
Effect.mapError((cause) =>
ProviderShared.eventError(
state.id,
`${state.name} returned a malformed error event`,
ProviderShared.encodeJson(event),
cause,
),
),
Effect.flatMap(() => providerFailure(event, `${state.name} stream error`)),
)
return Effect.succeed<StepResult>([state, NO_EVENTS])
}
@@ -1303,15 +1481,17 @@ export const step = (state: ParserState, input: Event) => {
* implementations compose this baseline with their own tools and event variants.
*/
export const initial = (request: LLMRequest, extension: Extension = BASE): ParserState => ({
provider: request.model.provider,
completedCompactions: new Set<string>(),
id: extension.id,
name: extension.name,
providerMetadataKey: request.model.route.providerMetadataKey ?? "openresponses",
hasFunctionCall: false,
tools: ToolStream.empty<string>(),
completedTools: new Set<string>(),
lifecycle: Lifecycle.initial(),
outputItems: {},
messageItems: new Set<string>(),
messagePhases: {},
message: undefined,
reasoningItems: {},
})
+60 -63
View File
@@ -3,14 +3,16 @@ import { Tool } from "@opencode-ai/schema/tool"
import { Route } from "../route/client.js"
import { Auth } from "../route/auth.js"
import { Endpoint } from "../route/endpoint.js"
import { Framing } from "../route/framing.js"
import { HttpTransport } from "../route/transport/index.js"
import { Protocol } from "../route/protocol.js"
import {
AIError,
InvalidProviderOutputReason,
AIErrorReason,
InvalidProviderOutputError,
LLMEvent,
ProviderInternalReason,
UnknownProviderReason,
ProviderInternalError,
UnknownProviderError,
Usage,
type FinishReason,
type FinishReasonDetails,
@@ -244,6 +246,8 @@ export const OpenAIChatEvent = Schema.StructWithRest(
[Schema.Record(Schema.String, Schema.Unknown)],
)
export type OpenAIChatEvent = Schema.Schema.Type<typeof OpenAIChatEvent>
const DONE = "[DONE]" as const
const OpenAIChatStreamEvent = Schema.Union([Schema.Literal(DONE), Protocol.jsonEvent(OpenAIChatEvent)])
type OpenAIChatRequestMessage = LLMRequest["messages"][number]
interface PendingToolDelta {
@@ -784,26 +788,22 @@ export const fromRequest = Effect.fn("OpenAIChat.fromRequest")(function* (
// Streaming parsers are small state machines: every event returns a new state
// plus the common `LLMEvent`s produced by that event. Tool calls are accumulated
// because OpenAI streams JSON arguments across multiple deltas.
const finishReasonError = (event: OpenAIChatEvent, reason: AIError["reason"]) =>
new AIError({
module: ADAPTER,
method: "stream",
body: ProviderShared.encodeJson(event),
reason,
})
const mapFinishReason = Effect.fn("OpenAIChat.mapFinishReason")(function* (event: OpenAIChatEvent, reason: string) {
switch (reason) {
case "error":
return yield* finishReasonError(
event,
new UnknownProviderReason({ message: "Provider reported an error (finish_reason: error)" }),
)
return yield* new AIError({
reason: new UnknownProviderError({
message: "Provider reported an error (finish_reason: error)",
body: ProviderShared.encodeJson(event),
}),
})
case "network_error":
return yield* finishReasonError(
event,
new ProviderInternalReason({ message: "Provider reported a network error (finish_reason: network_error)" }),
)
return yield* new AIError({
reason: new ProviderInternalError({
message: "Provider reported a network error (finish_reason: network_error)",
body: ProviderShared.encodeJson(event),
}),
})
case "stop":
case "end":
return "stop" as const
@@ -815,7 +815,12 @@ const mapFinishReason = Effect.fn("OpenAIChat.mapFinishReason")(function* (event
case "tool_calls":
return "tool-calls" as const
default:
return "unknown" as const
return yield* new AIError({
reason: new UnknownProviderError({
message: `Provider finish_reason: ${reason}`,
body: ProviderShared.encodeJson(event),
}),
})
}
})
@@ -936,12 +941,8 @@ const step = (state: ParserState, event: OpenAIChatEvent) =>
if (event.error) {
const body = ProviderShared.encodeJson(event)
return yield* new AIError({
module: ADAPTER,
method: "stream",
body,
reason: classifyProviderFailure({
message: event.error.message,
code: event.error.code === undefined || event.error.code === null ? undefined : String(event.error.code),
status: typeof event.error.code === "number" ? event.error.code : undefined,
rawBody: body,
}),
@@ -1004,33 +1005,12 @@ const step = (state: ParserState, event: OpenAIChatEvent) =>
lifecycle = Lifecycle.reasoningStart(lifecycle, events, "reasoning-0", deltaMetadata)
const reasoningEmitted = state.reasoningEmitted || lifecycle.reasoning.has("reasoning-0")
if (delta?.content) {
lifecycle = Lifecycle.reasoningEnd(
lifecycle,
events,
"reasoning-0",
reasoningMetadata(
state.providerMetadataKey,
reasoningField,
reasoningDetailsObserved ? state.reasoningDetails : undefined,
),
)
lifecycle = Lifecycle.textDelta(lifecycle, events, "text-0", delta.content)
}
// Reasoning is one response-wide channel: it stays open alongside text and
// refusal output so late reasoning deltas and details join the same block,
// and `finishEvents` closes it once with the complete metadata.
if (delta?.content) lifecycle = Lifecycle.textDelta(lifecycle, events, "text-0", delta.content)
if (delta?.refusal) {
lifecycle = Lifecycle.reasoningEnd(
lifecycle,
events,
"reasoning-0",
reasoningMetadata(
state.providerMetadataKey,
reasoningField,
reasoningDetailsObserved ? state.reasoningDetails : undefined,
),
)
lifecycle = Lifecycle.textDelta(lifecycle, events, "text-0", delta.refusal)
}
if (delta?.refusal) lifecycle = Lifecycle.textDelta(lifecycle, events, "text-0", delta.refusal)
// Compatible providers may omit indexes. Prefer durable identity, then use
// batch position for parallel deltas or the latest call for sparse chunks.
@@ -1066,23 +1046,38 @@ const step = (state: ParserState, event: OpenAIChatEvent) =>
"OpenAI Chat tool call delta is missing id or name",
)
if (ToolStream.isError(result))
return yield* ProviderShared.eventError(ADAPTER, result.reason.message, ProviderShared.encodeJson(event))
return yield* new AIError({
reason: AIErrorReason.make({
...result.reason,
message: result.message,
cause: result.reason.cause,
body: ProviderShared.encodeJson(event),
}),
})
tools = result.tools
if (result.events.length) lifecycle = Lifecycle.stepStart(lifecycle, events)
events.push(...result.events)
}
if (finishReason !== undefined && state.finishReason === undefined && Object.keys(pendingTools).length > 0)
const incompleteTools = finishReason?.normalized === "content-filter" || finishReason?.normalized === "length"
if (
finishReason !== undefined &&
!incompleteTools &&
state.finishReason === undefined &&
Object.keys(pendingTools).length
)
return yield* ProviderShared.eventError(
ADAPTER,
"OpenAI Chat tool call delta is missing id or name",
ProviderShared.encodeJson(event),
)
// Finalize accumulated tool inputs eagerly when finish_reason arrives so
// valid calls and malformed local calls settle independently.
// Filtering or truncation terminates the response without confirming pending tool calls.
const finished =
finishReason !== undefined && state.finishReason === undefined && Object.keys(tools).length > 0
finishReason !== undefined &&
!incompleteTools &&
state.finishReason === undefined &&
Object.keys(tools).length > 0
? yield* ToolStream.finishAll(ADAPTER, tools)
: undefined
@@ -1110,11 +1105,9 @@ const step = (state: ParserState, event: OpenAIChatEvent) =>
const finishEvents = Effect.fn("OpenAIChat.finishEvents")(function* (state: ParserState) {
if (state.finishReason === undefined && state.requireFinishReason)
return yield* new AIError({
module: ADAPTER,
method: "stream",
reason: new InvalidProviderOutputReason({
classification: "incomplete-stream",
reason: new InvalidProviderOutputError({
message: "OpenAI Chat stream ended without finish_reason",
classification: "incomplete-stream",
route: ADAPTER,
}),
})
@@ -1131,10 +1124,12 @@ const finishEvents = Effect.fn("OpenAIChat.finishEvents")(function* (state: Pars
state.finishReason.normalized === "stop" && hasToolCalls ? "tool-calls" : state.finishReason.normalized,
}
: { normalized: hasToolCalls ? ("tool-calls" as const) : ("stop" as const) }
// Snapshot details at publish time so the emitted event never observes later
// mutation of the accumulated `reasoningDetails` array.
const metadata = reasoningMetadata(
state.providerMetadataKey,
state.reasoningField,
state.reasoningDetailsObserved ? state.reasoningDetails : undefined,
state.reasoningDetailsObserved ? [...state.reasoningDetails] : undefined,
)
const started =
state.reasoningDetailsObserved && !state.reasoningEmitted
@@ -1168,7 +1163,7 @@ export const protocol = Protocol.make({
from: fromRequest,
},
stream: {
event: Protocol.jsonEvent(OpenAIChatEvent),
event: OpenAIChatStreamEvent,
initial: (request) => ({
providerMetadataKey: request.model.route.providerMetadataKey ?? String(request.model.provider),
tools: ToolStream.empty<number>(),
@@ -1182,12 +1177,14 @@ export const protocol = Protocol.make({
nextToolIndex: 0,
requireFinishReason: request.model.compatibility?.requireFinishReason ?? true,
}),
step,
step: (state: ParserState, event) => (event === DONE ? Effect.succeed([state, []] as const) : step(state, event)),
terminal: (event) => event === DONE,
onHalt: finishEvents,
},
})
export const httpTransport = HttpTransport.sseJson.with<OpenAIChatBody>()
export const framing = Framing.sseWithDone
export const httpTransport = HttpTransport.sseJson.with<OpenAIChatBody>().with({ framing })
export const route = Route.make({
id: ADAPTER,
@@ -1,6 +1,5 @@
import { Route, type RouteRoutedLanguageModelInput } from "../route/client.js"
import { Endpoint } from "../route/endpoint.js"
import { Framing } from "../route/framing.js"
import * as OpenAIChat from "./openai-chat.js"
const ADAPTER = "openai-compatible-chat"
@@ -19,7 +18,7 @@ export const route = Route.make({
providerMetadataKey: "openai",
protocol: OpenAIChat.protocol,
endpoint: Endpoint.path("/chat/completions"),
framing: Framing.sse,
framing: OpenAIChat.framing,
})
export * as OpenAICompatibleChat from "./openai-compatible-chat.js"
+12 -28
View File
@@ -9,14 +9,7 @@ import {
type ImageRoute,
} from "../image.js"
import { Auth, type Definition as AuthDefinition } from "../route/auth.js"
import {
InvalidProviderOutputReason,
AIError,
Usage,
mergeHttpOptions,
mergeJsonRecords,
type HttpOptions,
} from "../schema/index.js"
import { Usage, mergeHttpOptions, mergeJsonRecords, type HttpOptions } from "../schema/index.js"
import { ProviderShared } from "./shared.js"
import { ImageInputs } from "./utils/image-input.js"
import { OpenAIImage } from "./utils/openai-image.js"
@@ -84,13 +77,6 @@ const nativeOptions = (options: OpenAIImageOptions | undefined) => {
}
}
const invalidOutput = (message: string) =>
new AIError({
module: ADAPTER,
method: "generate",
reason: new InvalidProviderOutputReason({ message, route: ADAPTER }),
})
const applyQuery = (url: string, query: Record<string, string> | undefined) => {
if (!query) return url
const next = new URL(url)
@@ -104,12 +90,12 @@ export const model = (input: ModelInput) => {
generate: Effect.fn("OpenAIImages.generate")(function* (request: ImageRequestFor<OpenAIImageOptions>, execute) {
const mask = request.options?.mask
if (mask !== undefined && (request.images?.length ?? 0) === 0)
return yield* ImageInputs.invalid(ADAPTER, "An OpenAI image mask requires at least one input image")
return yield* ImageInputs.invalid("An OpenAI image mask requires at least one input image")
const http = mergeHttpOptions(request.model.http, request.http)
const sourceImages = request.images ?? []
const multipartImages = yield* Effect.forEach(sourceImages, (image) => {
if (image.type === "bytes") return Effect.succeed({ data: image.data, mediaType: image.mediaType })
if (image.type === "url") return ImageInputs.decodeDataUrl(image.url, ADAPTER)
if (image.type === "url") return ImageInputs.decodeDataUrl(image.url)
return Effect.undefined
})
const multipartMask =
@@ -118,7 +104,7 @@ export const model = (input: ModelInput) => {
: mask.type === "bytes"
? { data: mask.data, mediaType: mask.mediaType }
: mask.type === "url"
? yield* ImageInputs.decodeDataUrl(mask.url, ADAPTER)
? yield* ImageInputs.decodeDataUrl(mask.url)
: undefined
const useMultipart =
sourceImages.length > 0 &&
@@ -161,7 +147,7 @@ export const model = (input: ModelInput) => {
return undefined
})
if (references.some((image) => image === undefined))
return yield* ImageInputs.invalid(ADAPTER, "OpenAI Images accepts image URLs, data URLs, bytes, and file IDs")
return yield* ImageInputs.invalid("OpenAI Images accepts image URLs, data URLs, bytes, and file IDs")
const maskReference =
mask === undefined
? undefined
@@ -173,7 +159,7 @@ export const model = (input: ModelInput) => {
? { file_id: mask.id }
: undefined
if (mask !== undefined && maskReference === undefined)
return yield* ImageInputs.invalid(ADAPTER, "OpenAI Images accepts masks as URLs, data URLs, bytes, or file IDs")
return yield* ImageInputs.invalid("OpenAI Images accepts masks as URLs, data URLs, bytes, or file IDs")
const requestBody = mergeJsonRecords(
{
model: request.model.id,
@@ -209,11 +195,9 @@ const parseResponse = Effect.fn("OpenAIImages.parseResponse")(function* (
options: OpenAIImageOptions | undefined,
overlay: Record<string, unknown> | undefined,
) {
const payload = yield* response.json.pipe(
Effect.mapError(() => invalidOutput("Failed to read the OpenAI Images response")),
)
const decoded = yield* Schema.decodeUnknownEffect(OpenAIImageResponse)(payload).pipe(
Effect.mapError(() => invalidOutput("OpenAI Images returned an invalid response")),
const output = yield* ProviderShared.imageResponse(ADAPTER, "OpenAI Images", response)
const decoded = yield* Schema.decodeUnknownEffect(Schema.fromJsonString(OpenAIImageResponse))(output.body).pipe(
Effect.mapError((cause) => output.invalid("OpenAI Images returned an invalid response", cause)),
)
const requestBody = mergeJsonRecords(nativeOptions(options), overlay)
const format =
@@ -221,7 +205,7 @@ const parseResponse = Effect.fn("OpenAIImages.parseResponse")(function* (
const images = yield* Effect.forEach(decoded.data, (item, index) => {
if (item.b64_json)
return Effect.fromResult(Encoding.decodeBase64(item.b64_json)).pipe(
Effect.mapError(() => invalidOutput(`OpenAI Images result ${index} contains invalid base64 data`)),
Effect.mapError((cause) => output.invalid(`OpenAI Images result ${index} contains invalid base64 data`, cause)),
Effect.map(
(data) =>
new GeneratedImage({
@@ -241,9 +225,9 @@ const parseResponse = Effect.fn("OpenAIImages.parseResponse")(function* (
item.revised_prompt === undefined ? undefined : { openai: { revisedPrompt: item.revised_prompt } },
}),
)
return Effect.fail(invalidOutput(`OpenAI Images result ${index} has neither image data nor a URL`))
return Effect.fail(output.invalid(`OpenAI Images result ${index} has neither image data nor a URL`))
})
if (images.length === 0) return yield* invalidOutput("OpenAI Images returned no images")
if (images.length === 0) return yield* output.invalid("OpenAI Images returned no images")
return new ImageResponse({
images,
usage:
+25 -1
View File
@@ -12,6 +12,7 @@ import { OpenAIImage } from "./utils/openai-image.js"
import { ResponsesHostedTools } from "./utils/responses-hosted-tools.js"
import { ToolSchemaProjection } from "./utils/tool-schema.js"
import { OpenResponsesChannel } from "./open-responses-channel.js"
import { ResponsesCompaction } from "./utils/responses-compaction.js"
const ADAPTER = "openai-responses"
const NAME = "OpenAI Responses"
@@ -20,6 +21,14 @@ const WEBSOCKET_ROTATE_AFTER_MS = 55 * 60 * 1000
export const DEFAULT_BASE_URL = "https://api.openai.com/v1"
export const PATH = OpenResponses.PATH
export const ContextManagement = Schema.Array(
Schema.Struct({
type: Schema.Literal("compaction"),
compactThreshold: Schema.optional(Schema.Int.check(Schema.isGreaterThan(0))),
}),
)
export type ContextManagement = typeof ContextManagement.Type
const OpenAIResponsesImageGenerationTool = Schema.Struct({
type: Schema.tag("image_generation"),
action: Schema.optional(Schema.Literals(["auto", "generate", "edit"])),
@@ -78,6 +87,14 @@ const OpenAIResponsesCoreFields = {
input: Schema.Array(Schema.Union([OpenResponses.InputItem, OpenAIResponsesHostedToolItem])),
tools: optionalArray(OpenAIResponsesTools),
tool_choice: Schema.optional(OpenAIResponsesToolChoice),
context_management: Schema.optional(
Schema.Array(
Schema.Struct({
type: Schema.Literal("compaction"),
compact_threshold: Schema.optional(Schema.Int.check(Schema.isGreaterThan(0))),
}),
),
),
}
const OpenAIResponsesBody = Schema.Struct({
@@ -125,6 +142,9 @@ const lowerToolChoice = (toolChoice: NonNullable<LLMRequest["toolChoice"]>, tool
const decodeBody = ProviderShared.validateWith(Schema.decodeUnknownEffect(OpenAIResponsesBody))
const fromRequest = Effect.fn("OpenAIResponses.fromRequest")(function* (request: LLMRequest) {
const management = yield* ProviderShared.validateWith(
Schema.decodeUnknownEffect(Schema.UndefinedOr(ContextManagement)),
)(request.providerOptions?.contextManagement)
const body = yield* OpenResponses.fromRequestWithExtension(
LLMRequest.update(request, { tools: [], toolChoice: undefined }),
extension,
@@ -133,6 +153,7 @@ const fromRequest = Effect.fn("OpenAIResponses.fromRequest")(function* (request:
const parallelToolCalls = OpenResponses.resolveParallelToolCalls(request)
return yield* decodeBody({
...body,
context_management: management?.map((edit) => ({ type: edit.type, compact_threshold: edit.compactThreshold })),
...(parallelToolCalls === undefined ? {} : { parallel_tool_calls: parallelToolCalls }),
tools:
request.tools.length === 0
@@ -149,7 +170,9 @@ const hostedToolResult = Effect.fn("OpenAIResponses.hostedToolResult")(function*
const isError = item.error !== undefined && item.error !== null
if (item.type === "image_generation_call" && item.result) {
yield* Effect.fromResult(Encoding.decodeBase64(item.result)).pipe(
Effect.mapError(() => ProviderShared.eventError(ADAPTER, "OpenAI Responses returned invalid image base64")),
Effect.mapError((cause) =>
ProviderShared.eventError(ADAPTER, "OpenAI Responses returned invalid image base64", undefined, cause),
),
)
const format = item.output_format ?? "png"
return {
@@ -221,6 +244,7 @@ export const transport = channelTransport({
})
export const route = Route.make({
compact: ResponsesCompaction.execute,
id: ADAPTER,
provider: "openai",
providerMetadataKey: "openai",
+44 -18
View File
@@ -2,11 +2,12 @@ import { Buffer } from "node:buffer"
import { Tool } from "@opencode-ai/schema/tool"
import { Effect, Schema, Stream } from "effect"
import * as Sse from "effect/unstable/encoding/Sse"
import { Headers, HttpClientRequest } from "effect/unstable/http"
import { Headers, HttpClientRequest, HttpClientResponse } from "effect/unstable/http"
import {
InvalidProviderOutputReason,
InvalidRequestReason,
InvalidProviderOutputError,
InvalidRequestError,
AIError,
HttpContext,
type ContentPart,
type LLMRequest,
type MediaPart,
@@ -96,17 +97,15 @@ export const sumTokens = (...values: ReadonlyArray<number | undefined>): number
return values.reduce((acc: number, value) => acc + (value ?? 0), 0)
}
export const eventError = (route: string, message: string, raw?: string) =>
export const eventError = (route: string, message: string, body?: string, cause?: unknown) =>
new AIError({
module: "ProviderShared",
method: "stream",
reason: new InvalidProviderOutputReason({ route, message, raw }),
reason: new InvalidProviderOutputError({ route, message, body, cause }),
})
export const parseJson = (route: string, input: string, message: string) =>
Effect.try({
try: () => decodeJson(input),
catch: () => eventError(route, message, input),
catch: (cause) => eventError(route, message, input, cause),
})
/**
@@ -208,15 +207,16 @@ export const errorText = (error: unknown) => {
/**
* `framing` step for Server-Sent Events. Decodes UTF-8, runs the SSE channel
* decoder, optionally filters named events, and drops empty / `[DONE]`
* keep-alive events so the protocol event schema sees one JSON string per
* element. Retry control events are ignored without interrupting the stream.
* decoder, optionally filters named events, and drops empty events. `[DONE]`
* is dropped by default or retained for protocols that use it as their stream
* boundary. Retry control events are ignored without interrupting the stream.
* Decoder failures become provider output errors so the public error channel
* stays `AIError`.
*/
export const sseFraming = (
bytes: Stream.Stream<Uint8Array, AIError>,
events?: ReadonlySet<string>,
includeDone = false,
): Stream.Stream<string, AIError> =>
bytes.pipe(
Stream.decodeText(),
@@ -233,7 +233,7 @@ export const sseFraming = (
(state, chunk) =>
Effect.gen(function* () {
const error = state.parser.feed(chunk)
if (error) return yield* eventError("sse", error.message)
if (error) return yield* eventError("sse", error.message, chunk, error)
return [state, state.output.splice(0)] as const
}),
),
@@ -241,7 +241,7 @@ export const sseFraming = (
(event) =>
(events === undefined || events.has(event.event)) &&
event.data.length > 0 &&
(event.data !== "[DONE]" || (events !== undefined && event.event !== "message")),
(event.data !== "[DONE]" || includeDone || (events !== undefined && event.event !== "message")),
),
Stream.map((event) => event.data),
)
@@ -249,13 +249,39 @@ export const sseFraming = (
/**
* Canonical invalid-request constructor shared by protocol lowering.
*/
export const invalidRequest = (message: string) =>
export const invalidRequest = (message: string, cause?: unknown) =>
new AIError({
module: "ProviderShared",
method: "request",
reason: new InvalidRequestReason({ message }),
reason: new InvalidRequestError({ message, cause }),
})
export const imageResponse = Effect.fn("ProviderShared.imageResponse")(function* (
route: string,
name: string,
response: HttpClientResponse.HttpClientResponse,
) {
const http = new HttpContext({ url: response.request.url, status: response.status, headers: response.headers })
const body = yield* response.text.pipe(
Effect.mapError(
(cause) =>
new AIError({
reason: new InvalidProviderOutputError({
route,
message: `Failed to read the ${name} response`,
http,
cause,
}),
}),
),
)
return {
body,
invalid: (message: string, cause?: unknown) =>
new AIError({
reason: new InvalidProviderOutputError({ route, message, body, http, cause }),
}),
}
})
export const matchToolChoice = <Auto, None, Required, Tool>(
route: string,
toolChoice: NonNullable<LLMRequest["toolChoice"]>,
@@ -302,7 +328,7 @@ export const unsupportedContent = (
export const validateWith =
<A, I, E extends { readonly message: string }>(decode: (input: I) => Effect.Effect<A, E>) =>
(payload: I) =>
decode(payload).pipe(Effect.mapError((error) => invalidRequest(error.message)))
decode(payload).pipe(Effect.mapError((error) => invalidRequest(error.message, error)))
/**
* Build an HTTP POST with a JSON body. Sets `content-type: application/json`
@@ -1,4 +1,4 @@
import { Effect, Schema } from "effect"
import { Effect, Encoding, Schema } from "effect"
import type { MediaPart } from "../../schema/index.js"
import { ProviderShared } from "../shared.js"
@@ -57,6 +57,16 @@ const documentBlock = (name: string, format: DocumentFormat, bytes: string): Doc
},
})
const mediaBase64 = Effect.fn("BedrockMedia.mediaBase64")(function* (part: MediaPart) {
const media = ProviderShared.normalizeMedia(part)
const bytes = yield* Effect.fromResult(Encoding.decodeBase64(media.base64)).pipe(
Effect.mapError((cause) =>
ProviderShared.invalidRequest("Bedrock Converse media data must be valid base64", cause),
),
)
return Encoding.encodeBase64(bytes)
})
// Route by MIME. Known image/document formats lower into a typed block; anything
// else fails with a clear error instead of silently degrading to a malformed
// document block. Image MIME types not in `IMAGE_FORMATS` (e.g. `image/svg+xml`)
@@ -66,8 +76,7 @@ export const lower = Effect.fn("BedrockMedia.lower")(function* (part: MediaPart)
const mime = part.mediaType.toLowerCase()
const imageFormat = IMAGE_FORMATS[mime as keyof typeof IMAGE_FORMATS]
if (imageFormat) {
const media = ProviderShared.normalizeMedia(part)
return { image: { format: imageFormat, source: { bytes: media.base64 } } } satisfies ImageBlock
return { image: { format: imageFormat, source: { bytes: yield* mediaBase64(part) } } } satisfies ImageBlock
}
if (mime.startsWith("image/"))
return yield* ProviderShared.invalidRequest(`Bedrock Converse does not support image media type ${part.mediaType}`)
@@ -75,8 +84,7 @@ export const lower = Effect.fn("BedrockMedia.lower")(function* (part: MediaPart)
if (documentFormat) {
if (!part.filename)
return yield* ProviderShared.invalidRequest("Bedrock Converse document media requires a filename")
const media = ProviderShared.normalizeMedia(part)
return documentBlock(part.filename, documentFormat, media.base64)
return documentBlock(part.filename, documentFormat, yield* mediaBase64(part))
}
return yield* ProviderShared.invalidRequest(`Bedrock Converse does not support media type ${part.mediaType}`)
})
@@ -1,12 +1,10 @@
import { Effect, Encoding } from "effect"
import type { ImageInput } from "../../image.js"
import { InvalidRequestReason, AIError } from "../../schema/index.js"
import { InvalidRequestError, AIError } from "../../schema/index.js"
const invalid = (module: string, message: string) =>
const invalid = (message: string, cause?: unknown) =>
new AIError({
module,
method: "generate",
reason: new InvalidRequestReason({ message }),
reason: new InvalidRequestError({ message, cause }),
})
export const dataUrl = (input: Extract<ImageInput, { readonly type: "bytes" }>) =>
@@ -14,13 +12,12 @@ export const dataUrl = (input: Extract<ImageInput, { readonly type: "bytes" }>)
export const decodeDataUrl = (
url: string,
module: string,
): Effect.Effect<{ readonly mediaType: string; readonly data: Uint8Array } | undefined, AIError> => {
if (!url.startsWith("data:")) return Effect.undefined
const match = /^data:([^;,]+);base64,(.*)$/s.exec(url)
if (!match) return Effect.fail(invalid(module, "Image data URLs must contain a MIME type and base64 data"))
if (!match) return Effect.fail(invalid("Image data URLs must contain a MIME type and base64 data"))
return Effect.fromResult(Encoding.decodeBase64(match[2])).pipe(
Effect.mapError(() => invalid(module, "Image data URL contains invalid base64 data")),
Effect.mapError((cause) => invalid("Image data URL contains invalid base64 data", cause)),
Effect.map((data) => ({ mediaType: match[1], data })),
)
}
+15 -6
View File
@@ -62,22 +62,31 @@ export const reasoningEnd = (
events: LLMEvent[],
id: string,
providerMetadata?: ProviderMetadata,
/** Authoritative complete value; replaces accumulated deltas when present. */
text?: string,
): State => {
if (!state.reasoning.has(id)) return state
const stepped = stepStart(state, events)
events.push(LLMEvent.reasoningEnd({ id, providerMetadata }))
events.push(LLMEvent.reasoningEnd({ id, text, providerMetadata }))
const reasoning = new Set(stepped.reasoning)
reasoning.delete(id)
return { ...stepped, reasoning }
}
export const textEnd = (state: State, events: LLMEvent[], id: string, providerMetadata?: ProviderMetadata): State => {
export const textEnd = (
state: State,
events: LLMEvent[],
id: string,
providerMetadata?: ProviderMetadata,
/** Authoritative complete value; replaces accumulated deltas when present. */
text?: string,
): State => {
if (!state.text.has(id)) return state
const stepped = stepStart(state, events)
events.push(LLMEvent.textEnd({ id, providerMetadata }))
const text = new Set(stepped.text)
text.delete(id)
return { ...stepped, text }
events.push(LLMEvent.textEnd({ id, text, providerMetadata }))
const open = new Set(stepped.text)
open.delete(id)
return { ...stepped, text: open }
}
const closeOpenBlocks = (state: State, events: LLMEvent[]): State => {
@@ -0,0 +1,153 @@
import { Effect, Schema, Stream } from "effect"
import {
AIError,
InvalidProviderOutputError,
CompactionPart,
CompactionResponse,
HttpOptions,
LLMRequest,
Message,
type ContentPart,
mergeJsonRecords,
} from "../../schema/index.js"
import type { CompactOperation } from "../../route/client.js"
import { Endpoint } from "../../route/endpoint.js"
import { RequestExecutor } from "../../route/executor.js"
import { HttpTransport } from "../../route/transport/index.js"
import { OpenResponses } from "../open-responses.js"
import { JsonObject, ProviderShared } from "../shared.js"
const Body = Schema.Struct({
model: Schema.String,
input: Schema.Array(Schema.Unknown),
instructions: Schema.optional(Schema.String),
previous_response_id: Schema.optional(Schema.String),
})
const Text = Schema.Union([OpenResponses.OpenResponsesInputText, OpenResponses.OpenResponsesOutputText])
const File = Schema.Union([
Schema.Struct({ type: Schema.Literal("input_file"), filename: Schema.String, file_url: Schema.String }),
Schema.Struct({ type: Schema.Literal("input_file"), filename: Schema.String, file_data: Schema.String }),
])
const MessageFields = {
type: Schema.Literal("message"),
id: Schema.optional(Schema.String),
status: Schema.optional(Schema.String),
phase: Schema.optional(OpenResponses.MessagePhase),
}
const Response = Schema.Struct({
object: Schema.Literal("response.compaction"),
output: Schema.Array(
Schema.Union([
OpenResponses.CompactionItem,
OpenResponses.OpenResponsesReasoningItem,
Schema.Struct({
...MessageFields,
role: Schema.Literal("user"),
content: Schema.Array(Schema.Union([Text, OpenResponses.OpenResponsesInputImage, File])).check(
Schema.isMinLength(1),
),
}),
Schema.Struct({
...MessageFields,
role: Schema.Literal("assistant"),
content: Schema.Array(Text).check(Schema.isMinLength(1)),
}),
]),
),
usage: Schema.optional(Schema.StructWithRest(OpenResponses.OpenResponsesUsage, [JsonObject])),
})
export const execute: CompactOperation = Effect.fn("ResponsesCompaction.execute")(
function* (request, executor, options) {
const route = request.model.route
const native = yield* route.body.from(request)
const body = yield* ProviderShared.validateWith(Schema.decodeUnknownEffect(Body))(
mergeJsonRecords(native, request.http?.body),
)
const url = Endpoint.render(route.endpoint, { request, body: native })
url.pathname = `${url.pathname.replace(/\/$/, "")}/compact`
const parts = yield* HttpTransport.jsonRequestParts({
request: LLMRequest.update(request, {
http: request.http === undefined ? undefined : new HttpOptions({ ...request.http, body: undefined }),
}),
body,
endpoint: Endpoint.path(url.toString()),
auth: route.auth,
encodeBody: Schema.encodeSync(Schema.fromJsonString(Body)),
})
const response = yield* executor.execute(
ProviderShared.jsonPost({ url: parts.url, body: parts.bodyText, headers: parts.headers }),
options?.http,
)
const text = yield* RequestExecutor.responseStream(response).pipe(
Stream.decodeText(),
Stream.runFold(
() => "",
(text, chunk) => text + chunk,
),
)
const invalid = (message: string, cause?: unknown) =>
new AIError({
reason: new InvalidProviderOutputError({
route: route.id,
message,
body: text,
cause,
http: RequestExecutor.responseHttp(response),
}),
})
const result = yield* Schema.decodeUnknownEffect(Schema.fromJsonString(Response))(text).pipe(
Effect.mapError((cause) => invalid("Invalid compaction response", cause)),
)
if (!result.output.some((item) => item.type === "compaction"))
return yield* invalid("Compaction response did not contain a checkpoint")
return new CompactionResponse({
messages: result.output.map((item) => toMessage(item, request.model)),
usage: OpenResponses.mapUsage(result.usage, route.providerMetadataKey ?? String(request.model.provider)),
})
},
)
function toMessage(item: (typeof Response.Type.output)[number], model: LLMRequest["model"]): Message {
if (item.type === "compaction")
return Message.assistant(
CompactionPart.make({ provider: model.provider, id: item.id ?? undefined, encrypted: item.encrypted_content }),
)
const key = model.route.providerMetadataKey ?? String(model.provider)
if (item.type === "reasoning") {
const summary = item.summary.length ? item.summary : [{ text: "" }]
return Message.assistant(
summary.map((part) => ({
type: "reasoning" as const,
text: part.text,
providerMetadata: { [key]: { itemId: item.id, reasoningEncryptedContent: item.encrypted_content } },
})),
)
}
return Message.make({
role: item.role,
providerMetadata: { [key]: { itemId: item.id, type: item.type, status: item.status, phase: item.phase } },
content: item.content.map((part): ContentPart => {
if (part.type === "input_text" || part.type === "output_text") return { type: "text", text: part.text }
if (part.type === "input_image")
return {
type: "media",
data: part.image_url,
mediaType: /^data:([^;,]+)/.exec(part.image_url)?.[1] ?? "image/*",
providerMetadata: part.detail === undefined ? undefined : { [key]: { detail: part.detail } },
}
const data = "file_url" in part ? part.file_url : part.file_data
return {
type: "media",
data,
filename: part.filename,
mediaType: /^data:([^;,]+)/.exec(data)?.[1] ?? "application/octet-stream",
}
}),
})
}
export * as ResponsesCompaction from "./responses-compaction.js"
+10 -24
View File
@@ -2,14 +2,7 @@ import { Effect, Encoding, Schema } from "effect"
import { Headers, HttpClientRequest } from "effect/unstable/http"
import { GeneratedImage, ImageModel, ImageResponse, type ImageRequestFor, type ImageRoute } from "../image.js"
import { Auth, type Definition as AuthDefinition } from "../route/auth.js"
import {
InvalidProviderOutputReason,
AIError,
Usage,
mergeHttpOptions,
mergeJsonRecords,
type HttpOptions,
} from "../schema/index.js"
import { Usage, mergeHttpOptions, mergeJsonRecords, type HttpOptions } from "../schema/index.js"
import { ProviderShared, optionalNull } from "./shared.js"
import { ImageInputs } from "./utils/image-input.js"
@@ -94,13 +87,6 @@ const nativeOptions = (options: XAIImageOptions | undefined) => {
}
}
const invalidOutput = (message: string) =>
new AIError({
module: ADAPTER,
method: "generate",
reason: new InvalidProviderOutputReason({ message, route: ADAPTER }),
})
const applyQuery = (url: string, query: Record<string, string> | undefined) => {
if (!query) return url
const next = new URL(url)
@@ -120,7 +106,7 @@ export const model = (input: ModelInput) => {
return undefined
})
if (imageReferences.some((image) => image === undefined))
return yield* ImageInputs.invalid(ADAPTER, "xAI Images accepts image URLs, data URLs, bytes, and file IDs")
return yield* ImageInputs.invalid("xAI Images accepts image URLs, data URLs, bytes, and file IDs")
const requestBody = mergeJsonRecords(
{
model: request.model.id,
@@ -149,17 +135,17 @@ export const model = (input: ModelInput) => {
HttpClientRequest.bodyText(text, "application/json"),
),
)
const payload = yield* response.json.pipe(
Effect.mapError(() => invalidOutput("Failed to read the xAI Images response")),
)
const decoded = yield* Schema.decodeUnknownEffect(XAIImageResponse)(payload).pipe(
Effect.mapError(() => invalidOutput("xAI Images returned an invalid response")),
const output = yield* ProviderShared.imageResponse(ADAPTER, "xAI Images", response)
const decoded = yield* Schema.decodeUnknownEffect(Schema.fromJsonString(XAIImageResponse))(output.body).pipe(
Effect.mapError((cause) => output.invalid("xAI Images returned an invalid response", cause)),
)
const images = yield* Effect.forEach(decoded.data, (item, index) => {
const mediaType = item.mime_type ?? "application/octet-stream"
if (item.b64_json)
return Effect.fromResult(Encoding.decodeBase64(item.b64_json)).pipe(
Effect.mapError(() => invalidOutput(`xAI Images result ${index} contains invalid base64 data`)),
Effect.mapError((cause) =>
output.invalid(`xAI Images result ${index} contains invalid base64 data`, cause),
),
Effect.map(
(data) =>
new GeneratedImage({
@@ -183,9 +169,9 @@ export const model = (input: ModelInput) => {
: { xai: { revisedPrompt: item.revised_prompt } },
}),
)
return Effect.fail(invalidOutput(`xAI Images result ${index} has neither image data nor a URL`))
return Effect.fail(output.invalid(`xAI Images result ${index} has neither image data nor a URL`))
})
if (images.length === 0) return yield* invalidOutput("xAI Images returned no images")
if (images.length === 0) return yield* output.invalid("xAI Images returned no images")
const usage = ProviderShared.isRecord(decoded.usage) ? decoded.usage : undefined
return new ImageResponse({
images,
@@ -44,6 +44,10 @@ const extension = {
const decodeBody = ProviderShared.validateWith(Schema.decodeUnknownEffect(XAIResponsesBody))
const fromRequest = Effect.fn("XAIResponses.fromRequest")(function* (request: LLMRequest) {
if (request.providerOptions?.contextManagement !== undefined)
return yield* ProviderShared.invalidRequest(
"xAI requires explicit compaction through LLMClient.compact; automatic context management is not supported",
)
return yield* decodeBody(yield* OpenResponses.fromRequestWithExtension(request, extension))
})
+6 -21
View File
@@ -2,13 +2,7 @@ import { Effect, Schema } from "effect"
import { Headers, HttpClientRequest } from "effect/unstable/http"
import { GeneratedImage, ImageModel, ImageResponse, type ImageRequestFor, type ImageRoute } from "../image.js"
import { Auth, type Definition as AuthDefinition } from "../route/auth.js"
import {
InvalidProviderOutputReason,
AIError,
mergeHttpOptions,
mergeJsonRecords,
type HttpOptions,
} from "../schema/index.js"
import { mergeHttpOptions, mergeJsonRecords, type HttpOptions } from "../schema/index.js"
import { ProviderShared } from "./shared.js"
import { ImageInputs } from "./utils/image-input.js"
@@ -63,13 +57,6 @@ const nativeOptions = (options: ZAIImageOptions | undefined) => {
}
}
const invalidOutput = (message: string) =>
new AIError({
module: ADAPTER,
method: "generate",
reason: new InvalidProviderOutputReason({ message, route: ADAPTER }),
})
const applyQuery = (url: string, query: Record<string, string> | undefined) => {
if (!query) return url
const next = new URL(url)
@@ -82,7 +69,7 @@ export const model = (input: ModelInput) => {
id: ADAPTER,
generate: Effect.fn("ZAIImages.generate")(function* (request: ImageRequestFor<ZAIImageOptions>, execute) {
if ((request.images?.length ?? 0) > 0)
return yield* ImageInputs.invalid(ADAPTER, "Z.ai hosted image generation does not support image inputs")
return yield* ImageInputs.invalid("Z.ai hosted image generation does not support image inputs")
const http = mergeHttpOptions(request.model.http, request.http)
const requestBody = mergeJsonRecords(
{ model: request.model.id, prompt: request.prompt },
@@ -104,13 +91,11 @@ export const model = (input: ModelInput) => {
HttpClientRequest.bodyText(text, "application/json"),
),
)
const payload = yield* response.json.pipe(
Effect.mapError(() => invalidOutput("Failed to read the Z.ai Images response")),
const output = yield* ProviderShared.imageResponse(ADAPTER, "Z.ai Images", response)
const decoded = yield* Schema.decodeUnknownEffect(Schema.fromJsonString(ZAIImageResponse))(output.body).pipe(
Effect.mapError((cause) => output.invalid("Z.ai Images returned an invalid response", cause)),
)
const decoded = yield* Schema.decodeUnknownEffect(ZAIImageResponse)(payload).pipe(
Effect.mapError(() => invalidOutput("Z.ai Images returned an invalid response")),
)
if (decoded.data.length === 0) return yield* invalidOutput("Z.ai Images returned no images")
if (decoded.data.length === 0) return yield* output.invalid("Z.ai Images returned no images")
return new ImageResponse({
images: decoded.data.map(
(item) =>
+62 -62
View File
@@ -1,17 +1,16 @@
import { Option, Schema } from "effect"
import {
AuthenticationReason,
ContentPolicyReason,
InvalidRequestReason,
AuthenticationError,
ContentPolicyError,
InvalidRequestError,
AIError,
ProviderErrorEvent,
ProviderInternalReason,
QuotaExceededReason,
RateLimitReason,
UnknownProviderReason,
ProviderInternalError,
QuotaExceededError,
RateLimitError,
UnknownProviderError,
type HttpContext,
type HttpRateLimitDetails,
type ProviderMetadata,
} from "./schema/index.js"
const patterns = [
@@ -38,6 +37,7 @@ const patterns = [
/too large for model with \d+ maximum context length/i,
/prompt has [\d,]+ tokens?, but the configured context size is [\d,]+ tokens?/i,
/model_context_window_exceeded/i,
/range of input length should be/i,
/too many tokens/i,
/token limit exceeded/i,
/request_too_large/i,
@@ -60,6 +60,7 @@ export const isContextOverflowFailure = (failure: unknown) =>
const decodeJson = Schema.decodeUnknownOption(Schema.fromJsonString(Schema.Unknown))
const QUOTA_CODES = new Set(["insufficient_quota", "usage_not_included", "billing_error"])
const AUTH_CODES = new Set(["authentication_error", "permission_error"])
const SERVER_CODES = new Set([
"api_error",
"internal_error",
@@ -75,32 +76,38 @@ const INVALID_REQUEST_CODES = new Set(["invalid_prompt", "invalid_request_error"
const RATE_LIMIT_TEXT = /rate increased too quickly|rate[-_\s]?limit|too[_\s]?many[_\s]?requests/i
const QUOTA_TEXT = /insufficient[-_\s]?quota|quota[-_\s]?exceeded/i
const CONTENT_POLICY_TEXT = /content[-_\s]?policy|content_filter|safety/i
const NETWORK_ERROR_TEXT = /network[-_\s]error/i
const SERVER_ERROR_TEXT =
/\b(?:try again|(?:please |you can )?retry (?:the |this |your )?request|try (?:the |this |your )?request again|(?:currently |temporarily )?at capacity|overloaded|temporarily unavailable|service[-_\s]?unavailable|(?:server|internal)[-_\s]?error|server (?:is )?busy|provider returned (?:an )?error|resource[-_\s]?exhausted|upstream (?:connect|connection|request)|request buffer limit while retrying upstream)\b/i
export interface ProviderFailure {
readonly message: string
readonly status?: number | undefined
readonly code?: string | undefined
// Raw wire payload, scanned for failure signals (codes, overflow phrases)
// that the summary message does not carry. Not shown to users.
readonly rawBody?: string | undefined
// Some SDKs supply parsed error data separately from the original response text.
readonly data?: unknown
readonly http?: HttpContext | undefined
readonly cause?: unknown
readonly retryAfterMs?: number | undefined
readonly rateLimit?: HttpRateLimitDetails | undefined
readonly http?: HttpContext | undefined
readonly providerMetadata?: ProviderMetadata | undefined
}
// Keep HTTP failures and provider-reported stream failures on one typed path so
// session retry policy never needs provider-specific string matching.
// Classification records affirmative evidence about a failure. Deterministic
// failures need positive identification (a 4xx status, quota/auth/policy
// signals); anything unrecognized stays UnknownProvider, which the session
// retry policy treats as retry-eligible because transient failures arrive in
// unpredictable shapes while deterministic rejections almost always carry a
// status or known code.
export function classifyProviderFailure(input: ProviderFailure): AIError["reason"] {
const body = input.http?.body ?? input.rawBody ?? ""
const codes = [input.code, ...providerCodes(body), ...providerCodes(input.message)]
.filter((code): code is string => code !== undefined)
.map((code) => code.toLowerCase())
const details = { message: input.message, body: input.rawBody, http: input.http, cause: input.cause }
const body = input.rawBody ?? ""
const codes = [...providerCodes(input.data), ...providerCodes(body), ...providerCodes(input.message)].map((code) =>
code.toLowerCase(),
)
// Scan the raw payload too so signals missing from the summary message
// (e.g. overflow phrases nested in a JSON error body) still classify.
const text = [input.message, body].filter((value) => value.length > 0).join("\n")
const common = { message: input.message, providerMetadata: input.providerMetadata, http: input.http }
const clientScoped = input.status === undefined || (input.status >= 400 && input.status < 500)
if (
@@ -110,62 +117,55 @@ export function classifyProviderFailure(input: ProviderFailure): AIError["reason
codes.includes("request_too_large") ||
isContextOverflow(text))
)
return new InvalidRequestReason({ ...common, classification: "context-overflow" })
return new InvalidRequestError({ ...details, classification: "context-overflow" })
if (input.status === 413 || isPayloadTooLarge(text))
return new InvalidRequestReason({ ...common, classification: "payload-too-large" })
if (CONTENT_POLICY_TEXT.test(text)) return new ContentPolicyReason(common)
return new InvalidRequestError({ ...details, classification: "payload-too-large" })
if (CONTENT_POLICY_TEXT.test(text)) return new ContentPolicyError(details)
if (codes.some((code) => QUOTA_CODES.has(code)) || (input.status === 429 && QUOTA_TEXT.test(text)))
return new QuotaExceededReason(common)
if (input.status === 401) return new AuthenticationReason({ ...common, kind: "invalid" })
if (input.status === 403) return new AuthenticationReason({ ...common, kind: "insufficient-permissions" })
if (codes.includes("authentication_error")) return new AuthenticationReason({ ...common, kind: "invalid" })
if (codes.includes("permission_error"))
return new AuthenticationReason({ ...common, kind: "insufficient-permissions" })
return new QuotaExceededError(details)
if (input.status === 401 || input.status === 403 || codes.some((code) => AUTH_CODES.has(code)))
return new AuthenticationError(details)
if (
codes.some((code) => code.includes("rate_limit") || code === "too_many_requests" || code === "throttlingexception")
input.status === 429 ||
codes.some(
(code) => code.includes("rate_limit") || code === "too_many_requests" || code === "throttlingexception",
) ||
RATE_LIMIT_TEXT.test(text)
)
return new RateLimitReason({
...common,
return new RateLimitError({
...details,
retryAfterMs: input.retryAfterMs,
rateLimit: input.rateLimit,
})
if (RATE_LIMIT_TEXT.test(text))
return new RateLimitReason({
...common,
retryAfterMs: input.retryAfterMs,
rateLimit: input.rateLimit,
})
if (NETWORK_ERROR_TEXT.test(text)) return new ProviderInternalReason({ ...common, status: input.status })
if (codes.some((code) => SERVER_CODES.has(code) || code.includes("exhausted") || code.includes("unavailable")))
return new ProviderInternalReason({
...common,
status: input.status,
if (
input.status === 408 ||
input.status === 409 ||
(input.status !== undefined && input.status >= 500) ||
((input.status === undefined || input.status < 400) &&
!codes.some((code) => INVALID_REQUEST_CODES.has(code)) &&
SERVER_ERROR_TEXT.test(text)) ||
codes.some((code) => SERVER_CODES.has(code) || code.includes("exhausted") || code.includes("unavailable"))
)
return new ProviderInternalError({
...details,
retryAfterMs: input.retryAfterMs,
})
if (input.status === 429) {
return new RateLimitReason({
...common,
retryAfterMs: input.retryAfterMs,
rateLimit: input.rateLimit,
})
}
if (input.status === 408 || input.status === 409 || (input.status !== undefined && input.status >= 500))
return new ProviderInternalReason({
...common,
status: input.status,
retryAfterMs: input.retryAfterMs,
})
if (codes.some((code) => INVALID_REQUEST_CODES.has(code))) return new InvalidRequestReason(common)
if (input.status === 400 || input.status === 404 || input.status === 413 || input.status === 422)
return new InvalidRequestReason(common)
return new UnknownProviderReason({ ...common, status: input.status })
if (codes.some((code) => INVALID_REQUEST_CODES.has(code))) return new InvalidRequestError(details)
// Any remaining 4xx is a deterministic rejection of this request.
if (input.status !== undefined && input.status >= 400 && input.status < 500) return new InvalidRequestError(details)
return new UnknownProviderError(details)
}
function providerCodes(value: string) {
const decoded = Option.getOrUndefined(decodeJson(value))
function providerCodes(value: unknown) {
const decoded = typeof value === "string" ? Option.getOrUndefined(decodeJson(value)) : value
if (!isRecord(decoded)) return []
const error = isRecord(decoded.error) ? decoded.error : undefined
return [decoded.code, error?.code, error?.type].filter((value): value is string => typeof value === "string")
const response = isRecord(decoded.response) ? decoded.response : undefined
const responseError = response && isRecord(response.error) ? response.error : undefined
const exception = isRecord(decoded.exception) ? decoded.exception : undefined
return [decoded.code, error?.code, error?.type, error?.status, responseError?.code, exception?.type].filter(
(value): value is string => typeof value === "string",
)
}
function isRecord(value: unknown): value is Record<string, unknown> {
+21 -9
View File
@@ -1,9 +1,11 @@
import type { RouteDefaultsInput } from "../route/client.js"
import type { Route, RouteDefaultsInput } from "../route/client.js"
import { Auth } from "../route/auth.js"
import type { ProviderPackage } from "../provider-package.js"
import { ProviderID, type ModelID } from "../schema/index.js"
import * as BedrockConverse from "../protocols/bedrock-converse.js"
import type { BedrockCredentials } from "../protocols/bedrock-converse.js"
import { BedrockMessages } from "../protocols/bedrock-messages.js"
import type { AnthropicMessages } from "../protocols/anthropic-messages.js"
export const id = ProviderID.make("amazon-bedrock")
@@ -25,38 +27,40 @@ export interface Settings extends ProviderPackage.Settings {
readonly region?: string
readonly topP?: number
}
export const routes = [BedrockConverse.route]
export const routes = [BedrockConverse.route, BedrockMessages.route]
const bedrockBaseURL = (region: string) => `https://bedrock-runtime.${region}.amazonaws.com`
const configuredRoute = (input: Config) => {
const configuredRoute = <Body, Prepared>(route: Route<Body, Prepared>, input: Config) => {
const { apiKey, credentials, region, baseURL, ...rest } = input
const resolvedRegion = region ?? credentials?.region ?? "us-east-1"
return BedrockConverse.route.with({
return route.with({
...rest,
provider: id,
providerMetadataKey: "bedrock",
providerMetadataKey: route.providerMetadataKey,
endpoint: { baseURL: baseURL ?? bedrockBaseURL(resolvedRegion) },
auth: apiKey === undefined ? BedrockConverse.sigV4Auth(credentials) : Auth.bearer(apiKey),
})
}
export const configure = (input: Config = {}) => {
const route = configuredRoute(input)
const route = configuredRoute(BedrockConverse.route, input)
const messages = configuredRoute(BedrockMessages.route, input)
return {
id,
model: (modelID: string | ModelID) => route.model({ id: modelID }),
messages: (modelID: string | ModelID) => messages.model<AnthropicMessages.ProviderOptionsInput>({ id: modelID }),
configure,
}
}
export const provider = configure()
export const model: ProviderPackage.Definition<Settings>["model"] = (modelID, settings) => {
const config = (settings: Settings): Config => {
if (settings.auth === "bearer" && settings.apiKey === undefined)
throw new Error("Amazon Bedrock bearer auth requires apiKey")
if (settings.auth === "sigv4" && settings.apiKey !== undefined)
throw new Error("Amazon Bedrock SigV4 auth does not accept apiKey")
return configure({
return {
apiKey: settings.auth === "sigv4" ? undefined : settings.apiKey,
baseURL: settings.baseURL,
credentials: settings.credentials,
@@ -64,5 +68,13 @@ export const model: ProviderPackage.Definition<Settings>["model"] = (modelID, se
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
region: settings.region,
}).model(modelID)
}
}
export const model: ProviderPackage.Definition<Settings>["model"] = (modelID, settings) =>
configure(config(settings)).model(modelID)
export const messagesModel: ProviderPackage.Definition<
Settings & { readonly providerOptions?: AnthropicMessages.ProviderOptionsInput },
AnthropicMessages.ProviderOptionsInput
>["model"] = (modelID, settings) =>
configure({ ...config(settings), providerOptions: settings.providerOptions }).messages(modelID)
@@ -0,0 +1 @@
export { messagesModel as model } from "../amazon-bedrock.js"
@@ -57,7 +57,9 @@ const route = Route.make({
}),
endpoint: Endpoint.path(({ request }) => `/${request.model.id}:streamRawPredict`),
auth: Auth.none,
framing: AnthropicMessages.framing,
transport: AnthropicMessages.transport<
Omit<AnthropicMessages.AnthropicMessagesBody, "model"> & { readonly anthropic_version: typeof VERSION }
>(),
headers: () => ({ "anthropic-version": HEADER_VERSION }),
})
+1 -2
View File
@@ -5,7 +5,6 @@ import { ProviderShared } from "../protocols/shared.js"
import { AuthOptions, type ProviderAuthOption } from "../route/auth-options.js"
import { Route, type RouteDefaultsInput } from "../route/client.js"
import { Endpoint } from "../route/endpoint.js"
import { Framing } from "../route/framing.js"
import { Protocol } from "../route/protocol.js"
import { ProviderID, type ModelID, type LLMRequest } from "../schema/index.js"
import { profiles } from "./openai-compatible-profile.js"
@@ -75,7 +74,7 @@ export const route = Route.make({
providerMetadataKey: "openai",
protocol,
endpoint: Endpoint.path("/chat/completions", { baseURL: profiles.groq.baseURL }),
framing: Framing.sse,
framing: OpenAIChat.framing,
})
export const configure = (input: LanguageModelOptions = {}) => {
+1
View File
@@ -13,6 +13,7 @@ export * as GoogleVertexChat from "./google-vertex-chat.js"
export * as GoogleVertexMessages from "./google-vertex-messages.js"
export * as GoogleVertexResponses from "./google-vertex-responses.js"
export * as Groq from "./groq.js"
export * as Mistral from "./mistral.js"
export * as OpenAI from "./openai.js"
export * as OpenAICompatible from "./openai-compatible.js"
export * as OpenAICompatibleResponses from "./openai-compatible-responses.js"
+51
View File
@@ -0,0 +1,51 @@
import type { ProviderPackage } from "../provider-package.js"
import { MistralChat } from "../protocols/mistral-chat.js"
import { AuthOptions, type ProviderAuthOption } from "../route/auth-options.js"
import type { RouteDefaultsInput } from "../route/client.js"
import { ProviderID, type ModelID } from "../schema/index.js"
export const id = ProviderID.make("mistral")
export type ProviderOptions = MistralChat.ProviderOptionsInput
export type LanguageModelOptions = Omit<RouteDefaultsInput, "providerOptions"> &
ProviderAuthOption<"optional"> & {
readonly baseURL?: string
readonly providerOptions?: ProviderOptions
}
export interface Settings extends ProviderPackage.Settings {
readonly apiKey?: string
readonly baseURL?: string
readonly providerOptions?: ProviderOptions
}
export const route = MistralChat.route
export const routes = [route]
export const configure = (input: LanguageModelOptions = {}) => {
const { apiKey: _apiKey, auth: _auth, baseURL, ...defaults } = input
const configured = route.with({
...defaults,
endpoint: { baseURL: baseURL ?? MistralChat.DEFAULT_BASE_URL },
auth: AuthOptions.bearer(input, "MISTRAL_API_KEY"),
})
return {
id,
model: (modelID: string | ModelID) => configured.model<ProviderOptions>({ id: modelID }),
configure,
}
}
export const provider = configure()
export const model: ProviderPackage.Definition<Settings, ProviderOptions>["model"] = (modelID, settings) =>
configure({
apiKey: settings.apiKey,
baseURL: settings.baseURL,
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
providerOptions: settings.providerOptions,
}).model(modelID)
export * as Mistral from "./mistral.js"
@@ -1,10 +1,12 @@
import { mergeProviderOptions, type ProviderOptions } from "../schema/index.js"
import type { OpenAIServiceTier } from "../protocols/utils/openai-options.js"
import type { Options } from "../protocols/utils/open-responses-options.js"
import type { ContextManagement } from "../protocols/openai-responses.js"
export type { OpenAIResponseIncludable, OpenAIServiceTier } from "../protocols/utils/openai-options.js"
export type OpenAIOptionsInput = Omit<Options, "serviceTier"> & {
readonly contextManagement?: ContextManagement
readonly serviceTier?: OpenAIServiceTier
readonly [key: string]: unknown
}
+1 -2
View File
@@ -1,7 +1,6 @@
import { Effect, Schema } from "effect"
import { Route, type RouteDefaultsInput } from "../route/client.js"
import { Endpoint } from "../route/endpoint.js"
import { Framing } from "../route/framing.js"
import { Protocol } from "../route/protocol.js"
import { AuthOptions, type ProviderAuthOption } from "../route/auth-options.js"
import { ProviderID, type CacheHint, type ModelID } from "../schema/index.js"
@@ -167,7 +166,7 @@ export const route = Route.make({
providerMetadataKey: "openrouter",
protocol,
endpoint: Endpoint.path("/chat/completions", { baseURL: profile.baseURL }),
framing: Framing.sse,
framing: OpenAIChat.framing,
})
export const routes = [route]
+3 -1
View File
@@ -7,13 +7,14 @@ import * as OpenAICompatibleChat from "../protocols/openai-compatible-chat.js"
import * as OpenAIChat from "../protocols/openai-chat.js"
import { OpenResponsesChannel } from "../protocols/open-responses-channel.js"
import { XAIResponses } from "../protocols/xai-responses.js"
import { ResponsesCompaction } from "../protocols/utils/responses-compaction.js"
import { XAIImages } from "../protocols/xai-images.js"
import type { OpenAIOptionsInput } from "./openai-options.js"
import type { ProviderPackage } from "../provider-package.js"
export const id = ProviderID.make("xai")
export type XAIProviderOptionsInput = OpenAIOptionsInput
export type XAIProviderOptionsInput = OpenAIOptionsInput & { readonly contextManagement?: never }
export type LanguageModelOptions = Omit<RouteDefaultsInput, "providerOptions"> &
ProviderAuthOption<"optional"> & {
@@ -32,6 +33,7 @@ export type { XAIImageOptions } from "../protocols/xai-images.js"
const RESPONSES_WEBSOCKET_ROTATE_AFTER_MS = 24 * 60 * 1000
const responsesRoute = Route.make({
compact: ResponsesCompaction.execute,
id: "openai-responses",
provider: id,
providerMetadataKey: "xai",
+3 -5
View File
@@ -1,6 +1,6 @@
import { Config, Effect, Redacted } from "effect"
import { Headers } from "effect/unstable/http"
import { AuthenticationReason, InvalidRequestReason, AIError, type HttpOptions } from "../schema/index.js"
import { AuthenticationError, InvalidRequestError, AIError, type HttpOptions } from "../schema/index.js"
export class MissingCredentialError extends Error {
readonly _tag = "MissingCredentialError"
@@ -137,12 +137,10 @@ export function bearerHeader(name: string, source?: Secret | Credential) {
const toAIError = (error: AuthError): AIError => {
if (error instanceof MissingCredentialError || error instanceof Config.ConfigError) {
return new AIError({
module: "Auth",
method: "apply",
reason:
error instanceof MissingCredentialError
? new AuthenticationReason({ message: error.message, kind: "missing" })
: new InvalidRequestReason({ message: `Failed to resolve auth config: ${error.message}` }),
? new AuthenticationError({ message: error.message, cause: error })
: new InvalidRequestError({ message: `Failed to resolve auth config: ${error.message}`, cause: error }),
})
}
return error
+114 -22
View File
@@ -7,18 +7,21 @@ import { HttpTransport } from "./transport/index.js"
import type { HttpMiddleware, Transport, TransportRuntime, WebSocketChannelExecutor } from "./transport/index.js"
import type { Protocol } from "./protocol.js"
import { applyCachePolicy } from "../cache-policy.js"
import { normalizeToolHistory } from "../tool-history.js"
import { sanitizeSurrogates } from "../utils/sanitize.js"
import * as ProviderShared from "../protocols/shared.js"
import type { ProtocolID, ProviderOptions } from "../schema/index.js"
import {
AIError,
CompactionResponse,
AIErrorReason,
GenerationOptions,
HttpOptions,
LLMRequest,
LLMResponse,
LanguageModel,
LLMEvent,
InvalidProviderOutputReason,
InvalidProviderOutputError,
ProviderID,
mergeGenerationOptions,
mergeHttpOptions,
@@ -33,6 +36,7 @@ export interface RouteBody<Body> {
}
export interface Route<Body, Prepared = unknown> {
readonly compact?: CompactOperation
readonly id: string
readonly provider?: ProviderID
/** ProviderMetadata namespace emitted and consumed by this route. */
@@ -40,6 +44,8 @@ export interface Route<Body, Prepared = unknown> {
readonly protocol: ProtocolID
readonly endpoint: Endpoint.Definition<Body>
readonly auth: Auth.Definition
/** Deployment headers resolved once for every operation, before transport authentication. */
readonly headers?: (input: { readonly request: LLMRequest }) => Record<string, string>
readonly transport: Transport<Body, Prepared, unknown>
readonly defaults: RouteDefaults
readonly body: RouteBody<Body>
@@ -148,6 +154,10 @@ export const httpOptions = (input: HttpOptionsInput | undefined) => {
}
export interface Interface {
readonly compact: (
request: LLMRequest,
options?: Pick<StreamOptions, "http">,
) => Effect.Effect<CompactionResponse, AIError>
readonly stream: StreamMethod
readonly generate: GenerateMethod
}
@@ -165,24 +175,33 @@ export interface GenerateMethod {
(request: LLMRequest, options?: StreamOptions): Effect.Effect<LLMResponse, AIError>
}
export type CompactOperation = (
request: LLMRequest,
executor: RequestExecutor.Interface,
options?: Pick<StreamOptions, "http">,
) => Effect.Effect<CompactionResponse, AIError>
export class Service extends Context.Service<Service, Interface>()("@opencode/LLMClient") {}
const resolveRequestOptions = (request: LLMRequest) => {
const routeDefaults = request.model.route.defaults
const modelDefaults = request.model.defaults
const generation = mergeGenerationOptions(routeDefaults.generation, modelDefaults?.generation, request.generation)
return LLMRequest.update(request, {
const messages = normalizeToolHistory(request.messages)
const normalized = messages === request.messages ? request : LLMRequest.update(request, { messages })
const routeDefaults = normalized.model.route.defaults
const modelDefaults = normalized.model.defaults
const generation = mergeGenerationOptions(routeDefaults.generation, modelDefaults?.generation, normalized.generation)
return LLMRequest.update(normalized, {
generation: generation ?? new GenerationOptions({}),
providerOptions: mergeProviderOptions(
routeDefaults.providerOptions,
modelDefaults?.providerOptions,
request.providerOptions,
normalized.providerOptions,
),
http: mergeHttpOptions(routeDefaults.http, modelDefaults?.http, request.http),
http: mergeHttpOptions(routeDefaults.http, modelDefaults?.http, normalized.http),
})
}
export interface MakeInput<Body, Frame, Event, State> {
readonly compact?: CompactOperation
/** Route id used in diagnostics and prepared request metadata. */
readonly id: string
/** Provider identity for route-owned model construction. */
@@ -204,6 +223,7 @@ export interface MakeInput<Body, Frame, Event, State> {
}
export interface MakeTransportInput<Body, Prepared, Frame, Event, State> {
readonly compact?: CompactOperation
/** Route id used in diagnostics and prepared request metadata. */
readonly id: string
/** Provider identity for route-owned model construction. */
@@ -227,16 +247,14 @@ export interface MakeTransportInput<Body, Prepared, Frame, Event, State> {
const streamError = (route: string, message: string, cause: Cause.Cause<unknown>) => {
const failed = cause.reasons.find(Cause.isFailReason)?.error
if (failed instanceof AIError) return failed
return ProviderShared.eventError(route, message, Cause.pretty(cause))
return ProviderShared.eventError(route, message, undefined, cause)
}
const incompleteStreamError = (route: string) =>
new AIError({
module: "LLMClient",
method: "stream",
reason: new InvalidProviderOutputReason({
classification: "incomplete-stream",
reason: new InvalidProviderOutputError({
message: "The provider response ended unexpectedly.",
classification: "incomplete-stream",
route,
}),
})
@@ -265,11 +283,12 @@ function makeFromTransport<Body, Prepared, Frame, Event, State>(
const decodeEventEffect = Schema.decodeUnknownEffect(protocol.stream.event)
const decodeEvent = (route: string) => (frame: Frame) =>
decodeEventEffect(frame).pipe(
Effect.mapError(() =>
Effect.mapError((cause) =>
ProviderShared.eventError(
input.id,
`Invalid ${route} stream event`,
typeof frame === "string" ? frame : ProviderShared.encodeJson(frame),
cause,
),
),
)
@@ -280,12 +299,14 @@ function makeFromTransport<Body, Prepared, Frame, Event, State>(
const build = (routeInput: BuiltRouteInput): Route<Body, Prepared> => {
const route: Route<Body, Prepared> = {
compact: routeInput.compact,
id: routeInput.id,
provider: routeInput.provider === undefined ? undefined : ProviderID.make(routeInput.provider),
providerMetadataKey: routeInput.providerMetadataKey,
protocol: protocol.id,
endpoint: routeInput.endpoint,
auth: routeInput.auth ?? Auth.none,
headers: routeInput.headers,
transport: routeInput.transport,
defaults: routeInput.defaults ?? {},
body: protocol.body,
@@ -315,7 +336,6 @@ function makeFromTransport<Body, Prepared, Frame, Event, State>(
endpoint: routeInput.endpoint,
auth: routeInput.auth ?? Auth.none,
encodeBody,
headers: routeInput.headers,
middleware: options?.http,
webSocket: options?.webSocket,
}),
@@ -324,19 +344,48 @@ function makeFromTransport<Body, Prepared, Frame, Event, State>(
return Stream.unwrap(
routeInput.transport.execute(prepared, request, runtime, options).pipe(
Effect.map((execution) => {
const terminal = protocol.stream.terminal
// Preserve assembled inputs; replace only serialized event fallbacks with their original wire data.
const frameError =
(frame: Frame, event: Frame | Event = frame) =>
(error: AIError) =>
new AIError({
reason: AIErrorReason.make({
...error.reason,
message: error.reason.message,
cause: error.reason.cause,
body:
error.reason.body !== undefined && error.reason.body !== ProviderShared.encodeJson(event)
? error.reason.body
: (execution.body?.(frame) ??
(typeof frame === "string" ? frame : ProviderShared.encodeJson(frame))),
}),
})
const events = execution.frames.pipe(
Stream.mapEffect(decodeEvent(route)),
protocol.stream.terminal ? Stream.takeUntil(protocol.stream.terminal) : (stream) => stream,
Stream.mapEffect((frame) =>
decodeEvent(route)(frame).pipe(
Effect.catchCause((cause) =>
Effect.fail(streamError(route, `Failed to decode ${route} event`, cause)),
),
Effect.map((event) => ({ event, frame })),
Effect.mapError(frameError(frame)),
),
),
terminal ? Stream.takeUntil(({ event }) => terminal(event)) : (stream) => stream,
)
const stream = Stream.suspend(() => {
let state = protocol.stream.initial(request)
const parsed = events.pipe(
Stream.mapEffect((event) =>
Stream.mapEffect(({ event, frame }) =>
protocol.stream.step(state, event).pipe(
Effect.catchCause((cause) =>
Effect.fail(streamError(route, `Failed to parse ${route} event`, cause)),
),
Effect.map(([next, output]) => {
state = next
return output
}),
Effect.mapError(frameError(frame, event)),
),
),
Stream.flatMap(Stream.fromIterable),
@@ -352,6 +401,17 @@ function makeFromTransport<Body, Prepared, Frame, Event, State>(
}).pipe(
Stream.catchCause((cause) => Stream.fail(streamError(route, `Failed to read ${route} stream`, cause))),
requireTerminalEvent(route),
Stream.mapError(
(error) =>
new AIError({
reason: AIErrorReason.make({
...error.reason,
message: error.reason.message,
cause: error.reason.cause,
http: error.reason.http ?? execution.http,
}),
}),
),
)
return execution.complete ? stream.pipe(Stream.onEnd(execution.complete)) : stream
}),
@@ -392,6 +452,7 @@ export function make<Body, Prepared, Frame, Event, State>(
if ("transport" in input) return makeFromTransport(input)
const protocol = input.protocol
return makeFromTransport({
compact: input.compact,
id: input.id,
provider: input.provider,
providerMetadataKey: input.providerMetadataKey,
@@ -404,9 +465,19 @@ export function make<Body, Prepared, Frame, Event, State>(
})
}
const compile = Effect.fn("LLM.compile")(function* (request: LLMRequest, options?: StreamOptions) {
const prepareRequest = (request: LLMRequest) => {
const original = applyCachePolicy(resolveRequestOptions(request))
const resolved = LLMRequest.update(original, sanitizeSurrogates({ ...LLMRequest.input(original), model: undefined }))
const sanitized = LLMRequest.update(original, sanitizeSurrogates({ ...LLMRequest.input(original), model: undefined }))
const tools = [...new Map(sanitized.tools.map((tool) => [tool.name, tool])).values()]
const resolved = tools.length === sanitized.tools.length ? sanitized : LLMRequest.update(sanitized, { tools })
const headers = resolved.model.route.headers?.({ request: resolved })
return headers === undefined
? resolved
: LLMRequest.update(resolved, { http: mergeHttpOptions(new HttpOptions({ headers }), resolved.http) })
}
const compile = Effect.fn("LLM.compile")(function* (request: LLMRequest, options?: StreamOptions) {
const resolved = prepareRequest(request)
const route = resolved.model.route
const body = yield* route.body
@@ -465,6 +536,15 @@ export function generate(request: LLMRequest, options?: StreamOptions): Effect.E
})
}
export const compact = (
request: LLMRequest,
options?: Pick<StreamOptions, "http">,
): Effect.Effect<CompactionResponse, AIError, Service> =>
Effect.gen(function* () {
const client = yield* Service
return yield* client.compact(request, options)
})
export const streamRequest = (request: LLMRequest, options?: StreamOptions) =>
Stream.unwrap(
Effect.gen(function* () {
@@ -475,16 +555,28 @@ export const streamRequest = (request: LLMRequest, options?: StreamOptions) =>
export const layer: Layer.Layer<Service, never, RequestExecutor.Service> = Layer.effect(
Service,
Effect.gen(function* () {
const stream = streamRequestWith({
http: yield* RequestExecutor.Service,
const executor = yield* RequestExecutor.Service
const stream = streamRequestWith({ http: executor })
return Service.of({
stream,
generate: generateWith(stream),
compact: (request, options) =>
Effect.suspend(() => {
const operation = request.model.route.compact
if (!operation)
return ProviderShared.invalidRequest(
`${request.model.provider}/${request.model.route.id} does not support explicit compaction`,
)
return operation(prepareRequest(request), executor, options)
}),
})
return Service.of({ stream, generate: generateWith(stream) })
}),
)
export const Route = { make } as const
export const LLMClient = {
compact,
Service,
layer,
stream,
+48 -92
View File
@@ -7,14 +7,7 @@ import {
HttpClientRequest,
HttpClientResponse,
} from "effect/unstable/http"
import {
HttpContext,
HttpRateLimitDetails,
HttpRequestDetails,
HttpResponseDetails,
AIError,
TransportReason,
} from "../schema/index.js"
import { HttpContext, HttpRateLimitDetails, AIError, TransportError } from "../schema/index.js"
import { classifyProviderFailure } from "../provider-error.js"
export interface Interface {
@@ -97,24 +90,13 @@ const rateLimitDetails = (headers: Record<string, string>, retryAfter: number |
})
}
const requestDetails = (request: HttpClientRequest.HttpClientRequest) =>
new HttpRequestDetails({
method: request.method,
url: request.url,
headers: headerDetails(request.headers),
})
const responseDetails = (response: HttpClientResponse.HttpClientResponse) =>
new HttpResponseDetails({
export const responseHttp = (response: HttpClientResponse.HttpClientResponse) =>
new HttpContext({
url: response.request.url,
status: response.status,
headers: headerDetails(response.headers),
})
const responseBody = (body: string | void) => {
if (body === undefined) return {}
return { body }
}
const decodeProviderBody = Schema.decodeUnknownOption(
Schema.fromJsonString(
Schema.Struct({
@@ -132,77 +114,46 @@ const providerMessage = (status: number, body: string | void) => {
)
}
const responseHttp = (input: {
readonly request: HttpClientRequest.HttpClientRequest
readonly response: HttpClientResponse.HttpClientResponse
readonly body: ReturnType<typeof responseBody>
readonly rateLimit?: HttpRateLimitDetails | undefined
}) =>
new HttpContext({
request: requestDetails(input.request),
response: responseDetails(input.response),
...input.body,
rateLimit: input.rateLimit,
const statusError = (response: HttpClientResponse.HttpClientResponse) =>
Effect.gen(function* () {
if (response.status < 400) return response
const result = yield* response.text.pipe(Effect.result)
return yield* httpFailure({
message: providerMessage(response.status, result._tag === "Success" ? result.success : undefined),
url: response.request.url,
status: response.status,
responseHeaders: headerDetails(response.headers),
responseBody: result._tag === "Success" ? result.success : undefined,
cause: result._tag === "Failure" ? (result.failure.cause ?? result.failure) : undefined,
})
})
const statusError =
(request: HttpClientRequest.HttpClientRequest) => (response: HttpClientResponse.HttpClientResponse) =>
Effect.gen(function* () {
if (response.status < 400) return response
const body = yield* response.text.pipe(Effect.catch(() => Effect.void))
const headers = normalizedHeaders(response.headers)
const retryAfter = retryAfterMs(headers)
const rateLimit = rateLimitDetails(headers, retryAfter)
const details = responseBody(body)
return yield* new AIError({
module: "RequestExecutor",
method: "execute",
reason: classifyProviderFailure({
status: response.status,
message: providerMessage(response.status, body),
retryAfterMs: retryAfter,
rateLimit,
http: responseHttp({
request,
response,
body: details,
rateLimit,
}),
}),
})
})
// Classifies an HTTP failure captured outside the executor (for example by the
// AI SDK's own fetch) onto the same reason types and HttpContext that
// executor-driven requests produce. The originating request is not available on
// that path, so the method is assumed (language model calls are always POST),
// request headers are empty.
export const classifyHttpFailure = (input: {
/** Preserve HTTP diagnostics for executor and externally captured failures alike. */
export const httpFailure = (input: {
readonly message: string
readonly url: string
readonly url?: string | undefined
readonly status?: number | undefined
readonly code?: string | undefined
readonly data?: unknown
readonly responseHeaders?: Record<string, string> | undefined
readonly responseBody?: string | undefined
readonly cause?: unknown
}) => {
const headers = normalizedHeaders(Headers.fromInput(input.responseHeaders))
const retryAfter = retryAfterMs(headers)
const rateLimit = rateLimitDetails(headers, retryAfter)
const details = responseBody(input.responseBody)
return classifyProviderFailure({
message: input.message,
status: input.status,
code: input.code,
retryAfterMs: retryAfter,
rateLimit,
http: new HttpContext({
request: new HttpRequestDetails({ method: "POST", url: input.url, headers: {} }),
response:
input.status === undefined
? undefined
: new HttpResponseDetails({ status: input.status, headers: headerDetails(Headers.fromInput(headers)) }),
...details,
return new AIError({
reason: classifyProviderFailure({
message: input.message,
status: input.status,
data: input.data,
rawBody: input.responseBody,
retryAfterMs: retryAfter,
rateLimit,
cause: input.cause,
http:
input.status === undefined || input.url === undefined
? undefined
: new HttpContext({ url: input.url, status: input.status, headers }),
}),
})
}
@@ -229,25 +180,25 @@ const httpError = (input: {
readonly error: unknown
readonly request: HttpClientRequest.HttpClientRequest
readonly operation: HttpOperation
readonly http?: HttpContext
}) => {
const request = HttpClientError.isHttpClientError(input.error) ? input.error.request : input.request
const transportError = (failure: { readonly message: string; readonly code?: string | undefined }) =>
new AIError({
module: "RequestExecutor",
method: input.operation,
reason: new TransportReason({
reason: new TransportError({
message: failure.message,
cause: source,
http: input.http,
transport: "http",
operation: input.operation,
code: failure.code,
url: request.url,
http: new HttpContext({ request: requestDetails(request) }),
}),
})
const source =
HttpClientError.isHttpClientError(input.error) && "cause" in input.error.reason
? input.error.reason.cause
? (input.error.reason.cause ?? input.error)
: input.error
const native = nativeTransportFailure(source)
const code = native?.code
@@ -271,6 +222,13 @@ const httpError = (input: {
})
}
export const responseStream = (response: HttpClientResponse.HttpClientResponse): Stream.Stream<Uint8Array, AIError> =>
response.stream.pipe(
Stream.mapError((error) =>
httpError({ error, request: response.request, operation: "read", http: responseHttp(response) }),
),
)
export const stream = (
executor: Interface,
request: HttpClientRequest.HttpClientRequest,
@@ -279,9 +237,7 @@ export const stream = (
Stream.unwrap(
Effect.gen(function* () {
const response = yield* executor.execute(request, middleware)
return response.stream.pipe(
Stream.mapError((error) => httpError({ error, request: response.request, operation: "read" })),
)
return responseStream(response)
}),
)
@@ -294,7 +250,7 @@ export const layer: Layer.Layer<Service, never, HttpClient.HttpClient> = Layer.e
if (!middleware)
return yield* http.execute(request).pipe(
Effect.mapError((error) => httpError({ error, request, operation: "request" })),
Effect.flatMap(statusError(request)),
Effect.flatMap(statusError),
)
const response = yield* middleware(request, (input) =>
@@ -302,7 +258,7 @@ export const layer: Layer.Layer<Service, never, HttpClient.HttpClient> = Layer.e
.execute(input)
.pipe(Effect.mapError((cause) => (cause instanceof Error ? cause : new Error(String(cause))))),
).pipe(Effect.mapError((error) => httpError({ error, request, operation: "request" })))
return yield* statusError(response.request)(response)
return yield* statusError(response)
})
return Service.of({
execute: executeOnce,
+10 -2
View File
@@ -8,8 +8,8 @@ import type { AIError } from "../schema/index.js"
* `Framing` is the byte-stream-shaped seam between transport and protocol:
*
* - SSE (`Framing.sse`) — UTF-8 decode the body, run the SSE channel decoder,
* drop empty / `[DONE]` keep-alives. Each emitted frame is the JSON `data:`
* payload of one event.
* and emit the `data:` payload of each non-empty event. The default drops
* `[DONE]`; protocols that use it as a terminal select `sseWithDone`.
* - AWS event stream — length-prefixed binary frames with CRC checksums.
* Each emitted frame is one parsed binary event record.
*
@@ -19,11 +19,19 @@ import type { AIError } from "../schema/index.js"
export interface Definition<Frame> {
readonly id: string
readonly frame: (bytes: Stream.Stream<Uint8Array, AIError>) => Stream.Stream<Frame, AIError>
/** Original wire representation when framing transforms the provider payload. */
readonly body?: (frame: Frame) => string | undefined
}
/** Server-Sent Events framing. Used by every JSON-streaming HTTP provider. */
export const sse: Definition<string> = { id: "sse", frame: ProviderShared.sseFraming }
/** Server-Sent Events framing that retains the conventional `[DONE]` sentinel. */
export const sseWithDone: Definition<string> = {
id: "sse",
frame: (bytes) => ProviderShared.sseFraming(bytes, undefined, true),
}
/** SSE framing restricted to protocol-recognized event names. */
export const sseEvents = (events: ReadonlySet<string>): Definition<string> => ({
id: "sse",
+7 -2
View File
@@ -88,8 +88,13 @@ export const httpJson = <Body, Frame>(input: HttpJsonInput<Body, Frame>): HttpJs
}
}),
execute: (prepared, _request, runtime) =>
Effect.succeed({
frames: prepared.framing.frame(RequestExecutor.stream(runtime.http, prepared.request, prepared.middleware)),
Effect.gen(function* () {
const response = yield* runtime.http.execute(prepared.request, prepared.middleware)
return {
frames: prepared.framing.frame(RequestExecutor.responseStream(response)),
http: RequestExecutor.responseHttp(response),
body: prepared.framing.body,
}
}),
})
+3 -1
View File
@@ -3,7 +3,7 @@ import { Endpoint } from "../endpoint.js"
import { Auth } from "../auth.js"
import type { HttpMiddleware, Interface as RequestExecutorInterface } from "../executor.js"
import type { WebSocketChannelExecutor } from "./websocket-channel.js"
import type { AIError, LLMRequest } from "../../schema/index.js"
import type { AIError, HttpContext, LLMRequest } from "../../schema/index.js"
export interface TransportRuntime {
readonly http: RequestExecutorInterface
@@ -11,6 +11,8 @@ export interface TransportRuntime {
export interface TransportExecution<Frame> {
readonly frames: Stream.Stream<Frame, AIError>
readonly http?: HttpContext
body?(frame: Frame): string | undefined
/** Optional successful-consumption acknowledgement. HTTP leaves this absent. */
readonly complete?: Effect.Effect<void>
}
@@ -1,6 +1,6 @@
import type { Effect, Scope, Stream } from "effect"
import type { Headers } from "effect/unstable/http"
import type { AIError } from "../../schema/index.js"
import type { AIError, HttpContext } from "../../schema/index.js"
export interface WebSocketChannelExecutor {
readonly execute: (
@@ -10,6 +10,7 @@ export interface WebSocketChannelExecutor {
export interface WebSocketChannelExecution {
readonly frames: Stream.Stream<string, AIError>
readonly http?: HttpContext
/** Commits staged state after the decoded Route stream ends successfully. */
readonly complete: Effect.Effect<void>
}
+100 -33
View File
@@ -1,7 +1,13 @@
import { Cause, Effect, Queue, Stream } from "effect"
import { Headers } from "effect/unstable/http"
import { Socket } from "effect/unstable/socket"
import { AIError, TransportReason, type TransportOperation } from "../../schema/index.js"
import {
AIError,
AIErrorReason,
TransportError,
type HttpContext,
type TransportOperation,
} from "../../schema/index.js"
import * as HttpTransport from "./http.js"
import type { Transport } from "./index.js"
import type {
@@ -17,6 +23,7 @@ export interface WebSocketRequest {
}
export interface WebSocketConnection {
readonly http?: HttpContext
readonly sendText: (message: string) => Effect.Effect<void, AIError>
readonly messages: Stream.Stream<string | Uint8Array, AIError>
readonly close: Effect.Effect<void, never>
@@ -33,21 +40,22 @@ type WebSocketConstructorWithHeaders = (
const MAX_FRAME_BYTES = 16 * 1024 * 1024
const transportError = (
method: string,
message: string,
input: {
readonly operation: TransportOperation
readonly url?: string
readonly code?: string
readonly phase?: TransportReason["phase"]
readonly delivery?: TransportReason["delivery"]
readonly phase?: TransportError["phase"]
readonly delivery?: TransportError["delivery"]
readonly body?: string
readonly cause?: unknown
},
) =>
new AIError({
module: "WebSocketConnector",
method,
reason: new TransportReason({
reason: new TransportError({
message,
body: input.body,
cause: input.cause,
transport: "websocket",
operation: input.operation,
url: input.url,
@@ -59,22 +67,16 @@ const transportError = (
const annotateTransportError = (
error: AIError,
input: { readonly phase: TransportReason["phase"]; readonly delivery: TransportReason["delivery"] },
input: { readonly phase: TransportError["phase"]; readonly delivery: TransportError["delivery"] },
) =>
error.reason._tag === "Transport"
? new AIError({
module: error.module,
method: error.method,
reason: new TransportReason({
reason: new TransportError({
...error.reason,
message: error.reason.message,
transport: error.reason.transport,
operation: error.reason.operation,
code: error.reason.code,
url: error.reason.url,
http: error.reason.http,
cause: error.reason.cause,
phase: input.phase,
delivery: input.delivery,
recovery: error.reason.recovery,
}),
})
: error
@@ -95,7 +97,7 @@ const waitOpen = (ws: globalThis.WebSocket, input: WebSocketRequest) => {
if (ws.readyState === globalThis.WebSocket.OPEN) return Effect.void
if (ws.readyState === globalThis.WebSocket.CLOSING || ws.readyState === globalThis.WebSocket.CLOSED) {
return Effect.fail(
transportError("open", `WebSocket closed before opening (state ${ws.readyState})`, {
transportError(`WebSocket closed before opening (state ${ws.readyState})`, {
url: input.url,
operation: "request",
code: "closed",
@@ -124,7 +126,8 @@ const waitOpen = (ws: globalThis.WebSocket, input: WebSocketRequest) => {
cleanup()
resume(
Effect.fail(
transportError("open", `Failed to open WebSocket: ${eventMessage(event)}`, {
transportError(`Failed to open WebSocket: ${eventMessage(event)}`, {
cause: "error" in event ? (event.error ?? event) : event,
url: input.url,
operation: "request",
phase: "connect",
@@ -137,7 +140,9 @@ const waitOpen = (ws: globalThis.WebSocket, input: WebSocketRequest) => {
cleanup()
resume(
Effect.fail(
transportError("open", `WebSocket closed before opening with code ${event.code}`, {
transportError(`WebSocket closed before opening with code ${event.code}`, {
body: event.reason,
cause: event,
url: input.url,
operation: "request",
code: String(event.code),
@@ -169,7 +174,8 @@ export const toWebSocketUrl = (value: string) =>
throw new Error(`Unsupported WebSocket URL protocol ${url.protocol}`)
},
catch: (error) =>
transportError("prepare", error instanceof Error ? error.message : "Invalid WebSocket URL", {
transportError(error instanceof Error ? error.message : "Invalid WebSocket URL", {
cause: error,
url: value,
operation: "request",
code: "invalid-url",
@@ -189,7 +195,8 @@ export const open = (input: WebSocketRequest) =>
headers: input.headers,
}),
catch: (error) =>
transportError("open", error instanceof Error ? error.message : "Failed to construct WebSocket", {
transportError(error instanceof Error ? error.message : "Failed to construct WebSocket", {
cause: error,
url: input.url,
operation: "request",
phase: "connect",
@@ -214,7 +221,8 @@ export const fromWebSocket = (
Queue.failCauseUnsafe(
messages,
Cause.fail(
transportError("message", "WebSocket message exceeds the 16 MiB limit", {
transportError("WebSocket message exceeds the 16 MiB limit", {
body: typeof message === "string" ? message : new TextDecoder().decode(message),
url: input.url,
operation: "read",
code: "message-too-large",
@@ -231,7 +239,8 @@ export const fromWebSocket = (
Queue.failCauseUnsafe(
messages,
Cause.fail(
transportError("message", "WebSocket inbound queue overflow", {
transportError("WebSocket inbound queue overflow", {
body: typeof message === "string" ? message : new TextDecoder().decode(message),
url: input.url,
operation: "read",
code: "queue-overflow",
@@ -248,7 +257,8 @@ export const fromWebSocket = (
Queue.failCauseUnsafe(
messages,
Cause.fail(
transportError("message", "Unsupported WebSocket message payload", {
transportError("Unsupported WebSocket message payload", {
cause: event,
url: input.url,
operation: "read",
code: "message",
@@ -261,7 +271,8 @@ export const fromWebSocket = (
Queue.failCauseUnsafe(
messages,
Cause.fail(
transportError("message", `WebSocket error: ${eventMessage(event)}`, {
transportError(`WebSocket error: ${eventMessage(event)}`, {
cause: "error" in event ? (event.error ?? event) : event,
url: input.url,
operation: "read",
code: "message",
@@ -274,7 +285,9 @@ export const fromWebSocket = (
Queue.failCauseUnsafe(
messages,
Cause.fail(
transportError("message", `WebSocket closed with code ${event.code}`, {
transportError(`WebSocket closed with code ${event.code}`, {
body: event.reason,
cause: event,
url: input.url,
operation: "read",
code: String(event.code),
@@ -298,7 +311,7 @@ export const fromWebSocket = (
Effect.suspend(() => {
if (ws.readyState !== globalThis.WebSocket.OPEN)
return Effect.fail(
transportError("sendText", `WebSocket is not open (state ${ws.readyState})`, {
transportError(`WebSocket is not open (state ${ws.readyState})`, {
url: input.url,
operation: "write",
phase: "send",
@@ -308,7 +321,8 @@ export const fromWebSocket = (
return Effect.try({
try: () => ws.send(message),
catch: (error) =>
transportError("sendText", error instanceof Error ? error.message : "Failed to send WebSocket message", {
transportError(error instanceof Error ? error.message : "Failed to send WebSocket message", {
cause: error,
url: input.url,
operation: "write",
phase: "send",
@@ -349,10 +363,23 @@ export const makeDirect = (connector: WebSocketConnector): WebSocketChannelExecu
(connection) => connection.close,
)
const create = yield* exchange.driver.create(undefined)
yield* connection.sendText(create.message)
yield* connection.sendText(create.message).pipe(
Effect.mapError(
(error) =>
new AIError({
reason: AIErrorReason.make({
...error.reason,
message: error.reason.message,
cause: error.reason.cause,
http: error.reason.http ?? connection.http,
}),
}),
),
)
const decoder = new TextDecoder()
let observed = false
return {
http: connection.http,
frames: connection.messages.pipe(
Stream.map((message) => {
observed = true
@@ -364,9 +391,49 @@ export const makeDirect = (connector: WebSocketConnector): WebSocketChannelExecu
delivery: observed ? "accepted" : "ambiguous",
}),
),
Stream.mapEffect((frame) => exchange.driver.observe(create, frame)),
Stream.mapEffect((frame) =>
exchange.driver.observe(create, frame).pipe(
Effect.mapError(
(error) =>
new AIError({
reason: AIErrorReason.make({
...error.reason,
message: error.reason.message,
cause: error.reason.cause,
body: frame,
}),
}),
),
Effect.map((observation) =>
"error" in observation
? {
...observation,
error: new AIError({
reason: AIErrorReason.make({
...observation.error.reason,
message: observation.error.reason.message,
cause: observation.error.reason.cause,
body: frame,
}),
}),
}
: observation,
),
),
),
Stream.takeUntil(observationTerminal),
Stream.mapEffect(observationFrame),
Stream.mapError(
(error) =>
new AIError({
reason: AIErrorReason.make({
...error.reason,
message: error.reason.message,
cause: error.reason.cause,
http: error.reason.http ?? connection.http,
}),
}),
),
),
complete: Effect.void,
}
@@ -417,7 +484,7 @@ export const json = <Body, Message>(input: JsonInput<Body, Message>): JsonTransp
const webSocket = options?.webSocket
if (!webSocket) {
return Effect.fail(
transportError("json", "WebSocket JSON transport requires StreamOptions.webSocket", {
transportError("WebSocket JSON transport requires StreamOptions.webSocket", {
url: prepared.url,
operation: "request",
code: "unavailable",
@@ -435,7 +502,7 @@ export const json = <Body, Message>(input: JsonInput<Body, Message>): JsonTransp
connect: { url: prepared.url, headers: prepared.headers },
fallback: () =>
Stream.fail(
transportError("fallback", "WebSocket JSON transport does not provide HTTP fallback", {
transportError("WebSocket JSON transport does not provide HTTP fallback", {
url: prepared.url,
operation: "request",
code: "websocket",
+61 -95
View File
@@ -1,19 +1,13 @@
import { Schema } from "effect"
import { Tool } from "@opencode-ai/schema/tool"
import { ModelID, ProviderID, RouteID } from "./ids.js"
import { ProviderMetadata } from "./messages.js"
export const ProviderFailureClassification = Schema.Literals(["context-overflow", "payload-too-large"])
export type ProviderFailureClassification = typeof ProviderFailureClassification.Type
export class HttpRequestDetails extends Schema.Class<HttpRequestDetails>("AI.HttpRequestDetails")({
method: Schema.String,
export class HttpContext extends Schema.Class<HttpContext>("AI.HttpContext")({
url: Schema.String,
headers: Schema.Record(Schema.String, Schema.String),
}) {}
export class HttpResponseDetails extends Schema.Class<HttpResponseDetails>("AI.HttpResponseDetails")({
status: Schema.Number,
status: Schema.Int.check(Schema.isBetween({ minimum: 100, maximum: 599 })),
headers: Schema.Record(Schema.String, Schema.String),
}) {}
@@ -24,73 +18,58 @@ export class HttpRateLimitDetails extends Schema.Class<HttpRateLimitDetails>("AI
reset: Schema.optional(Schema.Record(Schema.String, Schema.String)),
}) {}
export class HttpContext extends Schema.Class<HttpContext>("AI.HttpContext")({
request: HttpRequestDetails,
response: Schema.optional(HttpResponseDetails),
body: Schema.optional(Schema.String),
bodyTruncated: Schema.optional(Schema.Boolean),
rateLimit: Schema.optional(HttpRateLimitDetails),
}) {}
export class InvalidRequestReason extends Schema.Class<InvalidRequestReason>("AI.Error.InvalidRequest")({
_tag: Schema.tag("InvalidRequest"),
const ReasonFields = {
message: Schema.String,
parameter: Schema.optional(Schema.String),
classification: Schema.optional(ProviderFailureClassification),
providerMetadata: Schema.optional(ProviderMetadata),
// Preserve the complete original response or triggering event before decoding narrows it.
body: Schema.optional(Schema.String),
http: Schema.optional(HttpContext),
}) {}
cause: Schema.optional(Schema.Defect({ includeStack: true })),
}
export class NoRouteReason extends Schema.Class<NoRouteReason>("AI.Error.NoRoute")({
_tag: Schema.tag("NoRoute"),
export class InvalidRequestError extends Schema.TaggedError<InvalidRequestError>("AI.Error.InvalidRequest")(
"InvalidRequest",
{
...ReasonFields,
parameter: Schema.optional(Schema.String),
classification: Schema.optional(ProviderFailureClassification),
},
) {}
export class NoRouteError extends Schema.TaggedError<NoRouteError>("AI.Error.NoRoute")("NoRoute", {
...ReasonFields,
route: RouteID,
provider: ProviderID,
model: ModelID,
}) {
get message() {
return `No AI route for ${this.provider}/${this.model} using ${this.route}`
}
}
export class AuthenticationReason extends Schema.Class<AuthenticationReason>("AI.Error.Authentication")({
_tag: Schema.tag("Authentication"),
message: Schema.String,
kind: Schema.Literals(["missing", "invalid", "expired", "insufficient-permissions", "unknown"]),
providerMetadata: Schema.optional(ProviderMetadata),
http: Schema.optional(HttpContext),
}) {}
export class RateLimitReason extends Schema.Class<RateLimitReason>("AI.Error.RateLimit")({
_tag: Schema.tag("RateLimit"),
message: Schema.String,
export class AuthenticationError extends Schema.TaggedError<AuthenticationError>("AI.Error.Authentication")(
"Authentication",
ReasonFields,
) {}
export class RateLimitError extends Schema.TaggedError<RateLimitError>("AI.Error.RateLimit")("RateLimit", {
...ReasonFields,
retryAfterMs: Schema.optional(Schema.Number),
rateLimit: Schema.optional(HttpRateLimitDetails),
providerMetadata: Schema.optional(ProviderMetadata),
http: Schema.optional(HttpContext),
}) {}
export class QuotaExceededReason extends Schema.Class<QuotaExceededReason>("AI.Error.QuotaExceeded")({
_tag: Schema.tag("QuotaExceeded"),
message: Schema.String,
providerMetadata: Schema.optional(ProviderMetadata),
http: Schema.optional(HttpContext),
}) {}
export class QuotaExceededError extends Schema.TaggedError<QuotaExceededError>("AI.Error.QuotaExceeded")(
"QuotaExceeded",
ReasonFields,
) {}
export class ContentPolicyReason extends Schema.Class<ContentPolicyReason>("AI.Error.ContentPolicy")({
_tag: Schema.tag("ContentPolicy"),
message: Schema.String,
providerMetadata: Schema.optional(ProviderMetadata),
http: Schema.optional(HttpContext),
}) {}
export class ContentPolicyError extends Schema.TaggedError<ContentPolicyError>("AI.Error.ContentPolicy")(
"ContentPolicy",
ReasonFields,
) {}
export class ProviderInternalReason extends Schema.Class<ProviderInternalReason>("AI.Error.ProviderInternal")({
_tag: Schema.tag("ProviderInternal"),
message: Schema.String,
status: Schema.optional(Schema.Number),
retryAfterMs: Schema.optional(Schema.Number),
providerMetadata: Schema.optional(ProviderMetadata),
http: Schema.optional(HttpContext),
}) {}
export class ProviderInternalError extends Schema.TaggedError<ProviderInternalError>("AI.Error.ProviderInternal")(
"ProviderInternal",
{
...ReasonFields,
retryAfterMs: Schema.optional(Schema.Number),
},
) {}
export const TransportType = Schema.Literals(["http", "websocket"])
export type TransportType = typeof TransportType.Type
@@ -98,14 +77,12 @@ export type TransportType = typeof TransportType.Type
export const TransportOperation = Schema.Literals(["request", "read", "write"])
export type TransportOperation = typeof TransportOperation.Type
export class TransportReason extends Schema.Class<TransportReason>("AI.Error.Transport")({
_tag: Schema.tag("Transport"),
message: Schema.String,
export class TransportError extends Schema.TaggedError<TransportError>("AI.Error.Transport")("Transport", {
...ReasonFields,
transport: TransportType,
operation: TransportOperation,
code: Schema.optional(Schema.String),
url: Schema.optional(Schema.String),
http: Schema.optional(HttpContext),
phase: Schema.optional(
Schema.Literals(["prepare", "queue", "connect", "send", "receive", "decode", "complete", "fallback", "close"]),
),
@@ -115,51 +92,40 @@ export class TransportReason extends Schema.Class<TransportReason>("AI.Error.Tra
),
}) {}
export class InvalidProviderOutputReason extends Schema.Class<InvalidProviderOutputReason>(
export class InvalidProviderOutputError extends Schema.TaggedError<InvalidProviderOutputError>(
"AI.Error.InvalidProviderOutput",
)({
_tag: Schema.tag("InvalidProviderOutput"),
message: Schema.String,
)("InvalidProviderOutput", {
...ReasonFields,
classification: Schema.optional(Schema.Literals(["incomplete-stream"])),
route: Schema.optional(Schema.String),
raw: Schema.optional(Schema.String),
providerMetadata: Schema.optional(ProviderMetadata),
}) {}
export class UnknownProviderReason extends Schema.Class<UnknownProviderReason>("AI.Error.UnknownProvider")({
_tag: Schema.tag("UnknownProvider"),
message: Schema.String,
status: Schema.optional(Schema.Number),
providerMetadata: Schema.optional(ProviderMetadata),
http: Schema.optional(HttpContext),
}) {}
export class UnknownProviderError extends Schema.TaggedError<UnknownProviderError>("AI.Error.UnknownProvider")(
"UnknownProvider",
ReasonFields,
) {}
export const AIErrorReason = Schema.Union([
InvalidRequestReason,
NoRouteReason,
AuthenticationReason,
RateLimitReason,
QuotaExceededReason,
ContentPolicyReason,
ProviderInternalReason,
TransportReason,
InvalidProviderOutputReason,
UnknownProviderReason,
InvalidRequestError,
NoRouteError,
AuthenticationError,
RateLimitError,
QuotaExceededError,
ContentPolicyError,
ProviderInternalError,
TransportError,
InvalidProviderOutputError,
UnknownProviderError,
]).pipe(Schema.toTaggedUnion("_tag"))
export type AIErrorReason = Schema.Schema.Type<typeof AIErrorReason>
export class AIError extends Schema.TaggedError<AIError>()("AI.Error", {
module: Schema.String,
method: Schema.String,
reason: AIErrorReason,
// Raw provider payload as a string, so classified failures never lose the
// original error detail even when the pretty message is a summary.
body: Schema.optional(Schema.String),
}) {
override readonly cause = this.reason
override get message() {
return `${this.module}.${this.method}: ${this.reason.message}`
override get message(): string {
return this.reason.message
}
}
+50 -15
View File
@@ -3,6 +3,7 @@ import { LLM } from "@opencode-ai/schema/llm"
import { ContentBlockID, ToolCallID } from "./ids.js"
import {
Message,
CompactionPart,
ProviderMetadata,
ToolCallPart,
ToolOutput,
@@ -62,6 +63,8 @@ export { ProviderMetadata } from "./messages.js"
* Matches the same escape-hatch field on `LLMEvent`.
*/
export class Usage extends Schema.Class<Usage>("AI.Usage")({
/** Effective input size of the final message iteration, when reported; not billed totals. */
contextTokens: Schema.optional(Schema.Number),
inputTokens: Schema.optional(Schema.Number),
outputTokens: Schema.optional(Schema.Number),
nonCachedInputTokens: Schema.optional(Schema.Number),
@@ -72,7 +75,7 @@ export class Usage extends Schema.Class<Usage>("AI.Usage")({
providerMetadata: Schema.optional(ProviderMetadata),
}) {
/**
* Visible output tokens — `outputTokens` minus `reasoningTokens`, clamped
* Non-reasoning output tokens (including compaction summaries) — `outputTokens` minus `reasoningTokens`, clamped
* to zero. The one place subtraction happens in this contract; the clamp
* means a provider reporting `reasoningTokens > outputTokens` produces a
* harmless zero rather than a negative that crashes downstream schemas.
@@ -88,6 +91,12 @@ export class Usage extends Schema.Class<Usage>("AI.Usage")({
export type UsageInput = Usage | ConstructorParameters<typeof Usage>[0]
/** A replacement context window. Replace prior history with these messages. */
export class CompactionResponse extends Schema.Class<CompactionResponse>("LLM.CompactionResponse")({
messages: Schema.Array(Message),
usage: Schema.optional(Usage),
}) {}
export const StepStart = Schema.Struct({
type: Schema.tag("step-start"),
index: Schema.Number,
@@ -112,6 +121,8 @@ export type TextDelta = Schema.Schema.Type<typeof TextDelta>
export const TextEnd = Schema.Struct({
type: Schema.tag("text-end"),
id: ContentBlockID,
/** Authoritative complete value; replaces accumulated deltas when present. */
text: Schema.optional(Schema.String),
providerMetadata: Schema.optional(ProviderMetadata),
}).annotate({ identifier: "LLM.Event.TextEnd" })
export type TextEnd = Schema.Schema.Type<typeof TextEnd>
@@ -134,6 +145,8 @@ export type ReasoningDelta = Schema.Schema.Type<typeof ReasoningDelta>
export const ReasoningEnd = Schema.Struct({
type: Schema.tag("reasoning-end"),
id: ContentBlockID,
/** Authoritative complete value; replaces accumulated deltas when present. */
text: Schema.optional(Schema.String),
providerMetadata: Schema.optional(ProviderMetadata),
}).annotate({ identifier: "LLM.Event.ReasoningEnd" })
export type ReasoningEnd = Schema.Schema.Type<typeof ReasoningEnd>
@@ -237,6 +250,7 @@ export const ProviderErrorEvent = Schema.Struct({
export type ProviderErrorEvent = Schema.Schema.Type<typeof ProviderErrorEvent>
const llmEventTagged = Schema.Union([
CompactionPart,
StepStart,
TextStart,
TextDelta,
@@ -270,6 +284,7 @@ const toolCallID = (value: ToolCallID | string) => ToolCallID.make(value)
* `events.filter(LLMEvent.guards["tool-call"])`.
*/
export const LLMEvent = Object.assign(llmEventTagged, {
compaction: CompactionPart.make,
stepStart: StepStart.make,
textStart: (input: WithID<TextStart, ContentBlockID>) => TextStart.make({ ...input, id: contentBlockID(input.id) }),
textDelta: (input: WithID<TextDelta, ContentBlockID>) => TextDelta.make({ ...input, id: contentBlockID(input.id) }),
@@ -307,6 +322,7 @@ export const LLMEvent = Object.assign(llmEventTagged, {
}),
providerError: ProviderErrorEvent.make,
is: {
compaction: llmEventTagged.guards.compaction,
stepStart: llmEventTagged.guards["step-start"],
textStart: llmEventTagged.guards["text-start"],
textDelta: llmEventTagged.guards["text-delta"],
@@ -328,17 +344,32 @@ export const LLMEvent = Object.assign(llmEventTagged, {
})
export type LLMEvent = Schema.Schema.Type<typeof llmEventTagged>
/** Joins deltas per fragment, letting an authoritative end value replace that fragment's accumulated deltas. */
const joinFragments = (
events: ReadonlyArray<LLMEvent>,
isDelta: (event: LLMEvent) => event is LLMEvent & { id: string; text: string },
isEnd: (event: LLMEvent) => event is LLMEvent & { id: string; text?: string },
) => {
const order: string[] = []
const parts = new Map<string, string>()
for (const event of events) {
if (isDelta(event)) {
if (!parts.has(event.id)) order.push(event.id)
parts.set(event.id, (parts.get(event.id) ?? "") + event.text)
}
if (isEnd(event) && event.text !== undefined) {
if (!parts.has(event.id)) order.push(event.id)
parts.set(event.id, event.text)
}
}
return order.map((id) => parts.get(id)).join("")
}
const responseText = (events: ReadonlyArray<LLMEvent>) =>
events
.filter(LLMEvent.is.textDelta)
.map((event) => event.text)
.join("")
joinFragments(events, LLMEvent.is.textDelta, LLMEvent.is.textEnd)
const responseReasoning = (events: ReadonlyArray<LLMEvent>) =>
events
.filter(LLMEvent.is.reasoningDelta)
.map((event) => event.text)
.join("")
joinFragments(events, LLMEvent.is.reasoningDelta, LLMEvent.is.reasoningEnd)
const responseUsage = (events: ReadonlyArray<LLMEvent>) =>
events.reduce<Usage | undefined>(
@@ -445,10 +476,11 @@ const reduceTextDelta = (state: ResponseState, event: TextDelta): ResponseState
const reduceTextEnd = (state: ResponseState, event: TextEnd): ResponseState => {
const current = state.textParts[event.id]
if (!current) return state
const text = event.text ?? current.text
const providerMetadata = event.providerMetadata ?? current.providerMetadata
return {
...replaceContent(state, current.contentIndex, textContent(current.text, providerMetadata)),
textParts: { ...state.textParts, [event.id]: { ...current, providerMetadata } },
...replaceContent(state, current.contentIndex, textContent(text, providerMetadata)),
textParts: { ...state.textParts, [event.id]: { ...current, text, providerMetadata } },
}
}
@@ -478,10 +510,11 @@ const reduceReasoningDelta = (state: ResponseState, event: ReasoningDelta): Resp
const reduceReasoningEnd = (state: ResponseState, event: ReasoningEnd): ResponseState => {
const current = state.reasoningParts[event.id]
if (!current) return state
const text = event.text ?? current.text
const providerMetadata = event.providerMetadata ?? current.providerMetadata
return {
...replaceContent(state, current.contentIndex, reasoningContent(current.text, providerMetadata)),
reasoningParts: { ...state.reasoningParts, [event.id]: { ...current, providerMetadata } },
...replaceContent(state, current.contentIndex, reasoningContent(text, providerMetadata)),
reasoningParts: { ...state.reasoningParts, [event.id]: { ...current, text, providerMetadata } },
}
}
@@ -542,6 +575,8 @@ const reduceToolCall = (state: ResponseState, event: ToolCall): ResponseState =>
const reduceResponseState = (state: ResponseState, event: LLMEvent): ResponseState => {
const next = appendEvent(state, event)
switch (event.type) {
case "compaction":
return appendContent(next, event)
case "text-start":
return ensureText(next, event.id, event.providerMetadata)
case "text-delta":
@@ -579,12 +614,12 @@ export class LLMResponse extends Schema.Class<LLMResponse>("LLM.Response")({
usage: Schema.optional(Usage),
finishReason: FinishReasonDetails,
}) {
/** Concatenated assistant text assembled from streamed `text-delta` events. */
/** Concatenated assistant text; each fragment's `text-end` value replaces its accumulated deltas when present. */
get text() {
return responseText(this.events)
}
/** Concatenated reasoning text assembled from streamed `reasoning-delta` events. */
/** Concatenated reasoning text; each fragment's `reasoning-end` value replaces its accumulated deltas when present. */
get reasoning() {
return responseReasoning(this.events)
}
+38 -9
View File
@@ -9,7 +9,7 @@ import {
LanguageModelSchema,
ProviderOptions,
} from "./options.js"
import { isRecord } from "../utils/record.js"
import { ProviderID } from "./ids.js"
export const MessageRole = Schema.Literals(["system", "user", "assistant", "tool"])
export type MessageRole = Schema.Schema.Type<typeof MessageRole>
@@ -53,14 +53,10 @@ export const MediaPart = Schema.Struct({
filename: Schema.optional(Schema.String),
cache: Schema.optional(CacheHint),
metadata: Schema.optional(Schema.Record(Schema.String, Schema.Unknown)),
providerMetadata: Schema.optional(ProviderMetadata),
}).annotate({ identifier: "LLM.Content.Media" })
export type MediaPart = Schema.Schema.Type<typeof MediaPart>
const isToolResultValue = (value: unknown): value is ToolResultValue =>
isRecord(value) &&
(value.type === "text" || value.type === "json" || value.type === "error" || value.type === "content") &&
"value" in value
const toolResultValueSchema = Schema.Union([
Schema.Struct({
type: Schema.Literal("json"),
@@ -80,6 +76,7 @@ const toolResultValueSchema = Schema.Union([
}),
]).annotate({ identifier: "LLM.ToolResult" })
export type ToolResultValue = Schema.Schema.Type<typeof toolResultValueSchema>
const isToolResultValue = Schema.is(toolResultValueSchema)
export const ToolResultValue = Object.assign(toolResultValueSchema, {
is: isToolResultValue,
@@ -190,9 +187,40 @@ export const ReasoningPart = Schema.Struct({
}).annotate({ identifier: "LLM.Content.Reasoning" })
export type ReasoningPart = Schema.Schema.Type<typeof ReasoningPart>
export const ContentPart = Schema.Union([TextPart, MediaPart, ToolCallPart, ToolResultPart, ReasoningPart]).pipe(
Schema.toTaggedUnion("type"),
)
/** A provider-generated context checkpoint, distinct from visible assistant text. */
type CompactionContent =
| { readonly encrypted: string; readonly text?: never }
| { readonly text: string | null; readonly encrypted?: never }
const compactionPartSchema = Schema.Struct({
type: Schema.Literal("compaction"),
provider: ProviderID,
id: Schema.optional(Schema.String),
encrypted: Schema.optional(Schema.String),
/** Null means the provider failed to produce a summary; prior history must be retained. */
text: Schema.optional(Schema.NullOr(Schema.String)),
})
.pipe(
Schema.refine(
(part): part is typeof part & CompactionContent => (part.encrypted !== undefined) !== (part.text !== undefined),
{ message: "Compaction requires either encrypted content or a summary" },
),
)
.annotate({ identifier: "LLM.Content.Compaction" })
export type CompactionPart = typeof compactionPartSchema.Type
export const CompactionPart = Object.assign(compactionPartSchema, {
make: (input: Omit<CompactionPart, "type" | "encrypted" | "text"> & CompactionContent): CompactionPart =>
Schema.decodeUnknownSync(compactionPartSchema)({ type: "compaction", ...input }),
})
export const ContentPart = Schema.Union([
TextPart,
MediaPart,
ToolCallPart,
ToolResultPart,
ReasoningPart,
CompactionPart,
]).pipe(Schema.toTaggedUnion("type"))
export type ContentPart = Schema.Schema.Type<typeof ContentPart>
export class Message extends Schema.Class<Message>("LLM.Message")({
@@ -200,6 +228,7 @@ export class Message extends Schema.Class<Message>("LLM.Message")({
role: MessageRole,
content: Schema.Array(ContentPart),
metadata: Schema.optional(Schema.Record(Schema.String, Schema.Unknown)),
providerMetadata: Schema.optional(ProviderMetadata),
native: Schema.optional(Schema.Record(Schema.String, Schema.Unknown)),
}) {}
+113 -51
View File
@@ -1,9 +1,10 @@
export * as TestLLM from "./testing.js"
import { LLMClient, type Interface as LLMClientShape } from "./route/client.js"
import { LLMClient } from "./route/client.js"
import {
LLMEvent,
LLMResponse,
CompactionResponse,
type FinishReasonDetails,
type AIError,
type LLMRequest,
@@ -16,13 +17,33 @@ export type Response = readonly LLMEvent[] | Stream.Stream<LLMEvent, AIError>
export type Gate = Readonly<{ started: Effect.Effect<void>; release: Effect.Effect<void> }>
type ClientInterface = Context.Service.Shape<typeof LLMClient.Service>
export type Responder = (request: LLMRequest) => Response
export interface TestInterface extends ClientInterface {
/** Returns a snapshot of requests observed at execution time. */
readonly requests: () => Effect.Effect<readonly LLMRequest[]>
readonly push: (...responses: readonly Response[]) => Effect.Effect<void>
/** Replaces the fallback without changing queued responses. */
readonly always: (response: Response) => Effect.Effect<void>
/** Answers requests after the one-shot queue is exhausted; receives the original request. */
readonly serve: (responder: Responder) => Effect.Effect<void>
/** Waits for request arrivals, not output or completion. */
readonly wait: (count: number) => Effect.Effect<void>
readonly gate: () => Effect.Effect<Gate, never, Scope.Scope>
}
export class Test extends Context.Service<Test, TestInterface>()("@opencode/ai/TestLLM/Test") {}
/** @deprecated Use TestInterface through Test and testLayer. */
export interface Interface {
readonly requests: LLMRequest[]
readonly push: (...responses: readonly Response[]) => Effect.Effect<void>
readonly always: (response: Response) => Effect.Effect<void>
readonly wait: (count: number) => Effect.Effect<void>
readonly gate: Effect.Effect<Gate, never, Scope.Scope>
readonly client: LLMClientShape
readonly client: ClientInterface
}
export interface LayerOptions {
@@ -31,6 +52,7 @@ export interface LayerOptions {
readonly fallback?: Response
}
/** @deprecated Use Test and testLayer for normal client methods and test controls. */
export class Service extends Context.Service<Service, Interface>()("@opencode/ai/TestLLM") {}
export const complete = (
@@ -80,59 +102,74 @@ export const hangAfter = (...events: readonly LLMEvent[]) => Stream.concat(Strea
const toStream = (response: Response) => (Stream.isStream(response) ? response : Stream.fromIterable(response))
export const layer = (options: LayerOptions = {}) =>
Layer.effect(
Service,
Effect.gen(function* () {
const requests: LLMRequest[] = []
const responses: Response[] = []
let started = Deferred.makeUnsafe<void>()
let fallback = options.fallback
let activeGate: { readonly started: Queue.Queue<void>; readonly release: Latch.Latch } | undefined
const wait = (count: number): Effect.Effect<void> =>
Effect.suspend(() =>
requests.length >= count ? Effect.void : Deferred.await(started).pipe(Effect.andThen(wait(count))),
)
const make = (options: LayerOptions) =>
Effect.sync(() => {
const requests: LLMRequest[] = []
const responses: Response[] = []
let started = Deferred.makeUnsafe<void>()
let fallback: Response | Responder | undefined = options.fallback
let activeGate: { readonly started: Queue.Queue<void>; readonly release: Latch.Latch } | undefined
const wait = (count: number): Effect.Effect<void> =>
Effect.suspend(() =>
requests.length >= count ? Effect.void : Deferred.await(started).pipe(Effect.andThen(wait(count))),
)
const stream = ((request: LLMRequest) => {
requests.push(options.transformRequest?.(request) ?? request)
const stream: ClientInterface["stream"] = (request) =>
Stream.suspend(() => {
const count = requests.push(options.transformRequest?.(request) ?? request)
const waiting = started
started = Deferred.makeUnsafe()
Deferred.doneUnsafe(waiting, Effect.void)
const response = responses.shift() ?? fallback
if (!response) return Stream.die(new Error(`TestLLM has no response for request ${requests.length}`))
const streamed = toStream(response)
const gate = activeGate
if (!gate) return streamed
return Stream.unwrap(
Queue.offer(gate.started, undefined).pipe(Effect.andThen(gate.release.await), Effect.as(streamed)),
)
}) as LLMClientShape["stream"]
const client = LLMClient.Service.of({
stream,
generate: (request) =>
stream(request).pipe(
Stream.runFold(LLMResponse.empty, LLMResponse.reduce),
Effect.flatMap((state) => {
const response = LLMResponse.complete(state)
if (response) return Effect.succeed(response)
return Effect.die("TestLLM response ended without a terminal finish event")
}),
),
try {
const response = responses.shift() ?? (typeof fallback === "function" ? fallback(request) : fallback)
if (!response) return Stream.die(new Error(`TestLLM has no response for request ${count}`))
const streamed = toStream(response)
if (!gate) return streamed
return Stream.unwrap(
Queue.offer(gate.started, undefined).pipe(Effect.andThen(gate.release.await), Effect.as(streamed)),
)
} finally {
// Waiters can resume synchronously; assign the reply and gate before notifying them.
Deferred.doneUnsafe(waiting, Effect.void)
}
})
return Service.of({
requests,
push: (...input) =>
Effect.sync(() => {
responses.push(...input)
const test = Test.of({
compact: (request) =>
stream(request).pipe(
Stream.runFold(LLMResponse.empty, LLMResponse.reduce),
Effect.flatMap((state) => {
const response = LLMResponse.complete(state)
if (!response?.message.content.some((part) => part.type === "compaction"))
return Effect.die("TestLLM compaction response must contain a checkpoint and terminal finish event")
return Effect.succeed(new CompactionResponse({ messages: [response.message], usage: response.usage }))
}),
always: (response) =>
Effect.sync(() => {
fallback = response
),
stream,
generate: (request) =>
stream(request).pipe(
Stream.runFold(LLMResponse.empty, LLMResponse.reduce),
Effect.flatMap((state) => {
const response = LLMResponse.complete(state)
if (response) return Effect.succeed(response)
return Effect.die("TestLLM response ended without a terminal finish event")
}),
wait,
gate: Effect.gen(function* () {
),
requests: () => Effect.sync(() => [...requests]),
push: (...input) =>
Effect.sync(() => {
responses.push(...input)
}),
always: (response) =>
Effect.sync(() => {
fallback = response
}),
serve: (responder) =>
Effect.sync(() => {
fallback = responder
}),
wait,
gate: () =>
Effect.gen(function* () {
const gate = {
started: yield* Effect.acquireRelease(Queue.unbounded<void>(), Queue.shutdown),
release: yield* Latch.make(),
@@ -147,11 +184,36 @@ export const layer = (options: LayerOptions = {}) =>
release,
}
}),
client,
})
}),
})
return { test, requests }
})
/** Provides one shared implementation under the normal client and test-control tags. */
export const testLayer = (options: LayerOptions = {}) =>
Layer.effectContext(
Effect.map(make(options), (implementation) =>
Context.make(LLMClient.Service, implementation.test).pipe(Context.add(Test, implementation.test)),
),
)
/** @deprecated Use testLayer; retained for published callers of the legacy control interface. */
export const layer = (options: LayerOptions = {}) =>
Layer.effect(
Service,
Effect.map(make(options), (implementation) =>
Service.of({
requests: implementation.requests,
push: implementation.test.push,
always: implementation.test.always,
wait: implementation.test.wait,
gate: implementation.test.gate(),
client: implementation.test,
}),
),
)
/** @deprecated testLayer provides LLMClient.Service directly. */
export const clientLayer = Layer.effect(
LLMClient.Service,
Effect.map(Service, (service) => service.client),
+75
View File
@@ -0,0 +1,75 @@
import { Message, ToolResultPart, type ToolCallPart } from "./schema/messages.js"
const EMPTY_TOOL_OUTPUT = "(no tool output)"
const MISSING_TOOL_RESULT = "Tool result missing"
export function normalizeToolHistory(messages: ReadonlyArray<Message>) {
const normalized: Message[] = []
const pending = new Map<string, ToolCallPart>()
const appendMissingResults = () => {
if (pending.size === 0) return
normalized.push(missingToolResults(pending.values()))
pending.clear()
}
for (const message of messages) {
if (message.role === "user" || message.role === "assistant") appendMissingResults()
if (message.role === "tool") {
const tool = normalizeToolMessage(message, pending)
if (tool) normalized.push(tool)
continue
}
normalized.push(message)
if (message.role !== "assistant") continue
for (const part of message.content) {
if (part.type === "tool-call" && part.providerExecuted !== true) pending.set(part.id, part)
}
}
return normalized.length === messages.length && normalized.every((message, index) => message === messages[index])
? messages
: normalized
}
function missingToolResults(calls: Iterable<ToolCallPart>) {
return new Message({
role: "tool",
content: [...calls].map((call) =>
ToolResultPart.make({ id: call.id, name: call.name, result: MISSING_TOOL_RESULT, resultType: "error" }),
),
})
}
function normalizeToolMessage(message: Message, pending: Map<string, ToolCallPart>): Message | undefined {
const content = message.content.map((part) => {
if (part.type !== "tool-result" || part.providerExecuted === true) return part
const call = pending.get(part.id)
if (call) pending.delete(part.id)
return normalizeToolResult(part, call?.name ?? part.name)
})
if (content.length === 0) return undefined
if (content.every((part, index) => part === message.content[index])) return message
return new Message({
id: message.id,
role: message.role,
content,
metadata: message.metadata,
providerMetadata: message.providerMetadata,
native: message.native,
})
}
function normalizeToolResult(part: ToolResultPart, name: string): ToolResultPart {
const named = part.name === name ? part : { ...part, name }
if (named.result.type === "text" && named.result.value === "")
return { ...named, result: { type: "text", value: EMPTY_TOOL_OUTPUT } }
if (named.result.type === "error" && named.result.value === "")
return { ...named, result: { type: "error", value: EMPTY_TOOL_OUTPUT } }
if (named.result.type !== "content") return named
const value = named.result.value.filter((item) => item.type !== "text" || item.text !== "")
if (value.length === 0) return { ...named, result: { type: "text", value: EMPTY_TOOL_OUTPUT } }
if (value.length === named.result.value.length) return named
return { ...named, result: { type: "content", value } }
}
+7 -9
View File
@@ -1,4 +1,4 @@
import { describe, expect } from "bun:test"
import { describe, expect, test } from "bun:test"
import { Effect, Schema, Stream } from "effect"
import { LLM, LLMRequest, LLMResponse } from "../src/index.js"
import { Route, Endpoint, LLMClient, Protocol, type FramingDef } from "../src/route.js"
@@ -148,15 +148,13 @@ describe("llm route", () => {
}),
)
it.effect("builds models from configured routes", () =>
Effect.gen(function* () {
const configured = fake.with({ provider: "fake-provider", endpoint: { baseURL: "https://fake.local" } })
test("builds models from configured routes", () => {
const configured = fake.with({ provider: "fake-provider", endpoint: { baseURL: "https://fake.local" } })
expect(configured.model({ id: "fake-model" })).toMatchObject({
provider: "fake-provider",
})
}),
)
expect(configured.model({ id: "fake-model" })).toMatchObject({
provider: "fake-provider",
})
})
it.effect("does not register duplicate route ids globally", () =>
Effect.gen(function* () {
+32
View File
@@ -126,6 +126,38 @@ describe("applyCachePolicy", () => {
}),
)
for (const fixture of [
{ name: "default", cache: undefined, control: { type: "ephemeral" } },
{ name: "auto", cache: "auto", control: { type: "ephemeral" } },
{
name: "explicit one-hour",
cache: { tools: true, system: true, messages: { tail: 1 }, ttlSeconds: 3600 },
control: { type: "ephemeral", ttl: "1h" },
},
{ name: "disabled", cache: "none", control: undefined },
] as const) {
it.effect(`Bedrock Messages respects ${fixture.name} caching`, () =>
Effect.gen(function* () {
const prepared = yield* compileRequest(
LLM.request({
model: AmazonBedrock.configure({ apiKey: "test" }).messages("anthropic.claude-opus-4-6-v1"),
system: "Stable instructions",
tools: [
{ name: "lookup", description: "Look up a value", inputSchema: { type: "object", properties: {} } },
],
prompt: "hello",
cache: fixture.cache,
}),
)
expect(prepared.body).toMatchObject({
tools: [{ name: "lookup", cache_control: fixture.control }],
system: [{ type: "text", text: "Stable instructions", cache_control: fixture.control }],
messages: [{ role: "user", content: [{ type: "text", text: "hello", cache_control: fixture.control }] }],
})
}),
)
}
it.effect("'auto' is a no-op on Gemini (out-of-band caching protocol)", () =>
Effect.gen(function* () {
const prepared = yield* compileRequest(
+97
View File
@@ -0,0 +1,97 @@
import { expect } from "bun:test"
import { Effect } from "effect"
import { FetchHttpClient } from "effect/unstable/http"
import { LLM, LLMRequest, Message } from "../src/index.js"
import { LLMClient } from "../src/route/client.js"
import { OpenAI } from "../src/providers.js"
import { testEffect } from "./lib/effect.js"
import { runtimeLayer } from "./lib/http.js"
import { sseEvents } from "./lib/sse.js"
testEffect(runtimeLayer(FetchHttpClient.layer)).live("compaction and a tool loop work end to end over HTTP", () =>
Effect.gen(function* () {
const checkpoint = { type: "compaction", id: "cmp_local", encrypted_content: "opaque-local-state" }
const calls: string[] = []
const server = yield* Effect.acquireRelease(
Effect.sync(() =>
Bun.serve({
hostname: "127.0.0.1",
port: 0,
async fetch(request) {
const path = new URL(request.url).pathname
calls.push(path)
const body = await request.json()
expect(request.headers.get("authorization")).toBe("Bearer fixture")
if (path === "/v1/responses/compact") {
expect(body.stream).toBeUndefined()
return Response.json({
object: "response.compaction",
output: [checkpoint],
usage: { input_tokens: 100, output_tokens: 10, total_tokens: 110 },
})
}
expect(body.input[0]).toEqual(checkpoint)
expect(body.stream).toBe(true)
if (calls.length === 2)
return new Response(
sseEvents(
{
type: "response.output_item.done",
item: { type: "function_call", id: "fc_1", call_id: "call_1", name: "lookup", arguments: "{}" },
},
{ type: "response.completed", response: { id: "resp_1" } },
),
{ headers: { "content-type": "text/event-stream" } },
)
expect(body.input.at(-2)).toMatchObject({ type: "function_call", call_id: "call_1" })
expect(body.input.at(-1)).toEqual({ type: "function_call_output", call_id: "call_1", output: "42" })
const output = sseEvents(
{ type: "response.output_item.added", item: { type: "message", id: "msg_1" } },
{ type: "response.output_text.delta", item_id: "msg_1", delta: "The answer is 42." },
{
type: "response.output_item.done",
item: { type: "message", id: "msg_1", content: [{ type: "output_text", text: "The answer is 42." }] },
},
{ type: "response.completed", response: { id: "resp_2" } },
)
return new Response(
new ReadableStream({
start(controller) {
controller.enqueue(new TextEncoder().encode(output.slice(0, 37)))
controller.enqueue(new TextEncoder().encode(output.slice(37)))
controller.close()
},
}),
{ headers: { "content-type": "text/event-stream" } },
)
},
}),
),
(server) => Effect.sync(() => server.stop(true)),
)
const model = OpenAI.configure({ apiKey: "fixture", baseURL: `http://127.0.0.1:${server.port}/v1` }).responses(
"fixture",
)
const request = LLM.request({
model,
prompt: "original",
tools: [{ name: "lookup", description: "Lookup a number", inputSchema: { type: "object", properties: {} } }],
})
const compacted = yield* LLMClient.compact(request)
const messages = [...compacted.messages, Message.user("Look up the answer")]
const first = yield* LLMClient.generate(LLMRequest.update(request, { messages }))
expect(first.toolCalls).toHaveLength(1)
const call = first.toolCalls[0]!
const last = yield* LLMClient.generate(
LLMRequest.update(request, {
messages: [
...messages,
first.message,
Message.tool({ id: call.id, name: call.name, result: "42", resultType: "text" }),
],
}),
)
expect(last.text).toBe("The answer is 42.")
expect(calls).toEqual(["/v1/responses/compact", "/v1/responses", "/v1/responses"])
}),
)
+49
View File
@@ -0,0 +1,49 @@
import { expect, test } from "bun:test"
import { Schema } from "effect"
import { CompactionPart, LLMEvent, LLMResponse, Message, ProviderID } from "../src/schema/index.js"
test("compaction survives event assembly and message serialization without becoming text", () => {
const part = CompactionPart.make({
provider: ProviderID.make("openai"),
id: "cmp_1",
encrypted: "opaque",
})
const response = LLMResponse.fromEvents([
LLMEvent.textStart({ id: "before" }),
LLMEvent.textDelta({ id: "before", text: "Before" }),
LLMEvent.textEnd({ id: "before" }),
part,
LLMEvent.textStart({ id: "after" }),
LLMEvent.textDelta({ id: "after", text: "After" }),
LLMEvent.textEnd({ id: "after" }),
LLMEvent.finish({ reason: { normalized: "stop" } }),
])!
expect(response.message.content.map((part) => part.type)).toEqual(["text", "compaction", "text"])
expect(response.text).toBe("BeforeAfter")
expect(response.reasoning).toBe("")
expect(response.events.filter(LLMEvent.is.compaction)).toEqual([part])
const codec = Schema.fromJsonString(Message)
expect(Schema.decodeSync(codec)(Schema.encodeSync(codec)(response.message))).toEqual(response.message)
})
test("compaction requires exactly one typed representation", () => {
const provider = ProviderID.make("anthropic")
expect(CompactionPart.make({ provider, text: null })).toEqual({ type: "compaction", provider, text: null })
const decode = Schema.decodeUnknownSync(CompactionPart)
expect(() => decode({ type: "compaction", provider })).toThrow()
expect(() => decode({ type: "compaction", provider, text: "summary", encrypted: "opaque" })).toThrow()
})
test("tagged content and event guards accept both checkpoint representations", () => {
for (const part of [
CompactionPart.make({ provider: ProviderID.make("openai"), encrypted: "opaque" }),
CompactionPart.make({ provider: ProviderID.make("anthropic"), text: "summary" }),
CompactionPart.make({ provider: ProviderID.make("anthropic"), text: null }),
]) {
expect(LLMEvent.is.compaction(part)).toBe(true)
expect(LLMEvent.guards.compaction(part)).toBe(true)
const codec = Schema.fromJsonString(Message)
const message = Message.assistant(part)
expect(Schema.decodeSync(codec)(Schema.encodeSync(codec)(message))).toEqual(message)
}
})
+50 -1
View File
@@ -1,7 +1,7 @@
import { describe, expect, test } from "bun:test"
import { Effect, Ref, Schema } from "effect"
import { HttpClientRequest, HttpClientResponse } from "effect/unstable/http"
import { LLM, Message, ToolCallPart, mergeProviderOptions } from "../src/index.js"
import { LLM, LLMRequest, Message, ToolCallPart, ToolDefinition, mergeProviderOptions } from "../src/index.js"
import { AnthropicMessages, OpenAIChat } from "../src/protocols.js"
import { Auth, LLMClient } from "../src/route.js"
import { compileRequest } from "../src/route/client.js"
@@ -77,6 +77,55 @@ describe("request option precedence", () => {
}),
)
it.effect("keeps the last tool definition for duplicate names", () =>
Effect.gen(function* () {
const request = LLM.request({
model: OpenAIChat.route.model({ id: "gpt-4o-mini" }),
prompt: "Use a tool.",
})
const prepared = yield* compileRequest(
LLMRequest.update(request, {
tools: [
ToolDefinition.make({ name: "lookup", description: "old", inputSchema: { type: "object" } }),
ToolDefinition.make({ name: "search", description: "search", inputSchema: { type: "object" } }),
ToolDefinition.make({ name: "lookup", description: "new", inputSchema: { type: "object" } }),
],
}),
)
expect(prepared.body.tools).toEqual([
{
type: "function",
function: { name: "lookup", description: "new", parameters: { type: "object" }, strict: false },
},
{
type: "function",
function: { name: "search", description: "search", parameters: { type: "object" }, strict: false },
},
])
}),
)
it.effect("normalizes tool history before protocol lowering", () =>
Effect.gen(function* () {
const prepared = yield* compileRequest(
LLM.request({
model: OpenAIChat.route.model({ id: "gpt-4o-mini" }),
messages: [
Message.assistant(ToolCallPart.make({ id: "call_1", name: "lookup", input: {} })),
Message.user("Continue."),
],
}),
)
expect(prepared.body.messages).toMatchObject([
{ role: "assistant", tool_calls: [{ id: "call_1", function: { name: "lookup" } }] },
{ role: "tool", tool_call_id: "call_1", content: "Tool result missing" },
{ role: "user", content: "Continue." },
])
}),
)
it.effect("applies model HTTP defaults before request HTTP overlays", () =>
LLMClient.generate(
LLM.request({
+356 -206
View File
@@ -1,10 +1,10 @@
import { describe, expect } from "bun:test"
import { Deferred, Effect, Fiber, Layer, Ref, Stream } from "effect"
import { Headers, HttpClient, HttpClientError, HttpClientRequest, HttpClientResponse } from "effect/unstable/http"
import { LLM, AIError } from "../src/index.js"
import { Deferred, Effect, Fiber, Ref, Stream } from "effect"
import { Headers, HttpClientError, HttpClientRequest } from "effect/unstable/http"
import { LLM, AIError, HttpContext, InvalidProviderOutputError, TransportError } from "../src/index.js"
import { LLMClient, RequestExecutor, WebSocketTransport, type WebSocketChannelExecutor } from "../src/route.js"
import * as OpenAIChat from "../src/protocols/openai-chat.js"
import * as OpenAI from "../src/providers/openai.js"
import { route } from "../src/protocols/openai-chat.js"
import { configure } from "../src/providers/openai.js"
import { dynamicResponse, fixedResponse, systemError } from "./lib/http.js"
import { deltaChunk } from "./lib/openai-chunks.js"
import { sseEvents, sseRaw } from "./lib/sse.js"
@@ -18,66 +18,86 @@ const secretRequest = HttpClientRequest.post("https://provider.test/v1/chat?api_
HttpClientRequest.setHeaders(Headers.fromInput({ authorization: "Bearer header-secret-456" })),
)
const responsesLayer = (responses: ReadonlyArray<Response>) =>
RequestExecutor.layer.pipe(
Layer.provide(
Layer.unwrap(
Effect.gen(function* () {
const cursor = yield* Ref.make(0)
return Layer.succeed(
HttpClient.HttpClient,
HttpClient.make((request) =>
Effect.gen(function* () {
const index = yield* Ref.getAndUpdate(cursor, (value) => value + 1)
return HttpClientResponse.fromWeb(request, responses[index] ?? responses[responses.length - 1])
}),
),
)
}),
),
),
)
const countedResponsesLayer = (attempts: Ref.Ref<number>, responses: ReadonlyArray<Response>) =>
RequestExecutor.layer.pipe(
Layer.provide(
Layer.unwrap(
Effect.gen(function* () {
const cursor = yield* Ref.make(0)
return Layer.succeed(
HttpClient.HttpClient,
HttpClient.make((request) =>
Effect.gen(function* () {
yield* Ref.update(attempts, (value) => value + 1)
const index = yield* Ref.getAndUpdate(cursor, (value) => value + 1)
return HttpClientResponse.fromWeb(request, responses[index] ?? responses[responses.length - 1])
}),
),
)
}),
),
),
)
const expectAIError = (error: unknown) => {
expect(error).toBeInstanceOf(AIError)
if (!(error instanceof AIError)) throw new Error("expected AIError")
expect(error.reason).toBeInstanceOf(Error)
expect(error.cause).toBe(error.reason)
return error
}
const errorHttp = (error: AIError) => ("http" in error.reason ? error.reason.http : undefined)
const largeProviderMessage = `Upstream request failed: ${"validation failed; ".repeat(1_000)}`
describe("RequestExecutor", () => {
it.effect("preserves externally captured HTTP errors without inventing response context", () =>
Effect.sync(() => {
const cause = new Error("upstream request failed")
const body = '{"error":{"message":"Rate limited","trace":"original"}}'
const error = RequestExecutor.httpFailure({
message: "Rate limited",
url: request.url,
status: 429,
responseHeaders: { "Retry-After": "2", "X-Request-ID": "req_external" },
responseBody: body,
cause,
})
expect(error.message).toBe("Rate limited")
expect(error.reason).toMatchObject({ _tag: "RateLimit", retryAfterMs: 2000 })
expect(error.reason.body).toBe(body)
expect(error.reason.cause).toBe(cause)
expect(error.reason.http).toEqual(
new HttpContext({
url: request.url,
status: 429,
headers: { "retry-after": "2", "x-request-id": "req_external" },
}),
)
expect(RequestExecutor.httpFailure({ message: "No response", url: request.url }).reason.http).toBeUndefined()
expect(RequestExecutor.httpFailure({ message: "No URL", status: 500 }).reason.http).toBeUndefined()
}),
)
it.effect("retains the original body-read failure on an HTTP status error", () =>
Effect.gen(function* () {
const cause = new Error("response body disconnected")
const error = yield* Effect.gen(function* () {
const executor = yield* RequestExecutor.Service
return yield* executor.execute(request).pipe(Effect.flip)
}).pipe(
Effect.provide(
fixedResponse(
new ReadableStream({
start(controller) {
controller.error(cause)
},
}),
{
status: 503,
headers: { "x-request-id": "req_failed_body" },
},
),
),
)
expect(error.reason._tag).toBe("ProviderInternal")
expect(error.reason.cause).toBe(cause)
expect(error.reason.body).toBeUndefined()
expect(error.reason.http).toMatchObject({ status: 503, headers: { "x-request-id": "req_failed_body" } })
}),
)
it.effect("parses response body failures at the executor seam", () =>
Effect.gen(function* () {
const executor = yield* RequestExecutor.Service
const error = yield* RequestExecutor.stream(executor, secretRequest).pipe(Stream.runDrain, Effect.flip)
expectAIError(error)
expect(error.message).toBe("ECONNRESET: disconnected query-secret-123 header-secret-456")
expect(error.reason.http).toMatchObject({ status: 200, url: secretRequest.url })
expect(error.reason.cause).toMatchObject({ code: "ECONNRESET" })
expect(error.reason).toMatchObject({
_tag: "Transport",
message: "ECONNRESET: disconnected query-secret-123 header-secret-456",
transport: "http",
operation: "read",
code: "ECONNRESET",
@@ -85,15 +105,14 @@ describe("RequestExecutor", () => {
})
}).pipe(
Effect.provide(
responsesLayer([
new Response(
new ReadableStream({
start(controller) {
controller.error(systemError("ECONNRESET", "disconnected query-secret-123 header-secret-456"))
},
}),
),
]),
fixedResponse(
new ReadableStream({
start(controller) {
controller.error(systemError("ECONNRESET", "disconnected query-secret-123 header-secret-456"))
},
}),
{},
),
),
),
)
@@ -104,23 +123,23 @@ describe("RequestExecutor", () => {
const error = yield* RequestExecutor.stream(executor, secretRequest).pipe(Stream.runDrain, Effect.flip)
expectAIError(error)
expect(error.message).toBe("ECONNRESET: socket closed")
expect(error.reason.cause).toBeInstanceOf(TypeError)
expect(error.reason).toMatchObject({
_tag: "Transport",
message: "ECONNRESET: socket closed",
operation: "read",
code: "ECONNRESET",
})
}).pipe(
Effect.provide(
responsesLayer([
new Response(
new ReadableStream({
pull(controller) {
controller.error(new TypeError("fetch failed", { cause: systemError("ECONNRESET", "socket closed") }))
},
}),
),
]),
fixedResponse(
new ReadableStream({
pull(controller) {
controller.error(new TypeError("fetch failed", { cause: systemError("ECONNRESET", "socket closed") }))
},
}),
{},
),
),
),
)
@@ -133,8 +152,10 @@ describe("RequestExecutor", () => {
.pipe(Effect.flip)
expectAIError(error)
expect(error.reason.message).toBe("plugin rejected request")
}).pipe(Effect.provide(responsesLayer([]))),
expect(error.message).toBe("plugin rejected request")
expect(error.reason.cause).toBeInstanceOf(Error)
expect(error.reason.http).toBeUndefined()
}).pipe(Effect.provide(dynamicResponse(() => Effect.die(new Error("unexpected HTTP request"))))),
)
it.effect("reports the request sent by middleware", () =>
@@ -152,16 +173,11 @@ describe("RequestExecutor", () => {
.pipe(Effect.flip)
expectAIError(error)
expect(error.message).toBe("ECONNRESET: proxy disconnected proxy-secret")
expect(error.reason.http).toBeUndefined()
expect(error.reason).toMatchObject({
_tag: "Transport",
message: "ECONNRESET: proxy disconnected proxy-secret",
url: "https://proxy.test/v1/chat?api_key=proxy-secret",
http: {
request: {
url: "https://proxy.test/v1/chat?api_key=proxy-secret",
headers: { authorization: "Bearer proxy-secret" },
},
},
})
}).pipe(
Effect.provide(
@@ -188,11 +204,9 @@ describe("RequestExecutor", () => {
expect(error.reason).toMatchObject({ _tag: "InvalidRequest", classification: "context-overflow" })
}).pipe(
Effect.provide(
responsesLayer([
new Response('{"error":{"code":"context_length_exceeded","message":"prompt too long"}}', {
status: 400,
}),
]),
fixedResponse('{"error":{"code":"context_length_exceeded","message":"prompt too long"}}', {
status: 400,
}),
),
),
)
@@ -206,9 +220,9 @@ describe("RequestExecutor", () => {
expect(error.reason).toMatchObject({
_tag: "InvalidRequest",
classification: "payload-too-large",
http: { response: { status: 413 } },
})
}).pipe(Effect.provide(responsesLayer([new Response("request too large", { status: 413 })]))),
expect(error.reason.http?.status).toBe(413)
}).pipe(Effect.provide(fixedResponse("request too large", { status: 413 }))),
)
it.effect("classifies Anthropic request_too_large as context overflow", () =>
@@ -220,15 +234,13 @@ describe("RequestExecutor", () => {
expect(error.reason).toMatchObject({
_tag: "InvalidRequest",
classification: "context-overflow",
http: { response: { status: 413 } },
})
expect(error.reason.http?.status).toBe(413)
}).pipe(
Effect.provide(
responsesLayer([
new Response('{"error":{"type":"request_too_large","message":"Request exceeds the maximum size"}}', {
status: 413,
}),
]),
fixedResponse('{"error":{"type":"request_too_large","message":"Request exceeds the maximum size"}}', {
status: 413,
}),
),
),
)
@@ -241,8 +253,8 @@ describe("RequestExecutor", () => {
expectAIError(error)
expect(error.reason).toMatchObject({ _tag: "InvalidRequest" })
expect("classification" in error.reason ? error.reason.classification : undefined).toBeUndefined()
expect(error.reason.message).toBe("Provider request failed with HTTP 400")
}).pipe(Effect.provide(responsesLayer([new Response("invalid parameter", { status: 400 })]))),
expect(error.message).toBe("Provider request failed with HTTP 400")
}).pipe(Effect.provide(fixedResponse("invalid parameter", { status: 400 }))),
)
it.effect("preserves structured provider messages from large error bodies", () =>
@@ -251,20 +263,18 @@ describe("RequestExecutor", () => {
const error = yield* executor.execute(request).pipe(Effect.flip)
expectAIError(error)
expect(error.reason).toMatchObject({ _tag: "InvalidRequest", message: largeProviderMessage })
expect(errorHttp(error)?.body).toContain(largeProviderMessage)
expect(errorHttp(error)?.bodyTruncated).toBeUndefined()
expect(error.reason).toMatchObject({ _tag: "InvalidRequest" })
expect(error.message).toBe(largeProviderMessage)
expect(error.reason.body).toContain(largeProviderMessage)
}).pipe(
Effect.provide(
responsesLayer([
new Response(
JSON.stringify({
model: "gpt-5.6-sol",
error: { type: "invalid_request", message: largeProviderMessage },
}),
{ status: 400 },
),
]),
fixedResponse(
JSON.stringify({
model: "test-model",
error: { type: "invalid_request", message: largeProviderMessage },
}),
{ status: 400 },
),
),
),
)
@@ -277,9 +287,9 @@ describe("RequestExecutor", () => {
expectAIError(error)
expect(error.reason).toMatchObject({
_tag: "InvalidRequest",
message: "Provider request failed with HTTP 400",
})
}).pipe(Effect.provide(responsesLayer([new Response('{"error":{"message":" "}}', { status: 400 })]))),
expect(error.message).toBe("Provider request failed with HTTP 400")
}).pipe(Effect.provide(fixedResponse('{"error":{"message":" "}}', { status: 400 }))),
)
it.effect("classifies provider rate limits hidden behind HTTP 400", () =>
@@ -291,7 +301,7 @@ describe("RequestExecutor", () => {
expectAIError(error)
expect(error.reason).toMatchObject({ _tag: "RateLimit" })
}).pipe(Effect.provide(responsesLayer([new Response(body, { status: 400 })])))
}).pipe(Effect.provide(fixedResponse(body, { status: 400 })))
yield* classify("Request rate increased too quickly")
yield* classify('{"type":"error","error":{"type":"too_many_requests"}}')
@@ -308,7 +318,7 @@ describe("RequestExecutor", () => {
expectAIError(error)
expect(error.reason).toMatchObject({ _tag: "ProviderInternal" })
}).pipe(Effect.provide(responsesLayer([new Response(body, { status: 400 })])))
}).pipe(Effect.provide(fixedResponse(body, { status: 400 })))
yield* classify('{"code":"resource_exhausted"}')
yield* classify('{"code":"service_unavailable"}')
@@ -321,37 +331,27 @@ describe("RequestExecutor", () => {
const error = yield* executor.execute(request).pipe(Effect.flip)
expectAIError(error)
expect(error).toMatchObject({
reason: {
_tag: "RateLimit",
retryAfterMs: 0,
rateLimit: { retryAfterMs: 0 },
http: {
request: {
method: "POST",
url: "https://provider.test/v1/chat?api_key=secret&key=secret&debug=1",
headers: { authorization: "Bearer secret", "x-safe": "visible" },
},
response: {
status: 429,
headers: {
"retry-after-ms": "0",
"x-request-id": "req_123",
"x-api-key": "secret",
},
},
expect(error.reason).toMatchObject({
_tag: "RateLimit",
retryAfterMs: 0,
rateLimit: { retryAfterMs: 0 },
http: {
url: "https://provider.test/v1/chat?api_key=secret&key=secret&debug=1",
status: 429,
headers: {
"retry-after-ms": "0",
"x-request-id": "req_123",
"x-api-key": "secret",
},
},
})
expect(errorHttp(error)?.body).toBe("rate limited")
expect(error.reason.body).toBe("rate limited")
}).pipe(
Effect.provide(
responsesLayer([
new Response("rate limited", {
status: 429,
headers: { "retry-after-ms": "0", "x-request-id": "req_123", "x-api-key": "secret" },
}),
]),
fixedResponse("rate limited", {
status: 429,
headers: { "retry-after-ms": "0", "x-request-id": "req_123", "x-api-key": "secret" },
}),
),
),
)
@@ -362,10 +362,9 @@ describe("RequestExecutor", () => {
const error = yield* executor.execute(request).pipe(Effect.flip)
expectAIError(error)
expect(errorHttp(error)?.request.headers["x-safe"]).toBe("visible")
expect(errorHttp(error)?.response?.headers["x-safe"]).toBe("response-secret")
expect(error.reason.http?.headers["x-safe"]).toBe("response-secret")
}).pipe(
Effect.provide(responsesLayer([new Response("bad", { status: 400, headers: { "x-safe": "response-secret" } })])),
Effect.provide(fixedResponse("bad", { status: 400, headers: { "x-safe": "response-secret" } })),
Effect.provideService(Headers.CurrentRedactedNames, ["x-safe"]),
),
)
@@ -385,20 +384,18 @@ describe("RequestExecutor", () => {
})
}).pipe(
Effect.provide(
responsesLayer([
new Response("rate limited", {
status: 429,
headers: {
"retry-after-ms": "0",
"x-ratelimit-limit-requests": "500",
"x-ratelimit-limit-tokens": "30000",
"x-ratelimit-remaining-requests": "499",
"x-ratelimit-remaining-tokens": "29900",
"x-ratelimit-reset-requests": "1s",
"x-ratelimit-reset-tokens": "10s",
},
}),
]),
fixedResponse("rate limited", {
status: 429,
headers: {
"retry-after-ms": "0",
"x-ratelimit-limit-requests": "500",
"x-ratelimit-limit-tokens": "30000",
"x-ratelimit-remaining-requests": "499",
"x-ratelimit-remaining-tokens": "29900",
"x-ratelimit-reset-requests": "1s",
"x-ratelimit-reset-tokens": "10s",
},
}),
),
),
)
@@ -409,8 +406,8 @@ describe("RequestExecutor", () => {
const error = yield* executor.execute(request).pipe(Effect.flip)
expectAIError(error)
expect(error.reason).toMatchObject({ _tag: "ProviderInternal" })
expect(errorHttp(error)?.rateLimit).toEqual({
expect(error.reason).toMatchObject({ _tag: "RateLimit" })
expect(error.reason._tag === "RateLimit" ? error.reason.rateLimit : undefined).toEqual({
retryAfterMs: 0,
limit: { requests: "100", "input-tokens": "10000" },
remaining: { requests: "12", "input-tokens": "9000" },
@@ -418,20 +415,18 @@ describe("RequestExecutor", () => {
})
}).pipe(
Effect.provide(
responsesLayer([
new Response("overloaded", {
status: 529,
headers: {
"retry-after-ms": "0",
"anthropic-ratelimit-requests-limit": "100",
"anthropic-ratelimit-requests-remaining": "12",
"anthropic-ratelimit-requests-reset": "2026-05-06T12:00:00Z",
"anthropic-ratelimit-input-tokens-limit": "10000",
"anthropic-ratelimit-input-tokens-remaining": "9000",
"anthropic-ratelimit-input-tokens-reset": "2026-05-06T12:00:10Z",
},
}),
]),
fixedResponse("rate limited", {
status: 429,
headers: {
"retry-after-ms": "0",
"anthropic-ratelimit-requests-limit": "100",
"anthropic-ratelimit-requests-remaining": "12",
"anthropic-ratelimit-requests-reset": "2026-05-06T12:00:00Z",
"anthropic-ratelimit-input-tokens-limit": "10000",
"anthropic-ratelimit-input-tokens-remaining": "9000",
"anthropic-ratelimit-input-tokens-reset": "2026-05-06T12:00:10Z",
},
}),
),
),
)
@@ -444,15 +439,20 @@ describe("RequestExecutor", () => {
return yield* executor.execute(request).pipe(Effect.flip)
}).pipe(
Effect.provide(
countedResponsesLayer(attempts, [
new Response("busy", { status: 503, headers: { "retry-after-ms": "0" } }),
new Response("ok", { status: 200 }),
]),
dynamicResponse((input) =>
Effect.gen(function* () {
const attempt = yield* Ref.getAndUpdate(attempts, (value) => value + 1)
return attempt === 0
? input.respond("busy", { status: 503, headers: { "retry-after-ms": "0" } })
: input.respond("ok", { status: 200 })
}),
),
),
)
expectAIError(error)
expect(error.reason).toMatchObject({ _tag: "ProviderInternal", status: 503 })
expect(error.reason).toMatchObject({ _tag: "ProviderInternal" })
expect(error.reason.http?.status).toBe(503)
expect(yield* Ref.get(attempts)).toBe(1)
}),
)
@@ -465,15 +465,14 @@ describe("RequestExecutor", () => {
const error = yield* executor.execute(request).pipe(Effect.flip)
expectAIError(error)
expect(error.reason).toMatchObject({ _tag: "ProviderInternal", status })
expect(error.reason).toMatchObject({ _tag: "ProviderInternal" })
expect(error.reason.http?.status).toBe(status)
}).pipe(
Effect.provide(
responsesLayer([
new Response("provider failure", {
status,
headers: { "retry-after-ms": "0" },
}),
]),
fixedResponse("provider failure", {
status,
headers: { "retry-after-ms": "0" },
}),
),
)
@@ -484,21 +483,28 @@ describe("RequestExecutor", () => {
it.effect("preserves large authentication error bodies", () =>
Effect.gen(function* () {
const executor = yield* RequestExecutor.Service
const error = yield* executor.execute(request).pipe(Effect.flip)
const attempts = yield* Ref.make(0)
const error = yield* Effect.gen(function* () {
const executor = yield* RequestExecutor.Service
return yield* executor.execute(request).pipe(Effect.flip)
}).pipe(
Effect.provide(
dynamicResponse((input) =>
Effect.gen(function* () {
const attempt = yield* Ref.getAndUpdate(attempts, (value) => value + 1)
return attempt === 0
? input.respond("x".repeat(20_000), { status: 401 })
: input.respond("should not retry", { status: 200 })
}),
),
),
)
expectAIError(error)
expect(error.reason).toMatchObject({ _tag: "Authentication" })
expect(errorHttp(error)?.bodyTruncated).toBeUndefined()
expect(errorHttp(error)?.body).toHaveLength(20_000)
}).pipe(
Effect.provide(
responsesLayer([
new Response("x".repeat(20_000), { status: 401 }),
new Response("should not retry", { status: 200 }),
]),
),
),
expect(error.reason.body).toHaveLength(20_000)
expect(yield* Ref.get(attempts)).toBe(1)
}),
)
it.effect("preserves response body fields", () =>
@@ -507,16 +513,12 @@ describe("RequestExecutor", () => {
const error = yield* executor.execute(request).pipe(Effect.flip)
expectAIError(error)
expect(errorHttp(error)?.body).toBe(
'{"error":{"message":"bad","key":"body-secret","detail":"api_key=query-secret"}}',
)
expect(error.reason.body).toBe('{"error":{"message":"bad","key":"body-secret","detail":"api_key=query-secret"}}')
}).pipe(
Effect.provide(
responsesLayer([
new Response('{"error":{"message":"bad","key":"body-secret","detail":"api_key=query-secret"}}', {
status: 400,
}),
]),
fixedResponse('{"error":{"message":"bad","key":"body-secret","detail":"api_key=query-secret"}}', {
status: 400,
}),
),
),
)
@@ -527,12 +529,10 @@ describe("RequestExecutor", () => {
const error = yield* executor.execute(secretRequest).pipe(Effect.flip)
expectAIError(error)
expect(errorHttp(error)?.body).toBe("provider echoed query-secret-123 and authorization header-secret-456")
expect(error.reason.body).toBe("provider echoed query-secret-123 and authorization header-secret-456")
}).pipe(
Effect.provide(
responsesLayer([
new Response("provider echoed query-secret-123 and authorization header-secret-456", { status: 400 }),
]),
fixedResponse("provider echoed query-secret-123 and authorization header-secret-456", { status: 400 }),
),
),
)
@@ -540,9 +540,7 @@ describe("RequestExecutor", () => {
it.effect("does not re-execute after a successful response reaches stream parsing", () =>
Effect.gen(function* () {
const attempts = yield* Ref.make(0)
const model = OpenAIChat.route
.with({ endpoint: { baseURL: "https://api.openai.test/v1" } })
.model({ id: "gpt-4o-mini" })
const model = route.with({ endpoint: { baseURL: "https://api.openai.test/v1" } }).model({ id: "gpt-4o-mini" })
const error = yield* LLMClient.generate(LLM.request({ model, prompt: "Say hello." })).pipe(
Effect.provide(
dynamicResponse((input) =>
@@ -564,19 +562,171 @@ describe("RequestExecutor", () => {
expectAIError(error)
expect(error.reason).toMatchObject({ _tag: "InvalidProviderOutput" })
expect(error.reason.body).toBe("not-json")
expect(error.reason.cause).toBeDefined()
expect(error.reason.http).toMatchObject({ status: 200, headers: { "content-type": "text/event-stream" } })
expect(yield* Ref.get(attempts)).toBe(1)
}),
)
})
describe("WebSocket channel execution", () => {
const model = OpenAI.configure({ baseURL: "https://api.openai.test/v1/", apiKey: "test" }).responses("gpt-4.1-mini")
const model = configure({ baseURL: "https://api.openai.test/v1/", apiKey: "test" }).responses("gpt-4.1-mini")
const request = LLM.request({ model, prompt: "Say hello." })
const frames = [
JSON.stringify({ type: "response.output_item.added", item: { type: "message", id: "msg_1" } }),
JSON.stringify({ type: "response.output_text.delta", item_id: "msg_1", delta: "Hi" }),
JSON.stringify({ type: "response.completed", response: { id: "resp_1" } }),
]
it.effect("preserves close reasons and native event causes without fabricated HTTP metadata", () =>
Effect.gen(function* () {
class TestSocket extends EventTarget {
readyState = globalThis.WebSocket.OPEN
send() {}
close() {}
}
const socket = new TestSocket()
const connection = yield* WebSocketTransport.fromWebSocket(
// oxlint-disable-next-line typescript-eslint/no-unsafe-type-assertion
socket as unknown as globalThis.WebSocket,
{ url: "wss://provider.test/responses", headers: Headers.empty },
)
const event = new CloseEvent("close", { code: 1011, reason: "upstream trace: req_close" })
socket.dispatchEvent(event)
const error = yield* connection.messages.pipe(Stream.runDrain, Effect.flip)
expect(error.reason).toMatchObject({ _tag: "Transport", code: "1011", phase: "close" })
expect(error.message).toBe("WebSocket closed with code 1011")
expect(error.reason.body).toBe(event.reason)
expect(error.reason.cause).toBe(event)
expect(error.reason.http).toBeUndefined()
yield* connection.close
}),
)
it.effect("preserves opening event errors and native send exceptions", () =>
Effect.gen(function* () {
const cause = new Error("native send failed")
class TestSocket extends EventTarget {
readyState = globalThis.WebSocket.CONNECTING
send() {
throw cause
}
close() {}
}
const socket = new TestSocket()
const open = WebSocketTransport.fromWebSocket(
// oxlint-disable-next-line typescript-eslint/no-unsafe-type-assertion
socket as unknown as globalThis.WebSocket,
{ url: "wss://provider.test/responses", headers: Headers.empty },
)
const fiber = yield* open.pipe(Effect.flip, Effect.forkChild({ startImmediately: true }))
const event = new ErrorEvent("error", { message: "handshake rejected", error: cause })
socket.dispatchEvent(event)
const error = yield* Fiber.join(fiber)
expect(error.reason.cause).toBe(cause)
expect(error.message).toContain("handshake rejected")
expect(error.reason.http).toBeUndefined()
socket.readyState = globalThis.WebSocket.OPEN
const connection = yield* open
const sent = yield* connection.sendText("create").pipe(Effect.flip)
expect(sent.reason.cause).toBe(cause)
expect(sent.message).toBe(cause.message)
yield* connection.close
}),
)
it.effect("preserves raw driver failures and known upgrade metadata", () =>
Effect.gen(function* () {
const cause = new Error("driver validation failed")
const frame = '{ "error": "failed", "trace": "original" }'
const http = new HttpContext({
url: "https://provider.test/responses",
status: 101,
headers: { upgrade: "websocket" },
})
const executor = WebSocketTransport.makeDirect({
open: () =>
Effect.succeed({
http,
sendText: () => Effect.void,
messages: Stream.make(frame),
close: Effect.void,
}),
})
const execution = yield* executor.execute({
id: "exchange_error",
connect: { url: "wss://provider.test/responses", headers: Headers.empty },
fallback: () => Stream.empty,
driver: {
create: () => Effect.succeed({ message: "create", mode: "full" }),
observe: () =>
Effect.succeed({
type: "provider-failure",
error: new AIError({
reason: new InvalidProviderOutputError({
message: "Driver failed",
cause,
body: "narrowed",
}),
}),
}),
},
})
const error = yield* execution.frames.pipe(Stream.runDrain, Effect.flip)
expect(error.message).toBe("Driver failed")
expect(error.reason.body).toBe(frame)
expect(error.reason.cause).toBe(cause)
expect(error.reason.http).toBe(http)
expect(execution.http).toBe(http)
}),
)
it.effect("retains diagnostic fields when annotating transport delivery", () =>
Effect.gen(function* () {
const cause = new Error("connection closed")
const executor = WebSocketTransport.makeDirect({
open: () =>
Effect.succeed({
sendText: () => Effect.void,
messages: Stream.fail(
new AIError({
reason: new TransportError({
message: "Socket closed",
transport: "websocket",
operation: "read",
phase: "close",
recovery: "retry-full",
body: "server close detail",
cause,
}),
}),
),
close: Effect.void,
}),
})
const execution = yield* executor.execute({
id: "exchange_closed",
connect: { url: "wss://provider.test/responses", headers: Headers.empty },
fallback: () => Stream.empty,
driver: {
create: () => Effect.succeed({ message: "create", mode: "full" }),
observe: (_create, frame) => Effect.succeed({ type: "frame", frame }),
},
})
const error = yield* execution.frames.pipe(Stream.runDrain, Effect.flip)
expect(error.message).toBe("Socket closed")
expect(error.reason.body).toBe("server close detail")
expect(error.reason.cause).toBe(cause)
expect(error.reason).toMatchObject({ phase: "close", delivery: "ambiguous", recovery: "retry-full" })
expect(error.reason.http).toBeUndefined()
}),
)
it.effect("runs a channel driver through the direct executor", () =>
Effect.gen(function* () {
const sent = yield* Ref.make("")
+2
View File
@@ -32,6 +32,8 @@ describe("public exports", () => {
expect(Provider.make).toBeFunction()
expect(ProviderSubpath.make).toBe(Provider.make)
expect(TestLLM.layer).toBeFunction()
expect(TestLLM.testLayer).toBeFunction()
expect(TestLLM.Test.of).toBeFunction()
})
test("route barrel exposes route-authoring APIs", () => {
@@ -10,7 +10,7 @@
"transport": "http",
"request": {
"method": "POST",
"url": "https://api.anthropic.com/v1/messages",
"url": "https://api.anthropic.com/v1/messages?beta=true",
"headers": {
"anthropic-version": "2023-06-01",
"content-type": "application/json"
@@ -29,7 +29,7 @@
"transport": "http",
"request": {
"method": "POST",
"url": "https://api.anthropic.com/v1/messages",
"url": "https://api.anthropic.com/v1/messages?beta=true",
"headers": {
"anthropic-version": "2023-06-01",
"content-type": "application/json"
@@ -10,7 +10,7 @@
"transport": "http",
"request": {
"method": "POST",
"url": "https://api.anthropic.com/v1/messages",
"url": "https://api.anthropic.com/v1/messages?beta=true",
"headers": {
"anthropic-version": "2023-06-01",
"content-type": "application/json"
@@ -29,7 +29,7 @@
"transport": "http",
"request": {
"method": "POST",
"url": "https://api.anthropic.com/v1/messages",
"url": "https://api.anthropic.com/v1/messages?beta=true",
"headers": {
"anthropic-version": "2023-06-01",
"content-type": "application/json"
@@ -24,7 +24,7 @@
"transport": "http",
"request": {
"method": "POST",
"url": "https://api.anthropic.com/v1/messages",
"url": "https://api.anthropic.com/v1/messages?beta=true",
"headers": {
"anthropic-version": "2023-06-01",
"content-type": "application/json"
@@ -0,0 +1,36 @@
{
"version": 1,
"metadata": {
"model": "zai-glm-5-2",
"tags": [
"prefix:mistral-chat-glm",
"provider:mistral",
"protocol:mistral-chat",
"hosted-model",
"tool",
"tool-call"
],
"name": "mistral-chat-glm/streams-an-indexed-tool-call",
"recordedAt": "2026-08-30T17:38:02.921Z"
},
"interactions": [
{
"transport": "http",
"request": {
"method": "POST",
"url": "https://api.mistral.ai/v1/chat/completions",
"headers": {
"content-type": "application/json"
},
"body": "{\"model\":\"zai-glm-5-2\",\"messages\":[{\"role\":\"system\",\"content\":\"Call lookup_weather exactly once with Paris.\"},{\"role\":\"user\",\"content\":\"What is the weather?\"}],\"tools\":[{\"type\":\"function\",\"function\":{\"name\":\"lookup_weather\",\"description\":\"Look up the current weather for a city\",\"parameters\":{\"type\":\"object\",\"properties\":{\"city\":{\"type\":\"string\",\"enum\":[\"Paris\"]}},\"required\":[\"city\"],\"additionalProperties\":false},\"strict\":false}}],\"tool_choice\":{\"type\":\"function\",\"function\":{\"name\":\"lookup_weather\"}},\"stream\":true,\"max_tokens\":256,\"temperature\":0}"
},
"response": {
"status": 200,
"headers": {
"content-type": "text/event-stream; charset=utf-8"
},
"body": "data: {\"id\":\"f139bf0e4b984e51aabf6a83c237674d\",\"object\":\"chat.completion.chunk\",\"created\":1788111482,\"model\":\"zai-glm-5-2\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"f139bf0e4b984e51aabf6a83c237674d\",\"object\":\"chat.completion.chunk\",\"created\":1788111482,\"model\":\"zai-glm-5-2\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"index\":0,\"content\":\"\"},\"finish_reason\":null,\"logprobs\":null}]}\n\ndata: {\"id\":\"f139bf0e4b984e51aabf6a83c237674d\",\"object\":\"chat.completion.chunk\",\"created\":1788111482,\"model\":\"zai-glm-5-2\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"id\":\"chatcmpl-tool-8cc4d8f9f07b298a\",\"type\":\"function\",\"function\":{\"name\":\"lookup_weather\",\"arguments\":\"{\\\"city\\\": \\\"\"},\"index\":0}],\"index\":0,\"content\":\"\"},\"finish_reason\":null,\"logprobs\":null}]}\n\ndata: {\"id\":\"f139bf0e4b984e51aabf6a83c237674d\",\"object\":\"chat.completion.chunk\",\"created\":1788111482,\"model\":\"zai-glm-5-2\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"type\":\"function\",\"function\":{\"name\":\"\",\"arguments\":\"Paris\\\"}\"},\"index\":0}],\"index\":0,\"content\":\"\"},\"finish_reason\":null,\"logprobs\":null}]}\n\ndata: {\"id\":\"f139bf0e4b984e51aabf6a83c237674d\",\"object\":\"chat.completion.chunk\",\"created\":1788111482,\"model\":\"zai-glm-5-2\",\"choices\":[{\"index\":0,\"delta\":{\"index\":0,\"content\":\"\"},\"finish_reason\":\"stop\",\"logprobs\":null}],\"usage\":{\"prompt_tokens\":171,\"total_tokens\":182,\"completion_tokens\":11,\"prompt_tokens_details\":{\"cached_tokens\":0}}}\n\ndata: [DONE]\n\n"
}
}
]
}
@@ -0,0 +1,47 @@
{
"version": 1,
"metadata": {
"model": "mistral-small-latest",
"tags": ["prefix:mistral-chat", "provider:mistral", "protocol:mistral-chat", "tool", "tool-loop", "usage"],
"name": "mistral-chat/drives-a-tool-loop",
"recordedAt": "2026-08-30T17:18:49.552Z"
},
"interactions": [
{
"transport": "http",
"request": {
"method": "POST",
"url": "https://api.mistral.ai/v1/chat/completions",
"headers": {
"content-type": "application/json"
},
"body": "{\"model\":\"mistral-small-latest\",\"messages\":[{\"role\":\"system\",\"content\":\"Call lookup_weather exactly once with Paris.\"},{\"role\":\"user\",\"content\":\"What is the weather?\"}],\"tools\":[{\"type\":\"function\",\"function\":{\"name\":\"lookup_weather\",\"description\":\"Look up the current weather for a city\",\"parameters\":{\"type\":\"object\",\"properties\":{\"city\":{\"type\":\"string\",\"enum\":[\"Paris\"]}},\"required\":[\"city\"],\"additionalProperties\":false},\"strict\":false}}],\"tool_choice\":{\"type\":\"function\",\"function\":{\"name\":\"lookup_weather\"}},\"stream\":true,\"max_tokens\":160,\"temperature\":0,\"reasoning_effort\":\"none\"}"
},
"response": {
"status": 200,
"headers": {
"content-type": "text/event-stream; charset=utf-8"
},
"body": "data: {\"id\":\"07491e37a5ed48f9987f1583753a466b\",\"object\":\"chat.completion.chunk\",\"created\":1788110328,\"model\":\"mistral-small-latest\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"07491e37a5ed48f9987f1583753a466b\",\"object\":\"chat.completion.chunk\",\"created\":1788110328,\"model\":\"mistral-small-latest\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"id\":\"ffJovBNqY\",\"type\":\"function\",\"function\":{\"name\":\"lookup_weather\",\"arguments\":\"{\\\"city\\\": \\\"Paris\\\"}\"},\"index\":0}]},\"finish_reason\":\"tool_calls\"}],\"usage\":{\"prompt_tokens\":110,\"total_tokens\":122,\"completion_tokens\":12,\"prompt_tokens_details\":{\"cached_tokens\":0},\"service_tier\":\"standard\"},\"p\":\"abcdefghijklm\"}\n\ndata: [DONE]\n\n"
}
},
{
"transport": "http",
"request": {
"method": "POST",
"url": "https://api.mistral.ai/v1/chat/completions",
"headers": {
"content-type": "application/json"
},
"body": "{\"model\":\"mistral-small-latest\",\"messages\":[{\"role\":\"system\",\"content\":\"Call lookup_weather exactly once with Paris.\"},{\"role\":\"user\",\"content\":\"What is the weather?\"},{\"role\":\"assistant\",\"content\":\"\",\"tool_calls\":[{\"id\":\"ffJovBNqY\",\"type\":\"function\",\"function\":{\"name\":\"lookup_weather\",\"arguments\":\"{\\\"city\\\":\\\"Paris\\\"}\"}}]},{\"role\":\"tool\",\"tool_call_id\":\"ffJovBNqY\",\"name\":\"lookup_weather\",\"content\":\"{\\\"condition\\\":\\\"sunny\\\",\\\"temperature\\\":\\\"18C\\\"}\"}],\"tools\":[{\"type\":\"function\",\"function\":{\"name\":\"lookup_weather\",\"description\":\"Look up the current weather for a city\",\"parameters\":{\"type\":\"object\",\"properties\":{\"city\":{\"type\":\"string\",\"enum\":[\"Paris\"]}},\"required\":[\"city\"],\"additionalProperties\":false},\"strict\":false}}],\"tool_choice\":\"none\",\"stream\":true,\"max_tokens\":160,\"temperature\":0,\"reasoning_effort\":\"none\"}"
},
"response": {
"status": 200,
"headers": {
"content-type": "text/event-stream; charset=utf-8"
},
"body": "data: {\"id\":\"8fcd293093b849139fc0893a48bbc7ce\",\"object\":\"chat.completion.chunk\",\"created\":1788110328,\"model\":\"mistral-small-latest\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"8fcd293093b849139fc0893a48bbc7ce\",\"object\":\"chat.completion.chunk\",\"created\":1788110328,\"model\":\"mistral-small-latest\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"The\"},\"finish_reason\":null}],\"p\":\"abcdefghijklmn\"}\n\ndata: {\"id\":\"8fcd293093b849139fc0893a48bbc7ce\",\"object\":\"chat.completion.chunk\",\"created\":1788110328,\"model\":\"mistral-small-latest\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" weather in Paris is\"},\"finish_reason\":null}],\"p\":\"abcdefghijklmn\"}\n\ndata: {\"id\":\"8fcd293093b849139fc0893a48bbc7ce\",\"object\":\"chat.completion.chunk\",\"created\":1788110328,\"model\":\"mistral-small-latest\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" currently sunny with\"},\"finish_reason\":null}],\"p\":\"abcdefghijklmnopqrstu\"}\n\ndata: {\"id\":\"8fcd293093b849139fc0893a48bbc7ce\",\"object\":\"chat.completion.chunk\",\"created\":1788110328,\"model\":\"mistral-small-latest\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" a temperature of \"},\"finish_reason\":null}],\"p\":\"abcdef\"}\n\ndata: {\"id\":\"8fcd293093b849139fc0893a48bbc7ce\",\"object\":\"chat.completion.chunk\",\"created\":1788110328,\"model\":\"mistral-small-latest\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"18°C\"},\"finish_reason\":null}],\"p\":\"abcdefghijklmnopqr\"}\n\ndata: {\"id\":\"8fcd293093b849139fc0893a48bbc7ce\",\"object\":\"chat.completion.chunk\",\"created\":1788110328,\"model\":\"mistral-small-latest\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\".\"},\"finish_reason\":\"stop\"}],\"usage\":{\"prompt_tokens\":57,\"total_tokens\":74,\"completion_tokens\":17,\"prompt_tokens_details\":{\"cached_tokens\":0},\"service_tier\":\"standard\"},\"p\":\"abcdefghijklmnopqrstuvwxyz\"}\n\ndata: [DONE]\n\n"
}
}
]
}
File diff suppressed because one or more lines are too long
@@ -0,0 +1,29 @@
{
"version": 1,
"metadata": {
"model": "mistral-small-latest",
"tags": ["prefix:mistral-chat", "provider:mistral", "protocol:mistral-chat", "text", "usage"],
"name": "mistral-chat/streams-text-with-usage",
"recordedAt": "2026-08-30T17:18:45.432Z"
},
"interactions": [
{
"transport": "http",
"request": {
"method": "POST",
"url": "https://api.mistral.ai/v1/chat/completions",
"headers": {
"content-type": "application/json"
},
"body": "{\"model\":\"mistral-small-latest\",\"messages\":[{\"role\":\"user\",\"content\":\"Reply with exactly one word: hello\"}],\"stream\":true,\"max_tokens\":40,\"temperature\":0,\"reasoning_effort\":\"none\"}"
},
"response": {
"status": 200,
"headers": {
"content-type": "text/event-stream; charset=utf-8"
},
"body": "data: {\"id\":\"9a4d16bdddb74e5e89c2cf9e9b91e065\",\"object\":\"chat.completion.chunk\",\"created\":1788110325,\"model\":\"mistral-small-latest\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"9a4d16bdddb74e5e89c2cf9e9b91e065\",\"object\":\"chat.completion.chunk\",\"created\":1788110325,\"model\":\"mistral-small-latest\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Hi\"},\"finish_reason\":null}],\"p\":\"abcdefghijklmnopqrs\"}\n\ndata: {\"id\":\"9a4d16bdddb74e5e89c2cf9e9b91e065\",\"object\":\"chat.completion.chunk\",\"created\":1788110325,\"model\":\"mistral-small-latest\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"\"},\"finish_reason\":\"stop\"}],\"usage\":{\"prompt_tokens\":22,\"total_tokens\":24,\"completion_tokens\":2,\"prompt_tokens_details\":{\"cached_tokens\":0},\"service_tier\":\"standard\"},\"p\":\"abcdefghijklmnopqrstuvwxyz0\"}\n\ndata: [DONE]\n\n"
}
}
]
}
@@ -23,7 +23,7 @@
"headers": {
"content-type": "application/json"
},
"body": "{\"model\":\"anthropic/claude-sonnet-4.6\",\"messages\":[{\"role\":\"system\",\"content\":\"Use the get_weather tool exactly once. After the tool result, reply exactly: Paris is sunny.\"},{\"role\":\"user\",\"content\":\"What is the weather in Paris?\"}],\"tools\":[{\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"description\":\"Get current weather for a city.\",\"parameters\":{\"type\":\"object\",\"properties\":{\"city\":{\"type\":\"string\"}},\"required\":[\"city\"],\"additionalProperties\":false}}}],\"stream\":true,\"stream_options\":{\"include_usage\":true},\"max_tokens\":1536,\"reasoning\":{\"max_tokens\":1024}}"
"body": "{\"model\":\"anthropic/claude-sonnet-4.6\",\"messages\":[{\"role\":\"system\",\"content\":\"Use the get_weather tool exactly once. After the tool result, reply exactly: Paris is sunny.\"},{\"role\":\"user\",\"content\":\"What is the weather in Paris?\"}],\"tools\":[{\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"description\":\"Get current weather for a city.\",\"parameters\":{\"type\":\"object\",\"properties\":{\"city\":{\"type\":\"string\"}},\"required\":[\"city\"],\"additionalProperties\":false},\"strict\":false}}],\"stream\":true,\"stream_options\":{\"include_usage\":true},\"reasoning\":{\"max_tokens\":1024},\"max_completion_tokens\":1536,\"store\":false,\"usage\":{\"include\":true}}"
},
"response": {
"status": 200,
@@ -41,7 +41,7 @@
"headers": {
"content-type": "application/json"
},
"body": "{\"model\":\"anthropic/claude-sonnet-4.6\",\"messages\":[{\"role\":\"system\",\"content\":\"Use the get_weather tool exactly once. After the tool result, reply exactly: Paris is sunny.\"},{\"role\":\"user\",\"content\":\"What is the weather in Paris?\"},{\"role\":\"assistant\",\"content\":\"Sure! Let me check the weather in Paris for you right now!\",\"tool_calls\":[{\"id\":\"toolu_01PaChhcyw3yu2P2bDS2bgAA\",\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\":\\\"Paris\\\"}\"}}],\"reasoning\":\"The user wants to know the weather in Paris. I'll use the get_weather tool.\",\"reasoning_details\":[{\"type\":\"reasoning.text\",\"text\":\"The user wants to know the weather in Paris. I'll use the get_weather tool.\",\"format\":\"anthropic-claude-v1\",\"index\":0,\"signature\":\"ErkCCosBCA8YAipAjKnRKpxkZ4eHrMPJ63IWEOYPSzb+XSHyG+vLK+2ks2O9T4N9M37Xn2kausQSH1rfsrdmKxgUlBg6yUFRgMVR7DIRY2xhdWRlLXNvbm5ldC00LTY4AEIIdGhpbmtpbmdaJDRjMGYwNDZmLTI1ZmQtNDVmYi1iZmIzLWEwOGE4ZTI0OWNhNxIMNFb5O6pb4nX0HojdGgyL5h+CAIpsxpdM1QgiMGm/i3ST6F5mAhxB+Uez0Cm95ra9yvQkrzHaA/AmWoXpdmPlczSn1S1RDk2IqeA57Spbf7JT44jygtLQt6yZmGzoTBHn3VkwaNZsuuAtbdo4B5QJXooa/AoKKs54QZ2kfS640vsv5flQVCg7CoQCFuLKjIeLMO7MnxVyuskXJr1DgesTa7I0ScF53U9JGhgB\"}]},{\"role\":\"tool\",\"tool_call_id\":\"toolu_01PaChhcyw3yu2P2bDS2bgAA\",\"content\":\"{\\\"temperature\\\":22,\\\"condition\\\":\\\"sunny\\\"}\"}],\"tools\":[{\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"description\":\"Get current weather for a city.\",\"parameters\":{\"type\":\"object\",\"properties\":{\"city\":{\"type\":\"string\"}},\"required\":[\"city\"],\"additionalProperties\":false}}}],\"stream\":true,\"stream_options\":{\"include_usage\":true},\"max_tokens\":1536,\"reasoning\":{\"max_tokens\":1024}}"
"body": "{\"model\":\"anthropic/claude-sonnet-4.6\",\"messages\":[{\"role\":\"system\",\"content\":\"Use the get_weather tool exactly once. After the tool result, reply exactly: Paris is sunny.\"},{\"role\":\"user\",\"content\":\"What is the weather in Paris?\"},{\"role\":\"assistant\",\"content\":\"Sure! Let me check the weather in Paris for you right now!\",\"tool_calls\":[{\"id\":\"toolu_01PaChhcyw3yu2P2bDS2bgAA\",\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\":\\\"Paris\\\"}\"}}],\"reasoning\":\"The user wants to know the weather in Paris. I'll use the get_weather tool.\",\"reasoning_details\":[{\"type\":\"reasoning.text\",\"text\":\"The user wants to know the weather in Paris. I'll use the get_weather tool.\",\"format\":\"anthropic-claude-v1\",\"index\":0,\"signature\":\"ErkCCosBCA8YAipAjKnRKpxkZ4eHrMPJ63IWEOYPSzb+XSHyG+vLK+2ks2O9T4N9M37Xn2kausQSH1rfsrdmKxgUlBg6yUFRgMVR7DIRY2xhdWRlLXNvbm5ldC00LTY4AEIIdGhpbmtpbmdaJDRjMGYwNDZmLTI1ZmQtNDVmYi1iZmIzLWEwOGE4ZTI0OWNhNxIMNFb5O6pb4nX0HojdGgyL5h+CAIpsxpdM1QgiMGm/i3ST6F5mAhxB+Uez0Cm95ra9yvQkrzHaA/AmWoXpdmPlczSn1S1RDk2IqeA57Spbf7JT44jygtLQt6yZmGzoTBHn3VkwaNZsuuAtbdo4B5QJXooa/AoKKs54QZ2kfS640vsv5flQVCg7CoQCFuLKjIeLMO7MnxVyuskXJr1DgesTa7I0ScF53U9JGhgB\"}]},{\"role\":\"tool\",\"tool_call_id\":\"toolu_01PaChhcyw3yu2P2bDS2bgAA\",\"content\":\"{\\\"temperature\\\":22,\\\"condition\\\":\\\"sunny\\\"}\"}],\"tools\":[{\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"description\":\"Get current weather for a city.\",\"parameters\":{\"type\":\"object\",\"properties\":{\"city\":{\"type\":\"string\"}},\"required\":[\"city\"],\"additionalProperties\":false},\"strict\":false}}],\"stream\":true,\"stream_options\":{\"include_usage\":true},\"reasoning\":{\"max_tokens\":1024},\"max_completion_tokens\":1536,\"store\":false,\"usage\":{\"include\":true}}"
},
"response": {
"status": 200,
@@ -15,7 +15,7 @@
"headers": {
"content-type": "application/json"
},
"body": "{\"model\":\"anthropic/claude-sonnet-4.6\",\"messages\":[{\"role\":\"system\",\"content\":\"Think through the arithmetic, then reply with only the final integer.\"},{\"role\":\"user\",\"content\":\"What is 173 multiplied by 219?\"}],\"stream\":true,\"stream_options\":{\"include_usage\":true},\"max_tokens\":1536,\"temperature\":0,\"reasoning\":{\"max_tokens\":1024}}"
"body": "{\"model\":\"anthropic/claude-sonnet-4.6\",\"messages\":[{\"role\":\"system\",\"content\":\"Think through the arithmetic, then reply with only the final integer.\"},{\"role\":\"user\",\"content\":\"What is 173 multiplied by 219?\"}],\"stream\":true,\"stream_options\":{\"include_usage\":true},\"temperature\":0,\"reasoning\":{\"max_tokens\":1024},\"max_completion_tokens\":1536,\"store\":false,\"usage\":{\"include\":true}}"
},
"response": {
"status": 200,
@@ -10,11 +10,11 @@
"transport": "http",
"request": {
"method": "POST",
"url": "https://api.anthropic.com/v1/messages",
"url": "https://api.anthropic.com/v1/messages?beta=true",
"headers": {
"content-type": "application/json"
},
"body": "{\"model\":\"claude-haiku-4-5-20251001\",\"system\":[{\"type\":\"text\",\"text\":\"Read the PDF returned by the tool and follow the user's response format exactly.\"}],\"messages\":[{\"role\":\"user\",\"content\":[{\"type\":\"text\",\"text\":\"Return only the verification code from the PDF.\"}]},{\"role\":\"assistant\",\"content\":[{\"type\":\"tool_use\",\"id\":\"call_pdf_1\",\"name\":\"read_pdf\",\"input\":{}}]},{\"role\":\"user\",\"content\":[{\"type\":\"tool_result\",\"tool_use_id\":\"call_pdf_1\",\"content\":[{\"type\":\"text\",\"text\":\"PDF read successfully\"},{\"type\":\"document\",\"source\":{\"type\":\"base64\",\"media_type\":\"application/pdf\",\"data\":\"JVBERi0xLjQKMSAwIG9iago8PCAvVHlwZSAvQ2F0YWxvZyAvUGFnZXMgMiAwIFIgPj4KZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFszIDAgUl0gL0NvdW50IDEgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL1R5cGUgL1BhZ2UgL1BhcmVudCAyIDAgUiAvTWVkaWFCb3ggWzAgMCA2MTIgNzkyXSAvUmVzb3VyY2VzIDw8IC9Gb250IDw8IC9GMSA1IDAgUiA+PiA+PiAvQ29udGVudHMgNCAwIFIgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0xlbmd0aCA3NSA+PgpzdHJlYW0KQlQKL0YxIDE4IFRmCjcyIDcyMCBUZAooUERGIGNhc3NldHRlIHZlcmlmaWNhdGlvbiBjb2RlOiBPUkNISUQtNzM5MSkgVGoKRVQKZW5kc3RyZWFtCmVuZG9iago1IDAgb2JqCjw8IC9UeXBlIC9Gb250IC9TdWJ0eXBlIC9UeXBlMSAvQmFzZUZvbnQgL0hlbHZldGljYSA+PgplbmRvYmoKeHJlZgowIDYKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDA5IDAwMDAwIG4gCjAwMDAwMDAwNTggMDAwMDAgbiAKMDAwMDAwMDExNSAwMDAwMCBuIAowMDAwMDAwMjQxIDAwMDAwIG4gCjAwMDAwMDAzNjUgMDAwMDAgbiAKdHJhaWxlcgo8PCAvU2l6ZSA2IC9Sb290IDEgMCBSID4+CnN0YXJ0eHJlZgo0MzUKJSVFT0YK\"}}]}]}],\"tools\":[{\"name\":\"read_pdf\",\"description\":\"Read the attached PDF.\",\"input_schema\":{\"type\":\"object\",\"properties\":{},\"additionalProperties\":false}}],\"stream\":true,\"max_tokens\":40,\"temperature\":0}"
"body": "{\"model\":\"claude-haiku-4-5-20251001\",\"system\":[{\"type\":\"text\",\"text\":\"Read the PDF returned by the tool and follow the user's response format exactly.\"}],\"messages\":[{\"role\":\"user\",\"content\":[{\"type\":\"text\",\"text\":\"Return only the verification code from the PDF.\"}]},{\"role\":\"assistant\",\"content\":[{\"type\":\"tool_use\",\"id\":\"call_pdf_1\",\"name\":\"read_pdf\",\"input\":{}}]},{\"role\":\"user\",\"content\":[{\"type\":\"tool_result\",\"tool_use_id\":\"call_pdf_1\",\"content\":[{\"type\":\"text\",\"text\":\"PDF read successfully\"},{\"type\":\"document\",\"source\":{\"type\":\"base64\",\"media_type\":\"application/pdf\",\"data\":\"JVBERi0xLjQKMSAwIG9iago8PCAvVHlwZSAvQ2F0YWxvZyAvUGFnZXMgMiAwIFIgPj4KZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFszIDAgUl0gL0NvdW50IDEgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL1R5cGUgL1BhZ2UgL1BhcmVudCAyIDAgUiAvTWVkaWFCb3ggWzAgMCA2MTIgNzkyXSAvUmVzb3VyY2VzIDw8IC9Gb250IDw8IC9GMSA1IDAgUiA+PiA+PiAvQ29udGVudHMgNCAwIFIgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0xlbmd0aCA3NSA+PgpzdHJlYW0KQlQKL0YxIDE4IFRmCjcyIDcyMCBUZAooUERGIGNhc3NldHRlIHZlcmlmaWNhdGlvbiBjb2RlOiBPUkNISUQtNzM5MSkgVGoKRVQKZW5kc3RyZWFtCmVuZG9iago1IDAgb2JqCjw8IC9UeXBlIC9Gb250IC9TdWJ0eXBlIC9UeXBlMSAvQmFzZUZvbnQgL0hlbHZldGljYSA+PgplbmRvYmoKeHJlZgowIDYKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDA5IDAwMDAwIG4gCjAwMDAwMDAwNTggMDAwMDAgbiAKMDAwMDAwMDExNSAwMDAwMCBuIAowMDAwMDAwMjQxIDAwMDAwIG4gCjAwMDAwMDAzNjUgMDAwMDAgbiAKdHJhaWxlcgo8PCAvU2l6ZSA2IC9Sb290IDEgMCBSID4+CnN0YXJ0eHJlZgo0MzUKJSVFT0YK\"},\"title\":\"verification.pdf\"}]}]}],\"tools\":[{\"name\":\"read_pdf\",\"description\":\"Read the attached PDF.\",\"input_schema\":{\"type\":\"object\",\"properties\":{},\"additionalProperties\":false}}],\"stream\":true,\"max_tokens\":40,\"temperature\":0}"
},
"response": {
"status": 200,
@@ -10,11 +10,11 @@
"transport": "http",
"request": {
"method": "POST",
"url": "https://api.anthropic.com/v1/messages",
"url": "https://api.anthropic.com/v1/messages?beta=true",
"headers": {
"content-type": "application/json"
},
"body": "{\"model\":\"claude-haiku-4-5-20251001\",\"messages\":[{\"role\":\"user\",\"content\":[{\"type\":\"document\",\"source\":{\"type\":\"base64\",\"media_type\":\"application/pdf\",\"data\":\"JVBERi0xLjQKMSAwIG9iago8PCAvVHlwZSAvQ2F0YWxvZyAvUGFnZXMgMiAwIFIgPj4KZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFszIDAgUl0gL0NvdW50IDEgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL1R5cGUgL1BhZ2UgL1BhcmVudCAyIDAgUiAvTWVkaWFCb3ggWzAgMCA2MTIgNzkyXSAvUmVzb3VyY2VzIDw8IC9Gb250IDw8IC9GMSA1IDAgUiA+PiA+PiAvQ29udGVudHMgNCAwIFIgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0xlbmd0aCA3NSA+PgpzdHJlYW0KQlQKL0YxIDE4IFRmCjcyIDcyMCBUZAooUERGIGNhc3NldHRlIHZlcmlmaWNhdGlvbiBjb2RlOiBPUkNISUQtNzM5MSkgVGoKRVQKZW5kc3RyZWFtCmVuZG9iago1IDAgb2JqCjw8IC9UeXBlIC9Gb250IC9TdWJ0eXBlIC9UeXBlMSAvQmFzZUZvbnQgL0hlbHZldGljYSA+PgplbmRvYmoKeHJlZgowIDYKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDA5IDAwMDAwIG4gCjAwMDAwMDAwNTggMDAwMDAgbiAKMDAwMDAwMDExNSAwMDAwMCBuIAowMDAwMDAwMjQxIDAwMDAwIG4gCjAwMDAwMDAzNjUgMDAwMDAgbiAKdHJhaWxlcgo8PCAvU2l6ZSA2IC9Sb290IDEgMCBSID4+CnN0YXJ0eHJlZgo0MzUKJSVFT0YK\"}},{\"type\":\"text\",\"text\":\"Return only the verification code from the PDF.\"}]}],\"stream\":true,\"max_tokens\":40,\"temperature\":0}"
"body": "{\"model\":\"claude-haiku-4-5-20251001\",\"messages\":[{\"role\":\"user\",\"content\":[{\"type\":\"document\",\"source\":{\"type\":\"base64\",\"media_type\":\"application/pdf\",\"data\":\"JVBERi0xLjQKMSAwIG9iago8PCAvVHlwZSAvQ2F0YWxvZyAvUGFnZXMgMiAwIFIgPj4KZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFszIDAgUl0gL0NvdW50IDEgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL1R5cGUgL1BhZ2UgL1BhcmVudCAyIDAgUiAvTWVkaWFCb3ggWzAgMCA2MTIgNzkyXSAvUmVzb3VyY2VzIDw8IC9Gb250IDw8IC9GMSA1IDAgUiA+PiA+PiAvQ29udGVudHMgNCAwIFIgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0xlbmd0aCA3NSA+PgpzdHJlYW0KQlQKL0YxIDE4IFRmCjcyIDcyMCBUZAooUERGIGNhc3NldHRlIHZlcmlmaWNhdGlvbiBjb2RlOiBPUkNISUQtNzM5MSkgVGoKRVQKZW5kc3RyZWFtCmVuZG9iago1IDAgb2JqCjw8IC9UeXBlIC9Gb250IC9TdWJ0eXBlIC9UeXBlMSAvQmFzZUZvbnQgL0hlbHZldGljYSA+PgplbmRvYmoKeHJlZgowIDYKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDA5IDAwMDAwIG4gCjAwMDAwMDAwNTggMDAwMDAgbiAKMDAwMDAwMDExNSAwMDAwMCBuIAowMDAwMDAwMjQxIDAwMDAwIG4gCjAwMDAwMDAzNjUgMDAwMDAgbiAKdHJhaWxlcgo8PCAvU2l6ZSA2IC9Sb290IDEgMCBSID4+CnN0YXJ0eHJlZgo0MzUKJSVFT0YK\"},\"title\":\"verification.pdf\"},{\"type\":\"text\",\"text\":\"Return only the verification code from the PDF.\"}]}],\"stream\":true,\"max_tokens\":40,\"temperature\":0}"
},
"response": {
"status": 200,
+17 -15
View File
@@ -44,21 +44,23 @@ describe("Tool.make (dynamic JSON Schema)", () => {
expect(definition?.inputSchema).toEqual(jsonSchema)
})
test("execute receives the raw input untouched", async () => {
const seen: unknown[] = []
const tool = Tool.make({
description: "echo",
jsonSchema: { type: "object" },
execute: (params) =>
Effect.sync(() => {
seen.push(params)
return { ok: true }
}),
})
const result = await Effect.runPromise(tool.execute({ hello: "world" }))
expect(seen).toEqual([{ hello: "world" }])
expect(result).toEqual({ ok: true })
})
it.effect("execute receives the raw input untouched", () =>
Effect.gen(function* () {
const seen: unknown[] = []
const tool = Tool.make({
description: "echo",
jsonSchema: { type: "object" },
execute: (params) =>
Effect.sync(() => {
seen.push(params)
return { ok: true }
}),
})
const result = yield* tool.execute({ hello: "world" })
expect(seen).toEqual([{ hello: "world" }])
expect(result).toEqual({ ok: true })
}),
)
})
describe("LLM.generateObject", () => {
+42 -16
View File
@@ -7,6 +7,37 @@ import { it } from "./lib/effect.js"
import { dynamicResponse } from "./lib/http.js"
describe("Image", () => {
for (const provider of [OpenAI, Google, XAI, ZAI]) {
const model = provider.configure({ apiKey: "test", baseURL: "https://image.test" }).image("image-model")
for (const body of ['{"data":42,"candidates":42,"opaque":{"nested":[1,2]},"trace":"outer"}', '{"invalid":']) {
it.effect(`retains ${model.provider} image response body and decode cause: ${body}`, () =>
Effect.gen(function* () {
const error = yield* Image.generate({ model, prompt: "hello" }).pipe(Effect.flip)
expect(error.reason._tag).toBe("InvalidProviderOutput")
expect(error.message).toContain("invalid response")
expect(error.reason.body).toBe(body)
expect(error.reason.http).toMatchObject({ status: 200, headers: { "x-image-trace": "trace-1" } })
expect(error.reason.http?.url).toStartWith("https://image.test/")
expect(error.reason.cause).toBeInstanceOf(Error)
}).pipe(
Effect.provide(
ImageClient.layer.pipe(
Layer.provide(
dynamicResponse((input) =>
Effect.succeed(
input.respond(body, {
headers: { "content-type": "application/json", "x-image-trace": "trace-1" },
}),
),
),
),
),
),
),
)
}
}
it.effect("generates images through the OpenAI Images API", () =>
Effect.gen(function* () {
const response = yield* Image.generate({
@@ -530,22 +561,17 @@ describe("Image", () => {
Effect.sync(() => {
expect(error.reason._tag).toBe("InvalidProviderOutput")
if (error.reason._tag !== "InvalidProviderOutput") return
expect(error.reason.message).toContain("finish reasons: IMAGE_SAFETY")
expect(error.reason.providerMetadata).toEqual({
google: {
promptFeedback: { blockReason: "SAFETY" },
candidates: [
{
index: 0,
finishReason: "IMAGE_SAFETY",
finishMessage: "The generated image was blocked by safety filters.",
safetyRatings: [{ category: "HARM_CATEGORY_DANGEROUS_CONTENT", blocked: true }],
citationMetadata: undefined,
groundingMetadata: undefined,
parts: [{ type: "text", text: "blocked", thought: false, thoughtSignature: undefined }],
},
],
},
expect(error.message).toContain("finish reasons: IMAGE_SAFETY")
expect(JSON.parse(error.reason.body ?? "")).toEqual({
promptFeedback: { blockReason: "SAFETY" },
candidates: [
{
finishReason: "IMAGE_SAFETY",
finishMessage: "The generated image was blocked by safety filters.",
safetyRatings: [{ category: "HARM_CATEGORY_DANGEROUS_CONTENT", blocked: true }],
content: { parts: [{ text: "blocked", thought: false }] },
},
],
})
}),
),
+81 -4
View File
@@ -11,6 +11,7 @@ describe("provider error classification", () => {
"Input length 131393 exceeds the maximum allowed input length of 131040 tokens.",
"The input (516368 tokens) is longer than the model's context length (262144 tokens).",
"Prompt has 5,958,968 tokens, but the configured context size is 256,000 tokens",
"Range of input length should be [1, 129024]",
"Too many tokens",
"Token limit exceeded",
]
@@ -80,6 +81,46 @@ describe("provider error classification", () => {
).toEqual(["ProviderInternal", "ProviderInternal", "ProviderInternal"])
})
test("classifies retryable server messages as provider internal", () => {
const message =
"The model is currently at capacity due to high demand. Please try again in a few minutes, or use a higher service tier for priority processing."
expect(
[
message,
"Try again",
"Please retry your request shortly.",
"You can retry the request.",
"Try your request again.",
"The service is temporarily at capacity.",
"The model is overloaded.",
"Service unavailable",
"Internal server error",
"The server is busy.",
"Provider returned error",
"Provider returned an error",
"ResourceExhausted",
"Upstream connection failed",
"Exceeded request buffer limit while retrying upstream",
].map((message) => classifyProviderFailure({ message })._tag),
).toEqual(Array(15).fill("ProviderInternal"))
expect(
classifyProviderFailure({ message: "Provider request failed", rawBody: "Please try again later." })._tag,
).toBe("ProviderInternal")
})
test("prioritizes specific failures over retryable server text", () => {
expect(
[
classifyProviderFailure({ message: "Invalid credentials, try again", status: 401 }),
classifyProviderFailure({ message: "Quota exceeded, try again", status: 429 }),
classifyProviderFailure({ message: "Rate limit exceeded, try again" }),
classifyProviderFailure({ message: "Upstream request failed: validation failed", status: 400 }),
classifyProviderFailure({ message: "Try again", status: 200 }),
].map((failure) => failure._tag),
).toEqual(["Authentication", "QuotaExceeded", "RateLimit", "InvalidRequest", "ProviderInternal"])
})
test("classifies transient client statuses as provider internal", () => {
expect([408, 409].map((status) => classifyProviderFailure({ message: `HTTP ${status}`, status })._tag)).toEqual([
"ProviderInternal",
@@ -87,10 +128,12 @@ describe("provider error classification", () => {
])
})
test("classifies network error text as provider internal", () => {
test("classifies any remaining 4xx status as an invalid request", () => {
expect(
["network error", "network-error", "network_error"].map((message) => classifyProviderFailure({ message })._tag),
).toEqual(["ProviderInternal", "ProviderInternal", "ProviderInternal"])
[400, 402, 404, 418, 422, 451].map(
(status) => classifyProviderFailure({ message: `HTTP ${status}`, status })._tag,
),
).toEqual(Array(6).fill("InvalidRequest"))
})
test("classifies nested provider codes when a top-level code is also present", () => {
@@ -103,14 +146,48 @@ describe("provider error classification", () => {
).toEqual(["QuotaExceeded", "ProviderInternal", "InvalidRequest"])
})
test("keeps unknown and malformed provider payloads non-retryable", () => {
test("leaves unrecognized failures unclassified for the retry default", () => {
expect(classifyProviderFailure({ message: '{"error":{"message":"no_kv_space"}}' })._tag).toBe("UnknownProvider")
expect(classifyProviderFailure({ message: '{"type":"error","error":{"code":123}}' })._tag).toBe("UnknownProvider")
expect(classifyProviderFailure({ message: "not-json" })._tag).toBe("UnknownProvider")
expect(classifyProviderFailure({ message: "network error" })._tag).toBe("UnknownProvider")
})
})
describe("provider error rawBody classification", () => {
test("classifies provider envelopes without separate code inputs", () => {
const cases = [
['{"type":"error","error":{"type":"overloaded_error","message":"Try again"}}', "ProviderInternal"],
['{"error":{"code":"insufficient_quota","message":"Request failed"}}', "QuotaExceeded"],
[
'{"type":"response.failed","response":{"error":{"code":"authentication_error","message":"Denied"}}}',
"Authentication",
],
['{"error":{"code":429,"status":"RESOURCE_EXHAUSTED","message":"Try again"}}', "ProviderInternal"],
['{"exception":{"type":"throttlingException","details":{"message":"Try again"}}}', "RateLimit"],
] as const
for (const [rawBody, expected] of cases) {
const reason = classifyProviderFailure({ message: "Request failed", rawBody })
expect(reason._tag).toBe(expected)
expect(reason.body).toBe(rawBody)
expect(reason).not.toHaveProperty("code")
}
})
test("classifies separately supplied SDK data without replacing the response body", () => {
const data = { error: { code: "authentication_error" } }
for (const value of [data, JSON.stringify(data)]) {
const reason = classifyProviderFailure({
message: "Request failed",
status: 400,
rawBody: '{"message":"Request failed"}',
data: value,
})
expect(reason._tag).toBe("Authentication")
expect(reason.body).toBe('{"message":"Request failed"}')
}
})
test("classifies overflow signals buried in the raw payload when the summary is vague", () => {
const reason = classifyProviderFailure({
message: "Request failed",

Some files were not shown because too many files have changed in this diff Show More