Compare commits

..
52 Commits
Author SHA1 Message Date
Jérôme BenoitandTest User 0caae608a2 chore(nix): update nixpkgs for Bun 1.4 (#50221)
Co-authored-by: Test User <test@test.com>
2026-09-27 21:34:52 -05:00
Kit Langton 96f23508be refactor(core): remove unused project discovery option (#51729) 2026-09-27 15:05:46 -07:00
Kit Langton 28bb0a7158 refactor(core): drop forwarding shim modules (#51670) 2026-09-27 14:47:19 -07:00
DS 3d109828ff fix(tui): truncate btw question preview (#51713) 2026-09-27 21:05:20 +02:00
Shoubhit Dash c0d49f101c feat(ai): retry transient failures on queued generation reads (#51635) 2026-09-27 19:47:25 +05:30
Shoubhit Dash be2446e188 feat(ai): add ElevenLabs Scribe transcription route (#51641) 2026-09-27 19:34:36 +05:30
Shoubhit Dash 107966eddd fix(ai): reject and throw with signal.reason on abort (#51633) 2026-09-27 19:24:45 +05:30
Kit Langton f5e580cde1 chore(core): remove dead modules and exports (#51667) 2026-09-27 06:38:43 -07:00
Shoubhit Dash 4428a77acd fix(ai): classify terminal generation failures by provider error code (#51632) 2026-09-27 18:44:21 +05:30
opencode-agent[bot] 01eb18144b chore(core): refresh bundled models.dev snapshot 2026-09-27 12:19:33 +00:00
Shoubhit Dash 01208048dc test(ai): cover queued media failures, resume, cancel, and transcription sources (#51626) 2026-09-27 17:43:24 +05:30
Shoubhit Dash 995f76cb63 feat(tui): add last turn source to diff viewer (#51639) 2026-09-27 17:39:41 +05:30
Shoubhit Dash c33de198e3 fix(ai): surface truncated Gemini speech and transcription results (#51379) 2026-09-27 14:29:36 +05:30
Shoubhit Dash 24224bcb57 fix(ai): describe speech assets in the format actually sent (#51377) 2026-09-27 14:16:23 +05:30
Aiden Cline 7913c8db59 feat(core): log provider rejections that trigger overflow compaction (#51582) 2026-09-26 23:11:33 -05:00
Aiden Cline d9987ef9c8 feat(core): fit output limits to the context window (#51271) 2026-09-26 22:58:28 -05:00
Jack f0ed4f67df docs: list LongCat 2.5 Preview Free in V2 Console (#51487) 2026-09-26 21:20:36 +08:00
opencode-agent[bot] 2109d68d39 chore(core): refresh bundled models.dev snapshot 2026-09-26 12:18:22 +00:00
Filip cefb2968e2 fix(cli): cancel auth attempts on ctrl+c (#51484) 2026-09-26 11:59:17 +00:00
opencode-agent[bot] 00d179015f chore: update nix node_modules hashes 2026-09-26 10:01:15 +00:00
1d4e1233e5 feat(cli): share grouped auth picker with MCP auth (#51006)
Co-authored-by: rekram1-node <rekram1-node@users.noreply.github.com>
Co-authored-by: thdxr <thdxr@users.noreply.github.com>
Co-authored-by: rekram1-node <63023139+rekram1-node@users.noreply.github.com>
Co-authored-by: Filip Hejmowski <fhejmowski@simplito.com>
2026-09-26 11:42:04 +02:00
d14f20b46e fix(tui): capitalize OpenCode web search provider (#49023)
Co-authored-by: vimtor <36263538+vimtor@users.noreply.github.com>
Co-authored-by: Victor Navarro <vn4varro@gmail.com>
2026-09-26 09:36:20 +00:00
opencode-agent[bot]andrekram1-node 37049a5a13 fix(tui): skip model selection after MCP connection (#51448)
Co-authored-by: rekram1-node <rekram1-node@users.noreply.github.com>
2026-09-26 00:00:25 -05:00
Aiden Cline a64bc3616e refactor(core): cleanup session compaction (#51447) 2026-09-26 00:00:11 -05:00
opencode 39021dfd67 sync release versions for v2.0.18 2026-09-25 23:57:39 +00:00
opencode-agent[bot] 709ddc0d79 chore: update nix node_modules hashes 2026-09-25 23:23:45 +00:00
Kit LangtonandAiden Cline 041885d838 fix(core): decode legacy media in compaction checkpoints (#51409)
Co-authored-by: Aiden Cline <aidenpcline@gmail.com>
2026-09-25 18:08:16 -05:00
Aiden Cline 29ce49db0f refactor(util): share a browser opener across cli, tui, and core (#51412) 2026-09-25 18:00:32 -05:00
opencode 00738c5b2d sync release versions for v2.0.17 2026-09-25 21:09:12 +00:00
Aiden Cline 6ec8ca920f feat(core): name Copilot sessions with the free utility model (#51237) 2026-09-25 14:53:16 -05:00
Shoubhit Dash 6585bb7105 fix(ai): keep OpenAI image output settings and Z.ai URL expiry (#51380) 2026-09-25 22:51:46 +05:30
Shoubhit Dash f954688fbb fix(ai): harden OpenAI transcription stream parsing (#51378) 2026-09-25 22:51:22 +05:30
Shoubhit Dash 1463dabde9 fix(ai): tighten media error consistency (#51374) 2026-09-25 22:46:04 +05:30
Shoubhit Dash 29ea6ee05b test(ai): cover media facade selectors and url asset edges (#51376) 2026-09-25 22:37:20 +05:30
Shoubhit Dash b170904731 docs(ai): describe transcription speakers as a constraint (#51382) 2026-09-25 22:36:02 +05:30
Shoubhit Dash 88e1fa9304 fix(ai): accept timestamps: false on speech routes without timestamps (#51375) 2026-09-25 22:33:59 +05:30
opencode-agent[bot]andvimtor ff1bf315ed docs(www): hide Console Usage API documentation (#51358)
Co-authored-by: vimtor <36263538+vimtor@users.noreply.github.com>
2026-09-25 19:01:48 +02:00
Shoubhit Dash 144ce00e00 fix(ai): bound generation event poll sleep by the deadline (#51372) 2026-09-25 22:31:35 +05:30
Shoubhit Dash ad504094f0 fix(ai): never delete a finished Runway task on cancel (#51373) 2026-09-25 22:24:16 +05:30
Shoubhit Dash bad6834a3e fix(ai): size fal Kontext by aspect ratio and decode sync_mode data URIs (#51371) 2026-09-25 22:23:14 +05:30
Shoubhit Dash 0c4bbc3cd1 feat(ai): keep prompt cache across effort switches on GPT-6 Sol and Luna (#51339) 2026-09-25 22:15:23 +05:30
Shoubhit Dash ae7dd82126 fix(ai): report Black Forest Labs submit cost as image usage (#51370) 2026-09-25 22:13:55 +05:30
Shoubhit Dash 65d5123ead fix(ai): enable AssemblyAI speaker labels when speakers is set (#51369) 2026-09-25 22:11:44 +05:30
Shoubhit Dash 4eb46a8885 fix(core): revert always-thinking variants for Claude Opus 5.5 (#51359) 2026-09-25 22:03:12 +05:30
Jack 1986e92842 docs(go): show permanent DeepSeek $60 allowance (#51363) 2026-09-26 00:17:08 +08:00
Shoubhit Dash 14fc63ba9e fix(core): keep thinking on for Claude Opus 5.5 variants (#51338) 2026-09-25 18:32:41 +05:30
opencode-agent[bot]andnexxeln c34ffa117e fix(ai): preserve Gemini 3.8 TTS WAV output (#51300)
Co-authored-by: nexxeln <95541290+nexxeln@users.noreply.github.com>
2026-09-25 18:11:36 +05:30
beeb14e910 feat(prompt): undo queued prompts back into the input (#51124)
Co-authored-by: vimtor <36263538+vimtor@users.noreply.github.com>
Co-authored-by: vimtor <vn4varro@gmail.com>
2026-09-25 14:36:29 +02:00
opencode-agent[bot] aae42e2e75 chore(core): refresh bundled models.dev snapshot 2026-09-25 12:21:04 +00:00
cc9011c1ae fix(tui): virtualize large added-file diffs (#51122)
Co-authored-by: vimtor <36263538+vimtor@users.noreply.github.com>
Co-authored-by: vimtor <vn4varro@gmail.com>
2026-09-25 13:51:00 +02:00
Victor Navarro 6cd938e1e9 feat(core): register Console-hosted MCP servers (#51325) 2026-09-25 13:16:04 +02:00
Jack 7de6b3fc15 docs(console): document Qwen3.8 Max (#51320) 2026-09-25 19:13:45 +08:00
269 changed files with 6214 additions and 6160 deletions
-5
View File
@@ -1,5 +0,0 @@
---
"@opencode/core": patch
---
Correct directory page headings when the read offset is zero.
+1 -1
View File
@@ -184,7 +184,7 @@ const table = sqliteTable("session", {
- Keep `SessionRunner`, model resolution, tool registry, permissions, and filesystem Location-scoped. Omitted `Location.workspaceID` means implicit-local placement; explicit workspace identity remains reserved for future placement semantics.
- Preserve one explicit `llm.stream(request)` call per Physical Attempt and reload projected history before durable continuation. A logical Step may use generic pre-output retries, one full-context retry after continuation rejection, incomplete-stream continuation, or one overflow-compaction rebuild. Generic retries retain the logical step number and do not consume another agent-step allowance. Do not delegate orchestration to an in-memory tool loop.
- Keep local Session drains process-local until clustering is implemented. `SessionRunCoordinator` joins explicit same-Session resumes, coalesces prompt wakeups, and allows different Sessions to run concurrently. A write-ahead execution claim marks a process-local busy period for restart recovery: terminal completion, failure, or user interruption releases it, while shutdown interruption and process death preserve it. Startup recovery resumes claimed top-level Sessions with durable per-execution attempt accounting. The claim is a recovery marker, not clustered ownership, fencing, or an exactly-once guarantee.
- Keep native compaction mechanisms out of `SessionCompaction`. Plugins register `native` strategies through the `SessionCompaction` editor that turn a prepared request into a replacement window (the built-in `NativeCompactionPlugin` handles `@opencode/ai` compaction operations); later registrations win. Core owns the provider-mode decision, route provenance, the retry policy, overflow recovery, interruption, usage accounting, and checkpoint persistence.
- Keep provider-specific native compaction mechanisms in `@opencode/ai` behind `LLMClient.compact`. `SessionCompaction` chooses a summary or native compaction from the model's `compaction` setting and owns route provenance, request shrinking, the retry policy, interruption, usage accounting, and checkpoint persistence.
- Keep delivery vocabulary explicit. Prompts steer by default. At safe step boundaries, steered compaction takes priority up to the first steered move control; other steers retain enqueue order. At an idle boundary, steers take priority; otherwise exactly one queued item delivers before the runner reevaluates continuation. Inbox items may be cancelled or changed between queue and steer before delivery. Promoting new user input resets the selected agent's step allowance; a batch of steers resets it once.
- One step is one logical LLM call; its durable record covers only the model-visible span. Do not write "provider turn", and do not use bare "turn" for a single call: "turn" is reserved for the future assistant-turn unit containing all steps from prompt promotion until the session would go idle.
- Keep event replay ownership separate from clustered Session execution ownership.
+71 -57
View File
@@ -32,7 +32,7 @@
},
"packages/ai": {
"name": "@opencode/ai",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@aws-sdk/credential-providers": "3.1057.0",
"@opencode/schema": "workspace:*",
@@ -54,7 +54,7 @@
},
"packages/app": {
"name": "@opencode/app",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@corvu/drawer": "catalog:",
"@dnd-kit/abstract": "0.5.0",
@@ -112,13 +112,14 @@
},
"packages/cli": {
"name": "@opencode/cli",
"version": "2.0.16",
"version": "2.0.18",
"bin": {
"opencode": "./bin/opencode.cjs",
"opencode2": "./bin/opencode2.cjs",
},
"dependencies": {
"@agentclientprotocol/sdk": "1.2.1",
"@clack/core": "1.0.0-alpha.1",
"@clack/prompts": "1.0.0-alpha.1",
"@effect/platform-node": "catalog:",
"@opencode-ai/pty": "0.1.13",
@@ -135,7 +136,7 @@
"effect": "catalog:",
"immer": "11.1.4",
"jsonc-parser": "3.3.1",
"open": "10.1.2",
"picocolors": "1.1.1",
"solid-js": "catalog:",
"tree-sitter-bash": "0.25.0",
"tree-sitter-powershell": "0.25.10",
@@ -177,7 +178,7 @@
},
"packages/client": {
"name": "@opencode/client",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@opencode/protocol": "workspace:*",
"@opencode/schema": "workspace:*",
@@ -203,7 +204,7 @@
},
"packages/codemode": {
"name": "@opencode/codemode",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"acorn": "8.15.0",
"effect": "catalog:",
@@ -216,7 +217,7 @@
},
"packages/console/app": {
"name": "@opencode/console-app",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@cloudflare/vite-plugin": "1.15.2",
"@ibm/plex": "6.4.1",
@@ -252,7 +253,7 @@
},
"packages/console/core": {
"name": "@opencode/console-core",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@aws-sdk/client-sts": "3.782.0",
"@jsx-email/render": "1.1.1",
@@ -279,7 +280,7 @@
},
"packages/console/function": {
"name": "@opencode/console-function",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@openauthjs/openauth": "0.0.0-20250322224806",
"@opencode/console-core": "workspace:*",
@@ -296,7 +297,7 @@
},
"packages/console/mail": {
"name": "@opencode/console-mail",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@jsx-email/all": "2.2.3",
"@jsx-email/cli": "1.4.3",
@@ -320,7 +321,7 @@
},
"packages/console/support": {
"name": "@opencode/console-support",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@cloudflare/vite-plugin": "1.15.2",
"@opencode/console-core": "workspace:*",
@@ -340,7 +341,7 @@
},
"packages/core": {
"name": "@opencode/core",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@ai-sdk/cohere": "3.0.27",
"@ai-sdk/gateway": "3.0.104",
@@ -408,7 +409,7 @@
},
"packages/desktop": {
"name": "@opencode/desktop",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@zip.js/zip.js": "2.7.62",
"electron-context-menu": "5.0.0",
@@ -457,7 +458,7 @@
},
"packages/enterprise": {
"name": "@opencode/enterprise",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@hono/standard-validator": "catalog:",
"@opencode-ai/sdk": "1.18.21",
@@ -494,7 +495,7 @@
},
"packages/function": {
"name": "@opencode/function",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@octokit/auth-app": "8.0.1",
"@octokit/rest": "catalog:",
@@ -510,7 +511,7 @@
},
"packages/http-recorder": {
"name": "@opencode/http-recorder",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@effect/platform-node-shared": "4.0.0-rc.112",
},
@@ -529,7 +530,7 @@
},
"packages/httpapi-codegen": {
"name": "@opencode/httpapi-codegen",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"effect": "catalog:",
"prettier": "3.6.2",
@@ -542,7 +543,7 @@
},
"packages/latex": {
"name": "@opencode/latex",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@opencode/plugin": "workspace:*",
"@opentui/core": "catalog:",
@@ -556,7 +557,7 @@
},
"packages/merman": {
"name": "@opencode/merman",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@opencode/plugin": "workspace:*",
"@opentui/core": "catalog:",
@@ -571,7 +572,7 @@
},
"packages/plugin": {
"name": "@opencode/plugin",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@ai-sdk/provider": "3.0.8",
"@opencode/ai": "workspace:*",
@@ -610,7 +611,7 @@
},
"packages/plugin-browser": {
"name": "@opencode/plugin-browser",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@opencode/plugin": "workspace:*",
"@opencode/schema": "workspace:*",
@@ -640,7 +641,7 @@
},
"packages/protocol": {
"name": "@opencode/protocol",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@opencode/schema": "workspace:*",
"effect": "catalog:",
@@ -655,7 +656,7 @@
},
"packages/schema": {
"name": "@opencode/schema",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@standard-schema/spec": "catalog:",
"effect": "catalog:",
@@ -679,7 +680,7 @@
},
"packages/sdk": {
"name": "@opencode/sdk",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@opencode/client": "workspace:*",
"@opencode/core": "workspace:*",
@@ -700,7 +701,7 @@
},
"packages/server": {
"name": "@opencode/server",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@effect/platform-node": "catalog:",
"@effect/platform-node-shared": "catalog:",
@@ -722,7 +723,7 @@
},
"packages/session-ui": {
"name": "@opencode/session-ui",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@kobalte/core": "catalog:",
"@opencode/client": "workspace:*",
@@ -757,7 +758,7 @@
},
"packages/simulation": {
"name": "@opencode/simulation",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@opencode/ai": "workspace:*",
"@opencode/core": "workspace:*",
@@ -777,7 +778,7 @@
},
"packages/stats/app": {
"name": "@opencode/stats-app",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@ibm/plex": "6.4.1",
"@kobalte/core": "catalog:",
@@ -811,7 +812,7 @@
},
"packages/stats/core": {
"name": "@opencode/stats-core",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@aws-sdk/client-athena": "3.933.0",
"@planetscale/database": "1.19.0",
@@ -830,7 +831,7 @@
},
"packages/stats/server": {
"name": "@opencode/stats-server",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@aws-sdk/client-firehose": "3.933.0",
"@effect/platform-node": "catalog:",
@@ -876,7 +877,7 @@
},
"packages/theme": {
"name": "@opencode/theme",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@opentui/core": "catalog:",
"effect": "catalog:",
@@ -890,7 +891,7 @@
},
"packages/tui": {
"name": "@opencode/tui",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@opencode/client": "workspace:*",
"@opencode/core": "workspace:*",
@@ -909,7 +910,6 @@
"effect": "catalog:",
"fuzzysort": "catalog:",
"get-east-asian-width": "catalog:",
"open": "10.1.2",
"opentui-spinner": "catalog:",
"remeda": "catalog:",
"solid-js": "catalog:",
@@ -925,7 +925,7 @@
},
"packages/ui": {
"name": "@opencode/ui",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@kobalte/core": "catalog:",
"@pierre/diffs": "catalog:",
@@ -960,7 +960,7 @@
},
"packages/util": {
"name": "@opencode/util",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@effect/opentelemetry": "catalog:",
"@effect/platform-node": "catalog:",
@@ -982,6 +982,7 @@
"mime-types": "3.0.2",
"minimatch": "10.2.5",
"npm-package-arg": "13.0.2",
"open": "11.0.4",
"pacote": "21.5.1",
},
"devDependencies": {
@@ -997,7 +998,7 @@
},
"packages/web": {
"name": "@opencode/web",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@astrojs/cloudflare": "12.6.3",
"@astrojs/markdown-remark": "6.3.1",
@@ -1038,7 +1039,7 @@
},
"services/update": {
"name": "@opencode/update",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"jose": "6.0.11",
"semver": "catalog:",
@@ -1098,6 +1099,7 @@
"@types/node": "catalog:",
"bun-types": "1.4.2",
"effect": "catalog:",
"open": "11.0.4",
"solid-js": "catalog:",
},
"catalog": {
@@ -4322,7 +4324,7 @@
"is-decimal": ["is-decimal@2.0.1", "", {}, "sha512-AAB9hiomQs5DXWcRB1rqsxGUstbRroFOPPVAomNk/3XHR5JyEZChOyTWe2oayKnsSsr/kcGqF+z6yuH6HHpN0A=="],
"is-docker": ["is-docker@3.0.0", "", { "bin": { "is-docker": "cli.js" } }, "sha512-eljcgEDlEns/7AXFosB5K/2nCM4P7FQPkGc/DWLy5rmFEWvZayGrik1d9/QIY5nJ4f9YsVvBkA6kJpHn9rISdQ=="],
"is-docker": ["is-docker@4.0.0", "", { "bin": { "is-docker": "cli.js" } }, "sha512-LHE+wROyG/Y/0ZnbktRCoTix2c1RhgWaZraMZ8o1Q7zCh0VSrICJQO5oqIIISrcSBtrXv0o233w1IYwsWCjTzA=="],
"is-document.all": ["is-document.all@1.0.0", "", { "dependencies": { "call-bound": "^1.0.4" } }, "sha512-+XSoyS05OdBbhFuELhgTCpFNHkpBOJqtsZfUFFpe5QTw+9Sjbh8zitxhQkYAo6wV7e1Vb8cAPvpCk9jGam/82g=="],
@@ -4340,6 +4342,8 @@
"is-hexadecimal": ["is-hexadecimal@2.0.1", "", {}, "sha512-DgZQp241c8oO6cA1SbTEWiXeoxV42vlcJxgH+B3hi1AiqqKruZR3ZGF8In3fj4+/y/7rHvlOZLZtgJ/4ttYGZg=="],
"is-in-ssh": ["is-in-ssh@1.0.0", "", {}, "sha512-jYa6Q9rH90kR1vKB6NM7qqd1mge3Fx4Dhw5TVlK1MUBqhEOuCagrEHMevNuCcbECmXZ0ThXkRm+Ymr51HwEPAw=="],
"is-inside-container": ["is-inside-container@1.0.0", "", { "dependencies": { "is-docker": "^3.0.0" }, "bin": { "is-inside-container": "cli.js" } }, "sha512-KIYLCCJghfHZxqjYBE7rEy0OBuTd5xCHS7tHVgvCLkx7StIoaxwNW3hCALgEUjFfeRk+MG/Qxmp/vtETEF3tRA=="],
"is-map": ["is-map@2.0.3", "", {}, "sha512-1Qed0/Hr2m+YqxnM09CjA2d/i6YZNfF6R2oRAOj36eUdS6qIV/huPJNSEpKbupewFs+ZsJlxsjjPbc0/afW6Lw=="],
@@ -4386,7 +4390,7 @@
"is-whitespace": ["is-whitespace@0.3.0", "", {}, "sha512-RydPhl4S6JwAyj0JJjshWJEFG6hNye3pZFBRZaTUfZFwGHxzppNaNOVgQuS/E/SlhrApuMXrpnK1EEIXfdo3Dg=="],
"is-wsl": ["is-wsl@3.1.1", "", { "dependencies": { "is-inside-container": "^1.0.0" } }, "sha512-e6rvdUCiQCAuumZslxRJWR/Doq4VpPR82kqclvcS0efgt430SlGIk05vdCN58+VrzgtIcfNODjozVielycD4Sw=="],
"is-wsl": ["is-wsl@2.2.0", "", { "dependencies": { "is-docker": "^2.0.0" } }, "sha512-fKzAra0rGJUUBwGBgNkHZuToZcn+TtXHpeCgmkMJMMYx1sQDYaCSyjJBSCa2nH1DGm7s3n1oBnohoVTBaN7Lww=="],
"isarray": ["isarray@1.0.0", "", {}, "sha512-VLghIWNM6ELQzo7zwmcg0NmTVyWKYjvIeM83yjp0wRDTmUnrM678fQbcKBo6n2CJEF0szoG//ytg+TKla89ALQ=="],
@@ -4850,7 +4854,7 @@
"oniguruma-to-es": ["oniguruma-to-es@4.3.6", "", { "dependencies": { "oniguruma-parser": "^0.12.2", "regex": "^6.1.0", "regex-recursion": "^6.0.2" } }, "sha512-csuQ9x3Yr0cEIs/Zgx/OEt9iBw9vqIunAPQkx19R/fiMq2oGVTgcMqO/V3Ybqefr1TBvosI6jU539ksaBULJyA=="],
"open": ["open@10.1.2", "", { "dependencies": { "default-browser": "^5.2.1", "define-lazy-prop": "^3.0.0", "is-inside-container": "^1.0.0", "is-wsl": "^3.1.0" } }, "sha512-cxN6aIDPz6rm8hbebcP7vrQNhvRcveZoJU72Y7vskh4oIm+BZwBECnx5nTmrlres1Qapvx27Qo1Auukpf8PKXw=="],
"open": ["open@11.0.4", "", { "dependencies": { "default-browser": "^5.5.1", "define-lazy-prop": "^3.0.0", "is-in-ssh": "^1.0.0", "is-inside-container": "^1.0.0", "powershell-utils": "^0.2.1", "wsl-utils": "^1.0.0" } }, "sha512-++Zlftm0kVLPmzC06t6epuWmcRMDbI4z5P3NNX979WA/k23+NtSOynEGzsVfZwguKw2mi5umVgnBlJQMwRz4Pg=="],
"openai": ["openai@6.49.0", "", { "peerDependencies": { "@aws-sdk/credential-provider-node": ">=3.972.0 <4", "@smithy/hash-node": ">=4.3.0 <5", "@smithy/signature-v4": ">=5.4.0 <6", "ws": "^8.18.0", "zod": "^3.25 || ^4.0" }, "optionalPeers": ["@aws-sdk/credential-provider-node", "@smithy/hash-node", "@smithy/signature-v4", "ws", "zod"] }, "sha512-aYCc0C6L864eR6WSYIwQGyXriw/nIyZx0ObvhzOEVuk0zoBDpynjSbrionWI7q65B5H8jJX0DXR9snEzM6bfPg=="],
@@ -4994,6 +4998,8 @@
"postject": ["postject@1.0.0-alpha.6", "", { "dependencies": { "commander": "^9.4.0" }, "bin": { "postject": "dist/cli.js" } }, "sha512-b9Eb8h2eVqNE8edvKdwqkrY6O7kAwmI8kcnBv1NScolYJbo59XUF0noFq+lxbC1yN20bmC0WBEbDC5H/7ASb0A=="],
"powershell-utils": ["powershell-utils@0.2.1", "", {}, "sha512-C+y9x90UElAddDZmV4qOx9W53B61PO7cIqWz2dQsWlwswuq4mr8NEwytdGKboYbQlGZ3awrkTeNvcZiZNHnQ8A=="],
"preact": ["preact@11.0.0-beta.0", "", {}, "sha512-IcODoASASYwJ9kxz7+MJeiJhvLriwSb4y4mHIyxdgaRZp6kPUud7xytrk/6GZw8U3y6EFJaRb5wi9SrEK+8+lg=="],
"preact-render-to-string": ["preact-render-to-string@6.6.5", "", { "peerDependencies": { "preact": ">=10 || >= 11.0.0-0" } }, "sha512-O6MHzYNIKYaiSX3bOw0gGZfEbOmlIDtDfWwN1JJdc/T3ihzRT6tGGSEWE088dWrEDGa1u7101q+6fzQnO9XCPA=="],
@@ -5814,7 +5820,7 @@
"ws": ["ws@8.21.0", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-Vsp28b7DRcimFQvrqu2Wek3z1iYxDCWqHYB8Qsnk/S4RfaCQzPGPyBNuVjJV3cd6UiKtUtp6sNM77gWvzcCH+g=="],
"wsl-utils": ["wsl-utils@0.1.0", "", { "dependencies": { "is-wsl": "^3.1.0" } }, "sha512-h3Fbisa2nKGPxCpm89Hk33lBLsnaGBvctQopaBSOW/uIs6FTe1ATyAnKFJrzVs9vpGdsTe73WF3V4lIsk4Gacw=="],
"wsl-utils": ["wsl-utils@1.0.0", "", { "dependencies": { "is-wsl": "^3.1.0", "powershell-utils": "^0.1.0" } }, "sha512-Hl0ZOAs672vg+06kfujwRhoS6/jehvULrlFkuF2dRu6pHgA8U06h3xqNIqNNU1LTXPcedxByAR4GS6pwQK0mgA=="],
"xdg-basedir": ["xdg-basedir@5.1.0", "", {}, "sha512-GCPAHLvrIH13+c0SuacwvRYj2SxJXQ4kaVTT5xgL3kPrz56XxkF21IGhjSE1+W0aw7gpBWRGXLCPnPby6lSpmQ=="],
@@ -5910,8 +5916,6 @@
"@astrojs/telemetry/ci-info": ["ci-info@4.4.0", "", {}, "sha512-77PSwercCZU2Fc4sX94eF8k8Pxte6JAwL4/ICZLFjJLqegs7kCuAsqqj/70NQF6TvDpgFjkubQB2FW2ZZddvQg=="],
"@astrojs/telemetry/is-docker": ["is-docker@4.0.0", "", { "bin": { "is-docker": "cli.js" } }, "sha512-LHE+wROyG/Y/0ZnbktRCoTix2c1RhgWaZraMZ8o1Q7zCh0VSrICJQO5oqIIISrcSBtrXv0o233w1IYwsWCjTzA=="],
"@aws-crypto/crc32/@aws-sdk/types": ["@aws-sdk/types@3.974.4", "", { "dependencies": { "@smithy/types": "^4.16.1", "tslib": "^2.6.2" } }, "sha512-dSFDNG00MEz0/xl5gxL62giLd1iYyJsTxZ1I1DOj6lC+bbgLB4TRsYClJg3b62dhXT1uATzsTNXPnC+33EJV3A=="],
"@aws-crypto/crc32c/@aws-sdk/types": ["@aws-sdk/types@3.974.4", "", { "dependencies": { "@smithy/types": "^4.16.1", "tslib": "^2.6.2" } }, "sha512-dSFDNG00MEz0/xl5gxL62giLd1iYyJsTxZ1I1DOj6lC+bbgLB4TRsYClJg3b62dhXT1uATzsTNXPnC+33EJV3A=="],
@@ -6394,8 +6398,6 @@
"builder-util/js-yaml": ["js-yaml@4.3.1", "", { "dependencies": { "argparse": "^2.0.1" }, "bin": { "js-yaml": "bin/js-yaml.js" } }, "sha512-CY6crGq313MX8GkwvB7tzgp99vjQxY1++5y10/BKN/GUfHqWaOGQMNZkBvqSzsZKWk/ijwHlWzzkLulsGHhjWQ=="],
"chrome-launcher/is-wsl": ["is-wsl@2.2.0", "", { "dependencies": { "is-docker": "^2.0.0" } }, "sha512-fKzAra0rGJUUBwGBgNkHZuToZcn+TtXHpeCgmkMJMMYx1sQDYaCSyjJBSCa2nH1DGm7s3n1oBnohoVTBaN7Lww=="],
"chromium-bidi/zod": ["zod@3.25.76", "", {}, "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ=="],
"clean-css/source-map": ["source-map@0.6.1", "", {}, "sha512-UjgapumWlbMhkBgzT7Ykc5YXUT46F0iKu8SGXq0bcwP5dz/h0Plj6enJqjz1Zbq2l5WaqYnrVbwWOWMyF3F47g=="],
@@ -6500,6 +6502,10 @@
"import-in-the-middle/es-module-lexer": ["es-module-lexer@2.3.2", "", {}, "sha512-poHGpORABojJJucnV9KbOavETW8lBVnphkW77ER5/BQ5Fz7oXSoCNek7IH3vR5nRjdsEz926ibFYX8KtLQmdyw=="],
"is-inside-container/is-docker": ["is-docker@3.0.0", "", { "bin": { "is-docker": "cli.js" } }, "sha512-eljcgEDlEns/7AXFosB5K/2nCM4P7FQPkGc/DWLy5rmFEWvZayGrik1d9/QIY5nJ4f9YsVvBkA6kJpHn9rISdQ=="],
"is-wsl/is-docker": ["is-docker@2.2.1", "", { "bin": { "is-docker": "cli.js" } }, "sha512-F+i2BKsFrH66iaUFc0woD8sLy8getkwTwtOBjvs56Cx4CgJDeKQeqfz8wAYiSb8JOprWhHH5p77PbmYCvvUuXQ=="],
"js-beautify/glob": ["glob@10.5.0", "", { "dependencies": { "foreground-child": "^3.1.0", "jackspeak": "^3.1.2", "minimatch": "^9.0.4", "minipass": "^7.1.2", "package-json-from-dist": "^1.0.0", "path-scurry": "^1.11.1" }, "bin": { "glob": "dist/esm/bin.mjs" } }, "sha512-DfXN8DfhJ7NH3Oe7cFmu3NCu1wKbkReJ8TorzSAFbSKrlNaQSKfIzqYqVY8zlbs2NLBbWpRiU52GX2PbaBVNkg=="],
"js-beautify/nopt": ["nopt@7.2.1", "", { "dependencies": { "abbrev": "^2.0.0" }, "bin": { "nopt": "bin/nopt.js" } }, "sha512-taM24ViiimT/XntxbPyJQzCG+p4EKOpgD3mxFwW38mGjVUrfERQOeY4EDHjdnptttfHuHQXFx+lTP08Q+mLa/w=="],
@@ -6510,8 +6516,6 @@
"lighthouse/devtools-protocol": ["devtools-protocol@0.0.1663043", "", {}, "sha512-33aOY3ZnBP1dgZsshgaL+/XlsQleiFZgyUaDtdZkEa1nbZhVY1MoDeWjk+wxg25fU924l1ZJfoGNmjjeA/5s1w=="],
"lighthouse/open": ["open@8.4.2", "", { "dependencies": { "define-lazy-prop": "^2.0.0", "is-docker": "^2.1.1", "is-wsl": "^2.2.0" } }, "sha512-7x81NCL719oNbsq/3mh+hVrAWmFuEYUqrq/Iw3kUzH8ReypT9QQ0BLoJS7/G9k6N81XjW4qHWtjWwe/9eLy1EQ=="],
"lighthouse/ws": ["ws@7.5.13", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": "^5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-rsKI6xDBFVf4r/x8XyChGK04QR/XHroxs/jUcoWvtEZM8TPU/X/uIY9B1CsSzYws9ZJb/6bbBu7dPhFW00CAoA=="],
"md-to-react-email/marked": ["marked@7.0.4", "", { "bin": { "marked": "bin/marked.js" } }, "sha512-t8eP0dXRJMtMvBojtkcsA7n48BkauktUKzfkPSCq85ZMTJ0v76Rke4DYz01omYpPTUh4p/f7HePgRo3ebG8+QQ=="],
@@ -6614,8 +6618,6 @@
"sst/jose": ["jose@5.2.3", "", {}, "sha512-KUXdbctm1uHVL8BYhnyHkgp3zDX5KW8ZhAKVFEfUbU2P8Alpzjb+48hHvjOdQIyPshoblhzsuqOwEEAbtHVirA=="],
"storybook/open": ["open@10.2.0", "", { "dependencies": { "default-browser": "^5.2.1", "define-lazy-prop": "^3.0.0", "is-inside-container": "^1.0.0", "wsl-utils": "^0.1.0" } }, "sha512-YgBpdJHPyQ2UE5x+hlSXcnejzAvD0b22U2OuAP+8OnlJT+PjWPxtgmGqKKc+RgTM63U9gN0YzrYc71R2WT/hTA=="],
"storybook-solidjs-vite/semver": ["semver@7.8.1", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-rkVq3IXh+4FDGch+KwzX3aV9W3kO54GyEgpvBzSyctDA6Xtd7RJQV1xmXbeQp5v7+VzLOfVqiutSE6GICgPFvg=="],
"storybook-solidjs-vite/vite": ["vite@7.1.11", "", { "dependencies": { "esbuild": "^0.25.0", "fdir": "^6.5.0", "picomatch": "^4.0.3", "postcss": "^8.5.6", "rollup": "^4.43.0", "tinyglobby": "^0.2.15" }, "optionalDependencies": { "fsevents": "~2.3.3" }, "peerDependencies": { "@types/node": "^20.19.0 || >=22.12.0", "jiti": ">=1.21.0", "less": "^4.0.0", "lightningcss": "^1.21.0", "sass": "^1.70.0", "sass-embedded": "^1.70.0", "stylus": ">=0.54.8", "sugarss": "^5.0.0", "terser": "^5.16.0", "tsx": "^4.8.1", "yaml": "^2.4.2" }, "optionalPeers": ["@types/node", "jiti", "less", "lightningcss", "sass", "sass-embedded", "stylus", "sugarss", "terser", "tsx", "yaml"], "bin": { "vite": "bin/vite.js" } }, "sha512-uzcxnSDVjAopEUjljkWh8EIrg6tlzrjFUfMcR1EVsRDGwf/ccef0qQPRyOrROwhrTDaApueq+ja+KLPlzR/zdg=="],
@@ -6702,6 +6704,10 @@
"write-file-atomic/signal-exit": ["signal-exit@4.1.0", "", {}, "sha512-bzyZ1e88w9O1iNJbKnOlvYTrWPDl46O1bG0D3XInv+9tkPrxrN8jUUTiFlDkkmKWgn1M6CfIA13SuGqOa9Korw=="],
"wsl-utils/is-wsl": ["is-wsl@3.1.1", "", { "dependencies": { "is-inside-container": "^1.0.0" } }, "sha512-e6rvdUCiQCAuumZslxRJWR/Doq4VpPR82kqclvcS0efgt430SlGIk05vdCN58+VrzgtIcfNODjozVielycD4Sw=="],
"wsl-utils/powershell-utils": ["powershell-utils@0.1.0", "", {}, "sha512-dM0jVuXJPsDN6DvRpea484tCUaMiXWjuCn++HGTqUWzGDjv5tZkEZldAJ/UMlqRYGFrD/etByo4/xOuC/snX2A=="],
"yaml-language-server/prettier": ["prettier@3.9.6", "", { "bin": { "prettier": "bin/prettier.cjs" } }, "sha512-OpN0zzVdiaiAhxpuuj5efpIS4sY9j7bY6uR5mnj5yPzGkdkjNKSJeUThPb60Jw29QuAZgA4o+/iB49kFiaBX6g=="],
"yaml-language-server/request-light": ["request-light@0.5.8", "", {}, "sha512-3Zjgh+8b5fhRJBQZoy+zbVKpAQGLyka0MPgW3zruTF4dFFJ8Fqcfu9YsAvi/rvdcaTeWG3MkbZv4WKxAn/84Lg=="],
@@ -7336,8 +7342,6 @@
"builder-util/js-yaml/argparse": ["argparse@2.0.1", "", {}, "sha512-8+9WqebbFzpX9OR+Wa6O29asIogeRMzcGtAINdpMHHyAg10f05aSFVBbcEqGf/PXw1EjAZ+q2/bEBg3DvurK3Q=="],
"chrome-launcher/is-wsl/is-docker": ["is-docker@2.2.1", "", { "bin": { "is-docker": "cli.js" } }, "sha512-F+i2BKsFrH66iaUFc0woD8sLy8getkwTwtOBjvs56Cx4CgJDeKQeqfz8wAYiSb8JOprWhHH5p77PbmYCvvUuXQ=="],
"cliui/string-width/emoji-regex": ["emoji-regex@8.0.0", "", {}, "sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A=="],
"cliui/strip-ansi/ansi-regex": ["ansi-regex@5.0.1", "", {}, "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ=="],
@@ -7398,12 +7402,6 @@
"lazystream/readable-stream/string_decoder": ["string_decoder@1.1.1", "", { "dependencies": { "safe-buffer": "~5.1.0" } }, "sha512-n/ShnvDi6FHbbVfviro+WojiFzv+s8MPMHBczVePfUpDJLwoLT0ht1l4YwBCbi8pJAveEEdnkHyPyTP/mzRfwg=="],
"lighthouse/open/define-lazy-prop": ["define-lazy-prop@2.0.0", "", {}, "sha512-Ds09qNh8yw3khSjiJjiUInaGX9xlqZDY7JVryGxdxV7NPeuqQfplOpQ66yJFZut3jLa5zOwkXw1g9EI2uKh4Og=="],
"lighthouse/open/is-docker": ["is-docker@2.2.1", "", { "bin": { "is-docker": "cli.js" } }, "sha512-F+i2BKsFrH66iaUFc0woD8sLy8getkwTwtOBjvs56Cx4CgJDeKQeqfz8wAYiSb8JOprWhHH5p77PbmYCvvUuXQ=="],
"lighthouse/open/is-wsl": ["is-wsl@2.2.0", "", { "dependencies": { "is-docker": "^2.0.0" } }, "sha512-fKzAra0rGJUUBwGBgNkHZuToZcn+TtXHpeCgmkMJMMYx1sQDYaCSyjJBSCa2nH1DGm7s3n1oBnohoVTBaN7Lww=="],
"miniflare/sharp/@img/sharp-darwin-arm64": ["@img/sharp-darwin-arm64@0.33.5", "", { "optionalDependencies": { "@img/sharp-libvips-darwin-arm64": "1.0.4" }, "os": "darwin", "cpu": "arm64" }, "sha512-UT4p+iz/2H4twwAoLCqfA9UH5pI6DggwKEGuaPy7nCVQ8ZsiY5PIcrRvD1DzuY3qYL07NtIQcWnBSY/heikIFQ=="],
"miniflare/sharp/@img/sharp-darwin-x64": ["@img/sharp-darwin-x64@0.33.5", "", { "optionalDependencies": { "@img/sharp-libvips-darwin-x64": "1.0.4" }, "os": "darwin", "cpu": "x64" }, "sha512-fyHac4jIc1ANYGRDxtiqelIbdWkIuQaI84Mv45KvGRRxSAa7o7d1ZKAOBaYbnepLC1WqxfpimdeWfvqqSGwR2Q=="],
@@ -7674,6 +7672,10 @@
"@astrojs/starlight/@astrojs/mdx/@astrojs/markdown-remark/shiki": ["shiki@3.23.0", "", { "dependencies": { "@shikijs/core": "3.23.0", "@shikijs/engine-javascript": "3.23.0", "@shikijs/engine-oniguruma": "3.23.0", "@shikijs/langs": "3.23.0", "@shikijs/themes": "3.23.0", "@shikijs/types": "3.23.0", "@shikijs/vscode-textmate": "^10.0.2", "@types/hast": "^3.0.4" } }, "sha512-55Dj73uq9ZXL5zyeRPzHQsK7Nbyt6Y10k5s7OjuFZGMhpp4r/rsLBH0o/0fstIzX1Lep9VxefWljK/SKCzygIA=="],
"@astrojs/starlight/astro/@astrojs/telemetry/is-docker": ["is-docker@3.0.0", "", { "bin": { "is-docker": "cli.js" } }, "sha512-eljcgEDlEns/7AXFosB5K/2nCM4P7FQPkGc/DWLy5rmFEWvZayGrik1d9/QIY5nJ4f9YsVvBkA6kJpHn9rISdQ=="],
"@astrojs/starlight/astro/@astrojs/telemetry/is-wsl": ["is-wsl@3.1.1", "", { "dependencies": { "is-inside-container": "^1.0.0" } }, "sha512-e6rvdUCiQCAuumZslxRJWR/Doq4VpPR82kqclvcS0efgt430SlGIk05vdCN58+VrzgtIcfNODjozVielycD4Sw=="],
"@astrojs/starlight/astro/p-queue/p-timeout": ["p-timeout@6.1.4", "", {}, "sha512-MyIV3ZA/PmyBN/ud8vV9XzwTrNtR4jFrObymZYnZqMmW0zA8Z17vnT0rBgFE/TlohB+YCHqXMgZzb3Csp49vqg=="],
"@astrojs/starlight/astro/sharp/@img/sharp-darwin-arm64": ["@img/sharp-darwin-arm64@0.33.5", "", { "optionalDependencies": { "@img/sharp-libvips-darwin-arm64": "1.0.4" }, "os": "darwin", "cpu": "arm64" }, "sha512-UT4p+iz/2H4twwAoLCqfA9UH5pI6DggwKEGuaPy7nCVQ8ZsiY5PIcrRvD1DzuY3qYL07NtIQcWnBSY/heikIFQ=="],
@@ -8076,6 +8078,10 @@
"@opencode/web/@astrojs/cloudflare/wrangler/workerd": ["workerd@1.20260708.1", "", { "optionalDependencies": { "@cloudflare/workerd-darwin-64": "1.20260708.1", "@cloudflare/workerd-darwin-arm64": "1.20260708.1", "@cloudflare/workerd-linux-64": "1.20260708.1", "@cloudflare/workerd-linux-arm64": "1.20260708.1", "@cloudflare/workerd-windows-64": "1.20260708.1" }, "bin": { "workerd": "bin/workerd" } }, "sha512-WAK+Kt/VVCSldH2qSr8lx46XCJ4Q+bdlHNaFqUtOHthBEIB8C1N8HVW+VOLrxDoTCk0NGNv0zajnBeQK4JOB9w=="],
"@opencode/web/astro/@astrojs/telemetry/is-docker": ["is-docker@3.0.0", "", { "bin": { "is-docker": "cli.js" } }, "sha512-eljcgEDlEns/7AXFosB5K/2nCM4P7FQPkGc/DWLy5rmFEWvZayGrik1d9/QIY5nJ4f9YsVvBkA6kJpHn9rISdQ=="],
"@opencode/web/astro/@astrojs/telemetry/is-wsl": ["is-wsl@3.1.1", "", { "dependencies": { "is-inside-container": "^1.0.0" } }, "sha512-e6rvdUCiQCAuumZslxRJWR/Doq4VpPR82kqclvcS0efgt430SlGIk05vdCN58+VrzgtIcfNODjozVielycD4Sw=="],
"@opencode/web/astro/js-yaml/argparse": ["argparse@2.0.1", "", {}, "sha512-8+9WqebbFzpX9OR+Wa6O29asIogeRMzcGtAINdpMHHyAg10f05aSFVBbcEqGf/PXw1EjAZ+q2/bEBg3DvurK3Q=="],
"@opencode/web/astro/p-queue/p-timeout": ["p-timeout@6.1.4", "", {}, "sha512-MyIV3ZA/PmyBN/ud8vV9XzwTrNtR4jFrObymZYnZqMmW0zA8Z17vnT0rBgFE/TlohB+YCHqXMgZzb3Csp49vqg=="],
@@ -8216,6 +8222,10 @@
"archiver-utils/glob/path-scurry/lru-cache": ["lru-cache@10.4.3", "", {}, "sha512-JNAzZcXrCt42VGLuYz0zfAzDfAvJWW6AfYlDBQyDV5DClI2m5sAmK+OIO7s59XfsRsWHp02jAJrRadPRGTt6SQ=="],
"astro-expressive-code/astro/@astrojs/telemetry/is-docker": ["is-docker@3.0.0", "", { "bin": { "is-docker": "cli.js" } }, "sha512-eljcgEDlEns/7AXFosB5K/2nCM4P7FQPkGc/DWLy5rmFEWvZayGrik1d9/QIY5nJ4f9YsVvBkA6kJpHn9rISdQ=="],
"astro-expressive-code/astro/@astrojs/telemetry/is-wsl": ["is-wsl@3.1.1", "", { "dependencies": { "is-inside-container": "^1.0.0" } }, "sha512-e6rvdUCiQCAuumZslxRJWR/Doq4VpPR82kqclvcS0efgt430SlGIk05vdCN58+VrzgtIcfNODjozVielycD4Sw=="],
"astro-expressive-code/astro/js-yaml/argparse": ["argparse@2.0.1", "", {}, "sha512-8+9WqebbFzpX9OR+Wa6O29asIogeRMzcGtAINdpMHHyAg10f05aSFVBbcEqGf/PXw1EjAZ+q2/bEBg3DvurK3Q=="],
"astro-expressive-code/astro/p-queue/p-timeout": ["p-timeout@6.1.4", "", {}, "sha512-MyIV3ZA/PmyBN/ud8vV9XzwTrNtR4jFrObymZYnZqMmW0zA8Z17vnT0rBgFE/TlohB+YCHqXMgZzb3Csp49vqg=="],
@@ -8318,6 +8328,10 @@
"temp/rimraf/glob/minimatch": ["minimatch@3.1.5", "", { "dependencies": { "brace-expansion": "^1.1.7" } }, "sha512-VgjWUsnnT6n+NUk6eZq77zeFdpW2LWDzP6zFGrCbHXiYNul5Dzqk2HHQ5uFH2DNW5Xbp8+jVzaeNt94ssEEl4w=="],
"toolbeam-docs-theme/astro/@astrojs/telemetry/is-docker": ["is-docker@3.0.0", "", { "bin": { "is-docker": "cli.js" } }, "sha512-eljcgEDlEns/7AXFosB5K/2nCM4P7FQPkGc/DWLy5rmFEWvZayGrik1d9/QIY5nJ4f9YsVvBkA6kJpHn9rISdQ=="],
"toolbeam-docs-theme/astro/@astrojs/telemetry/is-wsl": ["is-wsl@3.1.1", "", { "dependencies": { "is-inside-container": "^1.0.0" } }, "sha512-e6rvdUCiQCAuumZslxRJWR/Doq4VpPR82kqclvcS0efgt430SlGIk05vdCN58+VrzgtIcfNODjozVielycD4Sw=="],
"toolbeam-docs-theme/astro/js-yaml/argparse": ["argparse@2.0.1", "", {}, "sha512-8+9WqebbFzpX9OR+Wa6O29asIogeRMzcGtAINdpMHHyAg10f05aSFVBbcEqGf/PXw1EjAZ+q2/bEBg3DvurK3Q=="],
"toolbeam-docs-theme/astro/p-queue/p-timeout": ["p-timeout@6.1.4", "", {}, "sha512-MyIV3ZA/PmyBN/ud8vV9XzwTrNtR4jFrObymZYnZqMmW0zA8Z17vnT0rBgFE/TlohB+YCHqXMgZzb3Csp49vqg=="],
Generated
+3 -3
View File
@@ -2,11 +2,11 @@
"nodes": {
"nixpkgs": {
"locked": {
"lastModified": 1776683584,
"narHash": "sha256-NuTLMrr10Tng72hurYG8jYQ4XKK8wnpJmOGcPiis96g=",
"lastModified": 1790510107,
"narHash": "sha256-EVMNYv7hYDDD9TGVT/hIyTYgpiXA8y3m5xIEIxuGNU0=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "9dd5558b06dbdacbf635a3dd36dce1b1a7ee3a89",
"rev": "3181085bfd08663b6b9e60bc7a8395c2aaa741bd",
"type": "github"
},
"original": {
+4 -4
View File
@@ -1,8 +1,8 @@
{
"nodeModules": {
"x86_64-linux": "sha256-aQQQhaUlAhpfqzH0vNi0IJ1cg7FQHIKYzxeq5d8PZoU=",
"aarch64-linux": "sha256-r9aDFu3UYmudmmYPhzCrpFvQlaejXc8V1IzLtG3jZPc=",
"aarch64-darwin": "sha256-B0m41LelD7d61vPHGIZZSO/cU7gbjHDJHt6oxNRRM8Q=",
"x86_64-darwin": "sha256-9TWJsyI3Y6BMomtGSgqA1th9LpxpqP4F5Tl/GyexVYw="
"x86_64-linux": "sha256-7DgxTpKv6ITKTom0mhlJNPQfdEjtEHK5P8uyCr26HZw=",
"aarch64-linux": "sha256-PJxW1Ibfx6oS1neWPSHqzP1Pm1HT9my1TrrHmOb6/no=",
"aarch64-darwin": "sha256-VIme5VHfM8JxNiDSOykkr5FytghDLI0FxkhiOXUSyQw=",
"x86_64-darwin": "sha256-rQ/j0QkR1vxAq4jgUbr0nY4RDyiLTJqN8q1AfoiEqVQ="
}
}
+2 -1
View File
@@ -2,7 +2,7 @@
"$schema": "https://json.schemastore.org/package.json",
"name": "opencode",
"description": "AI-powered development tool",
"version": "2.0.16",
"version": "2.0.18",
"private": true,
"type": "module",
"packageManager": "bun@1.4.2",
@@ -162,6 +162,7 @@
"@types/node": "catalog:",
"bun-types": "1.4.2",
"effect": "catalog:",
"open": "11.0.4",
"solid-js": "catalog:"
},
"patchedDependencies": {
+5 -5
View File
@@ -12,9 +12,9 @@
Per-type constructors live on the type, not as top-level re-exports. Use `Message.system(...)`, `Message.user(...)`, `Message.assistant(...)`, `Message.tool(...)`, `Message.media(...)`, `LanguageModel.make(...)`, `ToolDefinition.make(...)`, `ToolCallPart.make(...)`, `ToolResultPart.make(...)`, `ToolChoice.make(...)`, `ToolChoice.named(...)`, `SystemPart.make(...)`, and `GenerationOptions.make(...)` directly. The top-level `LLM` namespace is reserved for request-shaped call APIs: `LLM.request`, `LLM.generate`, `LLM.stream`, and `LLM.generateObject`. `LLM.generate`/`LLM.stream` and Promise `ai.llm.generate`/`ai.llm.stream` accept ergonomic input or an `LLMRequest`; both paths use the same canonical request. Core still builds, logs, replays, and updates that durable `LLMRequest` boundary. Use `LLMRequest.update(...)` when deriving canonical request data; do not add a duplicate `LLM.updateRequest(...)` path.
Modality namespaces mirror `LLM` exactly: `Image.request`, `Image.generate`, `Image.stream` (later `Video`, `Speech`, `Transcription`). Common request fields (`images`, `mask`, `n`, `size`, `aspectRatio`, `seed`, `format`) lower natively or fail with a typed `AIError`; provider-native controls always live under `providerOptions`, never under a modality-specific `options` key.
Modality namespaces mirror `LLM` exactly: `Image.request`, `Image.generate`, `Image.stream`, and the same for `Video`, `Speech`, and `Transcription`. Common request fields (`images`, `mask`, `n`, `size`, `aspectRatio`, `seed`, `format`) lower natively or fail with a typed `AIError`; provider-native controls always live under `providerOptions`, never under a modality-specific `options` key.
Media payloads are always `Media.Asset` (`src/media.ts`). Construct them with `Media.bytes`, `Media.base64`, `Media.url`, `Media.ref`, `Media.fromDataUrl`, or `Media.file`; never introduce a parallel `data: string | Uint8Array` shape. `MediaPart.media`, `ImageRequest.images`/`mask`, `ImageResponse.images`, and the `media` `LLMEvent` all share it. Protocols branch on `asset.source.type` and `asset.kind` and use `ProviderShared.inlineMedia` / `requireInlineMedia` / `mediaUrl` / `MediaInput.refID` rather than re-deriving base64 or URL handling.
Media payloads are always `Media.Asset` (`src/media.ts`). Construct them with `Media.bytes`, `Media.base64`, `Media.url`, `Media.ref`, `Media.fromDataUrl`, or `Media.file`; never introduce a parallel `data: string | Uint8Array` shape. `MediaPart.media`, `ImageRequest.images`/`mask`, `ImageResponse.images`, and the `media` `LLMEvent` all share it. Protocols branch on `asset.source.type` and `asset.kind` and use `ProviderShared.requireInlineMedia` / `inlineRequired` / `mediaUrl` / `mediaReference` and `MediaInput.inlineBytes` / `refID` rather than re-deriving base64 or URL handling.
`schema/messages.ts → media.ts → route/executor-service.ts` is an accepted runtime dependency from the schema layer on the executor service tag: `Media.Asset.bytes()` must be able to download `url` sources, and the tag lives in that leaf module precisely so the schema barrel never imports the executor implementation (which imports the schema barrel back). Do not move the tag into `route/executor.ts` or import `route/executor.ts` from `src/schema/*` or `src/media.ts`.
@@ -98,11 +98,11 @@ When a provider supports multiple physical transports, selection remains executi
Media does not fit the SSE-frames-to-event-state-machine LLM route. `MediaRoute.inline(...)` / `queued(...)` / `stream(...)` (`src/route/media.ts`) compose a `MediaProtocol` kind with `Endpoint` and `Auth` and own the transport plumbing: `http` option merging, URL/query rendering, auth headers, JSON vs multipart encoding, and handing the response back to the protocol. `MediaProtocol.inline` (`src/route/media-protocol.ts`) is `body.from(request)` plus `response.decode(response, context)`; each protocol declares `const route = MediaProtocol.identity({ id, name, provider })` once and decodes through `route.decodeJson` / `route.text` / `route.decodeStarted` so decode failures retain the raw body and HTTP context, raising `route.unsupported(operation, message)` for requests it cannot lower, and passes `route` as the first argument to `MediaProtocol.inline` / `queued` / `stream`. `Generation` (`src/generation.ts`) is the provider-neutral handle for a queued generation over a `GenerationRoute` (`status`, `result`, `cancel`). Image protocol files follow the same section order as LLM protocols and declare unsupported common fields once through the protocol's `unsupported` list.
`MediaProtocol.queued` is the submit-then-poll kind every video route uses: `start` (body + decode into `{ token, snapshot }`), `status`, `result`, and optional `cancel`, each addressed by a route-owned `token` whose `Schema.Codec` makes it serializable. `MediaRoute.inline` and `MediaRoute.queued` compose the two kinds with `Endpoint` and `Auth`; the queued route decodes the token once at the boundary (`start` output or `resume` input) and closes over it in a token-free `GenerationRoute` (`status`/`result`/`cancel` are plain Effects), so `Generation` never sees the token's shape and only carries the encoded JSON for persistence. Polls reuse the route's auth and deployment headers plus the request's `http` overlay after `start`, and resolve relative paths against the route base URL (provider-issued absolute URLs such as fal's `status_url` pass through). `result` is always its own GET even when the provider returns output inside the status document, so `Generation.await` behaves the same after `start` and after `resume`. `PollContext.auth` carries only what `Auth` added or changed so protocols can hand download credentials to output assets as transient `Media.Asset.headers` (Veo) — never part of `source` or JSON. Status strings map through a per-protocol `STATUS` table via `MediaProtocol.status`; terminal generations without output fail through `output.ended` / `output.contentPolicy` with the provider document on `reason.body`. `GenerationAwaitOptions` (`AwaitOptions` in `src/generation.ts`, `{ poll?: Poll }`) is the one options type for `await`, `events`, `Video.generate`, and `Video.stream`.
`MediaProtocol.queued` is the submit-then-poll kind every video route uses: `start` (body + decode into `{ token, snapshot }`), `status`, `result`, and optional `cancel` (with `activeOnly` when the provider's cancel endpoint deletes finished work, as Runway's does: the route refreshes status first and skips terminal generations), each addressed by a route-owned `token` whose `Schema.Codec` makes it serializable. `MediaRoute.inline` and `MediaRoute.queued` compose the two kinds with `Endpoint` and `Auth`; the queued route decodes the token once at the boundary (`start` output or `resume` input) and closes over it in a token-free `GenerationRoute` (`status`/`result`/`cancel` are plain Effects), so `Generation` never sees the token's shape and only carries the encoded JSON for persistence. Polls reuse the route's auth and deployment headers plus the request's `http` overlay after `start`, and resolve relative paths against the route base URL (provider-issued absolute URLs such as fal's `status_url` pass through). `result` is always its own GET even when the provider returns output inside the status document, so `Generation.await` behaves the same after `start` and after `resume`. `PollContext.auth` carries only what `Auth` added or changed so protocols can hand download credentials to output assets as transient `Media.Asset.headers` (Veo) — never part of `source` or JSON. Status strings map through a per-protocol `STATUS` table via `MediaProtocol.status`; terminal generations without output fail through `output.ended` / `output.contentPolicy` with the provider document on `reason.body`; a `failed` generation maps the provider's error code through a per-protocol `FAILURE` table via `MediaProtocol.failure` so rejected inputs are not reported as retryable `ProviderInternal`. `GenerationAwaitOptions` (`AwaitOptions` in `src/generation.ts`, `{ poll?: Poll }`) is the one options type for `await`, `events`, `Video.generate`, and `Video.stream`.
`MediaProtocol.stream` is the incremental kind every speech route uses, with the same discipline as LLM protocols. `MediaRoute.stream` submits the caller's request as `MediaProtocol.Addressed<Request>` (`{ ...request, mode }`, `mode: "generate" | "stream"`), so one provider stays one protocol: `body.from`, the endpoint path, and `frames` read `request.mode` to pick the body, path, and framing. `frames(bytes, context)` returns frames — `Framing.sse`, `Framing.lines`, `Framing.document` (a single-document response shaped like a streamed record), or the raw `bytes` for chunked audio. `initial()` is fresh per-response parser state; `step` folds each frame into it and emits modality events; `finish(state, context)` runs once after the last frame with the request, body, and observed `http` (header-only usage lives there) and emits exactly one terminal event or fails with `route.incomplete()`. Keep parser state to real accumulators and derive anything the request or body determines in `finish`. `generate` runs the same stream and folds it with the modality's `collect`. Request-derived URL parameters go on the body's `query` (array values repeat the parameter), applied before route and caller `http.query`. Decode frames with `route.decodeFrame` and raise stream-time failures with `route.frameError` (the frame stays on `reason.body`); protocols never thread HTTP context, because the route fills `reason.http` on stream errors that lack it. Speech protocols share `protocols/utils/speech-stream.ts` for deltas, timestamps, voice ids, PCM and container descriptions, and the terminal asset.
Every modality route is the inline | stream | queued union (transcription uses all three: OpenAI and Gemini stream, Deepgram is inline, AssemblyAI is queued), every client is `MediaClient.make(Service, { modality, responseEvents })` (`src/media-client.ts`), which dispatches on the route's `kind`, and every model composes through `composeRoute`. fal queue protocols come from `protocols/utils/fal-queue.ts`, bodies are `json`, `multipart`, or `binary` (a raw upload), and a queued protocol that must upload media before submitting implements `start.prepare` (`MediaProtocol.Prepare`; AssemblyAI `/v2/upload`).
Every modality route is the inline | stream | queued union (transcription uses all three: OpenAI and Gemini stream, Deepgram and ElevenLabs are inline, AssemblyAI is queued), every client is `MediaClient.make(Service, { modality, responseEvents })` (`src/media-client.ts`), which dispatches on the route's `kind`, and every model composes through `composeRoute`. fal queue protocols come from `protocols/utils/fal-queue.ts`, bodies are `json`, `multipart`, or `binary` (a raw upload), and a queued protocol that must upload media before submitting implements `start.prepare` (`MediaProtocol.Prepare`; AssemblyAI `/v2/upload`).
### URL Construction
@@ -112,7 +112,7 @@ For providers where the URL is derived from typed inputs (Azure resource name, B
### Provider Facades
Provider-facing APIs are configured facades over route values. Endpoint/auth/resource/API-version setup happens before model selection, and model selectors accept only a model or deployment id. Media models use per-modality selectors on the same facade (`openai.image(id)`, later `.video` / `.speech` / `.transcription`) that mirror `openai.responses(id)`; the one-word overlap with the request namespace is accepted over a second construction path:
Provider-facing APIs are configured facades over route values. Endpoint/auth/resource/API-version setup happens before model selection, and model selectors accept only a model or deployment id. Media models use per-modality selectors on the same facade (`openai.image(id)`, `.speech(id)`, `.transcription(id)`, `google.video(id)`) that mirror `openai.responses(id)`; the one-word overlap with the request namespace is accepted over a second construction path:
```ts
const openai = OpenAI.configure({ apiKey, baseURL })
+41 -36
View File
@@ -475,18 +475,18 @@ const program = Effect.gen(function* () {
Common fields are portable in shape, not in support. Unsupported fields fail with a typed `AIError` before any network
call rather than being dropped, so check this table before swapping only the `model`:
| Provider | `n` | `size` | `aspectRatio` | `seed` | `format` | `images` | `mask` |
| --------------------- | --- | --------- | ------------- | ------ | -------- | ------------------------- | ------------------- |
| OpenAI | ✓¹ | ✓ | ✗ | ✗ | ✓ | ✓ | ✓ |
| Google (Gemini) | 1 | ✗ | ✓ | ✓ | ✗ | ✓ (no public URLs) | ✗ |
| xAI | ✓ | ✗ | ✓ | ✗ | ✗ | ✓ | ✗ |
| Z.ai | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ |
| Meta | ✓ | ✓ (hint) | ✗ | ✗ | ✓ | ✓ | ✗ |
| Black Forest Labs | 1 | per model | per model | ✓ | ✓ | per model (1–8) | `flux-pro-1.0-fill` |
| fal | ✓ | per model | per model | ✓ | ✓ | 1 (several on `/edit`) | ✓ |
| Replicate | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ (use `providerOptions`) | ✗ |
| Stability `image` | 1 | ✗ | ✓ | ✓ | ✓ | 1 (not on `core`) | ✗ |
| Stability `upscale()` | ✗ | ✗ | ✗ | ✓ | ✓ | exactly 1 (required) | ✗ |
| Provider | `n` | `size` | `aspectRatio` | `seed` | `format` | `images` | `mask` |
| --------------------- | --- | --------- | ------------- | ------ | -------- | -------------------------------- | ------------------- |
| OpenAI | ✓¹ | ✓ | ✗ | ✗ | ✓ | ✓ | ✓ |
| Google (Gemini) | 1 | ✗ | ✓ | ✓ | ✗ | ✓ (no public URLs) | ✗ |
| xAI | ✓ | ✗ | ✓ | ✗ | ✗ | ✓ | ✗ |
| Z.ai | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ |
| Meta | ✓ | ✓ (hint) | ✗ | ✗ | ✓ | ✓ | ✗ |
| Black Forest Labs | 1 | per model | per model | ✓ | ✓ | per model (1–8) | `flux-pro-1.0-fill` |
| fal | ✓ | per model | per model | ✓ | ✓ | 1 (several on `/edit`, `/multi`) | ✓ |
| Replicate | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ (use `providerOptions`) | ✗ |
| Stability `image` | 1 | ✗ | ✓ | ✓ | ✓ | 1 (not on `core`) | ✗ |
| Stability `upscale()` | ✗ | ✗ | ✗ | ✓ | ✓ | exactly 1 (required) | ✗ |
✓ lowers natively; ✗ fails whenever the field is set (including `n: 1`); `1` means `n > 1` fails. ¹ `Image.stream` on OpenAI generates one image. fal
rejects `size` and `aspectRatio` together; which one a fal or BFL model takes depends on the model.
@@ -621,8 +621,7 @@ persist the bytes promptly if they must remain available.
### Partial images
OpenAI's GPT image models stream previews. `Image.stream` sends `stream: true` with `partialImages` (0–3, default 2)
and emits `image-partial` events before each final `image`; `Image.generate` keeps the plain JSON request.
`dall-e-*` models do not stream and fail typed:
and emits `image-partial` events before each final `image`; `Image.generate` keeps the plain JSON request:
```ts
import { Stream } from "effect"
@@ -699,7 +698,7 @@ const program = Effect.gen(function* () {
})
```
The hosted result is represented as a provider-executed tool call and tool result, and the generated image is also emitted as a first-class `media` `LLMEvent` (`response.message` then carries a `media` part). Gemini image-capable models emit the same `media` event for inline image output. Retaining `response.message` preserves the generated image for continuation on both routes.
The hosted result is represented as a provider-executed tool call and a tool result whose content carries the generated image as a file. Gemini image-capable models instead emit a first-class `media` `LLMEvent` for inline image output (`response.message` then carries a `media` part). Retaining `response.message` preserves the generated image for continuation on both routes.
## Video generation
@@ -753,7 +752,10 @@ const events = Video.stream({ model: Runway.configure({ apiKey }).video("gen4.5"
Status polls, result fetches, cancels, and asset downloads all run through the same request executor with the route's
auth. `Generation.await` and `Generation.events` fail with a
`Timeout` reason when `poll.timeout` (default 10 minutes) elapses. Failed,
`Timeout` reason when `poll.timeout` (default 10 minutes) elapses. Status polls and result fetches retry transient
failures (rate limits, provider 5xx, network errors) with backoff that honors `retry-after`, always within
`poll.timeout`; submits and cancels never retry. Interrupting a wait (or aborting its `signal`) does not cancel the
provider job, which keeps running and billing: call `cancel()` to stop it. Failed,
cancelled, and expired generations fail typed with the provider's terminal document on `reason.body`; moderation
outcomes (Veo `raiMediaFilteredReasons`, xAI `respect_moderation`, Runway `SAFETY.*` codes) surface as `notices` when
a video is still returned and as a `ContentPolicy` reason when nothing is.
@@ -774,7 +776,9 @@ Provider notes:
The promise client exposes the same surface: `ai.video.start(...)` resolves to a handle with `await`, `events`,
`result`, `refresh`, `cancel`, and `token`; `ai.video.generate`, `ai.video.resume(model, token)`, and
`ai.video.stream` mirror the Effect API. The handle's `status` and `progress` are a snapshot from when it was
created; `refresh()` resolves to a new handle.
created; `refresh()` resolves to a new handle. Every promise method and stream accepts `{ signal }`: like `fetch`,
aborting rejects the Promise or throws from the `for await` loop with `signal.reason` (an `AbortError` `DOMException`
unless `abort(reason)` passed one), while `break` stops a stream without throwing.
```ts
import { ai } from "@opencode/ai/promise"
@@ -789,7 +793,7 @@ await ai.write(video.video, "./kite.mp4")
Speech (text-to-speech) is one request whose response is parsed incrementally, so every route supports both
`Speech.generate` (the whole file) and `Speech.stream` (audio chunks as they arrive). Models come from `.speech(...)`
selectors on the `OpenAI`, `Google` (Gemini TTS), `ElevenLabs`, `Cartesia`, `Deepgram`, and `XAI` facades. Common fields
selectors on the `OpenAI`, `Google` (Gemini TTS), `ElevenLabs`, `Cartesia`, and `Deepgram` facades. Common fields
(`voice`, `format`, `speed`, `language`, `instructions`, `timestamps`) lower natively or fail with a typed `AIError`
before any network call; provider-native controls live under `providerOptions`, inferred from the selected model.
@@ -840,9 +844,10 @@ Provider notes:
- **OpenAI** streams over SSE (`stream_format: "sse"`), which is also the only place it reports token usage; `tts-1`
and `tts-1-hd` do not support SSE and stream the raw audio body instead. `pcm` is 24 kHz 16-bit mono. `language`
and `timestamps` are not supported.
- **Gemini TTS** returns raw 16-bit PCM only (`audio/L16;codec=pcm;rate=24000`), so any `format` other than `pcm`
fails typed; wrap the samples yourself. Style is directed in the text, so `instructions` and `speed` fail typed.
Only `gemini-3.1-flash-tts-preview` and later support streaming. Two-speaker audio goes through
- **Gemini TTS** returns the provider's default output: WAV for Gemini 3.8 TTS `generate`, raw 16-bit PCM
(`audio/L16;codec=pcm;rate=24000`) otherwise. `pcm` is the only explicit `format` it accepts, and it fails typed on
Gemini 3.8 `generate`; the route never wraps PCM as WAV. Style is directed in the text, so `instructions` and
`speed` fail typed. Only `gemini-3.1-flash-tts-preview` and later support streaming. Two-speaker audio goes through
`providerOptions.speechConfig.multiSpeakerVoiceConfig`.
- **ElevenLabs** requires `voice` (the path voice id) and authenticates with `xi-api-key`. `format` maps to the
`output_format` query parameter (`mp3_44100_128`, `pcm_24000`, `wav_24000`, `opus_48000_64`);
@@ -854,10 +859,6 @@ Provider notes:
- **Deepgram** Aura's voice is the model id (`aura-2-thalia-en`), so `voice` and `language` fail typed. `format`
and `providerOptions` lower to query parameters (`encoding`, `container`, `sample_rate`, `bit_rate`); `pcm` is
`linear16` without a container. Auth is `Authorization: Token <DEEPGRAM_API_KEY>`.
- **xAI** (`POST /v1/tts`) has no model field, so the `.speech(...)` id (for example `"grok-tts"`) only names the
model. `voice` is the `voice_id` (default `eve`), `language` defaults to `auto`, and `format` is the codec (`mp3`,
`wav`, `pcm`, `mulaw`, `alaw`); `providerOptions.sampleRate` and `bitRate` complete `output_format`. Both
`generate` and `stream` read the raw audio body. `instructions` and `timestamps` are not supported.
The promise client mirrors the Effect API; `ai.speech.stream` is an `AsyncIterable`.
@@ -875,11 +876,12 @@ for await (const event of ai.speech.stream({ model, text: "Hello from OpenCode."
## Transcription
Transcription (speech-to-text) is the one modality whose providers use every route kind: OpenAI and Gemini stream,
Deepgram answers inline, and AssemblyAI is queued. `Transcription.generate` and `Transcription.stream` work on all of
them; `Transcription.start` / `resume` return a `Generation` on queued routes and fail with `UnsupportedOperation`
elsewhere. Models come from `.transcription(...)` selectors on the `OpenAI`, `Google`, `Deepgram`, `AssemblyAI`, and
`XAI` facades. Common fields (`language`, `prompt`, `timestamps: "none" | "segment" | "word"`, `diarize`, `speakers`) lower
natively or fail with a typed `AIError` before any network call; a route may return more than asked.
Deepgram and ElevenLabs answer inline, and AssemblyAI is queued. `Transcription.generate` and `Transcription.stream`
work on all of them; `Transcription.start` / `resume` return a `Generation` on queued routes and fail with
`UnsupportedOperation` elsewhere. Models come from `.transcription(...)` selectors on the `OpenAI`, `Google`,
`Deepgram`, `ElevenLabs`, and `AssemblyAI` facades. Common fields (`language`, `prompt`,
`timestamps: "none" | "segment" | "word"`, `diarize`, `speakers`) lower natively or fail with a typed `AIError` before
any network call; a route may return more than asked.
```ts
import { Console, Effect, Stream } from "effect"
@@ -891,7 +893,7 @@ const openai = OpenAI.configure({ apiKey: process.env.OPENAI_API_KEY })
const program = Effect.gen(function* () {
const audio = yield* Media.file("./call.mp3")
// Speaker-labelled segments; labels are provider-native strings ("A", "0", "spk:0").
// Speaker-labelled segments; labels are provider-native strings ("A", "0", "spk:0", "speaker_0").
const response = yield* Transcription.generate({
model: Deepgram.configure({ apiKey }).transcription("nova-3"),
audio,
@@ -901,7 +903,7 @@ const program = Effect.gen(function* () {
response.text // "Hello from OpenCode."
response.segments // [{ text, startSeconds, endSeconds, speaker: "0" }]
response.words // [{ text, startSeconds, endSeconds, speaker, confidence }]
response.language // the provider's own value, lowercased ("en", "english", "en_us")
response.language // the provider's own value, lowercased ("en", "eng", "english", "en_us")
// Text deltas as the model transcribes, then one finish carrying the whole transcript.
yield* Transcription.stream({ model: openai.transcription("gpt-4o-mini-transcribe"), audio }).pipe(
@@ -925,10 +927,12 @@ Provider notes:
- **OpenAI** takes inline audio only; `diarize` needs `gpt-4o-transcribe-diarize`, timestamps need `whisper-1`, and `whisper-1` does not stream.
- **Gemini** needs a transcribe model (`gemini-3.5-transcribe`); `prompt` and `speakers` fail typed.
- **Deepgram** detects the language unless `language` is set; vocabulary goes in `providerOptions.keyterm`.
- **AssemblyAI** uploads inline audio before submitting and is the only route that accepts `speakers`.
- **xAI** (`grok-voice-transcribe-2.0`) answers inline and always returns words; `diarize` or `timestamps: "segment"`
groups them into speaker-turn segments. Vocabulary goes in `providerOptions.keyterm`, and headerless PCM uploads
send `audio_format` and `sample_rate` from `audio.info`. `prompt` and `speakers` fail typed.
- **ElevenLabs** (`scribe_v2`) uploads inline audio as the multipart `file` and sends a URL as `source_url`. Words
always carry timestamps, and segments are speaker turns, so `diarize`, `timestamps: "segment"`, or `speakers` turns
on diarization. `speakers` is an upper bound (`num_speakers`); `prompt` fails typed (vocabulary goes in
`providerOptions.keyterms`), as do webhook delivery and per-channel output (`use_multi_channel` without
`multichannel_output_style: "combined"`).
- **AssemblyAI** uploads inline audio before submitting and treats `speakers` as the exact speaker count.
The promise client mirrors the Effect API:
@@ -951,6 +955,7 @@ const transcript = await generation.await({ poll: { interval: 3_000 } })
- **`ImageClient`** — Effect service and layer for image execution, parallel to `LLMClient`.
- **`Media`** — the shared asset type (`Media.Asset`, `Media.Source`) and constructors used by messages, tool results, and media requests.
- **`Generation`** — provider-neutral handle for an in-flight media generation (`await`, `refresh`, `cancel`, `events`) used by queued media routes.
- **`Video.request` / `generate` / `stream` / `start` / `resume`** — queued video generation through a provider-neutral request; `VideoClient` is its Effect service and layer.
- **`Speech.request` / `Speech.generate` / `Speech.stream`** — text-to-speech through a provider-neutral request; `SpeechClient` is its Effect service and layer.
- **`Transcription.request` / `generate` / `stream` / `start` / `resume`** — speech-to-text over inline, streaming, and queued routes; `TranscriptionClient` is its Effect service and layer.
- **`AIClient.layer` / `AIClient.layerWith(executor)`** — every modality client plus the request executor in one layer.
+57 -43
View File
@@ -40,7 +40,7 @@ The design below is derived from a survey of the raw provider APIs (OpenAI, Gemi
### Model selection
A model value is built as `OpenAI.configure({ apiKey }).responses("gpt-5")` or `.image("gpt-image-2")`: `configure` fixes credentials, endpoint, and defaults; the selector fixes which of the provider's APIs to hit and binds the typed `providerOptions` generic. Media follows the same shape with one selector per modality — `openai.image(id)` today, `.video(id)` / `.speech(id)` / `.transcription(id)` as those modalities land — mirroring `openai.responses(id)`. `Image.request` accepts `ImageModel` only, exactly as `LLM.request` accepts `LanguageModel`.
A model value is built as `OpenAI.configure({ apiKey }).responses("gpt-5")` or `.image("gpt-image-2")`: `configure` fixes credentials, endpoint, and defaults; the selector fixes which of the provider's APIs to hit and binds the typed `providerOptions` generic. Media follows the same shape with one selector per modality — `.image(id)`, `.video(id)`, `.speech(id)`, `.transcription(id)` on the facades that offer each — mirroring `openai.responses(id)`. `Image.request` accepts `ImageModel` only, exactly as `LLM.request` accepts `LanguageModel`.
```ts
import { OpenAI, Google } from "@opencode/ai/providers"
@@ -135,8 +135,8 @@ Effect.gen(function* () {
})
```
`size` and `aspectRatio` are not interchangeable; each route rejects fields it cannot lower — see the README's Image
portability matrix.
`size` and `aspectRatio` are not interchangeable; each route rejects fields it cannot lower — see the portability table
in the README's Image generation section.
Editing is not a separate function; `images`/`mask` on the request select the edit path in the route (OpenAI `/images/edits`, Gemini multimodal parts, xAI `/images/edits`). Routes that cannot honor `mask` fail with `Unsupported`.
@@ -166,8 +166,8 @@ Effect.gen(function* () {
// Simple: wait for it.
const response = yield* Video.generate(request, { poll: { interval: "10 seconds", timeout: "10 minutes" } })
response.video // Media.Asset: url with expiresAt (+ transient `headers` for Veo downloads)
response.usage // credits on Runway; the other three report none
response.video // Media.Asset: url (expiresAt on Veo and Runway; transient `headers` for Veo downloads)
response.usage // credits on Runway; the other three report none (xAI's usage.cost_in_usd_ticks is not decoded)
response.notices // Veo raiMediaFilteredReasons → filtered, xAI respect_moderation → moderated
yield* response.video.materialize() // pull bytes before the URL expires
@@ -175,7 +175,7 @@ Effect.gen(function* () {
const generation = yield* Video.start(request) // Generation<VideoResponse>
generation.id; generation.status; generation.progress; generation.position; generation.token
yield* generation.await({ poll }) // VideoResponse
yield* generation.cancel() // fal PUT cancel_url, Runway DELETE /tasks/{id}; no-op for Veo and xAI
yield* generation.cancel() // fal PUT cancel_url, Runway DELETE /tasks/{id}; Veo and xAI succeed without a request
// Resume from another process. The token is validated against the route's codec and refreshed once. It carries no
// route identity, so persist the provider and model ID alongside it: `resume` needs the model.
@@ -188,10 +188,11 @@ Effect.gen(function* () {
Tokens are route-owned JSON: Veo `{ operation }`, xAI `{ requestID }`, Runway `{ taskID }`, fal
`{ requestID, statusURL, responseURL, cancelURL }` (fal's follow-up URLs are authoritative and absolute). Common-field
lowering per provider: Veo takes inline media only and rejects `audio: false` and `n > 1`; xAI rejects `seed` and
`negativePrompt` and routes a `video` input to edits or (`providerOptions.mode: "extend"`) extensions; fal rejects
`durationSeconds`, `references`, and `frames.last` because the field names and enums differ per model; Runway passes
`aspectRatio` through as its pixel `ratio` and rejects `n`.
lowering per provider: Veo takes inline media only, rejects `audio: false` and `n > 1`, and requires `frames.first`
when `frames.last` is set; xAI rejects `n`, `seed`, and `negativePrompt` and routes a `video` input to edits or
(`providerOptions.mode: "extend"`) extensions; fal rejects `n`, plus `durationSeconds`, `references`, and `frames.last`
because the field names and enums differ per model; Runway passes `aspectRatio` through as its pixel `ratio` and
rejects `n`.
Deferred: `Video.complete(model, token, webhook)` (finish from a webhook payload without polling) and provider poll
hints (none of the four providers emit one). Later providers: Luma, Kling, MiniMax, Replicate.
@@ -199,8 +200,7 @@ hints (none of the four providers emit one). Later providers: Luma, Kling, MiniM
#### Speech (TTS)
Shipped in phase 3 (`src/speech.ts`, `src/speech-client.ts`, protocols `openai-speech`, `google-speech`,
`elevenlabs-speech`, `cartesia-speech`, `deepgram-speech`, `xai-speech`; new `ElevenLabs`, `Cartesia`, and `Deepgram`
facades).
`elevenlabs-speech`, `cartesia-speech`, `deepgram-speech`; new `ElevenLabs`, `Cartesia`, and `Deepgram` facades).
```ts
const request = Speech.request({
@@ -242,14 +242,16 @@ name→id resolution. Multi-speaker (Gemini `speechConfig.multiSpeakerVoiceConfi
`opus_48000_64`, Cartesia `{ container, encoding, sample_rate }`, Deepgram `encoding`+`container`) and declares the
asset's media type rather than sniffing, because headerless PCM can look like an MPEG frame sync. Headerless PCM
always carries `info.encoding`, `info.sampleRate`, and `info.channels`; its media type is the provider's declaration
(Gemini `audio/L16;codec=pcm;rate=24000`, Deepgram's `content-type`) or `audio/pcm`. Gemini returns PCM only, so any
other `format` is rejected rather than wrapped as WAV by the route. Every `format` value a route cannot produce (unknown
to it, a container on Cartesia SSE, WAV on an ElevenLabs stream, anything but PCM on Gemini) fails the same way as an
unsupported field: `UnsupportedOperation` with `operation: "media.format"`.
(Gemini `audio/L16;codec=pcm;rate=24000`, Deepgram's `content-type`) or `audio/pcm`. Gemini's asset follows the
provider's declared type: WAV for Gemini 3.8 TTS `generate`, headerless PCM otherwise. The route never wraps PCM as WAV,
so `pcm` is the only explicit `format` it accepts, and not on Gemini 3.8 `generate`. Every `format` value a route cannot
produce (unknown to it, a container on Cartesia SSE, WAV on an ElevenLabs stream, anything but `pcm` on Gemini, `pcm` on
Gemini 3.8 `generate`) fails the same way as an unsupported field: `UnsupportedOperation` with
`operation: "media.format"`.
**Timestamps.** `timestamps: true` on the request asks for alignment. ElevenLabs selects the `with-timestamps`
endpoints (character-level, NDJSON when streaming); Cartesia sets `add_timestamps` on `/tts/sse` (word-level; a
`generate` with timestamps collects the SSE stream). OpenAI, Gemini, Deepgram, and xAI reject it.
`generate` with timestamps collects the SSE stream). OpenAI, Gemini, and Deepgram reject it.
Common-field lowering per provider:
@@ -260,7 +262,6 @@ Common-field lowering per provider:
| ElevenLabs | path voice id (required) | `voice_settings.speed` | `language_code` | unsupported | `with-timestamps` | `credits` from `character-cost` header |
| Cartesia | `voice` (required) | `generation_config.speed` | `language` | unsupported | `add_timestamps` | none |
| Deepgram | unsupported (voice is the model) | `speed` query | unsupported | unsupported | unsupported | `characters` from `dg-char-count` header |
| xAI | `voice_id` (defaults to `eve`) | `speed` | `language` (`auto` when omitted) | unsupported | unsupported | none |
Deferred: `Speech.session(...)` — input-streaming TTS where text arrives incrementally over a WebSocket (ElevenLabs
`stream-input`, Cartesia WebSocket contexts, Deepgram WebSocket speak) — is a separate scoped resource, not part of
@@ -269,8 +270,8 @@ Deferred: `Speech.session(...)` — input-streaming TTS where text arrives incre
#### Transcription (STT)
Shipped as the second half of phase 3 (`src/transcription.ts`, `src/transcription-client.ts`, protocols
`openai-transcription`, `google-transcription`, `deepgram-transcription`, `assemblyai-transcription`,
`xai-transcription`; new `AssemblyAI` facade).
`openai-transcription`, `google-transcription`, `deepgram-transcription`, `elevenlabs-transcription`,
`assemblyai-transcription`; new `AssemblyAI` facade).
```ts
const request = Transcription.request({
@@ -279,7 +280,7 @@ const request = Transcription.request({
language: "en", // provider-native passthrough
timestamps: "segment", // none | segment | word
diarize: true,
speakers: 2, // expected count, hint only (AssemblyAI)
speakers: 2, // speaker count (AssemblyAI exact, ElevenLabs maximum)
providerOptions: { known_speaker_names: ["agent"] },
})
@@ -293,10 +294,11 @@ yield* Transcription.resume(model, token)
Transcription is the first modality whose providers span all three protocol kinds, and it needed no fourth kind.
Every `MediaRoute` now carries its `kind`; `TranscriptionRoute` is the union of the inline, stream, and queued routes;
`TranscriptionModel.fromRoute` is overloaded per protocol kind (arity picks the overload: `<Options>`,
`<Options, Frame, State>`, `<Options, Token>`) and composes through `MediaRoute.inline` / `stream` / `queued`; and
`TranscriptionClient` dispatches on `route.kind`. `generate` on a queued route is `start` then `await`; `stream` on an
inline route is the response as a single `finish`, and on a queued route it is the status observations followed by
`finish`. `start` / `resume` on a non-queued route fail with `UnsupportedOperation` (`transcription.start`). The
`<Options, Frame, State>`, `<Options, Token>`) and composes through the shared `composeRoute` (`src/media-model.ts`),
which picks `MediaRoute.inline` / `stream` / `queued`; and `TranscriptionClient`, like every modality client, is
`MediaClient.make` (`src/media-client.ts`), which dispatches on `route.kind`. `generate` on a queued route is `start`
then `await`; `stream` on an inline route is the response as a single `finish`, and on a queued route it is the status
observations followed by `finish`. `start` / `resume` on a non-queued route fail with `UnsupportedOperation` (`transcription.start`). The
`finish` event carries the whole transcript (text, segments, words, language, duration, usage), so the stream route's
`collect` is just "take `finish`".
@@ -306,16 +308,23 @@ upload); `packages/ai/AGENTS.md` (Media Routes) describes both.
Settled rules:
- **Timestamps.** A granularity the selected route or model cannot produce fails as `UnsupportedOperation`
(`media.timestamps`), following Speech; a route that returns more than asked (Deepgram and AssemblyAI always return
words) is not stripped. Segments always carry start and end times: Gemini times each transcription part from its
(`media.timestamps`), following Speech; a route that returns more than asked (Deepgram, ElevenLabs, and AssemblyAI
always return words) is not stripped. Segments always carry start and end times: Gemini times each transcription part from its
word offsets, so segment timestamps and diarization also request word offsets there.
- **Diarization.** `diarize` means segments (and words, where the provider labels them) carry `speaker`. Labels are
provider-native strings — OpenAI `A` or a known speaker name, Deepgram `0`, Gemini `spk:0`, AssemblyAI `A` — with no
cross-provider speaker model. `speakers` is a hint; only AssemblyAI (`speakers_expected`) accepts it.
provider-native strings — OpenAI `A` or a known speaker name, Deepgram `0`, Gemini `spk:0`, AssemblyAI `A`,
ElevenLabs `speaker_0` — with no cross-provider speaker model. `speakers` is the number of speakers to label:
AssemblyAI (`speakers_expected`) treats it as an exact constraint rather than a hint, and ElevenLabs
(`num_speakers`) as the maximum. Both turn on diarization for it; the other routes reject it.
- **Segments from words.** ElevenLabs returns only a token list (`word`, `spacing`, `audio_event`), so its segments
are speaker turns: consecutive words and spacing with one `speaker_id`, text joined from the provider's own spacing
tokens. `words` drops spacing and audio events. Segments therefore need diarization, which `timestamps: "segment"`
turns on, as AssemblyAI's utterances need speaker labels.
- **Language** is passed through (`language`, OpenAI `gpt-transcribe` `languages[]`, Gemini `languageCodes`,
AssemblyAI `language_code`). `response.language` is the provider's own value, lowercased but not normalized: an
ISO code on most routes, `english` from whisper-1, `en_us` from AssemblyAI. Deepgram and AssemblyAI assume English
unless asked to detect, so a missing `language` enables their detection.
AssemblyAI and ElevenLabs `language_code`). `response.language` is the provider's own value, lowercased but not
normalized: an ISO code on most routes (AssemblyAI's detection returns `en`, ElevenLabs ISO 639-3 `eng`), `english`
from whisper-1. Deepgram and AssemblyAI assume English unless asked to detect, so a missing `language` enables their
detection.
- **Gemini** requires a transcribe model; other model ids fail with `UnsupportedOperation` before the call, because
general models ignore `audioTranscriptionConfig` and answer conversationally. Streamed chunks carry whole speaker
turns (one part per turn), which join with a space.
@@ -325,15 +334,15 @@ Settled rules:
| Provider | Kind | Audio input | `timestamps` | `diarize` | Unsupported | Usage |
|---|---|---|---|---|---|---|
| OpenAI | stream (`stream: true` in `stream` mode) | multipart `file` (inline only) | `whisper-1` (`verbose_json`); diarize model: `segment` | `gpt-4o-transcribe-diarize` (`diarized_json`) | `speakers`; `prompt` on the diarize model; streaming on `whisper-1` | `tokens` or `seconds` |
| OpenAI | stream (`stream: true` in `stream` mode; `whisper-1` ignores `stream`, so it emits only `finish`) | multipart `file` (inline only) | `whisper-1` (`verbose_json`); diarize model: `segment` | `gpt-4o-transcribe-diarize` (`diarized_json`) | `speakers`; `prompt` on the diarize model | `tokens` or `seconds` |
| Gemini | stream (`generateContent` / `streamGenerateContent`) | `inlineData` or Gemini Files `fileData` | `audioTranscriptionConfig.wordTimestamp` | `audioTranscriptionConfig.diarization` | `prompt`, `speakers` | `tokens` |
| Deepgram | inline | raw body, or JSON `{ url }` | words always; `segment` → `utterances` | `diarize_model=latest` + `utterances` | `prompt`, `speakers` | `seconds` (`metadata.duration`) |
| ElevenLabs | inline | multipart `file`, or `source_url` | words always; `segment` → `diarize` (speaker turns) | `diarize` | `prompt`; `webhook`, per-channel `use_multi_channel` | `seconds` (`audio_duration_secs`) |
| AssemblyAI | queued (upload → submit → poll) | `/v2/upload` then `audio_url`, or a URL | words always; `segment` → `speaker_labels` | `speaker_labels` | — | `seconds` (`audio_duration`) |
| xAI | inline (batch `/v1/stt`) | multipart `file` (last field), or `url` | words always; `segment` → `diarize` speaker turns | `diarize` | `prompt`, `speakers` | `seconds` (`duration`) |
Deferred: `Transcription.session(...)` — realtime STT over WebSocket (Deepgram live, AssemblyAI streaming, ElevenLabs
realtime, OpenAI realtime transcription) — is the same future scoped `session` shape as input-streaming TTS and ships
with the realtime work in phase 5. ElevenLabs Scribe is not implemented yet.
with the realtime work in phase 5.
### `Generation` — shared async execution
@@ -356,7 +365,11 @@ GenerationAwaitOptions = { poll?: Poll }
Poll = { interval?: Duration; timeout?: Duration }
```
`Generation` is not video-specific. Image routes on BFL, fal, and Replicate are queued; `Image.start` exists for them. A route declares itself `inline` or `queued`; `generate` on a queued route is `start` then `await`.
`Generation` is not video-specific. Image routes on BFL, fal, Replicate, and Stability `upscale()` are queued; `Image.start` exists for them. A route declares itself `inline` or `queued`; `generate` on a queued route is `start` then `await`.
Status polls and result reads retry transient failures (rate limits, provider 5xx, and transport errors, classified by the same `isRetryable` the Session runner uses) inside `MediaRoute.queued`. Only the HTTP exchange retries, never the decoded document: a terminal `failed` generation also surfaces as `ProviderInternal` and must not be re-read. Gaps grow exponentially from 1s with jitter, up to 30s each, honoring a provider `retry-after` up to that cap, for at most 8 retries. `await`, `events`, and `Video.stream` cut retries off at `poll.timeout` and fail with `Timeout`, so retries never extend the caller's deadline; a direct `result()` or `resume` read is bounded by the retry cap alone. `start` and `cancel` never retry: a repeated submit can start and bill a second job. The policy is internal; there is no option for it.
Interrupting `await`, `events`, or `Video.stream` (or aborting the promise API's `signal`) stops waiting only. The provider job keeps running and billing; call `cancel()` explicitly to stop it.
### Usage
@@ -399,18 +412,19 @@ for await (const event of ai.llm.stream(request)) { … }
await ai.dispose()
```
Streams become `AsyncIterable` via `Stream.toAsyncIterable`. `AIError` is thrown as-is. `AbortSignal` maps to interruption. Nothing in `src/*` except this entrypoint knows about promises.
Streams become `AsyncIterable` via `Stream.toAsyncIterable`. `AIError` is thrown as-is. Aborting an `AbortSignal` interrupts the work and, like `fetch`, rejects the Promise or throws from the stream with `signal.reason` instead of ending the stream as if complete. Nothing in `src/*` except this entrypoint knows about promises.
### Providers
Existing facades gain per-modality selectors; the modality routes each facade provides:
Existing facades gain per-modality selectors; the modality routes each facade provides (*italics* are not
implemented):
| Facade | llm | image | video | speech | transcription | other |
|---|---|---|---|---|---|---|
| `OpenAI` | responses (default), chat | Images API (stream) | Sora (deprecated 2026-09-24) | ✓ | ✓ | |
| `OpenAI` | responses (default), chat | Images API (stream) | *Sora skipped (decision 8)* | ✓ | ✓ | |
| `Google` | Gemini | Gemini-native | Veo | Gemini TTS | `gemini-3.5-transcribe` | |
| `XAI` | ✓ | ✓ | ✓ | ✓ | ✓ (batch) | |
| `ElevenLabs` | | | | ✓ | Scribe | soundEffect, music |
| `XAI` | ✓ | ✓ | ✓ | | | |
| `ElevenLabs` | | | | ✓ | Scribe | *soundEffect, music (phase 5)* |
| `Cartesia` | | | | ✓ | | |
| `Deepgram` | | | | Aura | ✓ | |
| `Fal` | | ✓ (queued) | ✓ | | | |
@@ -419,7 +433,7 @@ Existing facades gain per-modality selectors; the modality routes each facade pr
| `Replicate` | | ✓ (queued) | | | | |
| `Stability` | | `image` (inline), `upscale()` (queued) | | | | |
| `Runway` | | | ✓ | | | |
| `Luma`, `Kling`, `MiniMax` | | per provider | | | | |
| `Luma`, `Kling`, `MiniMax` | | *deferred* | *deferred* | | | |
New facades follow the existing one-file-per-provider rule. The facade selector is the public path for media models; modality-specific package entrypoints (for example `@opencode/ai/providers/openai/images`) are deferred until Core has a modality-aware model resolver.
@@ -463,7 +477,7 @@ Foundation + Image ship together as the reference implementation, serially. Vide
1. **Foundation** — per-modality selectors, `Media`, `Generation`, `Poll`, `Usage` union, `MediaProtocol` kinds, `@opencode/ai/promise` with `llm` + `image`. Port the five existing image protocols onto it. Unify `MediaPart` and add the `media` LLM event (fixes Gemini image output being dropped).
2. **Video** — ✅ Veo, xAI, fal, Runway shipped (`MediaProtocol.queued`, `Video.start/generate/resume/stream`, promise `ai.video`). Deferred: `Video.complete` (webhooks), Luma, Kling, MiniMax, Replicate.
3. **Speech + Transcription** — ✅ Speech: OpenAI, Gemini TTS, ElevenLabs, Cartesia, Deepgram, xAI shipped (`MediaProtocol.stream`, `Speech.generate/stream`, promise `ai.speech`). ✅ Transcription: OpenAI, Gemini, Deepgram, AssemblyAI, xAI shipped across all three route kinds (`Transcription.generate/stream/start/resume`, promise `ai.transcription`). Pending: ElevenLabs Scribe. Deferred: `Speech.session` and `Transcription.session` (WebSocket streaming).
3. **Speech + Transcription** — ✅ Speech: OpenAI, Gemini TTS, ElevenLabs, Cartesia, Deepgram shipped (`MediaProtocol.stream`, `Speech.generate/stream`, promise `ai.speech`). ✅ Transcription: OpenAI, Gemini, Deepgram, ElevenLabs Scribe, AssemblyAI shipped across all three route kinds (`Transcription.generate/stream/start/resume`, promise `ai.transcription`). Deferred: `Speech.session` and `Transcription.session` (WebSocket streaming).
4. **Image queued routes and partials** — ✅ BFL, fal, Replicate, and Stability creative upscale queued; Stability generate inline; OpenAI `partial_images` streaming (`image-partial` restored). Imagen dropped: shut down on the Gemini API and discontinued on Vertex (2026-06-30). Deferred: Stability's synchronous edit and fast/conservative upscale endpoints.
5. **Later** — ElevenLabs music/SFX, Lyria, `Speech.session` / `Transcription.session`, realtime.
+1 -1
View File
@@ -1,6 +1,6 @@
{
"$schema": "https://json.schemastore.org/package.json",
"version": "2.0.16",
"version": "2.0.18",
"name": "@opencode/ai",
"type": "module",
"license": "MIT",
+58 -27
View File
@@ -53,6 +53,8 @@ export type Event = Observation | { readonly type: "generation-finished"; readon
const TERMINAL: ReadonlySet<Status> = new Set(["completed", "failed", "cancelled", "expired"])
export const isTerminal = (status: Status) => TERMINAL.has(status)
export class Generation<Response> {
readonly id: string
readonly status: Status
@@ -81,7 +83,7 @@ export class Generation<Response> {
}
get terminal() {
return TERMINAL.has(this.status)
return isTerminal(this.status)
}
refresh(): Effect.Effect<Generation<Response>, AIError> {
@@ -100,7 +102,7 @@ export class Generation<Response> {
return settled.pipe(
// Non-completed terminal states also go through `result` so the route can surface its provider failure body.
Effect.flatMap((generation) => generation.result()),
Effect.timeoutOrElse({ duration: timeout, orElse: () => this.timeoutError(timeout) }),
Effect.timeoutOrElse({ duration: timeout, orElse: () => timeoutError(this.id, timeout) }),
)
}
@@ -109,9 +111,10 @@ export class Generation<Response> {
}
/**
* Status observations as a stream, ending after the first terminal observation. Each poll is bounded by the time
* remaining until `poll.timeout`, so a hung status request fails the stream instead of stalling it. (`Stream.interruptWhen`
* would express this directly but deadlocks under `TestClock` when the source completes while the timer sleeps.)
* Status observations as a stream, ending after the first terminal observation. Each poll and each sleep between polls
* is bounded by the time remaining until `poll.timeout`, so a hung status request or a long interval fails the stream at
* the deadline instead of stalling it. (`Stream.interruptWhen` would express this directly but deadlocks under
* `TestClock` when the source completes while the timer sleeps.)
*/
events(options?: AwaitOptions): Stream.Stream<Event, AIError> {
if (this.terminal) return Stream.make(this.event())
@@ -120,17 +123,13 @@ export class Generation<Response> {
Clock.currentTimeMillis.pipe(
Effect.map((start) => {
const deadline = start + Duration.toMillis(timeout)
const refresh = Clock.currentTimeMillis.pipe(
Effect.flatMap((now) =>
this.refresh().pipe(
Effect.timeoutOrElse({
duration: Duration.millis(Math.max(0, deadline - now)),
orElse: () => this.timeoutError(timeout),
}),
),
const refresh = within(this.refresh(), this.id, timeout, deadline)
const schedule = this.schedule(options?.poll).pipe(
Schedule.modifyDelay((meta) =>
Effect.succeed(Duration.min(meta.duration, Duration.millis(Math.max(0, deadline - meta.now)))),
),
)
return Stream.fromEffectSchedule(refresh, this.schedule(options?.poll)).pipe(
return Stream.fromEffectSchedule(refresh, schedule).pipe(
Stream.takeUntil((generation) => generation.terminal),
Stream.map((generation) => generation.event()),
)
@@ -145,15 +144,6 @@ export class Generation<Response> {
return { type: "generation-progress", id: this.id, progress: this.progress }
}
private timeoutError(timeout: Duration.Duration) {
return new AIError({
reason: new TimeoutError({
message: `Generation ${this.id} did not finish within ${Duration.format(timeout)}`,
timeoutMs: Duration.toMillis(timeout),
}),
})
}
private poll(poll: Poll | undefined) {
return this.refresh().pipe(
Effect.repeat({ schedule: this.schedule(poll), until: (generation) => generation.terminal }),
@@ -165,12 +155,53 @@ export class Generation<Response> {
}
}
/** `events` followed by the expanded result, with the result fetch bounded by the same `poll.timeout` deadline. */
export const resultEvents = <Response, A>(
generation: Generation<Response>,
expand: (response: Response) => ReadonlyArray<A>,
options?: AwaitOptions,
): Stream.Stream<Observation | A, AIError> =>
generation.events(options).pipe(
Stream.filter((event): event is Observation => event.type !== "generation-finished"),
Stream.concat(Stream.fromIterableEffect(Effect.map(generation.result(), expand))),
): Stream.Stream<Observation | A, AIError> => {
const timeout = Duration.fromInputUnsafe(options?.poll?.timeout ?? DEFAULT_POLL_TIMEOUT)
return Stream.unwrap(
Clock.currentTimeMillis.pipe(
Effect.map((start) =>
generation.events(options).pipe(
Stream.filter((event): event is Observation => event.type !== "generation-finished"),
Stream.concat(
Stream.fromIterableEffect(
within(generation.result(), generation.id, timeout, start + Duration.toMillis(timeout)).pipe(
Effect.map(expand),
),
),
),
),
),
),
)
}
/**
* Run `effect` within the time left until `deadline`. Fails before starting once the deadline has passed: a fast
* request could otherwise win the zero-budget race and schedule another zero-delay poll.
*/
const within = <A>(effect: Effect.Effect<A, AIError>, id: string, timeout: Duration.Duration, deadline: number) =>
Clock.currentTimeMillis.pipe(
Effect.flatMap((now) =>
now >= deadline
? Effect.fail(timeoutError(id, timeout))
: effect.pipe(
Effect.timeoutOrElse({
duration: Duration.millis(deadline - now),
orElse: () => Effect.fail(timeoutError(id, timeout)),
}),
),
),
)
const timeoutError = (id: string, timeout: Duration.Duration) =>
new AIError({
reason: new TimeoutError({
message: `Generation ${id} did not finish within ${Duration.format(timeout)}`,
timeoutMs: Duration.toMillis(timeout),
}),
})
+1 -1
View File
@@ -4,7 +4,7 @@ export { ImageClient } from "./image-client.js"
export { Auth } from "./route/auth.js"
export { Provider } from "./provider.js"
export { ProviderPackage } from "./provider-package.js"
export { isContextOverflow, isContextOverflowFailure } from "./provider-error.js"
export { isContextOverflow, isContextOverflowFailure, isRetryable } from "./provider-error.js"
export type {
RouteLanguageModelInput,
RouteRoutedLanguageModelInput,
+7 -6
View File
@@ -42,7 +42,7 @@ export type GenerationHandle<Response> = Snapshot & {
/** Serializable JSON; pass it back to `resume` from another process. */
readonly token: unknown
readonly await: (options?: AwaitOptions & RunOptions) => Promise<Response>
/** Status observations until the first terminal one, polling like `await`; abort ends iteration without throwing. */
/** Status observations until the first terminal one, polling like `await`; abort throws `signal.reason`. */
readonly events: (options?: AwaitOptions & RunOptions) => AsyncIterable<Event>
/** The result without polling; fails when the generation has not completed. */
readonly result: (options?: RunOptions) => Promise<Response>
@@ -50,15 +50,16 @@ export type GenerationHandle<Response> = Snapshot & {
readonly cancel: (options?: RunOptions) => Promise<void>
}
// Fails with `signal.reason` so aborted calls reject and aborted streams throw like `fetch`: an `AbortError` by default.
const abortEffect = (signal: AbortSignal | undefined) =>
signal === undefined
? Effect.never
: Effect.callback<void>((resume) => {
: Effect.callback<never, unknown>((resume) => {
if (signal.aborted) {
resume(Effect.void)
resume(Effect.fail(signal.reason))
return
}
const onAbort = () => resume(Effect.void)
const onAbort = () => resume(Effect.fail(signal.reason))
signal.addEventListener("abort", onAbort, { once: true })
return Effect.sync(() => signal.removeEventListener("abort", onAbort))
})
@@ -68,14 +69,14 @@ export const make = (options: Options = {}) => {
/** Run any package Effect (for example `LLMClient.compact(...)`) inside this runtime. */
const run = <A, E>(effect: Effect.Effect<A, E, Services>, options?: RunOptions) =>
runtime.runPromise(effect, { signal: options?.signal })
runtime.runPromise(Effect.raceFirst(effect, abortEffect(options?.signal)))
const iterate = <A, E>(stream: Stream.Stream<A, E, Services>, options?: RunOptions): AsyncIterable<A> =>
Stream.toAsyncIterable(
Stream.unwrap(
runtime.contextEffect.pipe(
Effect.map(
(context): Stream.Stream<A, E> =>
(context): Stream.Stream<A, unknown> =>
stream.pipe(Stream.interruptWhen(abortEffect(options?.signal)), Stream.provideContext(context)),
),
),
@@ -110,8 +110,11 @@ const fromRequest = Effect.fn("AssemblyAITranscription.fromRequest")(function* (
language_code: request.language,
language_detection: request.language === undefined ? true : undefined,
prompt: request.prompt,
// Turn-level `utterances`, the only segments AssemblyAI returns, require speaker labels.
speaker_labels: request.diarize === true || request.timestamps === "segment" ? true : undefined,
// Turn-level `utterances`, the only segments AssemblyAI returns, and `speakers_expected` require speaker labels.
speaker_labels:
request.diarize === true || request.timestamps === "segment" || request.speakers !== undefined
? true
: undefined,
speakers_expected: request.speakers,
},
request.providerOptions,
@@ -155,8 +158,7 @@ const decodeResult = Effect.fn("AssemblyAITranscription.decodeResult")(function*
const error = transcript.error ?? undefined
if (status === "failed")
return yield* output.ended("failed", `${route.name} transcription failed${error === undefined ? "" : `: ${error}`}`)
if (status !== "completed")
return yield* output.invalid(`${route.name} transcript ${context.token.transcriptID} has not finished`)
if (status !== "completed") return yield* output.pending(context.token.transcriptID)
const duration = transcript.audio_duration ?? undefined
return new TranscriptionResponse({
text: transcript.text ?? "",
+20 -6
View File
@@ -31,13 +31,21 @@ export type Request = ImageRequestFor<BlackForestLabsImageOptions>
// 2. Token and response schemas
// ---------------------------------------------------------------------------
/** Regional clusters answer on different hosts, so the returned `polling_url` is followed verbatim. */
export const Token = Schema.Struct({ id: Schema.String, pollingURL: Schema.String })
/**
* Regional clusters answer on different hosts, so the returned `polling_url` is followed verbatim. BFL reports the
* credit cost on submit, so it rides on the token; it is optional so tokens persisted before it existed still decode.
*/
export const Token = Schema.Struct({
id: Schema.String,
pollingURL: Schema.String,
cost: Schema.optionalKey(Schema.Number),
})
export type Token = Schema.Schema.Type<typeof Token>
const StartResponse = Schema.Struct({
id: Schema.String,
polling_url: Schema.String,
cost: optionalNull(Schema.Number),
})
const Result = Schema.Struct({
@@ -145,7 +153,11 @@ const fromRequest = Effect.fn("BlackForestLabsImages.fromRequest")(function* (re
// ---------------------------------------------------------------------------
const decodeStart = route.decodeStarted(StartResponse, (value) => ({
token: { id: value.id, pollingURL: value.polling_url },
token: {
id: value.id,
pollingURL: value.polling_url,
...(value.cost === undefined || value.cost === null ? {} : { cost: value.cost }),
},
snapshot: { id: value.id, status: "queued" },
}))
@@ -169,14 +181,16 @@ const decodeResult = Effect.fn("BlackForestLabsImages.decodeResult")(function* (
if (isModerated(document.status)) return yield* output.contentPolicy(`${route.name} moderated the generation`)
if (status === "failed" || status === "expired")
return yield* output.ended(status, `${route.name} generation ${context.token.id} ended with ${document.status}`)
if (status !== "completed" || document.result === undefined || document.result === null)
if (status !== "completed") return yield* output.pending(context.token.id)
if (document.result === undefined || document.result === null)
return yield* output.invalid(`${route.name} generation ${context.token.id} has no result`)
const { sample, seed, prompt, ...rest } = document.result
// A settled `cost` on the result supersedes the submit-time cost carried on the token.
const cost = document.cost ?? context.token.cost
return new ImageResponse({
// `sample` is a signed URL that expires 10 minutes after the result is ready, so it is downloaded now.
images: [yield* context.materialize(Media.url(sample))],
usage:
document.cost === undefined || document.cost === null ? undefined : { type: "credits", credits: document.cost },
usage: cost === undefined ? undefined : { type: "credits", credits: cost },
providerMetadata: {
bfl: { id: context.token.id, seed: seed ?? undefined, prompt: prompt ?? undefined, ...rest },
},
+20 -9
View File
@@ -67,6 +67,9 @@ const queryParameters = (request: Request) => {
}
const fromRequest = Effect.fn("DeepgramSpeech.fromRequest")(function* (request: Request) {
// Not in `unsupported`: that list would also reject `timestamps: false`, which asks for nothing.
if (request.timestamps === true)
return yield* route.unsupported("media.timestamps", `${route.name} does not return timestamps`)
if (
request.format !== undefined &&
FORMATS[request.format] === undefined &&
@@ -86,24 +89,32 @@ const fromRequest = Effect.fn("DeepgramSpeech.fromRequest")(function* (request:
// 6. Stream parsing
// ---------------------------------------------------------------------------
const HEADERLESS_ENCODINGS: Readonly<Record<string, SpeechStream.PcmEncoding>> = {
linear16: "pcm_s16le",
mulaw: "pcm_mulaw",
alaw: "pcm_alaw",
/** Deepgram wraps raw encodings in WAV unless `container` is `none`, and defaults their sample rate per encoding. */
const HEADERLESS_ENCODINGS: Readonly<
Record<string, { readonly encoding: SpeechStream.PcmEncoding; readonly sampleRate: number }>
> = {
linear16: { encoding: "pcm_s16le", sampleRate: 24000 },
mulaw: { encoding: "pcm_mulaw", sampleRate: 8000 },
alaw: { encoding: "pcm_alaw", sampleRate: 8000 },
}
const finish = (state: State, context: MediaProtocol.ResponseContext<Request>) => {
const headers = context.http.headers
const mediaType = headers["content-type"]
const format = audioFormat(context.request)
const encoding = HEADERLESS_ENCODINGS[format.encoding ?? ""]
const headerless = HEADERLESS_ENCODINGS[format.encoding ?? ""]
const container = format.container ?? (headerless === undefined ? undefined : "wav")
const requestID = headers["dg-request-id"]
const modelName = headers["dg-model-name"]
return SpeechStream.finish(route, state, {
...(format.container === "none" && encoding !== undefined
? SpeechStream.pcm(encoding, SpeechStream.sampleRate(mediaType), mediaType)
...(container === "none" && headerless !== undefined
? SpeechStream.pcm(
headerless.encoding,
SpeechStream.sampleRate(mediaType) ?? context.request.providerOptions?.sampleRate ?? headerless.sampleRate,
mediaType,
)
: // Deepgram's default encoding is MP3; WAV is a container around any encoding.
{ mediaType, info: { format: format.container === "wav" ? "wav" : (format.encoding ?? "mp3") } }),
{ mediaType, info: { format: container === "wav" ? "wav" : (format.encoding ?? "mp3") } }),
usage: SpeechStream.headerUsage("characters", headers["dg-char-count"]),
providerMetadata:
requestID === undefined && modelName === undefined
@@ -117,7 +128,7 @@ const finish = (state: State, context: MediaProtocol.ResponseContext<Request>) =
// ---------------------------------------------------------------------------
export const protocol = MediaProtocol.stream<Request, SpeechEvent, Uint8Array, State>(route, {
unsupported: ["voice", "language", "instructions", "timestamps"],
unsupported: ["voice", "language", "instructions"],
body: { from: fromRequest },
frames: (bytes) => bytes,
initial: () => ({ chunks: [] }),
@@ -6,6 +6,7 @@ import { mergeJsonRecords, type OpenString } from "../schema/index.js"
import { TranscriptionModel, TranscriptionResponse, type TranscriptionRequestFor } from "../transcription.js"
import { ProviderShared } from "./shared.js"
import { MediaInput } from "./utils/media-input.js"
import { SpeakerTurns } from "./utils/speaker-turns.js"
const route = MediaProtocol.identity({ id: "deepgram-transcription", name: "Deepgram", provider: "deepgram" })
export const DEFAULT_BASE_URL = "https://api.deepgram.com"
@@ -115,16 +116,6 @@ const speaker = (value: number | undefined) => (value === undefined ? undefined
const wordText = (word: typeof Word.Type) => word.punctuated_word ?? word.word
// Utterances split on pauses, not speakers: the v2 diarizer labels a whole utterance with one speaker even when its
// words change speaker, so segments split each utterance at speaker changes.
const speakerTurns = (words: ReadonlyArray<typeof Word.Type>) =>
words.reduce<Array<Array<typeof Word.Type>>>((turns, word) => {
const last = turns.at(-1)
if (last === undefined || last[0].speaker !== word.speaker) return [...turns, [word]]
last.push(word)
return turns
}, [])
const decodeResponse = Effect.fn("DeepgramTranscription.decodeResponse")(function* (
response: HttpClientResponse.HttpClientResponse,
) {
@@ -136,6 +127,8 @@ const decodeResponse = Effect.fn("DeepgramTranscription.decodeResponse")(functio
const requestID = output.value.metadata?.request_id
return new TranscriptionResponse({
text: alternative.transcript,
// Utterances split on pauses, not speakers: the v2 diarizer labels a whole utterance with one speaker even when
// its words change speaker, so segments split each utterance at speaker changes.
segments: output.value.results.utterances?.flatMap((utterance) =>
utterance.words === undefined || utterance.words.length === 0
? [
@@ -146,7 +139,7 @@ const decodeResponse = Effect.fn("DeepgramTranscription.decodeResponse")(functio
speaker: speaker(utterance.speaker),
},
]
: speakerTurns(utterance.words).map((turn) => ({
: SpeakerTurns.group(utterance.words, (word) => word.speaker).map((turn) => ({
text: turn.map(wordText).join(" "),
startSeconds: turn[0].start,
endSeconds: turn[turn.length - 1].end,
@@ -0,0 +1,211 @@
import { Effect, Schema } from "effect"
import type { HttpClientResponse } from "effect/unstable/http"
import { MediaProtocol } from "../route/media-protocol.js"
import { MediaRoute } from "../route/media.js"
import { mergeJsonRecords, type OpenString } from "../schema/index.js"
import { TranscriptionModel, TranscriptionResponse, type TranscriptionRequestFor } from "../transcription.js"
import { mediaTypeExtension } from "../utils/media-type.js"
import { ProviderShared, optionalNull } from "./shared.js"
import { MediaInput } from "./utils/media-input.js"
import { SpeakerTurns } from "./utils/speaker-turns.js"
const route = MediaProtocol.identity({
id: "elevenlabs-transcription",
name: "ElevenLabs Transcription",
provider: "elevenlabs",
})
export const DEFAULT_BASE_URL = "https://api.elevenlabs.io"
export const PATH = "/v1/speech-to-text"
// ---------------------------------------------------------------------------
// 1. Public model input
// ---------------------------------------------------------------------------
export type ElevenLabsTranscriptionOptions = {
readonly tag_audio_events?: boolean
readonly timestamps_granularity?: OpenString<"none" | "word" | "character">
readonly diarization_threshold?: number
readonly file_format?: OpenString<"pcm_s16le_16" | "other">
readonly temperature?: number
readonly seed?: number
readonly keyterms?: ReadonlyArray<string>
readonly no_verbatim?: boolean
readonly detect_speaker_roles?: boolean
readonly use_speaker_library?: boolean
readonly entity_detection?: string | ReadonlyArray<string>
readonly entity_redaction?: string | ReadonlyArray<string>
readonly entity_redaction_mode?: OpenString<"redacted" | "entity_type" | "enumerated_entity_type">
} & Record<string, unknown>
export type Request = TranscriptionRequestFor<ElevenLabsTranscriptionOptions>
// ---------------------------------------------------------------------------
// 2. Response schema
// ---------------------------------------------------------------------------
/** `type` is `word`, `spacing` (the whitespace between words), or `audio_event` (`(laughter)`). */
const Token = Schema.Struct({
text: Schema.String,
type: Schema.String,
start: optionalNull(Schema.Number),
end: optionalNull(Schema.Number),
speaker_id: optionalNull(Schema.String),
logprob: optionalNull(Schema.Number),
})
type Token = Schema.Schema.Type<typeof Token>
const Transcript = Schema.Struct({
language_code: optionalNull(Schema.String),
text: Schema.String,
words: optionalNull(Schema.Array(Token)),
transcription_id: optionalNull(Schema.String),
audio_duration_secs: optionalNull(Schema.Number),
})
// ---------------------------------------------------------------------------
// 5. Request body construction
// ---------------------------------------------------------------------------
/** Speaker turns are the only segments ElevenLabs can produce, and `num_speakers` only applies to diarization. */
const diarizes = (request: Request) =>
request.diarize === true || request.timestamps === "segment" || request.speakers !== undefined
const RESERVED_FORM_FIELDS = new Set([
"file",
"cloud_storage_url",
"source_url",
"model_id",
"language_code",
"diarize",
"num_speakers",
])
const validate = (request: Request, overlay: Record<string, unknown>) => {
// Webhook requests return 202 with no transcript; the result arrives at a configured webhook instead.
if (overlay.webhook === true)
return Effect.fail(route.unsupported("transcription.webhook", `${route.name} does not deliver to webhooks`))
// Separate multichannel output replaces the transcript with one transcript per channel.
if (overlay.use_multi_channel === true && overlay.multichannel_output_style !== "combined")
return Effect.fail(
route.unsupported(
"transcription.multichannel",
`${route.name} returns a single transcript; set multichannel_output_style: "combined" to merge channels`,
),
)
if (overlay.timestamps_granularity === "none" && (request.timestamps === "word" || diarizes(request)))
return Effect.fail(
route.unsupported(
"media.timestamps",
`${route.name} cannot return word timestamps or speaker turns with timestamps_granularity: "none"`,
),
)
return Effect.void
}
const fromRequest = Effect.fn("ElevenLabsTranscription.fromRequest")(function* (request: Request) {
const overlay = mergeJsonRecords(request.providerOptions, request.http?.body) ?? {}
yield* validate(request, overlay)
const form = new FormData()
const url = ProviderShared.mediaUrl(request.audio)
if (url === undefined) {
const extension = mediaTypeExtension(request.audio.mediaType)
const audio = yield* MediaInput.inlineBytes(route.id, request.audio)
form.append(
"file",
MediaInput.blob(audio, request.audio.mediaType),
extension === undefined ? "audio" : `audio.${extension}`,
)
}
MediaInput.appendFields(
form,
{
model_id: request.model.id,
// `cloud_storage_url` is deprecated in favor of `source_url`, which accepts any hosted audio or video URL.
source_url: url,
language_code: request.language,
diarize: diarizes(request) ? true : undefined,
num_speakers: request.speakers,
},
{ overlay, reserved: RESERVED_FORM_FIELDS, repeatArrays: "key" },
)
return MediaProtocol.multipart(form)
})
// ---------------------------------------------------------------------------
// 6. Response decoding
// ---------------------------------------------------------------------------
const decodeTranscript = route.decodeJson(Transcript)
type TimedWord = Token & { readonly start: number; readonly end: number }
const isTimedWord = (token: Token): token is TimedWord =>
token.type === "word" && typeof token.start === "number" && typeof token.end === "number"
/** Turn text keeps the provider's own spacing tokens, so languages written without spaces are not re-spaced. */
const speakerTurns = (tokens: ReadonlyArray<Token>) =>
SpeakerTurns.group(
tokens.filter((token) => token.type === "word" || token.type === "spacing"),
(token) => token.speaker_id,
).flatMap((turn) => {
const words = turn.filter(isTimedWord)
if (words.length === 0) return []
return [
{
text: turn
.map((token) => token.text)
.join("")
.trim(),
startSeconds: words[0].start,
endSeconds: words[words.length - 1].end,
speaker: turn[0].speaker_id ?? undefined,
},
]
})
const decodeResponse = Effect.fn("ElevenLabsTranscription.decodeResponse")(function* (
response: HttpClientResponse.HttpClientResponse,
context: MediaProtocol.DecodeContext<Request>,
) {
const output = yield* decodeTranscript(response)
const transcript = output.value
const tokens = transcript.words ?? []
const duration = transcript.audio_duration_secs ?? undefined
const transcriptionID = transcript.transcription_id ?? undefined
return new TranscriptionResponse({
text: transcript.text,
segments: diarizes(context.request) ? speakerTurns(tokens) : undefined,
words: tokens.filter(isTimedWord).map((word) => ({
text: word.text,
startSeconds: word.start,
endSeconds: word.end,
speaker: word.speaker_id ?? undefined,
confidence: typeof word.logprob === "number" ? Math.exp(word.logprob) : undefined,
})),
language: transcript.language_code?.toLowerCase(),
durationSeconds: duration,
usage: duration === undefined ? undefined : { type: "seconds", seconds: duration },
providerMetadata: transcriptionID === undefined ? undefined : { elevenlabs: { transcriptionId: transcriptionID } },
})
})
// ---------------------------------------------------------------------------
// 7. Protocol and route
// ---------------------------------------------------------------------------
export const protocol = MediaProtocol.inline<Request, TranscriptionResponse>(route, {
unsupported: ["prompt"],
body: { from: fromRequest },
response: { decode: decodeResponse },
})
export const model = (input: MediaRoute.ModelInput) =>
TranscriptionModel.fromRoute<ElevenLabsTranscriptionOptions>(
{ protocol, baseURL: DEFAULT_BASE_URL, path: PATH },
input,
)
export const ElevenLabsTranscription = {
protocol,
model,
} as const
+20 -14
View File
@@ -49,7 +49,7 @@ const QueueResult = Schema.StructWithRest(
// ---------------------------------------------------------------------------
const sizing = (model: string) => {
if (/^fal-ai\/(nano-banana|flux-pro\/v1\.1-ultra)/.test(model)) return "aspect_ratio"
if (/^fal-ai\/(nano-banana|flux-pro\/(v1\.1-ultra|kontext))/.test(model)) return "aspect_ratio"
if (model.startsWith("fal-ai/flux")) return "image_size"
return undefined
}
@@ -63,20 +63,24 @@ const validate = (request: Request) => {
return Effect.fail(route.unsupported("media.size", `${id} sizes by aspectRatio`))
if (request.aspectRatio !== undefined && field === "image_size")
return Effect.fail(route.unsupported("media.aspectRatio", `${id} sizes by size (image_size)`))
if ((request.images?.length ?? 0) > 1 && !isEdit(id))
if ((request.images?.length ?? 0) > 1 && !takesImageList(id))
return Effect.fail(
route.unsupported("media.images", `${id} takes one image_url; use an /edit endpoint for several images`),
route.unsupported(
"media.images",
`${id} takes one image_url; use an /edit or /multi endpoint for several images`,
),
)
return Effect.void
}
// `/edit` endpoints take an `image_urls` list; image-to-image, fill, and Ultra take one `image_url` (beside `mask_url`).
const isEdit = (model: string) => model.endsWith("/edit")
// `/edit` and `/multi` (Kontext) endpoints take an `image_urls` list; image-to-image, fill, and Ultra take one
// `image_url` (beside `mask_url`).
const takesImageList = (model: string) => model.endsWith("/edit") || model.endsWith("/multi")
const fromRequest = Effect.fn("FalImages.fromRequest")(function* (request: Request) {
yield* validate(request)
const images = yield* Effect.forEach(request.images ?? [], (image) => FalQueue.mediaUrl(image, route.name))
const edit = isEdit(request.model.id)
const list = takesImageList(request.model.id)
return MediaProtocol.json(
mergeJsonRecords(
{
@@ -86,8 +90,8 @@ const fromRequest = Effect.fn("FalImages.fromRequest")(function* (request: Reque
image_size: request.size === undefined ? undefined : MediaInput.dimensions(request.size),
aspect_ratio: request.aspectRatio,
output_format: request.format,
image_urls: edit && images.length > 0 ? images : undefined,
image_url: edit ? undefined : images[0],
image_urls: list && images.length > 0 ? images : undefined,
image_url: list ? undefined : images[0],
mask_url: request.mask === undefined ? undefined : yield* FalQueue.mediaUrl(request.mask, route.name),
},
request.providerOptions,
@@ -112,12 +116,14 @@ const decodeResult = Effect.fn("FalImages.decodeResult")(function* (
// With the safety checker on, flagged images come back blacked out rather than omitted.
const flagged = (has_nsfw_concepts ?? []).flatMap((value, index) => (value ? [index] : []))
return new ImageResponse({
images: images.map((image) =>
Media.url(image.url, {
mediaType: image.content_type ?? undefined,
info: { width: image.width ?? undefined, height: image.height ?? undefined },
}),
),
images: images.map((image) => {
const info = { width: image.width ?? undefined, height: image.height ?? undefined }
// `sync_mode: true` returns data URIs instead of hosted URLs.
return (
Media.parseDataUrl(image.url, { info }) ??
Media.url(image.url, { mediaType: image.content_type ?? undefined, info })
)
}),
notices:
flagged.length === 0
? undefined
+1 -13
View File
@@ -526,19 +526,7 @@ const mapFinishReason = (finishReason: string | undefined, hasToolCalls: boolean
if (finishReason === undefined) return hasToolCalls ? "tool-calls" : "unknown"
if (finishReason === "STOP") return hasToolCalls ? "tool-calls" : "stop"
if (finishReason === "MAX_TOKENS") return "length"
if (
finishReason === "IMAGE_SAFETY" ||
finishReason === "RECITATION" ||
finishReason === "SAFETY" ||
finishReason === "BLOCKLIST" ||
finishReason === "PROHIBITED_CONTENT" ||
finishReason === "SPII" ||
finishReason === "MODEL_ARMOR" ||
finishReason === "IMAGE_PROHIBITED_CONTENT" ||
finishReason === "IMAGE_RECITATION" ||
finishReason === "LANGUAGE"
)
return "content-filter"
if (GeminiGenerateContent.contentFiltered(finishReason)) return "content-filter"
if (
finishReason === "MALFORMED_FUNCTION_CALL" ||
finishReason === "UNEXPECTED_TOOL_CALL" ||
+1 -1
View File
@@ -101,7 +101,7 @@ const generationConfig = (request: Request) => {
const fromRequest = Effect.fn("GoogleImages.fromRequest")(function* (request: Request) {
if (request.n !== undefined && request.n > 1)
return yield* route.unsupported(
"image.n",
"media.n",
`${route.name} generates one image per request; call it once per image instead of n=${request.n}`,
)
const parts = yield* Effect.forEach(request.images ?? [], (image) =>
+27 -6
View File
@@ -56,10 +56,18 @@ interface State extends SpeechStream.Audio, GeminiGenerateContent.Metadata {
// ---------------------------------------------------------------------------
const fromRequest = Effect.fn("GoogleSpeech.fromRequest")(function* (request: MediaProtocol.Addressed<Request>) {
// Not in `unsupported`: that list would also reject `timestamps: false`, which asks for nothing.
if (request.timestamps === true)
return yield* route.unsupported("media.timestamps", `${route.name} does not return timestamps`)
if (request.format === "pcm" && request.mode === "generate" && /^gemini-3\.8-.*-tts(?:-|$)/.test(request.model.id))
return yield* route.unsupported(
"media.format",
`${route.name} returns WAV by default for Gemini 3.8 TTS unary requests; omit the format to accept it`,
)
if (request.format !== undefined && request.format !== "pcm")
return yield* route.unsupported(
"media.format",
`${route.name} only returns raw PCM; request format "pcm" or omit it, then wrap the samples yourself`,
`${route.name} only accepts raw PCM as an explicit format; omit it to accept the provider's default output`,
)
const voiceName = SpeechStream.voiceID(request.voice)
return MediaProtocol.json(
@@ -94,16 +102,29 @@ const step = Effect.fn("GoogleSpeech.step")(function* (state: State, frame: stri
part.inlineData === undefined ? [] : [part.inlineData],
)
const next: State = { ...GeminiGenerateContent.track(state, chunk), mimeType: state.mimeType ?? audio[0]?.mimeType }
return [next, audio.flatMap((part) => SpeechStream.delta(next, part.data)[1])] as const
const events = audio.flatMap((part) => SpeechStream.delta(next, part.data)[1])
const withheld = next.chunks.length === 0 ? GeminiGenerateContent.withheld(route.name, chunk, frame) : undefined
if (withheld !== undefined) return yield* withheld
return [next, events] as const
})
const finish = (state: State) => {
const finish = (state: State, context: MediaProtocol.ResponseContext<Request>) => {
if (state.finishReason === undefined) return Effect.fail(route.incomplete())
const sampleRate = SpeechStream.sampleRate(state.mimeType) ?? DEFAULT_SAMPLE_RATE
const output =
state.mimeType?.split(";")[0]?.toLowerCase() === "audio/wav"
? SpeechStream.container("wav", sampleRate)
: SpeechStream.pcm("pcm_s16le", sampleRate, state.mimeType ?? `audio/L16;codec=pcm;rate=${sampleRate}`)
if (context.request.format === "pcm" && output.info.format !== "pcm")
return Effect.fail(
route.frameError(`Google Speech returned ${output.info.format} instead of the requested raw PCM`),
)
return SpeechStream.finish(route, state, {
...SpeechStream.pcm("pcm_s16le", sampleRate, state.mimeType ?? `audio/L16;codec=pcm;rate=${sampleRate}`),
...output,
usage: GeminiGenerateContent.usage(state.usage),
notices: GeminiGenerateContent.notices(route.name, state),
providerMetadata: GeminiGenerateContent.providerMetadata(state),
detail: state.finishReason === undefined ? undefined : `finish reason: ${state.finishReason}`,
detail: `finish reason: ${state.finishReason}`,
})
}
@@ -112,7 +133,7 @@ const finish = (state: State) => {
// ---------------------------------------------------------------------------
export const protocol = MediaProtocol.stream<Request, SpeechEvent, string, State>(route, {
unsupported: ["instructions", "speed", "timestamps"],
unsupported: ["instructions", "speed"],
body: { from: fromRequest },
frames: (bytes, context) => GeminiGenerateContent.frames(bytes, context.request.mode),
initial: () => ({ chunks: [] }),
@@ -154,6 +154,9 @@ const step = Effect.fn("GoogleTranscription.step")(function* (state: State, fram
.filter((item) => item.length > 0)
.join(" ")
const delta = text.length === 0 || state.text.length === 0 ? text : ` ${text}`
const withheld =
state.text.length + delta.length === 0 ? GeminiGenerateContent.withheld(route.name, chunk, frame) : undefined
if (withheld !== undefined) return yield* withheld
const events: ReadonlyArray<TranscriptionEvent> = [
...(delta.length === 0 ? [] : [TranscriptionTextDeltaEvent.make({ delta })]),
...segments.map((segment) => TranscriptionSegmentEvent.make({ segment })),
@@ -169,6 +172,7 @@ const finish = (state: State) => {
segments: state.segments.length === 0 ? undefined : state.segments,
words: state.words.length === 0 ? undefined : state.words,
usage: GeminiGenerateContent.usage(state.usage),
notices: GeminiGenerateContent.notices(route.name, state),
providerMetadata: GeminiGenerateContent.providerMetadata(state),
}),
])
+15 -3
View File
@@ -36,7 +36,9 @@ const StartResponse = Schema.Struct({ name: Schema.String })
const Operation = Schema.Struct({
done: Schema.optional(Schema.Boolean),
error: Schema.optional(Schema.Struct({ message: Schema.optional(Schema.String) })),
error: Schema.optional(
Schema.Struct({ code: Schema.optional(Schema.Number), message: Schema.optional(Schema.String) }),
),
response: Schema.optional(
Schema.Struct({
generateVideoResponse: Schema.optional(
@@ -60,6 +62,16 @@ const Operation = Schema.Struct({
metadata: Schema.optional(Schema.Unknown),
})
// Operation errors are `google.rpc.Status`; unlisted codes (INTERNAL, UNAVAILABLE, ...) are provider-side.
const FAILURE = {
3: "InvalidRequest", // INVALID_ARGUMENT
7: "Authentication", // PERMISSION_DENIED
8: "RateLimit", // RESOURCE_EXHAUSTED
9: "InvalidRequest", // FAILED_PRECONDITION
11: "InvalidRequest", // OUT_OF_RANGE
16: "Authentication", // UNAUTHENTICATED
} as const satisfies Record<number, MediaProtocol.Failure>
// ---------------------------------------------------------------------------
// 5. Request body construction
// ---------------------------------------------------------------------------
@@ -149,12 +161,12 @@ const decodeResult = Effect.fn("GoogleVideo.decodeResult")(function* (
const output = yield* decodeOperation(response)
const operation = output.value
const status = statusOf(operation)
if (status === "running")
return yield* output.invalid(`${route.name} operation ${context.token.operation} has not finished`)
if (status === "running") return yield* output.pending(context.token.operation)
if (status === "failed")
return yield* output.ended(
"failed",
`${route.name} operation failed${operation.error?.message === undefined ? "" : `: ${operation.error.message}`}`,
MediaProtocol.failure(FAILURE, operation.error?.code),
)
const generated = operation.response?.generateVideoResponse
// Downloads require the same API key as the poll; the asset carries it transiently and follows the redirect.
+54 -24
View File
@@ -48,15 +48,17 @@ const Usage = Schema.Struct({
output_tokens_details: Schema.optional(Schema.Record(Schema.String, Schema.Unknown)),
})
const OpenAIImageResponse = Schema.Struct({
data: Schema.Array(
Schema.Struct({
b64_json: Schema.optional(Schema.String),
url: Schema.optional(Schema.String),
revised_prompt: Schema.optional(Schema.String),
}),
),
/** What the provider actually rendered; it can differ from the request when `auto` or a default applied. */
const Settings = {
output_format: Schema.optional(Schema.String),
size: Schema.optional(Schema.String),
quality: Schema.optional(Schema.String),
background: Schema.optional(Schema.String),
}
const OpenAIImageResponse = Schema.Struct({
data: Schema.Array(Schema.Struct({ b64_json: Schema.String })),
...Settings,
usage: Schema.optional(Usage),
})
@@ -69,11 +71,13 @@ const StreamEvent = Schema.Union([
type: Schema.Literals(["image_generation.partial_image", "image_edit.partial_image"]),
b64_json: Schema.String,
partial_image_index: Schema.Number,
...Settings,
output_format: Schema.String,
}),
Schema.Struct({
type: Schema.Literals(["image_generation.completed", "image_edit.completed"]),
b64_json: Schema.String,
...Settings,
output_format: Schema.String,
usage: Schema.optional(Usage),
}),
@@ -92,6 +96,9 @@ type Frame = string | { readonly document: string; readonly requested: string |
interface State {
readonly completed: number
readonly format?: string
readonly size?: string
readonly quality?: string
readonly background?: string
readonly usage?: MediaUsage
}
@@ -110,10 +117,6 @@ const nativeOptions = (options: OpenAIImageOptions | undefined) => {
const streamOptions = (request: MediaProtocol.Addressed<Request>) => {
if (request.mode !== "stream") return Effect.succeed(undefined)
if (request.model.id.startsWith("dall-e"))
return Effect.fail(
route.unsupported("media.stream", `${request.model.id} does not stream; use Image.generate or a GPT image model`),
)
if (request.n !== undefined && request.n > 1)
return Effect.fail(
route.unsupported("media.n", `${route.name} streams one image; use Image.generate for n=${request.n}`),
@@ -194,21 +197,34 @@ const usage = (value: Schema.Schema.Type<typeof Usage> | undefined): MediaUsage
details: { openai: value },
}
const eventImage = (frame: string, label: string, data: string, format: string) =>
/** `size` echoes the rendered `WIDTHxHEIGHT`; `auto` or any other value leaves the dimensions unknown. */
const info = (format: string, size: string | undefined): Media.Info => {
const match = size?.match(/^(\d+)x(\d+)$/)
return match ? { format, width: Number(match[1]), height: Number(match[2]) } : { format }
}
const eventImage = (frame: string, label: string, data: string, format: string, size: string | undefined) =>
MediaInput.decodedAsset((message, cause) => route.frameError(message, frame, cause), label, data, `image/${format}`, {
info: { format },
info: info(format, size),
})
const onEvent = Effect.fn("OpenAIImages.onEvent")(function* (state: State, frame: string) {
const event = yield* decodeEvent(frame)
const format = event.output_format
if ("partial_image_index" in event) {
const image = yield* eventImage(frame, `${route.name} partial image`, event.b64_json, format)
const image = yield* eventImage(frame, `${route.name} partial image`, event.b64_json, format, event.size)
return [state, [ImagePartialEvent.make({ index: event.partial_image_index, image })]] as const
}
const image = yield* eventImage(frame, `${route.name} result ${state.completed}`, event.b64_json, format)
const image = yield* eventImage(frame, `${route.name} result ${state.completed}`, event.b64_json, format, event.size)
return [
{ ...state, completed: state.completed + 1, format, usage: usage(event.usage) },
{
completed: state.completed + 1,
format,
size: event.size,
quality: event.quality,
background: event.background,
usage: usage(event.usage),
},
[ImageOutputEvent.make({ index: state.completed, image })],
] as const
})
@@ -219,16 +235,20 @@ const onDocument = Effect.fn("OpenAIImages.onDocument")(function* (frame: Exclud
Effect.mapError((cause) => invalid(`${route.name} returned an invalid response`, cause)),
)
const format = decoded.output_format ?? frame.requested ?? "png"
const mediaType = `image/${format}`
const images = yield* Effect.forEach(decoded.data, (item, index) =>
MediaInput.imageOutput(invalid, `${route.name} result ${index}`, item, mediaType, {
info: { format },
providerMetadata:
item.revised_prompt === undefined ? undefined : { openai: { revisedPrompt: item.revised_prompt } },
MediaInput.decodedAsset(invalid, `${route.name} result ${index}`, item.b64_json, `image/${format}`, {
info: info(format, decoded.size),
}),
)
if (images.length === 0) return yield* invalid(`${route.name} returned no images`)
const state: State = { completed: images.length, format, usage: usage(decoded.usage) }
const state: State = {
completed: images.length,
format,
size: decoded.size,
quality: decoded.quality,
background: decoded.background,
usage: usage(decoded.usage),
}
return [state, images.map((image, index) => ImageOutputEvent.make({ index, image }))] as const
})
@@ -237,7 +257,17 @@ const step = (state: State, frame: Frame) => (typeof frame === "string" ? onEven
const finish = (state: State) => {
if (state.completed === 0) return Effect.fail(route.incomplete())
return Effect.succeed([
ImageFinishEvent.make({ usage: state.usage, providerMetadata: { openai: { outputFormat: state.format } } }),
ImageFinishEvent.make({
usage: state.usage,
providerMetadata: {
openai: {
outputFormat: state.format,
size: state.size,
quality: state.quality,
background: state.background,
},
},
}),
])
}
@@ -143,12 +143,14 @@ const adapter = {
restoreHostedToolItem: (item: unknown) => (Schema.is(OpenAIResponsesHostedToolItem)(item) ? item : undefined),
} satisfies OpenResponses.ProviderAdapter
// Only GPT-6 Astra accepts `configuration_update`, and never alongside automatic `context_management` compaction.
// GPT-6 Astra, Sol, and Luna accept `configuration_update` only in standard mode (not `reasoning.mode: "pro"` or
// `-pro` slugs), and never alongside automatic `context_management` compaction.
const supportsEffortUpdates = (request: LLMRequest) => {
if (request.providerOptions?.contextManagement !== undefined) return false
if (Schema.is(Schema.Struct({ mode: Schema.Literal("pro") }))(request.http?.body?.reasoning)) return false
const override = request.model.compatibility?.supportsEffortUpdates
if (override !== undefined) return override
return /(?:^|\/)gpt-6-astra$/i.test(request.model.id)
return /(?:^|\/)gpt-6-(?:astra|sol|luna)$/i.test(request.model.id)
}
const nativeImageToolInput = (tool: ToolDefinition) => {
+14 -2
View File
@@ -60,7 +60,17 @@ interface State extends SpeechStream.Audio {
// `sse` is not supported for `tts-1` or `tts-1-hd`; those models stream the raw audio body instead.
const supportsSse = (model: string) => !/^tts-1(-hd)?(-|$)/.test(model)
const FORMATS = new Set(["mp3", "opus", "aac", "flac", "wav", "pcm"])
const fromRequest = Effect.fn("OpenAISpeech.fromRequest")(function* (request: MediaProtocol.Addressed<Request>) {
// Not in `unsupported`: that list would also reject `timestamps: false`, which asks for nothing.
if (request.timestamps === true)
return yield* route.unsupported("media.timestamps", `${route.name} does not return timestamps`)
if (request.format !== undefined && !FORMATS.has(request.format))
return yield* route.unsupported(
"media.format",
`${route.name} supports the mp3, opus, aac, flac, wav, and pcm formats, not "${request.format}"`,
)
return MediaProtocol.json(
mergeJsonRecords(
{
@@ -109,7 +119,9 @@ const onEvent = Effect.fn("OpenAISpeech.onEvent")(function* (state: State, frame
const finish = (state: State, context: MediaProtocol.ResponseContext<Request>) => {
if (isSse(context.body) && !state.done) return Effect.fail(route.incomplete())
const format = context.request.format ?? "mp3"
// The sent body reflects `providerOptions` and `http.body` overrides of `format`.
const sent = context.body.type === "json" ? context.body.value.response_format : undefined
const format = typeof sent === "string" ? sent : "mp3"
return SpeechStream.finish(route, state, {
...(format === "pcm" ? SpeechStream.pcm("pcm_s16le", PCM_SAMPLE_RATE) : SpeechStream.container(format)),
usage: state.usage,
@@ -121,7 +133,7 @@ const finish = (state: State, context: MediaProtocol.ResponseContext<Request>) =
// ---------------------------------------------------------------------------
export const protocol = MediaProtocol.stream<Request, SpeechEvent, string | Uint8Array, State>(route, {
unsupported: ["language", "timestamps"],
unsupported: ["language"],
body: { from: fromRequest },
frames: (bytes, context) => (isSse(context.body) ? Framing.sse.frame(bytes) : bytes),
initial: () => ({ chunks: [], done: false }),
@@ -1,8 +1,9 @@
import { Effect, Schema, Stream } from "effect"
import { classifyProviderFailure } from "../provider-error.js"
import { Framing } from "../route/framing.js"
import { MediaProtocol } from "../route/media-protocol.js"
import { MediaRoute } from "../route/media.js"
import { mergeJsonRecords, type MediaUsage } from "../schema/index.js"
import { AIError, mergeJsonRecords, type MediaUsage } from "../schema/index.js"
import {
TranscriptionFinishEvent,
TranscriptionModel,
@@ -59,6 +60,9 @@ const Usage = Schema.Union([
input_tokens: Schema.optional(Schema.Number),
output_tokens: Schema.optional(Schema.Number),
total_tokens: Schema.optional(Schema.Number),
input_token_details: Schema.optional(
Schema.Struct({ audio_tokens: Schema.optional(Schema.Number), text_tokens: Schema.optional(Schema.Number) }),
),
}),
Schema.Struct({ type: Schema.Literal("duration"), seconds: Schema.Number }),
])
@@ -75,14 +79,23 @@ const transcriptFields = {
usage: Schema.optional(Usage),
}
/** OpenAI may add stream event types; frames outside `EVENT_TYPES` are ignored. */
const EventType = Schema.Struct({ type: Schema.String })
const Event = Schema.Union([
Schema.Struct({ type: Schema.Literal("transcript.text.delta"), delta: Schema.String }),
Schema.Struct({ type: Schema.Literal("transcript.text.segment"), ...Segment.fields }),
Schema.Struct({ type: Schema.Literal("transcript.text.done"), ...transcriptFields }),
Schema.Struct({
type: Schema.Literal("error"),
message: Schema.optional(Schema.String),
error: Schema.optional(Schema.Struct({ message: Schema.optional(Schema.String) })),
}),
])
const EVENT_TYPES = new Set(["transcript.text.delta", "transcript.text.segment", "transcript.text.done", "error"])
const Transcript = Schema.Struct(transcriptFields)
type Transcript = Schema.Schema.Type<typeof Transcript>
const decodeEventType = route.decodeFrame(EventType)
const decodeEvent = route.decodeFrame(Event)
const decodeTranscript = route.decodeFrame(Transcript)
@@ -118,10 +131,12 @@ const capabilities = (model: string): Capabilities => {
return TRANSCRIBE
}
/** whisper-1 ignores `stream`, so its `stream` mode sends a plain request and emits only `finish`. */
const streamsEvents = (request: MediaProtocol.Addressed<Request>) =>
request.mode === "stream" && capabilities(request.model.id).stream
const validate = (request: MediaProtocol.Addressed<Request>, model: Capabilities) => {
const id = request.model.id
if (request.mode === "stream" && !model.stream)
return Effect.fail(route.unsupported("media.stream", `${id} does not stream; use Transcription.generate`))
if (request.diarize === true && !model.diarize)
return Effect.fail(route.unsupported("media.diarize", `${id} does not diarize; use gpt-4o-transcribe-diarize`))
if (request.prompt !== undefined && model.diarize)
@@ -173,7 +188,7 @@ const fromRequest = Effect.fn("OpenAITranscription.fromRequest")(function* (requ
timestamp_granularities: responseFormat === "verbose_json" ? [request.timestamps] : undefined,
// Diarizing audio longer than 30 seconds requires a chunking strategy.
chunking_strategy: model.diarize ? "auto" : undefined,
stream: request.mode === "stream" ? true : undefined,
stream: streamsEvents(request) ? true : undefined,
},
{
overlay: mergeJsonRecords(request.providerOptions, request.http?.body),
@@ -196,7 +211,15 @@ const segment = (value: Schema.Schema.Type<typeof Segment>): TranscriptionSegmen
})
const onEvent = Effect.fn("OpenAITranscription.onEvent")(function* (state: State, frame: string) {
if (!EVENT_TYPES.has((yield* decodeEventType(frame)).type)) return [state, []] as const
const event = yield* decodeEvent(frame)
if (event.type === "error")
return yield* new AIError({
reason: classifyProviderFailure({
message: `${route.name} stream failed: ${event.message ?? event.error?.message ?? "unknown error"}`,
rawBody: frame,
}),
})
if (event.type === "transcript.text.done") return [{ ...state, transcript: event }, []] as const
if (event.type === "transcript.text.delta")
return [state, event.delta.length === 0 ? [] : [TranscriptionTextDeltaEvent.make({ delta: event.delta })]] as const
@@ -246,7 +269,7 @@ export const protocol = MediaProtocol.stream<Request, TranscriptionEvent, Frame,
unsupported: ["speakers"],
body: { from: fromRequest },
frames: (bytes, context) =>
context.request.mode === "stream"
streamsEvents(context.request)
? Framing.sse.frame(bytes)
: Framing.document.frame(bytes).pipe(Stream.map((document) => ({ document }))),
initial: () => ({ segments: [] }),
@@ -132,8 +132,7 @@ const decodeResult = Effect.fn("ReplicateImages.decodeResult")(function* (
status,
`${route.name} prediction ${context.token.id} ${prediction.status}${typeof prediction.error === "string" ? `: ${prediction.error}` : ""}`,
)
if (status !== "completed")
return yield* output.invalid(`${route.name} prediction ${context.token.id} has not finished`)
if (status !== "completed") return yield* output.pending(context.token.id)
if (prediction.data_removed === true)
return yield* output.ended("expired", `${route.name} removed the output of prediction ${context.token.id}`)
if (!isOutput(prediction.output))
+8 -4
View File
@@ -137,12 +137,16 @@ const decodeResult = Effect.fn("RunwayVideo.decodeResult")(function* (
const message = `${route.name} task failed${code === undefined ? "" : ` (${code})`}${task.failure ? `: ${task.failure}` : ""}`
// Runway failure codes are dotted paths; every moderation outcome carries a SAFETY segment.
if (code !== undefined && /(^|\.)SAFETY(\.|$)/.test(code)) return yield* output.contentPolicy(message)
return yield* output.ended("failed", message)
// ASSET.INVALID rejects the caller's input media; Runway documents it as not retryable.
return yield* output.ended(
"failed",
message,
code !== undefined && /^ASSET\.INVALID(\.|$)/.test(code) ? "InvalidRequest" : "ProviderInternal",
)
}
if (status === "cancelled")
return yield* output.ended("cancelled", `${route.name} task ${context.token.taskID} was cancelled`)
if (status !== "completed")
return yield* output.invalid(`${route.name} task ${context.token.taskID} has not finished`)
if (status !== "completed") return yield* output.pending(context.token.taskID)
const urls = task.output ?? []
if (urls.length === 0) return yield* output.invalid(`${route.name} task succeeded without any output`)
return new VideoResponse({
@@ -171,7 +175,7 @@ export const protocol = MediaProtocol.queued<Request, VideoResponse, Token>(rout
start: { body: { from: fromRequest }, decode: decodeStart },
status: { path: taskPath, decode: decodeStatus },
result: { path: taskPath, decode: decodeResult },
cancel: { method: "DELETE", path: taskPath },
cancel: { method: "DELETE", path: taskPath, activeOnly: true },
})
const startPath = (request: Request) => {
@@ -175,7 +175,7 @@ const decodeUpscaleResult = Effect.fn("StabilityImages.decodeUpscaleResult")(fun
) {
if (response.status === 202) {
const output = yield* upscaleRoute.text(response)
return yield* output.invalid(`${upscaleRoute.name} upscale ${context.token.id} has not finished`)
return yield* output.pending(context.token.id)
}
return yield* decodeUpscaleImage(response)
})
@@ -72,6 +72,44 @@ export const blocked = (name: string, chunk: Chunk, frame: string) => {
})
}
const CONTENT_FILTER_REASONS = new Set([
"IMAGE_SAFETY",
"RECITATION",
"SAFETY",
"BLOCKLIST",
"PROHIBITED_CONTENT",
"SPII",
"MODEL_ARMOR",
"IMAGE_PROHIBITED_CONTENT",
"IMAGE_RECITATION",
"LANGUAGE",
])
/** Finish reasons for which Gemini stops output on safety or policy grounds. */
export const contentFiltered = (finishReason: string | undefined) =>
finishReason !== undefined && CONTENT_FILTER_REASONS.has(finishReason)
/** Callers check that the response produced no output: a policy stop after output is a partial result instead. */
export const withheld = (name: string, chunk: Chunk, frame: string) => {
const finishReason = chunk.candidates?.[0]?.finishReason
if (!contentFiltered(finishReason)) return undefined
return new AIError({
reason: new ContentPolicyError({ message: `${name} withheld its output (${finishReason})`, body: frame }),
})
}
/** Any finish reason other than `STOP` means the output may be cut short, so it is surfaced rather than dropped. */
export const notices = (name: string, state: Metadata): ReadonlyArray<Media.Notice> | undefined =>
state.finishReason === undefined || state.finishReason === "STOP"
? undefined
: [
{
type: contentFiltered(state.finishReason) ? "filtered" : "other",
message: `${name} finished with ${state.finishReason}`,
providerMetadata: { google: { finishReason: state.finishReason } },
},
]
export const usage = (usage: UsageMetadata | undefined): MediaUsage | undefined =>
usage === undefined
? undefined
@@ -71,8 +71,9 @@ export const imageOutput = (
}
/**
* Append multipart text fields: strings as-is, other values as JSON, or arrays as repeated parts named `key[]` or
* `key` with `repeatArrays`. `overlay` keys in `reserved` are dropped so `http.body` cannot replace route-owned fields.
* Append multipart text fields: strings as-is, other values as JSON, or scalar arrays as one part per item with
* `repeatArrays`, named `key[]` or `key`. `overlay` keys in `reserved` are dropped so `http.body` cannot replace
* route-owned fields.
*/
export const appendFields = (
form: FormData,
@@ -85,7 +86,7 @@ export const appendFields = (
) => {
const overlay = Object.entries(options.overlay ?? {}).filter(([key]) => !options.reserved.has(key))
Object.entries(mergeJsonRecords(fields, Object.fromEntries(overlay)) ?? {}).forEach(([key, value]) => {
if (Array.isArray(value) && options.repeatArrays !== undefined)
if (Array.isArray(value) && value.every(isScalar) && options.repeatArrays !== undefined)
return value.forEach((item) => form.append(options.repeatArrays === "key[]" ? `${key}[]` : key, String(item)))
form.append(key, typeof value === "string" ? value : encodeJson(value))
})
@@ -0,0 +1,10 @@
/** Split an ordered token list into runs of consecutive tokens with the same speaker. */
export const group = <Item>(items: ReadonlyArray<Item>, speaker: (item: Item) => unknown) =>
items.reduce<Array<Array<Item>>>((turns, item) => {
const last = turns.at(-1)
if (last === undefined || speaker(last[0]) !== speaker(item)) return [...turns, [item]]
last.push(item)
return turns
}, [])
export * as SpeakerTurns from "./speaker-turns.js"
@@ -85,6 +85,7 @@ export const finish = (
readonly mediaType: string | undefined
readonly info?: Media.Info
readonly usage?: MediaUsage
readonly notices?: ReadonlyArray<Media.Notice>
readonly providerMetadata?: ProviderMetadata
readonly detail?: string
},
@@ -97,6 +98,7 @@ export const finish = (
SpeechFinishEvent.make({
audio: Media.bytes(concatBytes(state.chunks), output.mediaType, { info: output.info }),
usage: output.usage,
notices: output.notices,
providerMetadata: output.providerMetadata,
}),
])
+2 -1
View File
@@ -101,7 +101,8 @@ const decodeResponse = Effect.fn("XAIImages.decodeResponse")(function* (
)
if (images.length === 0) return yield* output.invalid(`${route.name} returned no images`)
const usage = ProviderShared.isRecord(decoded.usage) ? decoded.usage : undefined
// xAI reports image counts rather than tokens, seconds, or credits; the raw record stays in provider metadata.
// xAI reports a USD cost (`cost_in_usd_ticks`) rather than tokens, seconds, or credits; the raw record stays in
// provider metadata.
return new ImageResponse({
images,
providerMetadata: usage === undefined ? undefined : { xai: { usage } },
-112
View File
@@ -1,112 +0,0 @@
import { Effect } from "effect"
import { MediaProtocol } from "../route/media-protocol.js"
import { MediaRoute } from "../route/media.js"
import { mergeJsonRecords } from "../schema/index.js"
import { SpeechModel, type SpeechEvent, type SpeechRequestFor } from "../speech.js"
import { SpeechStream } from "./utils/speech-stream.js"
const route = MediaProtocol.identity({ id: "xai-speech", name: "xAI Speech", provider: "xai" })
export const DEFAULT_BASE_URL = "https://api.x.ai/v1"
export const PATH = "/tts"
const DEFAULT_SAMPLE_RATE = 24000
// ---------------------------------------------------------------------------
// 1. Public model input
// ---------------------------------------------------------------------------
/** `voice`, `format`, `speed`, and `language` are common request fields; other native body fields pass through. */
export type XAISpeechOptions = {
readonly sampleRate?: 8000 | 16000 | 22050 | 24000 | 44100 | 48000
/** MP3 only. */
readonly bitRate?: 32000 | 64000 | 96000 | 128000 | 192000
readonly optimize_streaming_latency?: number
readonly text_normalization?: boolean
readonly replace?: Readonly<Record<string, string>>
} & Record<string, unknown>
export type Request = SpeechRequestFor<XAISpeechOptions>
// ---------------------------------------------------------------------------
// 4. Parser state
// ---------------------------------------------------------------------------
type State = SpeechStream.Audio
// ---------------------------------------------------------------------------
// 5. Request body construction
// ---------------------------------------------------------------------------
/** `output_format.codec` values; headerless codecs map to the PCM encoding of their samples. */
const CODECS = new Map<string, SpeechStream.PcmEncoding | undefined>([
["mp3", undefined],
["wav", undefined],
["pcm", "pcm_s16le"],
["mulaw", "pcm_mulaw"],
["alaw", "pcm_alaw"],
])
const outputFormat = Effect.fn("XAISpeech.outputFormat")(function* (request: Request) {
const codec = request.format ?? "mp3"
if (!CODECS.has(codec))
return yield* route.unsupported(
"media.format",
`${route.name} supports the mp3, wav, pcm, mulaw, and alaw formats, not "${codec}"`,
)
return { codec, sample_rate: request.providerOptions?.sampleRate, bit_rate: request.providerOptions?.bitRate }
})
// The TTS API has no model field, so the selected model id only names the model.
const fromRequest = Effect.fn("XAISpeech.fromRequest")(function* (request: MediaProtocol.Addressed<Request>) {
const { sampleRate: _sampleRate, bitRate: _bitRate, ...native } = request.providerOptions ?? {}
return MediaProtocol.json(
mergeJsonRecords(
{
text: request.text,
voice_id: SpeechStream.voiceID(request.voice),
// `language` is required; `auto` detects it from the text.
language: request.language ?? "auto",
output_format: yield* outputFormat(request),
speed: request.speed,
},
native,
request.http?.body,
) ?? {},
)
})
// ---------------------------------------------------------------------------
// 6. Stream parsing
// ---------------------------------------------------------------------------
const finish = Effect.fn("XAISpeech.finish")(function* (state: State, context: MediaProtocol.ResponseContext<Request>) {
const format = yield* outputFormat(context.request)
const sampleRate = format.sample_rate ?? DEFAULT_SAMPLE_RATE
const encoding = CODECS.get(format.codec)
return yield* SpeechStream.finish(
route,
state,
encoding === undefined ? SpeechStream.container(format.codec, sampleRate) : SpeechStream.pcm(encoding, sampleRate),
)
})
// ---------------------------------------------------------------------------
// 7. Protocol and route
// ---------------------------------------------------------------------------
/** The response body is the raw audio in both modes, so `stream` forwards body chunks as they arrive. */
export const protocol = MediaProtocol.stream<Request, SpeechEvent, Uint8Array, State>(route, {
unsupported: ["instructions", "timestamps"],
body: { from: fromRequest },
frames: (bytes) => bytes,
initial: () => ({ chunks: [] }),
step: (state, frame) => Effect.succeed(SpeechStream.delta(state, frame)),
finish,
})
export const model = (input: MediaRoute.ModelInput) =>
SpeechModel.fromRoute<XAISpeechOptions, Uint8Array, State>({ protocol, baseURL: DEFAULT_BASE_URL, path: PATH }, input)
export const XAISpeech = {
protocol,
model,
} as const
@@ -1,179 +0,0 @@
import { Effect, Schema } from "effect"
import type { HttpClientResponse } from "effect/unstable/http"
import { MediaProtocol } from "../route/media-protocol.js"
import { MediaRoute } from "../route/media.js"
import { mergeJsonRecords, type OpenString } from "../schema/index.js"
import {
TranscriptionModel,
TranscriptionResponse,
type TranscriptionRequestFor,
type TranscriptionSegment,
type TranscriptionWord,
} from "../transcription.js"
import { mediaTypeExtension } from "../utils/media-type.js"
import { ProviderShared } from "./shared.js"
import { MediaInput } from "./utils/media-input.js"
const route = MediaProtocol.identity({ id: "xai-transcription", name: "xAI Transcription", provider: "xai" })
export const DEFAULT_BASE_URL = "https://api.x.ai/v1"
export const PATH = "/stt"
// ---------------------------------------------------------------------------
// 1. Public model input
// ---------------------------------------------------------------------------
export type XAITranscriptionOptions = {
/** Inverse text normalization ("one hundred dollars" → "$100"); requires `language`. */
readonly format?: boolean
readonly keyterm?: ReadonlyArray<string>
readonly filler_words?: boolean
/** Headerless audio only; derived from `audio.info.encoding` and `audio.info.sampleRate` when omitted. */
readonly audio_format?: OpenString<"pcm" | "mulaw" | "alaw">
readonly sample_rate?: 8000 | 16000 | 22050 | 24000 | 44100 | 48000
readonly multichannel?: boolean
readonly channels?: number
readonly vad_threshold?: number
} & Record<string, unknown>
export type Request = TranscriptionRequestFor<XAITranscriptionOptions>
// ---------------------------------------------------------------------------
// 2. Response schema
// ---------------------------------------------------------------------------
const Word = Schema.Struct({
text: Schema.String,
start: Schema.Number,
end: Schema.Number,
confidence: Schema.optional(Schema.Number),
speaker: Schema.optional(Schema.Number),
})
const SttResponse = Schema.Struct({
text: Schema.String,
language: Schema.optional(Schema.String),
duration: Schema.optional(Schema.Number),
words: Schema.optional(Schema.Array(Word)),
channels: Schema.optional(
Schema.Array(
Schema.Struct({
index: Schema.Number,
text: Schema.String,
language: Schema.optional(Schema.String),
words: Schema.optional(Schema.Array(Word)),
}),
),
),
})
// ---------------------------------------------------------------------------
// 5. Request body construction
// ---------------------------------------------------------------------------
/** xAI returns only words, so segments are diarized speaker turns, as with AssemblyAI utterances. */
const wantsSegments = (request: Request) => request.diarize === true || request.timestamps === "segment"
const RAW_AUDIO_FORMATS: Readonly<Record<string, string>> = {
pcm_s16le: "pcm",
pcm_mulaw: "mulaw",
pcm_alaw: "alaw",
}
const RESERVED_FORM_FIELDS = new Set(["file", "url", "model", "language", "diarize"])
const fromRequest = Effect.fn("XAITranscription.fromRequest")(function* (request: Request) {
const audioFormat = RAW_AUDIO_FORMATS[request.audio.info?.encoding ?? ""]
const form = new FormData()
MediaInput.appendFields(
form,
{
model: request.model.id,
language: request.language,
diarize: wantsSegments(request) ? true : undefined,
audio_format: audioFormat,
sample_rate: audioFormat === undefined ? undefined : request.audio.info?.sampleRate,
},
{
overlay: mergeJsonRecords(request.providerOptions, request.http?.body),
reserved: RESERVED_FORM_FIELDS,
repeatArrays: "key",
},
)
// `file` must be the last field: options after it may be ignored for streamed uploads.
const url = ProviderShared.mediaUrl(request.audio)
if (url !== undefined) {
form.append("url", url)
return MediaProtocol.multipart(form)
}
const audio = yield* MediaInput.inlineBytes(route.id, request.audio)
const extension = mediaTypeExtension(request.audio.mediaType)
form.append(
"file",
MediaInput.blob(audio, request.audio.mediaType),
extension === undefined ? "audio" : `audio.${extension}`,
)
return MediaProtocol.multipart(form)
})
// ---------------------------------------------------------------------------
// 6. Response decoding
// ---------------------------------------------------------------------------
const decodeStt = route.decodeJson(SttResponse)
const word = (value: typeof Word.Type): TranscriptionWord => ({
text: value.text,
startSeconds: value.start,
endSeconds: value.end,
speaker: value.speaker === undefined ? undefined : String(value.speaker),
confidence: value.confidence,
})
const speakerSegments = (words: ReadonlyArray<TranscriptionWord>) =>
words.reduce<Array<TranscriptionSegment>>((turns, next) => {
const last = turns.at(-1)
if (last === undefined || last.speaker !== next.speaker)
return [
...turns,
{ text: next.text, startSeconds: next.startSeconds, endSeconds: next.endSeconds, speaker: next.speaker },
]
turns[turns.length - 1] = { ...last, text: `${last.text} ${next.text}`, endSeconds: next.endSeconds }
return turns
}, [])
const decodeResponse = Effect.fn("XAITranscription.decodeResponse")(function* (
response: HttpClientResponse.HttpClientResponse,
context: MediaProtocol.DecodeContext<Request>,
) {
const output = yield* decodeStt(response)
const transcript = output.value
const words = transcript.words?.map(word)
const duration = transcript.duration
return new TranscriptionResponse({
text: transcript.text,
segments: words === undefined || !wantsSegments(context.request) ? undefined : speakerSegments(words),
words,
language: transcript.language?.toLowerCase(),
durationSeconds: duration,
usage: duration === undefined ? undefined : { type: "seconds", seconds: duration },
providerMetadata: transcript.channels === undefined ? undefined : { xai: { channels: transcript.channels } },
})
})
// ---------------------------------------------------------------------------
// 7. Protocol and route
// ---------------------------------------------------------------------------
export const protocol = MediaProtocol.inline<Request, TranscriptionResponse>(route, {
unsupported: ["prompt", "speakers"],
body: { from: fromRequest },
response: { decode: decodeResponse },
})
export const model = (input: MediaRoute.ModelInput) =>
TranscriptionModel.fromRoute<XAITranscriptionOptions>({ protocol, baseURL: DEFAULT_BASE_URL, path: PATH }, input)
export const XAITranscription = {
protocol,
model,
} as const
+9 -2
View File
@@ -65,6 +65,13 @@ const STATUS = {
expired: "expired",
} as const satisfies Record<string, Status>
// Documented video error codes; `service_unavailable`, `internal_error`, and unknown codes are provider-side.
const FAILURE = {
invalid_argument: "InvalidRequest",
failed_precondition: "InvalidRequest",
permission_denied: "Authentication",
} as const satisfies Record<string, MediaProtocol.Failure>
// ---------------------------------------------------------------------------
// 5. Request body construction
// ---------------------------------------------------------------------------
@@ -136,14 +143,14 @@ const decodeResult = Effect.fn("XAIVideo.decodeResult")(function* (
const output = yield* decodeVideoStatus(response)
const decoded = output.value
const status = yield* MediaProtocol.status(STATUS, decoded.status, output)
if (status === "running")
return yield* output.invalid(`${route.name} request ${context.token.requestID} has not finished`)
if (status === "running") return yield* output.pending(context.token.requestID)
if (status === "failed") {
const code = decoded.error?.code ?? undefined
const message = decoded.error?.message ?? undefined
return yield* output.ended(
"failed",
`${route.name} generation failed${code === undefined ? "" : ` (${code})`}${message === undefined ? "" : `: ${message}`}`,
MediaProtocol.failure(FAILURE, code),
)
}
if (status !== "completed")
+3 -3
View File
@@ -1,7 +1,6 @@
import { Effect, Schema } from "effect"
import { Duration, Effect, Schema } from "effect"
import type { HttpClientResponse } from "effect/unstable/http"
import { ImageModel, ImageResponse, type ImageRequestFor } from "../image.js"
import { Media } from "../media.js"
import { MediaProtocol } from "../route/media-protocol.js"
import { MediaRoute } from "../route/media.js"
import { mergeJsonRecords, type OpenString } from "../schema/index.js"
@@ -9,6 +8,7 @@ import { mergeJsonRecords, type OpenString } from "../schema/index.js"
const route = MediaProtocol.identity({ id: "zai-images", name: "Z.ai Images", provider: "zai" })
export const DEFAULT_BASE_URL = "https://api.z.ai/api/paas/v4"
export const PATH = "/images/generations"
const OUTPUT_RETENTION = Duration.days(30)
// ---------------------------------------------------------------------------
// 1. Public model input
@@ -76,7 +76,7 @@ const decodeResponse = Effect.fn("ZAIImages.decodeResponse")(function* (
const filters = decoded.content_filter ?? []
return new ImageResponse({
// Z.ai returns only URLs and no content type; the media type resolves when the asset is materialized.
images: decoded.data.map((item) => Media.url(item.url)),
images: yield* Effect.forEach(decoded.data, (item) => MediaProtocol.expiringUrl(item.url, OUTPUT_RETENTION)),
// Z.ai reports applied content filters alongside a successful result; surface them instead of dropping them.
notices:
filters.length === 0
+41
View File
@@ -58,6 +58,47 @@ export const isContextOverflowFailure = (failure: unknown) =>
? failure.reason._tag === "InvalidRequest" && failure.reason.classification === "context-overflow"
: Schema.is(ProviderErrorEvent)(failure) && failure.classification === "context-overflow"
/**
* Whether a failed call may succeed when sent again: rate limits, provider-side failures, transport failures that did
* not deliver an accepted write, and unrecognized failures. Callers decide which calls are safe to repeat.
*/
export const isRetryable = (error: AIError) => {
const override = error.reason.http?.headers["x-should-retry"]
if (override === "true") return true
if (override === "false") return false
switch (error.reason._tag) {
case "RateLimit":
case "ProviderInternal":
return true
// A WebSocket acknowledgment marks delivery accepted before model output may exist.
// Read failures can still recover; the caller chooses retry versus continuation from durable output.
case "Transport":
return (
error.reason.delivery !== "rejected" &&
(error.reason.delivery !== "accepted" || error.reason.operation === "read")
)
case "InvalidProviderOutput":
return error.reason.classification === "incomplete-stream"
// Unrecognized failures retry: classification records affirmative
// deterministic evidence, and transient failures are exactly the ones
// that arrive in shapes no classifier anticipates.
case "UnknownProvider":
return true
case "Authentication":
case "QuotaExceeded":
case "ContentPolicy":
case "InvalidRequest":
case "UnsupportedOperation":
case "NoRoute":
case "Timeout":
return false
default: {
const exhaustive: never = error.reason
return exhaustive
}
}
}
const decodeJson = Schema.decodeUnknownOption(Schema.fromJsonString(Schema.Unknown))
// OpenCode Zen reports account caps as typed 429/402 errors that are not throttles.
const QUOTA_CODES = new Set([
+5
View File
@@ -3,8 +3,10 @@ import type { ProviderAuthOption } from "../route/auth-options.js"
import { MediaRoute } from "../route/media.js"
import { type HttpOptions, ProviderID, type ModelID } from "../schema/index.js"
import { ElevenLabsSpeech } from "../protocols/elevenlabs-speech.js"
import { ElevenLabsTranscription } from "../protocols/elevenlabs-transcription.js"
export type { ElevenLabsOutputFormat, ElevenLabsSpeechOptions } from "../protocols/elevenlabs-speech.js"
export type { ElevenLabsTranscriptionOptions } from "../protocols/elevenlabs-transcription.js"
export const id = ProviderID.make("elevenlabs")
@@ -24,12 +26,15 @@ const auth = (options: ProviderAuthOption<"optional">) => {
export const configure = (input: Config = {}) => {
const media = MediaRoute.deployment(input, auth(input))
const speech = (modelID: string | ModelID) => ElevenLabsSpeech.model({ ...media, id: modelID })
const transcription = (modelID: string | ModelID) => ElevenLabsTranscription.model({ ...media, id: modelID })
return {
id,
speech,
transcription,
configure,
}
}
export const provider = configure()
export const speech = provider.speech
export const transcription = provider.transcription
-8
View File
@@ -7,8 +7,6 @@ import { OpenAIChat } from "../protocols/openai-chat.js"
import { OpenResponsesChannel } from "../protocols/open-responses-channel.js"
import { XAIResponses } from "../protocols/xai-responses.js"
import { XAIImages } from "../protocols/xai-images.js"
import { XAISpeech } from "../protocols/xai-speech.js"
import { XAITranscription } from "../protocols/xai-transcription.js"
import { XAIVideo } from "../protocols/xai-video.js"
import type { OpenAIOptionsInput } from "./openai-options.js"
import type { ProviderPackage } from "../provider-package.js"
@@ -31,8 +29,6 @@ export type Settings = ProviderPackage.Settings &
}
export type { XAIImageOptions } from "../protocols/xai-images.js"
export type { XAISpeechOptions } from "../protocols/xai-speech.js"
export type { XAITranscriptionOptions } from "../protocols/xai-transcription.js"
export type { XAIVideoOptions } from "../protocols/xai-video.js"
const RESPONSES_WEBSOCKET_ROTATE_AFTER_MS = 24 * 60 * 1000
@@ -102,8 +98,6 @@ export const configure = (input: LanguageModelOptions = {}) => {
chat,
image: (modelID: string | ModelID) => XAIImages.model({ ...media, id: modelID }),
video: (modelID: string | ModelID) => XAIVideo.model({ ...media, id: modelID }),
speech: (modelID: string | ModelID) => XAISpeech.model({ ...media, id: modelID }),
transcription: (modelID: string | ModelID) => XAITranscription.model({ ...media, id: modelID }),
configure,
}
}
@@ -125,5 +119,3 @@ export const responses = provider.responses
export const chat = provider.chat
export const image = provider.image
export const video = provider.video
export const speech = provider.speech
export const transcription = provider.transcription
+2 -6
View File
@@ -88,19 +88,15 @@ const decodeProviderBody = Schema.decodeUnknownOption(
Schema.fromJsonString(
Schema.Struct({
message: Schema.optionalKey(Schema.String),
// xAI sends `{ code, error }` with the readable reason as a plain string.
error: Schema.optionalKey(
Schema.Union([Schema.String, Schema.Struct({ message: Schema.optionalKey(Schema.String) })]),
),
error: Schema.optionalKey(Schema.Struct({ message: Schema.optionalKey(Schema.String) })),
}),
),
)
const providerMessage = (status: number, body: string | void) => {
const decoded = body === undefined ? undefined : Option.getOrUndefined(decodeProviderBody(body))
const error = typeof decoded?.error === "string" ? decoded.error : decoded?.error?.message
return (
[error, decoded?.message].find((message) => message?.trim()) ??
[decoded?.error?.message, decoded?.message].find((message) => message?.trim()) ??
`Provider request failed with HTTP ${status}`
)
}
+41 -7
View File
@@ -5,12 +5,14 @@ import { Media } from "../media.js"
import type { AuthInput } from "./auth.js"
import {
AIError,
AuthenticationError,
ContentPolicyError,
HttpContext,
InvalidProviderOutputError,
InvalidRequestError,
ProviderID,
ProviderInternalError,
RateLimitError,
UnsupportedOperationError,
} from "../schema/index.js"
@@ -137,6 +139,11 @@ export interface Queued<Request, Response, Token> {
readonly cancel?: {
readonly method: AuthInput["method"]
readonly path: (token: Token) => string
/**
* Fetch a fresh status first and skip the call for terminal generations, for providers whose cancel endpoint
* destroys finished work (Runway's `DELETE /v1/tasks/{id}` deletes completed tasks and their outputs).
*/
readonly activeOnly?: boolean
}
}
@@ -183,6 +190,16 @@ export const stream = <Request, Event, Frame, State>(
// Response helpers
// ---------------------------------------------------------------------------
/** Reasons a provider can report for a `failed` generation; anything it does not classify is `ProviderInternal`. */
const FAILURES = {
InvalidRequest: InvalidRequestError,
Authentication: AuthenticationError,
RateLimit: RateLimitError,
ProviderInternal: ProviderInternalError,
}
export type Failure = keyof typeof FAILURES
const context = (response: HttpClientResponse.HttpClientResponse) =>
new HttpContext({ url: response.request.url, status: response.status, headers: response.headers })
@@ -194,8 +211,10 @@ export const identity = (input: { readonly id: string; readonly name: string; re
/**
* Read a text body while retaining the original payload and HTTP context on every downstream error. `invalid` is a
* malformed provider document; `ended` is a generation that reached a terminal status without output (`failed` is
* provider-side, `cancelled`/`expired` mean the result will never exist); `contentPolicy` is a moderated result.
* malformed provider document; `ended` is a generation that reached a terminal status without output (`failed`
* carries the provider's classification, defaulting to `ProviderInternal`; `cancelled`/`expired` mean the result
* will never exist); `pending` is a `result()` read before the generation finished, which is caller misuse;
* `contentPolicy` is a moderated result.
*/
const text = Effect.fn("MediaProtocol.text")(function* (response: HttpClientResponse.HttpClientResponse) {
const http = context(response)
@@ -217,13 +236,25 @@ export const identity = (input: { readonly id: string; readonly name: string; re
http,
invalid: (message: string, cause?: unknown) =>
new AIError({ reason: new InvalidProviderOutputError({ route: input.id, message, body, http, cause }) }),
ended: (status: Exclude<Status, "queued" | "running" | "completed">, message: string) =>
ended: (
status: Exclude<Status, "queued" | "running" | "completed">,
message: string,
failure: Failure = "ProviderInternal",
) =>
new AIError({
reason:
status === "failed"
? new ProviderInternalError({ message, body, http })
? new FAILURES[failure]({ message, body, http })
: new InvalidRequestError({ message, body, http }),
}),
pending: (id: string) =>
new AIError({
reason: new InvalidRequestError({
message: `${input.name} generation ${id} has not finished; await it before reading the result`,
body,
http,
}),
}),
contentPolicy: (message: string) => new AIError({ reason: new ContentPolicyError({ message, body, http }) }),
}
})
@@ -285,11 +316,14 @@ export const status = <Table extends Record<string, Status>>(
raw: string,
output: Output,
): Effect.Effect<Status, AIError> => {
const normalized: Status | undefined = table[raw]
if (normalized === undefined) return Effect.fail(output.invalid(`Unknown generation status "${raw}"`))
return Effect.succeed(normalized)
if (!Object.hasOwn(table, raw)) return Effect.fail(output.invalid(`Unknown generation status "${raw}"`))
return Effect.succeed(table[raw])
}
/** Map a provider error code through the protocol's table; missing or unmapped codes are `ProviderInternal`. */
export const failure = (table: Readonly<Record<string, Failure>>, code: string | number | undefined): Failure =>
code !== undefined && Object.hasOwn(table, code) ? table[code] : "ProviderInternal"
/** A `url` asset whose provider-declared retention window starts now. */
export const expiringUrl = (url: string, retention: Duration.Duration, options?: Parameters<typeof Media.url>[1]) =>
Clock.currentTimeMillis.pipe(
+49 -10
View File
@@ -1,12 +1,13 @@
import { Effect, Schema, Stream } from "effect"
import { Duration, Effect, Schedule, Schema, Stream } from "effect"
import { Headers, HttpClientRequest, type HttpClientResponse } from "effect/unstable/http"
import { Auth, type AuthInput } from "./auth.js"
import { Endpoint } from "./endpoint.js"
import { RequestExecutorService, type Interface } from "./executor-service.js"
import { RequestExecutor } from "./executor.js"
import { MediaProtocol } from "./media-protocol.js"
import { Generation } from "../generation.js"
import { Generation, isTerminal } from "../generation.js"
import type { Media } from "../media.js"
import { isRetryable } from "../provider-error.js"
import {
AIError,
AIErrorReason,
@@ -137,6 +138,32 @@ export const inline = <Request extends MediaRequest, Response>(
}
}
const READ_RETRY_MAX_DELAY = Duration.seconds(30)
/**
* Status and result reads retry transient failures; `start` and `cancel` never do. Gaps grow exponentially from 1s,
* jittered, up to 30s each, for at most 8 retries (about two minutes when every attempt fails), so a direct
* `Generation.result()` stays bounded; `await` and `events` also cut retries off at `poll.timeout`. A provider
* `retryAfterMs` raises the gap, still capped at 30s.
*/
const READ_RETRY = Schedule.max([
Schedule.min([Schedule.exponential("1 second"), Schedule.spaced(READ_RETRY_MAX_DELAY)]),
Schedule.recurs(8),
]).pipe(
Schedule.jittered,
Schedule.setInputType<AIError>(),
Schedule.modifyDelay(({ input, duration }) =>
Effect.succeed(
Duration.min(
input.reason._tag === "RateLimit" || input.reason._tag === "ProviderInternal"
? Duration.max(duration, Duration.millis(input.reason.retryAfterMs ?? 0))
: duration,
READ_RETRY_MAX_DELAY,
),
),
),
)
/**
* Compose a queued media protocol the same way, adding `start`/`resume` handles whose polls reuse the route's auth,
* deployment headers, and (for `start`) the request's `http` overlay. The token is decoded once at the boundary and
@@ -154,6 +181,8 @@ export const queued = <Request extends MediaRequest, Response, Token>(
const generationRoute = (token: Token, http: HttpOptions | undefined, execute: Execute) => {
const materialize = (asset: Media.Asset) =>
asset.materialize().pipe(Effect.provideService(RequestExecutorService, { execute }))
// Only the GET exchange retries: a decoded terminal failure (`output.ended`) can be a `ProviderInternal` too, and
// re-reading it would spin until the caller's deadline.
const poll = <A>(operation: {
readonly path: (token: Token) => string
readonly decode: (
@@ -161,17 +190,23 @@ export const queued = <Request extends MediaRequest, Response, Token>(
context: MediaProtocol.PollContext<Token>,
) => Effect.Effect<A, AIError>
}) =>
transport
.call("GET", operation.path(token), http, execute)
.pipe(Effect.flatMap((sent) => operation.decode(sent.response, { token, auth: sent.auth, materialize })))
transport.call("GET", operation.path(token), http, execute).pipe(
Effect.retry({ schedule: READ_RETRY, while: isRetryable }),
Effect.flatMap((sent) => operation.decode(sent.response, { token, auth: sent.auth, materialize })),
)
const status = poll(protocol.status)
const cancel = protocol.cancel
const send =
cancel === undefined
? undefined
: transport.call(cancel.method, cancel.path(token), http, execute).pipe(Effect.asVoid)
return {
status: poll(protocol.status),
status,
result: poll(protocol.result),
cancel:
cancel === undefined
? undefined
: transport.call(cancel.method, cancel.path(token), http, execute).pipe(Effect.asVoid),
send !== undefined && cancel?.activeOnly
? status.pipe(Effect.flatMap((snapshot) => (isTerminal(snapshot.status) ? Effect.void : send)))
: send,
}
}
@@ -380,7 +415,11 @@ const encode = (body: MediaProtocol.Body | undefined, headers: Headers.Headers)
}
}
/** Common fields are never silently dropped: a present field the protocol declared unsupported fails typed. */
/**
* Common fields are never silently dropped: a present field the protocol declared unsupported fails typed. `false`
* counts as present because some booleans mean something when false (video `audio`); protocols reject opt-in
* booleans such as speech `timestamps` with `=== true` in `body.from` instead of listing them.
*/
const rejectUnsupported = <Request extends object>(
route: string,
provider: ProviderID,
+1 -1
View File
@@ -103,7 +103,7 @@ export type TranscriptionRequestInput<Model extends TranscriptionModel = Transcr
// Response and events
// ---------------------------------------------------------------------------
/** Speaker labels are provider-native (`A`, `0`, `spk:0`, or a known speaker name). */
/** Speaker labels are provider-native (`A`, `0`, `spk:0`, `speaker_0`, or a known speaker name). */
export const TranscriptionSegment = Schema.Struct({
text: Schema.String,
startSeconds: Schema.Number,
+24
View File
@@ -351,10 +351,34 @@ describe("OpenAI Responses effort updates", () => {
}),
)
it.effect("strips markers when the body overlay selects pro reasoning mode", () =>
Effect.gen(function* () {
const prepared = yield* compileRequest(
LLM.request({
model: OpenAI.configure({ apiKey: "fixture", http: { body: { reasoning: { mode: "pro" } } } }).responses(
"gpt-6-sol",
),
messages: conversation,
providerOptions: { reasoningEffort: "low" },
}),
)
expect(updates(prepared.body)).toEqual([])
expect(prepared.body.reasoning).toEqual({ effort: "low" })
}),
)
for (const [id, supported] of [
["gpt-6-astra", true],
["openai/gpt-6-astra", true],
["gpt-6-sol", true],
["openai/gpt-6-sol", true],
["gpt-6-luna", true],
["openai/gpt-6-luna", true],
["gpt-6-astra-2026-09-01", false],
["gpt-6-sol-pro", false],
["gpt-6-luna-pro", false],
["gpt-6-sol-fast", false],
["gpt-5.6-sol", false],
] as const) {
it.effect(`${supported ? "lowers" : "strips"} markers for ${id}`, () =>
-20
View File
@@ -299,26 +299,6 @@ describe("RequestExecutor", () => {
),
)
it.effect("reads provider messages sent as a plain error string", () =>
Effect.gen(function* () {
const executor = yield* RequestExecutor.Service
const error = yield* executor.execute(request).pipe(Effect.flip)
expectAIError(error)
expect(error.message).toBe("Your team has no credits for this endpoint")
}).pipe(
Effect.provide(
fixedResponse(
JSON.stringify({
code: "The caller does not have permission to execute the specified operation",
error: "Your team has no credits for this endpoint",
}),
{ status: 403 },
),
),
),
)
it.effect("falls back when structured provider messages are empty", () =>
Effect.gen(function* () {
const executor = yield* RequestExecutor.Service
+36 -4
View File
@@ -23,6 +23,7 @@ import { Provider as ProviderSubpath } from "@opencode/ai/provider"
import {
AssemblyAI,
Baseten,
BlackForestLabs,
Cartesia,
CloudflareAIGateway,
CloudflareWorkersAI,
@@ -32,14 +33,18 @@ import {
Fal,
Fireworks,
Google,
Meta,
OpenCodeZen,
OpenAI,
OpenAICompatible,
OpenRouter,
Replicate,
Runway,
Stability,
TypeSafeAI,
VercelAIGateway,
XAI,
ZAI,
} from "@opencode/ai/providers"
import {
OpenAIChat,
@@ -151,12 +156,34 @@ describe("public exports", () => {
expect(XAI.provider.chat).toBe(XAI.chat)
expect(XAI.configure({ apiKey: "fixture" }).responses("grok-4.3").route.id).toBe("openai-responses")
expect(XAI.configure({ apiKey: "fixture" }).chat("grok-4.3").route.id).toBe("openai-compatible-chat")
expect(OpenAI.configure({ apiKey: "fixture" }).image("gpt-image-2").route.id).toBe("openai-images")
expect(OpenAI.provider.image).toBe(OpenAI.image)
expect(Google.configure({ apiKey: "fixture" }).image("imagen-4.0-generate-001").route.id).toBe("google-images")
expect(Google.provider.image).toBe(Google.image)
expect(XAI.configure({ apiKey: "fixture" }).image("grok-imagine-image").route.id).toBe("xai-images")
expect(XAI.provider.image).toBe(XAI.image)
expect(Fal.configure({ apiKey: "fixture" }).image("fal-ai/flux/dev").route.id).toBe("fal-images")
expect(Fal.provider.image).toBe(Fal.image)
expect(BlackForestLabs.configure({ apiKey: "fixture" }).image("flux-2-pro").route.id).toBe("bfl-images")
expect(BlackForestLabs.provider.image).toBe(BlackForestLabs.image)
expect(Replicate.configure({ apiKey: "fixture" }).image("black-forest-labs/flux-schnell").route.id).toBe(
"replicate-images",
)
expect(Replicate.provider.image).toBe(Replicate.image)
expect(Stability.configure({ apiKey: "fixture" }).image("sd3.5-large").route.id).toBe("stability-images")
expect(Stability.provider.image).toBe(Stability.image)
expect(Stability.configure({ apiKey: "fixture" }).upscale().route.id).toBe("stability-upscale")
expect(Stability.provider.upscale).toBe(Stability.upscale)
expect(Meta.configure({ apiKey: "fixture" }).image("muse-image").route.id).toBe("meta-images")
expect(Meta.provider.image).toBe(Meta.image)
expect(ZAI.configure({ apiKey: "fixture" }).image("glm-image").route.id).toBe("zai-images")
expect(ZAI.provider.image).toBe(ZAI.image)
expect(XAI.configure({ apiKey: "fixture" }).video("grok-imagine-video-1.5").route.id).toBe("xai-video")
expect(XAI.configure({ apiKey: "fixture" }).speech("grok-tts").route.id).toBe("xai-speech")
expect(XAI.configure({ apiKey: "fixture" }).transcription("grok-voice-transcribe-2.0").route.kind).toBe("inline")
expect(XAI.provider.speech).toBe(XAI.speech)
expect(XAI.provider.transcription).toBe(XAI.transcription)
expect(XAI.provider.video).toBe(XAI.video)
expect(Google.configure({ apiKey: "fixture" }).video("veo-3.1-generate-preview").route.id).toBe("google-video")
expect(Google.provider.video).toBe(Google.video)
expect(Fal.configure({ apiKey: "fixture" }).video("fal-ai/veo3.1").route.id).toBe("fal-video")
expect(Fal.provider.video).toBe(Fal.video)
expect(Runway.configure({ apiKey: "fixture" }).video("gen4.5").route.id).toBe("runway-video")
expect(Runway.provider.video).toBe(Runway.video)
expect(OpenAI.configure({ apiKey: "fixture" }).speech("gpt-4o-mini-tts").route.id).toBe("openai-speech")
@@ -170,6 +197,11 @@ describe("public exports", () => {
expect(Google.configure({ apiKey: "fixture" }).transcription("gemini-3.5-transcribe").route.kind).toBe("stream")
expect(Deepgram.configure({ apiKey: "fixture" }).transcription("nova-3").route.kind).toBe("inline")
expect(AssemblyAI.configure({ apiKey: "fixture" }).transcription("universal-3-5-pro").route.kind).toBe("queued")
expect(ElevenLabs.configure({ apiKey: "fixture" }).transcription("scribe_v2").route.id).toBe(
"elevenlabs-transcription",
)
expect(ElevenLabs.configure({ apiKey: "fixture" }).transcription("scribe_v2").route.kind).toBe("inline")
expect(ElevenLabs.provider.transcription).toBe(ElevenLabs.transcription)
})
test("protocol barrels expose supported low-level routes", () => {
@@ -0,0 +1,32 @@
{
"version": 1,
"metadata": {
"tags": [
"prefix:elevenlabs-transcription",
"provider:elevenlabs",
"protocol:elevenlabs-transcription"
],
"name": "elevenlabs-transcription/groups-diarized-words-into-speaker-turns",
"recordedAt": "2026-09-27T09:35:28.265Z"
},
"interactions": [
{
"transport": "http",
"request": {
"method": "POST",
"url": "https://api.elevenlabs.io/v1/speech-to-text",
"headers": {
"content-type": "multipart/form-data; boundary=----WebKitFormBoundary356bdc14864a477dbacbfcf60d1ecceb"
},
"body": "--BOUNDARY\r\nContent-Disposition: form-data; name=\"file\"; filename=\"audio.mp3\"\r\nContent-Type: audio/mpeg\r\n\r\n[audio]\r\n--BOUNDARY\r\nContent-Disposition: form-data; name=\"model_id\"\r\n\r\nscribe_v2\r\n--BOUNDARY\r\nContent-Disposition: form-data; name=\"diarize\"\r\n\r\ntrue\r\n--BOUNDARY--\r\n"
},
"response": {
"status": 200,
"headers": {
"content-type": "application/json"
},
"body": "{\"language_code\":\"eng\",\"language_probability\":0.9495430588722229,\"text\":\"Did the release ship? Yes, it shipped this morning\",\"words\":[{\"text\":\"Did\",\"start\":0.34,\"end\":0.44,\"type\":\"word\",\"speaker_id\":\"speaker_0\",\"logprob\":-1.7881377516459906e-6},{\"text\":\" \",\"start\":0.44,\"end\":0.48,\"type\":\"spacing\",\"speaker_id\":\"speaker_0\",\"logprob\":-1.1920928244535389e-7},{\"text\":\"the\",\"start\":0.48,\"end\":0.56,\"type\":\"word\",\"speaker_id\":\"speaker_0\",\"logprob\":-1.1920928244535389e-7},{\"text\":\" \",\"start\":0.56,\"end\":0.6,\"type\":\"spacing\",\"speaker_id\":\"speaker_0\",\"logprob\":-7.152531907195225e-6},{\"text\":\"release\",\"start\":0.6,\"end\":0.92,\"type\":\"word\",\"speaker_id\":\"speaker_0\",\"logprob\":-7.152531907195225e-6},{\"text\":\" \",\"start\":0.92,\"end\":0.94,\"type\":\"spacing\",\"speaker_id\":\"speaker_0\",\"logprob\":-8.344646857949556e-7},{\"text\":\"ship?\",\"start\":0.94,\"end\":1.26,\"type\":\"word\",\"speaker_id\":\"speaker_0\",\"logprob\":-7.414704032271402e-6},{\"text\":\" \",\"start\":1.26,\"end\":1.26,\"type\":\"spacing\",\"speaker_id\":\"speaker_0\",\"logprob\":-0.0009363081189803779},{\"text\":\"Yes,\",\"start\":1.68,\"end\":2.02,\"type\":\"word\",\"speaker_id\":\"speaker_1\",\"logprob\":-0.003542040009030245},{\"text\":\" \",\"start\":2.02,\"end\":2.48,\"type\":\"spacing\",\"speaker_id\":\"speaker_1\",\"logprob\":-3.933898824470816e-6},{\"text\":\"it\",\"start\":2.48,\"end\":2.62,\"type\":\"word\",\"speaker_id\":\"speaker_1\",\"logprob\":-3.933898824470816e-6},{\"text\":\" \",\"start\":2.62,\"end\":2.64,\"type\":\"spacing\",\"speaker_id\":\"speaker_1\",\"logprob\":-0.000013589766240329482},{\"text\":\"shipped\",\"start\":2.66,\"end\":2.9,\"type\":\"word\",\"speaker_id\":\"speaker_1\",\"logprob\":-0.000013589766240329482},{\"text\":\" \",\"start\":2.9,\"end\":2.94,\"type\":\"spacing\",\"speaker_id\":\"speaker_1\",\"logprob\":0.0},{\"text\":\"this\",\"start\":2.94,\"end\":3.12,\"type\":\"word\",\"speaker_id\":\"speaker_1\",\"logprob\":0.0},{\"text\":\" \",\"start\":3.12,\"end\":3.18,\"type\":\"spacing\",\"speaker_id\":\"speaker_1\",\"logprob\":-3.576278118089249e-7},{\"text\":\"morning\",\"start\":3.18,\"end\":3.5,\"type\":\"word\",\"speaker_id\":\"speaker_1\",\"logprob\":-3.576278118089249e-7}],\"transcription_id\":\"cs3I2282TH8hjw12brNg\",\"audio_duration_secs\":3.5526875}"
}
}
]
}
@@ -0,0 +1,50 @@
{
"version": 1,
"metadata": {
"tags": [
"prefix:elevenlabs-transcription",
"provider:elevenlabs",
"protocol:elevenlabs-transcription"
],
"name": "elevenlabs-transcription/transcribes-audio-with-word-timestamps",
"recordedAt": "2026-09-27T09:35:27.686Z"
},
"interactions": [
{
"transport": "http",
"request": {
"method": "POST",
"url": "https://api.elevenlabs.io/v1/speech-to-text",
"headers": {
"content-type": "multipart/form-data; boundary=----WebKitFormBoundarye2be7b31e94441bbbeb35a9c890a9d74"
},
"body": "--BOUNDARY\r\nContent-Disposition: form-data; name=\"file\"; filename=\"audio.mp3\"\r\nContent-Type: audio/mpeg\r\n\r\n[audio]\r\n--BOUNDARY\r\nContent-Disposition: form-data; name=\"model_id\"\r\n\r\nscribe_v2\r\n--BOUNDARY--\r\n"
},
"response": {
"status": 200,
"headers": {
"content-type": "application/json"
},
"body": "{\"language_code\":\"eng\",\"language_probability\":0.6618340611457825,\"text\":\"Hello from OpenCode\",\"words\":[{\"text\":\"Hello\",\"start\":0.4,\"end\":0.66,\"type\":\"word\",\"logprob\":-0.000014781842764932662},{\"text\":\" \",\"start\":0.66,\"end\":0.74,\"type\":\"spacing\",\"logprob\":-3.814689989667386e-6},{\"text\":\"from\",\"start\":0.74,\"end\":0.84,\"type\":\"word\",\"logprob\":-3.814689989667386e-6},{\"text\":\" \",\"start\":0.84,\"end\":0.9,\"type\":\"spacing\",\"logprob\":-0.018268775194883347},{\"text\":\"OpenCode\",\"start\":0.9,\"end\":1.44,\"type\":\"word\",\"logprob\":-0.1251817401498556}],\"transcription_id\":\"D4VfnANM2ArCHTujIb9q\",\"audio_duration_secs\":1.54125}"
}
},
{
"transport": "http",
"request": {
"method": "POST",
"url": "https://api.elevenlabs.io/v1/speech-to-text",
"headers": {
"content-type": "multipart/form-data; boundary=----WebKitFormBoundaryfb80d0e44d9e44d299416ed546a04056"
},
"body": "--BOUNDARY\r\nContent-Disposition: form-data; name=\"file\"; filename=\"audio.mp3\"\r\nContent-Type: audio/mpeg\r\n\r\n[audio]\r\n--BOUNDARY\r\nContent-Disposition: form-data; name=\"model_id\"\r\n\r\nscribe_v2\r\n--BOUNDARY--\r\n"
},
"response": {
"status": 200,
"headers": {
"content-type": "application/json"
},
"body": "{\"language_code\":\"eng\",\"language_probability\":0.6618340611457825,\"text\":\"Hello from OpenCode\",\"words\":[{\"text\":\"Hello\",\"start\":0.4,\"end\":0.66,\"type\":\"word\",\"logprob\":-0.000023007127310847864},{\"text\":\" \",\"start\":0.66,\"end\":0.74,\"type\":\"spacing\",\"logprob\":-2.3841830625315197e-6},{\"text\":\"from\",\"start\":0.74,\"end\":0.84,\"type\":\"word\",\"logprob\":-2.3841830625315197e-6},{\"text\":\" \",\"start\":0.84,\"end\":0.9,\"type\":\"spacing\",\"logprob\":-0.008306833915412426},{\"text\":\"OpenCode\",\"start\":0.9,\"end\":1.44,\"type\":\"word\",\"logprob\":-0.1075385226868093}],\"transcription_id\":\"SkYplzfq1DW8Ae3bWnoy\",\"audio_duration_secs\":1.54125}"
}
}
]
}
+20
View File
@@ -97,6 +97,26 @@ describe("Generation", () => {
}),
)
it.effect("fails an event stream at the deadline when the poll interval is longer than the timeout", () =>
Effect.gen(function* () {
const scripted = yield* scriptedRoute(["running"], "never")
const generation = new Generation(scripted.route, "t", { id: "gen_1", status: "queued" })
const fiber = yield* Effect.forkChild(
generation
.events({ poll: { interval: "30 seconds", timeout: "10 seconds" } })
.pipe(Stream.runCollect, Effect.flip),
)
yield* TestClock.adjust("9 seconds")
expect(fiber.pollUnsafe()).toBeUndefined()
yield* TestClock.adjust("1 second")
const error = yield* Fiber.join(fiber)
expect(error.reason._tag).toBe("Timeout")
expect(yield* Ref.get(scripted.polls)).toBe(1)
}),
)
it.effect("surfaces the route failure body for failed generations", () =>
Effect.gen(function* () {
const scripted = yield* scriptedRoute(["running", "failed"], "unused")
+159 -4
View File
@@ -78,8 +78,11 @@ describe("Image", () => {
mediaType: "image/webp",
})
expect(yield* response.image.bytes()).toEqual(Uint8Array.from([1, 2, 3]))
expect(response.image.providerMetadata).toEqual({ openai: { revisedPrompt: "A precise robot" } })
expect(response.image.info).toEqual({ format: "webp", width: 2048, height: 2048 })
expect(response.usage).toMatchObject({ type: "tokens", total: 12 })
expect(response.providerMetadata).toEqual({
openai: { outputFormat: "webp", size: "2048x2048", quality: "high", background: "opaque" },
})
}).pipe(
Effect.provide(
ImageClient.layer.pipe(
@@ -107,8 +110,11 @@ describe("Image", () => {
})
return input.respond(
JSON.stringify({
data: [{ b64_json: "AQID", revised_prompt: "A precise robot" }, { b64_json: "BAUG" }],
data: [{ b64_json: "AQID" }, { b64_json: "BAUG" }],
output_format: "webp",
size: "2048x2048",
quality: "high",
background: "opaque",
usage: { input_tokens: 4, output_tokens: 8, total_tokens: 12 },
}),
{ headers: { "content-type": "application/json" } },
@@ -144,6 +150,7 @@ describe("Image", () => {
),
)
expect(response.image.source).toEqual({ type: "bytes", data: Uint8Array.from([1, 2, 3]), mediaType: "image/png" })
expect(response.image.info).toEqual({ format: "png" })
}),
)
@@ -725,6 +732,7 @@ describe("Image", () => {
const errors = yield* Effect.all(
[
Image.start({ model: Google.configure({ apiKey: "test" }).image("gemini-3.1-flash-image"), prompt }),
Image.generate({ model: Google.configure({ apiKey: "test" }).image("gemini-3.1-flash-image"), prompt, n: 2 }),
Image.start({
model: BlackForestLabs.configure({ apiKey: "test" }).image("flux-2-pro"),
prompt,
@@ -735,7 +743,6 @@ describe("Image", () => {
prompt,
size: "512x512",
}),
Stream.runCollect(Image.stream({ model: openai.image("dall-e-3"), prompt })),
Stream.runCollect(Image.stream({ model: openai.image("gpt-image-2"), prompt, n: 2 })),
Image.start({ model: replicate, prompt, seed: 7 }),
Image.start({
@@ -749,9 +756,9 @@ describe("Image", () => {
expect(errors.map((error) => [error.reason._tag, "operation" in error.reason && error.reason.operation])).toEqual(
[
["UnsupportedOperation", "image.start"],
["UnsupportedOperation", "media.n"],
["UnsupportedOperation", "media.aspectRatio"],
["UnsupportedOperation", "media.size"],
["UnsupportedOperation", "media.stream"],
["UnsupportedOperation", "media.n"],
["UnsupportedOperation", "media.seed"],
["InvalidRequest", false],
@@ -761,6 +768,104 @@ describe("Image", () => {
}).pipe(Effect.provide(layer(() => Effect.die("an unsupported request reached the network")))),
)
const falToken = {
requestID: "r1",
statusURL: "https://queue.fal.test/fal-ai/flux/requests/r1/status",
responseURL: "https://queue.fal.test/fal-ai/flux/requests/r1",
cancelURL: "https://queue.fal.test/fal-ai/flux/requests/r1/cancel",
}
const falSubmitted = {
request_id: falToken.requestID,
status_url: falToken.statusURL,
response_url: falToken.responseURL,
cancel_url: falToken.cancelURL,
}
const bodies: Array<unknown> = []
it.effect("sizes fal Kontext by aspect ratio and sends several images to /multi", () =>
Effect.gen(function* () {
const fal = Fal.configure({ apiKey: "test", baseURL: "https://queue.fal.test" })
const images = [Media.url("https://example.test/a.png"), Media.url("https://example.test/b.png")]
const rejected = yield* Image.start({
model: fal.image("fal-ai/flux-pro/kontext"),
prompt: "A lighthouse",
size: "512x512",
}).pipe(Effect.flip)
yield* Image.start({
model: fal.image("fal-ai/flux-pro/kontext"),
prompt: "A lighthouse",
images: images.slice(0, 1),
aspectRatio: "16:9",
})
yield* Image.start({ model: fal.image("fal-ai/flux-pro/kontext/max/multi"), prompt: "A lighthouse", images })
expect(rejected.reason).toMatchObject({ _tag: "UnsupportedOperation", operation: "media.size" })
expect(bodies).toEqual([
{ prompt: "A lighthouse", aspect_ratio: "16:9", image_url: "https://example.test/a.png" },
{ prompt: "A lighthouse", image_urls: ["https://example.test/a.png", "https://example.test/b.png"] },
])
}).pipe(
Effect.provide(
layer((input) => {
bodies.push(JSON.parse(input.text))
return Effect.succeed(json(input, falSubmitted))
}),
),
),
)
it.effect("decodes fal sync_mode data URIs as inline images", () =>
Effect.gen(function* () {
const generation = yield* Image.resume(Fal.configure({ apiKey: "test" }).image("fal-ai/flux/schnell"), falToken)
const response = yield* generation.await()
expect(response.images.map((image) => image.source)).toEqual([
{ type: "base64", data: "AQID", mediaType: "image/png" },
{ type: "url", url: "https://v3.fal.media/out.jpg", mediaType: "image/jpeg" },
])
expect(response.image.info).toEqual({ width: 512, height: 512 })
expect(yield* response.image.bytes()).toEqual(Uint8Array.from([1, 2, 3]))
}).pipe(
Effect.provide(
layer((input) =>
Effect.succeed(
input.request.url === falToken.statusURL
? json(input, { status: "COMPLETED" })
: json(input, {
images: [
{ url: "data:image/png;base64,AQID", width: 512, height: 512, content_type: "image/png" },
{ url: "https://v3.fal.media/out.jpg", width: 512, height: 512, content_type: "image/jpeg" },
],
}),
),
),
),
),
)
const falDetail = { detail: [{ loc: ["body", "prompt"], msg: "Invalid input", type: "value_error" }] }
it.effect(
"fails a fal await whose COMPLETED status carries an error with the response_url body and HTTP context",
() =>
Effect.gen(function* () {
const generation = yield* Image.resume(Fal.configure({ apiKey: "test" }).image("fal-ai/flux/schnell"), falToken)
expect(generation.status).toBe("failed")
const error = yield* generation.await().pipe(Effect.flip)
expect(error.reason._tag).toBe("InvalidRequest")
expect(error.reason.body).toBe(JSON.stringify(falDetail))
expect(error.reason.http).toMatchObject({ url: falToken.responseURL, status: 422 })
}).pipe(
Effect.provide(
layer((input) =>
Effect.succeed(
input.request.url === falToken.statusURL
? json(input, { status: "COMPLETED", error: "Invalid input", error_type: "ValidationError" })
: json(input, falDetail, { status: 422 }),
),
),
),
),
)
const moderated = { id: "req_1", status: "Content Moderated" }
const prediction = {
id: "p_1",
@@ -768,6 +873,56 @@ describe("Image", () => {
output: { text: "not an image" },
urls: { get: "https://replicate.test/p_1", cancel: "https://replicate.test/p_1/cancel" },
}
for (const pending of [
{
model: BlackForestLabs.configure({ apiKey: "test" }).image("flux-2-pro"),
token: { id: "req_1", pollingURL: "https://bfl.test/v1/get_result?id=req_1" },
status: 200,
body: { id: "req_1", status: "Pending" },
message: "Black Forest Labs generation req_1",
},
{
model: Replicate.configure({ apiKey: "test" }).image("owner/model"),
token: { id: "p_1", getURL: "https://replicate.test/p_1", cancelURL: "https://replicate.test/p_1/cancel" },
status: 200,
body: {
id: "p_1",
status: "processing",
urls: { get: "https://replicate.test/p_1", cancel: "https://replicate.test/p_1/cancel" },
},
message: "Replicate generation p_1",
},
{
model: Stability.configure({ apiKey: "test", baseURL: "https://stability.test" }).upscale(),
token: { id: "up_1" },
status: 202,
body: { id: "up_1", status: "in-progress" },
message: "Stability AI generation up_1",
},
]) {
it.effect(`rejects reading a ${pending.model.provider} result before the generation finishes`, () =>
Effect.gen(function* () {
const generation = yield* Image.resume(pending.model, pending.token)
const error = yield* generation.result().pipe(Effect.flip)
expect(error.reason._tag).toBe("InvalidRequest")
expect(error.message).toBe(`${pending.message} has not finished; await it before reading the result`)
expect(error.reason.body).toBe(JSON.stringify(pending.body))
expect(error.reason.http?.status).toBe(pending.status)
}).pipe(
Effect.provide(
layer((input) =>
Effect.succeed(
input.respond(JSON.stringify(pending.body), {
status: pending.status,
headers: { "content-type": "application/json" },
}),
),
),
),
),
)
}
it.effect("classifies terminal outcomes the recordings never saw", () =>
Effect.gen(function* () {
const bfl = yield* Image.resume(BlackForestLabs.configure({ apiKey: "test" }).image("flux-2-pro"), {
+51 -1
View File
@@ -3,7 +3,7 @@ import { NodeFileSystem } from "@effect/platform-node"
import { Effect, Ref, Schema } from "effect"
import { FileSystem } from "effect"
import { HttpClientRequest } from "effect/unstable/http"
import { Media, Message } from "../src/index.js"
import { AIError, Media, Message } from "../src/index.js"
import { it } from "./lib/effect.js"
import { dynamicResponse, scriptedResponses } from "./lib/http.js"
@@ -161,6 +161,56 @@ describe("Media", () => {
}),
)
it.effect("keeps transient url download headers out of toJSON and AssetSchema encoding", () =>
Effect.sync(() => {
const asset = Media.url("https://cdn.example.test/video.mp4", {
mediaType: "video/mp4",
expiresAt: 42,
headers: { "x-goog-api-key": "secret" },
})
expect(asset.headers).toEqual({ "x-goog-api-key": "secret" })
const source = { type: "url", url: "https://cdn.example.test/video.mp4", mediaType: "video/mp4", expiresAt: 42 }
expect(asset.toJSON()).not.toHaveProperty("headers")
expect(JSON.stringify(asset)).not.toContain("secret")
expect(asset.toJSON().source).toEqual(source)
const encoded = Schema.encodeSync(Media.AssetSchema)(asset)
expect(encoded).not.toHaveProperty("headers")
expect(encoded.source).toEqual(source)
const codec = Schema.fromJsonString(Media.AssetSchema)
const json = Schema.encodeSync(codec)(asset)
expect(json).not.toContain("secret")
const restored = Schema.decodeSync(codec)(json)
expect(restored).toBeInstanceOf(Media.Asset)
expect(restored.source).toEqual(source)
expect(restored.expiresAt).toBe(42)
expect(restored.headers).toBeUndefined()
}),
)
it.effect("fails url downloads with non-2xx status as a typed AIError keeping http and body", () =>
Effect.gen(function* () {
const body = JSON.stringify({ error: { message: "file expired" } })
const error = yield* Media.url("https://cdn.example.test/expired.png")
.bytes()
.pipe(
Effect.flip,
Effect.provide(
dynamicResponse((input) =>
Effect.succeed(input.respond(body, { status: 404, headers: { "content-type": "application/json" } })),
),
),
)
expect(error).toBeInstanceOf(AIError)
expect(error.message).toContain("file expired")
expect(error.reason.http?.status).toBe(404)
expect(error.reason.http?.url).toBe("https://cdn.example.test/expired.png")
expect(error.reason.body).toBe(body)
}),
)
it.effect("reads files with sniffed media types and writes materialized assets", () =>
Effect.gen(function* () {
const fs = yield* FileSystem.FileSystem
+85 -1
View File
@@ -22,7 +22,8 @@ const chatBody = sseEvents(
/**
* Executor layer that answers chat completions with SSE text, image generations with one base64 PNG, Runway video
* tasks with a queued submission that succeeds on the second poll, speech with raw audio or SSE audio deltas, OpenAI
* transcription with JSON or SSE text deltas, and AssemblyAI transcripts that complete on the first poll.
* transcription with JSON or SSE text deltas, AssemblyAI transcripts that complete on the first poll, and `slow.test`
* chat completions that send one text delta and never finish.
*/
const executor = (seen: Array<string>) =>
RequestExecutor.layer.pipe(
@@ -55,6 +56,18 @@ const executor = (seen: Array<string>) =>
output: "https://replicate.test/a.webp",
urls: { get: "https://replicate.test/p_1", cancel: "https://replicate.test/p_1/cancel" },
})
if (web.url.startsWith("https://slow.test"))
return input.respond(
new ReadableStream({
start: (controller) =>
controller.enqueue(
new TextEncoder().encode(
`data: ${JSON.stringify({ choices: [{ delta: { content: "Hello" } }] })}\n\n`,
),
),
}),
{ headers: { "content-type": "text/event-stream" } },
)
if (web.url.endsWith("/chat/completions"))
return input.respond(chatBody, { headers: { "content-type": "text/event-stream" } })
if (web.url.endsWith("/audio/speech"))
@@ -304,6 +317,77 @@ describe("AI promise client", () => {
await ai.dispose()
})
test("aborted calls reject and aborted streams throw with the signal's reason", async () => {
const ai = AI.make({ layer: executor([]) })
const slow = OpenAI.configure({ apiKey: "test", baseURL: "https://slow.test/v1" }).chat("gpt-4o-mini")
const aborted = new AbortController()
aborted.abort()
const reason = new Error("mine")
const rejected = await ai.run(Effect.never, { signal: aborted.signal }).catch((error: unknown) => error)
expect(rejected).toBe(aborted.signal.reason)
expect(rejected).toMatchObject({ name: "AbortError" })
const inFlight = new AbortController()
setTimeout(() => inFlight.abort(reason), 10)
expect(
await ai.llm
.generate({ model: slow, prompt: "Hello" }, { signal: inFlight.signal })
.catch((error: unknown) => error),
).toBe(reason)
const preAborted = await Array.fromAsync(
ai.speech.stream({ model: openai.speech("gpt-4o-mini-tts"), text: "Hello" }, { signal: aborted.signal }),
).catch((error: unknown) => error)
expect(preAborted).toBe(aborted.signal.reason)
expect(preAborted).toMatchObject({ name: "AbortError" })
const midStream = new AbortController()
const deltas: Array<string> = []
const midStreamFailure = await Array.fromAsync(
ai.llm.stream({ model: slow, prompt: "Hello" }, { signal: midStream.signal }),
(event) => {
if (!LLMEvent.is.textDelta(event)) return
deltas.push(event.text)
midStream.abort()
},
).catch((error: unknown) => error)
expect(deltas).toEqual(["Hello"])
expect(midStreamFailure).toBe(midStream.signal.reason)
expect(midStreamFailure).toMatchObject({ name: "AbortError" })
const model = Runway.configure({ apiKey: "test", baseURL: "https://runway.test/v1" }).video("gen4.5")
const generation = await ai.video.start({ model, prompt: "A kite" })
const polling = new AbortController()
const events: Array<string> = []
const eventsFailure = await Array.fromAsync(
generation.events({ poll: { interval: 60_000 }, signal: polling.signal }),
(event) => {
events.push(event.type)
polling.abort(reason)
},
).catch((error: unknown) => error)
expect(events).toEqual(["generation-progress"])
expect(eventsFailure).toBe(reason)
await ai.dispose()
})
test("breaking out of an abortable stream cleans up without throwing", async () => {
const ai = AI.make({ layer: executor([]) })
const slow = OpenAI.configure({ apiKey: "test", baseURL: "https://slow.test/v1" }).chat("gpt-4o-mini")
const controller = new AbortController()
const deltas: Array<string> = []
for await (const event of ai.llm.stream({ model: slow, prompt: "Hello" }, { signal: controller.signal })) {
if (!LLMEvent.is.textDelta(event)) continue
deltas.push(event.text)
break
}
controller.abort()
expect(deltas).toEqual(["Hello"])
await ai.dispose()
})
test("the default client is created lazily and can be disposed", async () => {
expect(typeof AI.ai.llm.generate).toBe("function")
expect(typeof AI.ai.image.generate).toBe("function")
@@ -33,6 +33,8 @@ describe("Black Forest Labs Images recorded", () => {
expect(response.image.source.type).toBe("bytes")
expect(dimensions(yield* response.image.bytes())).toEqual({ width: 512, height: 512 })
// BFL reports cost on submit only; the Ready result omits it.
expect(response.usage).toEqual({ type: "credits", credits: 1.4000000000000001 })
}),
{ timeout: 15 * 60 * 1000 },
)
@@ -0,0 +1,50 @@
import { describe, expect } from "bun:test"
import { Effect, Stream } from "effect"
import { Transcription } from "../../src/index.js"
import { ElevenLabs } from "../../src/providers.js"
import { recordedTests } from "../recorded-test.js"
import { TRANSCRIPT, audio, audioRecording, dialog } from "./transcription-recording.js"
const model = ElevenLabs.configure({ apiKey: process.env.ELEVENLABS_API_KEY ?? "fixture" }).transcription("scribe_v2")
const recorded = recordedTests({
prefix: "elevenlabs-transcription",
provider: "elevenlabs",
protocol: "elevenlabs-transcription",
requires: ["ELEVENLABS_API_KEY"],
options: audioRecording,
})
describe("ElevenLabs Transcription recorded", () => {
recorded.effect("transcribes audio with word timestamps", () =>
Effect.gen(function* () {
const request = Transcription.request({ model, audio: yield* audio, timestamps: "word" })
const response = yield* Transcription.generate(request)
expect(response.text).toMatch(TRANSCRIPT)
expect(response.words?.map((word) => word.text)).toEqual(["Hello", "from", "OpenCode"])
expect(response.words?.every((word) => word.speaker === undefined && (word.confidence ?? 0) > 0)).toBe(true)
expect(response.segments).toBeUndefined()
expect(response.language).toBe("eng")
expect(response.durationSeconds).toBeGreaterThan(0)
expect(response.usage).toEqual({ type: "seconds", seconds: response.durationSeconds })
expect(response.providerMetadata?.elevenlabs?.transcriptionId).toEqual(expect.any(String))
const events = Array.from(yield* Stream.runCollect(Transcription.stream(request)))
expect(events.map((event) => event.type)).toEqual(["finish"])
}),
)
recorded.effect("groups diarized words into speaker turns", () =>
Effect.gen(function* () {
const response = yield* Transcription.generate({ model, audio: yield* dialog, diarize: true })
expect(response.segments?.map((segment) => segment.speaker)).toEqual(["speaker_0", "speaker_1"])
expect(response.segments?.[0].text).toMatch(/^Did the release ship\?$/)
expect(response.segments?.[1].text).toMatch(/^Yes, it shipped this morning\.?$/)
expect(response.segments?.map((segment) => segment.text).join(" ")).toBe(response.text)
expect(response.words?.some((word) => word.text.trim() === "")).toBe(false)
expect(new Set(response.words?.map((word) => word.speaker))).toEqual(new Set(["speaker_0", "speaker_1"]))
}),
)
})
@@ -29,7 +29,11 @@ describe("OpenAI Images recorded", () => {
expect(response.images).toHaveLength(1)
expect(response.image.mediaType).toBe("image/jpeg")
expect(response.image.info).toEqual({ format: "jpeg", width: 1024, height: 1024 })
expect((yield* response.image.bytes()).length).toBeGreaterThan(0)
expect(response.providerMetadata).toEqual({
openai: { outputFormat: "jpeg", size: "1024x1024", quality: "low", background: "opaque" },
})
}),
)
@@ -76,8 +80,13 @@ describe("OpenAI Images recorded", () => {
expect(events.map((event) => event.type)).toEqual(["image-partial", "image", "finish"])
const image = events.find(ImageEvent.is.image)
expect(image?.image.mediaType).toBe("image/jpeg")
expect(image?.image.info).toEqual({ format: "jpeg", width: 1024, height: 1024 })
expect(dimensions(yield* image!.image.bytes())).toEqual({ width: 1024, height: 1024 })
expect(events.find(ImageEvent.is.finish)?.usage).toMatchObject({ type: "tokens" })
const finish = events.find(ImageEvent.is.finish)
expect(finish?.usage).toMatchObject({ type: "tokens" })
expect(finish?.providerMetadata).toEqual({
openai: { outputFormat: "jpeg", size: "1024x1024", quality: "low", background: "opaque" },
})
}),
)
})
@@ -44,7 +44,12 @@ describe("OpenAI Transcription recorded", () => {
expect(deltas.length).toBeGreaterThan(1)
expect(deltas.join("")).toBe(finish.text)
expect(finish.text).toMatch(TRANSCRIPT)
expect(finish.usage).toMatchObject({ type: "tokens", input: expect.any(Number), output: expect.any(Number) })
expect(finish.usage).toMatchObject({
type: "tokens",
input: expect.any(Number),
output: expect.any(Number),
details: { openai: { input_token_details: { audio_tokens: expect.any(Number) } } },
})
}),
)
@@ -1,37 +0,0 @@
import { describe, expect } from "bun:test"
import { Effect } from "effect"
import { Speech } from "../../src/index.js"
import { XAI } from "../../src/providers.js"
import { recordedTests } from "../recorded-test.js"
import { TEXT, collectSpeech } from "./speech-recording.js"
const model = XAI.configure({ apiKey: process.env.XAI_API_KEY ?? "fixture" }).speech("grok-tts")
const recorded = recordedTests({
prefix: "xai-speech",
provider: "xai",
protocol: "xai-speech",
requires: ["XAI_API_KEY"],
})
describe("xAI Speech recorded", () => {
recorded.effect("generates speech", () =>
Effect.gen(function* () {
const response = yield* Speech.generate({ model, text: TEXT, voice: "eve", language: "en" })
expect(response.audio.mediaType).toBe("audio/mpeg")
expect((yield* response.audio.bytes()).length).toBeGreaterThan(0)
}),
)
recorded.effect("streams speech", () =>
Effect.gen(function* () {
const { finish } = yield* collectSpeech(
Speech.stream({ model, text: TEXT, voice: "eve", language: "en", format: "pcm" }),
)
expect(finish.audio.mediaType).toBe("audio/pcm")
expect(finish.audio.info).toMatchObject({ encoding: "pcm_s16le", sampleRate: 24000, channels: 1 })
}),
)
})
@@ -1,146 +0,0 @@
import { describe, expect } from "bun:test"
import { Effect, Layer, Stream } from "effect"
import { Speech, SpeechClient, SpeechEvent } from "../../src/index.js"
import { XAI } from "../../src/providers.js"
import { it } from "../lib/effect.js"
import { dynamicResponse, type Call, type Handler, observe } from "../lib/http.js"
const layer = (handler: Handler) => SpeechClient.layer.pipe(Layer.provideMerge(dynamicResponse(handler)))
const xai = XAI.configure({ apiKey: "test", baseURL: "https://api.xai.test/v1" })
const model = xai.speech("grok-tts")
const respondAudio = (calls: Array<Call>, body: Uint8Array | ReadableStream<Uint8Array>, contentType: string) =>
layer((input) =>
observe(calls, input).pipe(Effect.map(() => input.respond(body, { headers: { "content-type": contentType } }))),
)
describe("xAI Speech", () => {
it.effect("lowers common fields into the TTS body and describes the requested container", () => {
const calls: Array<Call> = []
return Effect.gen(function* () {
const wav = Uint8Array.from([0x52, 0x49, 0x46, 0x46, 0, 0, 0, 0, 0x57, 0x41, 0x56, 0x45])
const response = yield* Speech.generate({
model,
text: "Hello from OpenCode.",
voice: { id: "nlbqfwie" },
speed: 1.2,
language: "en",
format: "wav",
providerOptions: { sampleRate: 44100, text_normalization: true },
http: { body: { replace: { OpenCode: "Open Code" } } },
}).pipe(Effect.provide(respondAudio(calls, wav, "audio/wav")))
expect(calls.map((call) => [call.method, call.url, call.headers.get("authorization")])).toEqual([
["POST", "https://api.xai.test/v1/tts", "Bearer test"],
])
expect(JSON.parse(calls[0].body)).toEqual({
text: "Hello from OpenCode.",
voice_id: "nlbqfwie",
language: "en",
output_format: { codec: "wav", sample_rate: 44100 },
speed: 1.2,
text_normalization: true,
replace: { OpenCode: "Open Code" },
})
expect(response.audio.mediaType).toBe("audio/wav")
expect(response.audio.info).toEqual({ format: "wav", sampleRate: 44100 })
expect(yield* response.audio.bytes()).toEqual(wav)
expect(response.usage).toBeUndefined()
})
})
it.effect("defaults to auto-detected language and 24 kHz MP3", () => {
const calls: Array<Call> = []
return Effect.gen(function* () {
const response = yield* Speech.generate({ model, text: "Hi" }).pipe(
Effect.provide(respondAudio(calls, Uint8Array.from([0x49, 0x44, 0x33, 4]), "audio/mpeg")),
)
expect(JSON.parse(calls[0].body)).toEqual({ text: "Hi", language: "auto", output_format: { codec: "mp3" } })
expect(response.audio.mediaType).toBe("audio/mpeg")
expect(response.audio.info).toEqual({ format: "mp3", sampleRate: 24000 })
})
})
it.effect("streams raw body chunks as audio deltas and describes headerless PCM", () => {
const calls: Array<Call> = []
return Effect.gen(function* () {
const body = new ReadableStream<Uint8Array>({
start(controller) {
controller.enqueue(Uint8Array.from([1, 2]))
controller.enqueue(Uint8Array.from([3, 4, 5]))
controller.close()
},
})
const events = Array.from(
yield* Stream.runCollect(Speech.stream({ model, text: "Hi", voice: "eve", format: "pcm" })).pipe(
Effect.provide(respondAudio(calls, body, "audio/pcm")),
),
)
expect(JSON.parse(calls[0].body)).toEqual({
text: "Hi",
voice_id: "eve",
language: "auto",
output_format: { codec: "pcm" },
})
expect(events.map((event) => event.type)).toEqual(["audio-delta", "audio-delta", "finish"])
expect(events.filter(SpeechEvent.is.audioDelta).map((event) => Array.from(event.chunk))).toEqual([
[1, 2],
[3, 4, 5],
])
const finish = events.find(SpeechEvent.is.finish)
expect(finish?.audio.mediaType).toBe("audio/pcm")
expect(finish?.audio.info).toEqual({ format: "pcm", encoding: "pcm_s16le", sampleRate: 24000, channels: 1 })
expect(yield* finish!.audio.bytes()).toEqual(Uint8Array.from([1, 2, 3, 4, 5]))
})
})
it.effect("describes telephony codecs at the requested sample rate", () =>
Effect.gen(function* () {
const response = yield* Speech.generate({
model,
text: "Hi",
format: "mulaw",
providerOptions: { sampleRate: 8000 },
}).pipe(Effect.provide(respondAudio([], Uint8Array.from([0xff, 0x7f]), "audio/basic")))
expect(response.audio.mediaType).toBe("audio/mulaw")
expect(response.audio.info).toEqual({ format: "pcm", encoding: "pcm_mulaw", sampleRate: 8000, channels: 1 })
}),
)
it.effect("rejects what xAI cannot lower before sending anything", () =>
Effect.gen(function* () {
const errors = yield* Effect.all(
[
Speech.generate({ model, text: "Hi", instructions: "Warm." }),
Speech.generate({ model, text: "Hi", timestamps: true }),
Stream.runCollect(Speech.stream({ model, text: "Hi", format: "opus" })),
].map((effect) => Effect.flip(effect)),
)
expect(errors.map((error) => [error.reason._tag, "operation" in error.reason && error.reason.operation])).toEqual(
[
["UnsupportedOperation", "media.instructions"],
["UnsupportedOperation", "media.timestamps"],
["UnsupportedOperation", "media.format"],
],
)
expect(errors[2].reason).toMatchObject({ provider: "xai", route: "xai-speech" })
}).pipe(Effect.provide(layer(() => Effect.die("an unsupported request reached the network")))),
)
it.effect("fails typed when the provider returns no audio", () =>
Effect.gen(function* () {
const error = yield* Speech.generate({ model, text: "Hi" }).pipe(
Effect.provide(respondAudio([], new Uint8Array(), "audio/mpeg")),
Effect.flip,
)
expect(error.reason).toMatchObject({ _tag: "InvalidProviderOutput", route: "xai-speech" })
expect(error.reason.http?.status).toBe(200)
}),
)
})
@@ -1,39 +0,0 @@
import { describe, expect } from "bun:test"
import { Effect } from "effect"
import { Transcription } from "../../src/index.js"
import { XAI } from "../../src/providers.js"
import { recordedTests } from "../recorded-test.js"
import { TRANSCRIPT, audio, audioRecording, dialog } from "./transcription-recording.js"
const model = XAI.configure({ apiKey: process.env.XAI_API_KEY ?? "fixture" }).transcription("grok-voice-transcribe-2.0")
const recorded = recordedTests({
prefix: "xai-transcription",
provider: "xai",
protocol: "xai-transcription",
requires: ["XAI_API_KEY"],
options: audioRecording,
})
describe("xAI Transcription recorded", () => {
recorded.effect("transcribes with word timestamps", () =>
Effect.gen(function* () {
const response = yield* Transcription.generate({ model, audio: yield* audio, language: "en" })
expect(response.text).toMatch(TRANSCRIPT)
expect(response.words?.length).toBeGreaterThan(2)
expect(response.words?.every((word) => word.endSeconds >= word.startSeconds)).toBe(true)
expect(response.language).toBe("en")
expect(response.usage).toEqual({ type: "seconds", seconds: response.durationSeconds })
}),
)
recorded.effect("diarizes speakers into segments", () =>
Effect.gen(function* () {
const response = yield* Transcription.generate({ model, audio: yield* dialog, diarize: true })
expect(new Set(response.segments?.map((segment) => segment.speaker)).size).toBe(2)
expect(response.words?.every((word) => word.speaker !== undefined)).toBe(true)
}),
)
})
@@ -1,168 +0,0 @@
import { describe, expect } from "bun:test"
import { Effect, Layer, Stream } from "effect"
import { HttpClientRequest } from "effect/unstable/http"
import { Media, Transcription, TranscriptionClient } from "../../src/index.js"
import { XAI } from "../../src/providers.js"
import { it } from "../lib/effect.js"
import { dynamicResponse, json, type Handler } from "../lib/http.js"
const layer = (handler: Handler) => TranscriptionClient.layer.pipe(Layer.provideMerge(dynamicResponse(handler)))
const model = XAI.configure({ apiKey: "test", baseURL: "https://api.xai.test/v1" }).transcription(
"grok-voice-transcribe-2.0",
)
const audio = Media.bytes(Uint8Array.from([0x49, 0x44, 0x33, 1, 2, 3]), "audio/mpeg")
interface Upload {
readonly url: string
readonly authorization: string | null
readonly form: FormData
}
/** Reply with `body` and keep every request's parsed multipart form. */
const respondTranscript = (uploads: Array<Upload>, body: unknown) =>
layer((input) =>
Effect.gen(function* () {
const web = yield* HttpClientRequest.toWeb(input.request).pipe(Effect.orDie)
const form = yield* Effect.promise(() => web.formData())
uploads.push({ url: web.url, authorization: web.headers.get("authorization"), form })
return json(input, body)
}),
)
describe("xAI Transcription", () => {
it.effect("uploads inline audio as multipart with file last and splits diarized words into speaker turns", () => {
const uploads: Array<Upload> = []
return Effect.gen(function* () {
const request = Transcription.request({
model,
audio,
language: "en",
diarize: true,
timestamps: "word",
providerOptions: { format: true, keyterm: ["OpenCode", "Grok"], filler_words: false },
http: { body: { vad_threshold: 0.3, model: "ignored" } },
})
const response = yield* Transcription.generate(request)
const events = Array.from(yield* Stream.runCollect(Transcription.stream(request)))
const upload = uploads[0]
expect([upload.url, upload.authorization]).toEqual(["https://api.xai.test/v1/stt", "Bearer test"])
expect(Array.from(upload.form.keys())).toEqual([
"model",
"language",
"diarize",
"format",
"keyterm",
"keyterm",
"filler_words",
"vad_threshold",
"file",
])
expect(upload.form.get("model")).toBe("grok-voice-transcribe-2.0")
expect(upload.form.get("language")).toBe("en")
expect(upload.form.get("diarize")).toBe("true")
expect(upload.form.get("format")).toBe("true")
expect(upload.form.getAll("keyterm")).toEqual(["OpenCode", "Grok"])
expect(upload.form.get("filler_words")).toBe("false")
expect(upload.form.get("vad_threshold")).toBe("0.3")
const file = upload.form.get("file")
if (!(file instanceof File)) throw new Error("Expected a file upload")
expect([file.name, file.type]).toEqual(["audio.mp3", "audio/mpeg"])
expect(new Uint8Array(yield* Effect.promise(() => file.arrayBuffer()))).toEqual(yield* audio.bytes())
expect(response.text).toBe("Did it ship? Yes.")
expect(response.words).toEqual([
{ text: "Did", startSeconds: 0.2, endSeconds: 0.4, speaker: "0", confidence: 0.9 },
{ text: "it", startSeconds: 0.4, endSeconds: 0.5, speaker: "0", confidence: undefined },
{ text: "ship?", startSeconds: 0.5, endSeconds: 0.9, speaker: "0", confidence: undefined },
{ text: "Yes.", startSeconds: 1.2, endSeconds: 1.6, speaker: "1", confidence: 0.8 },
])
expect(response.segments).toEqual([
{ text: "Did it ship?", startSeconds: 0.2, endSeconds: 0.9, speaker: "0" },
{ text: "Yes.", startSeconds: 1.2, endSeconds: 1.6, speaker: "1" },
])
expect(response).toMatchObject({
language: "en",
durationSeconds: 1.75,
usage: { type: "seconds", seconds: 1.75 },
})
expect(events.map((event) => event.type)).toEqual(["finish"])
}).pipe(
Effect.provide(
respondTranscript(uploads, {
text: "Did it ship? Yes.",
language: "EN",
duration: 1.75,
words: [
{ text: "Did", start: 0.2, end: 0.4, confidence: 0.9, speaker: 0 },
{ text: "it", start: 0.4, end: 0.5, speaker: 0 },
{ text: "ship?", start: 0.5, end: 0.9, speaker: 0 },
{ text: "Yes.", start: 1.2, end: 1.6, confidence: 0.8, speaker: 1 },
],
}),
),
)
})
it.effect("sends remote audio by URL and describes headerless PCM uploads", () => {
const uploads: Array<Upload> = []
return Effect.gen(function* () {
const remote = yield* Transcription.generate({
model,
audio: Media.url("https://cdn.test/call.mp3", { mediaType: "audio/mpeg" }),
timestamps: "segment",
})
const pcm = yield* Transcription.generate({
model,
audio: Media.bytes(Uint8Array.from([0, 1, 0, 1]), "audio/pcm", {
info: { format: "pcm", encoding: "pcm_s16le", sampleRate: 16000, channels: 1 },
}),
})
expect(Array.from(uploads[0].form.entries())).toEqual([
["model", "grok-voice-transcribe-2.0"],
["diarize", "true"],
["url", "https://cdn.test/call.mp3"],
])
expect(Array.from(uploads[1].form.keys())).toEqual(["model", "audio_format", "sample_rate", "file"])
expect(uploads[1].form.get("audio_format")).toBe("pcm")
expect(uploads[1].form.get("sample_rate")).toBe("16000")
expect(remote.segments).toBeUndefined()
expect(pcm).toMatchObject({ text: "Hi", words: undefined, segments: undefined })
}).pipe(Effect.provide(respondTranscript(uploads, { text: "Hi", language: "en", duration: 0.5 })))
})
it.effect("rejects what xAI cannot lower before sending anything", () =>
Effect.gen(function* () {
const errors = yield* Effect.all(
[
Transcription.generate({ model, audio, prompt: "OpenCode" }),
Transcription.generate({ model, audio, speakers: 2 }),
Transcription.start({ model, audio }),
Transcription.generate({ model, audio: Media.ref("file_1", { provider: "xai", mediaType: "audio/mpeg" }) }),
].map((effect) => Effect.flip(effect)),
)
expect(errors.map((error) => [error.reason._tag, "operation" in error.reason && error.reason.operation])).toEqual(
[
["UnsupportedOperation", "media.prompt"],
["UnsupportedOperation", "media.speakers"],
["UnsupportedOperation", "transcription.start"],
["InvalidRequest", false],
],
)
expect(errors[0].reason).toMatchObject({ provider: "xai", route: "xai-transcription" })
}).pipe(Effect.provide(layer(() => Effect.die("an unsupported request reached the network")))),
)
it.effect("keeps the raw body when the transcript cannot be decoded", () => {
const uploads: Array<Upload> = []
return Effect.gen(function* () {
const error = yield* Transcription.generate({ model, audio }).pipe(Effect.flip)
expect(error.reason).toMatchObject({ _tag: "InvalidProviderOutput", body: JSON.stringify({ words: [] }) })
expect(error.reason.http?.status).toBe(200)
}).pipe(Effect.provide(respondTranscript(uploads, { words: [] })))
})
})
+6 -1
View File
@@ -32,7 +32,12 @@ describe("Z.ai Images", () => {
expect(response.images).toHaveLength(1)
expect(response.image.mediaType).toBe("application/octet-stream")
expect(response.image.source).toEqual({ type: "url", url: "https://cdn.z.ai/generated.png" })
// Z.ai documents that output URLs expire 30 days after generation; the test clock starts at 0.
expect(response.image.source).toEqual({
type: "url",
url: "https://cdn.z.ai/generated.png",
expiresAt: 30 * 24 * 60 * 60 * 1000,
})
expect(response.notices).toEqual([
{
type: "moderated",
+153 -1
View File
@@ -28,15 +28,94 @@ const cartesia = Cartesia.configure({ apiKey: "test", baseURL: "https://cartesia
const google = Google.configure({ apiKey: "test", baseURL: "https://google.test/v1beta" }).speech(
"gemini-2.5-flash-preview-tts",
)
const google38 = Google.configure({ apiKey: "test", baseURL: "https://google.test/v1beta" }).speech(
"gemini-3.8-flash-tts",
)
const google38Lite = Google.configure({ apiKey: "test", baseURL: "https://google.test/v1beta" }).speech(
"gemini-3.8-flash-lite-tts",
)
const deepgram = Deepgram.configure({ apiKey: "test", baseURL: "https://deepgram.test" }).speech("aura-2-thalia-en")
const voice = "JBFqnCBsd6RMkjVDRZzb"
describe("Speech", () => {
it.effect("preserves Google's WAV output instead of describing it as raw PCM", () =>
Effect.gen(function* () {
const bytes = new TextEncoder().encode("RIFF....WAVEfmt ")
const response = yield* Speech.generate({ model: google38, text: "Hi" }).pipe(
Effect.provide(
respond(
JSON.stringify({
candidates: [
{
content: { parts: [{ inlineData: { mimeType: "audio/wav", data: Encoding.encodeBase64(bytes) } }] },
finishReason: "STOP",
},
],
}),
"application/json",
),
),
)
expect(response.audio.mediaType).toBe("audio/wav")
expect(response.audio.info?.format).toBe("wav")
expect(response.audio.info?.encoding).toBeUndefined()
expect(yield* response.audio.bytes()).toEqual(bytes)
}),
)
it.effect("describes OpenAI audio in the format the request body actually asked for", () =>
Effect.gen(function* () {
const [pcm, wav] = yield* Effect.all([
Speech.generate({ model: openai, text: "Hi", format: "mp3", providerOptions: { response_format: "pcm" } }),
Speech.generate({ model: openai, text: "Hi", http: { body: { response_format: "wav" } } }),
]).pipe(Effect.provide(respond("\u0001\u0002", "application/octet-stream")))
expect(pcm.audio.mediaType).toBe("audio/pcm")
expect(pcm.audio.info).toEqual({ format: "pcm", encoding: "pcm_s16le", sampleRate: 24000, channels: 1 })
expect(wav.audio.mediaType).toBe("audio/wav")
expect(wav.audio.info?.format).toBe("wav")
}),
)
it.effect("describes Deepgram raw encodings in their default WAV container", () =>
Effect.gen(function* () {
const response = yield* Speech.generate({ model: deepgram, text: "Hi", providerOptions: { encoding: "mulaw" } })
expect(response.audio.mediaType).toBe("audio/wav")
expect(response.audio.info?.format).toBe("wav")
}).pipe(Effect.provide(respond("RIFF....WAVEfmt ", "audio/wav"))),
)
it.effect("always gives headerless Deepgram PCM a sample rate", () =>
Effect.gen(function* () {
const [requested, defaulted] = yield* Effect.all([
Speech.generate({
model: deepgram,
text: "Hi",
providerOptions: { encoding: "mulaw", container: "none", sampleRate: 16000 },
}),
Speech.generate({ model: deepgram, text: "Hi", providerOptions: { encoding: "alaw", container: "none" } }),
]).pipe(Effect.provide(respond("\u0001\u0002", "audio/basic")))
expect(requested.audio.info).toEqual({ format: "pcm", encoding: "pcm_mulaw", sampleRate: 16000, channels: 1 })
expect(defaulted.audio.info).toEqual({ format: "pcm", encoding: "pcm_alaw", sampleRate: 8000, channels: 1 })
}),
)
it.effect("rejects raw PCM for Gemini 3.8 unary requests before sending", () =>
Effect.gen(function* () {
const errors = yield* Effect.all(
[google38, google38Lite].map((model) =>
Speech.generate({ model, text: "Hi", format: "pcm" }).pipe(Effect.flip),
),
).pipe(Effect.provide(layer(() => Effect.die("An unsupported request reached the network"))))
expect(errors.map((error) => error.reason._tag)).toEqual(["UnsupportedOperation", "UnsupportedOperation"])
}),
)
it.effect("rejects what a provider cannot produce before sending anything", () =>
Effect.gen(function* () {
const errors = yield* Effect.all(
[
Speech.generate({ model: openai, text: "Hi", timestamps: true }),
Speech.generate({ model: openai, text: "Hi", format: "ogg" }),
Speech.generate({ model: google, text: "Hi", format: "mp3" }),
Speech.generate({ model: google, text: "Hi", instructions: "Warm." }),
collect(Speech.stream({ model: elevenlabs, text: "Hi", voice, format: "wav" })),
@@ -48,16 +127,57 @@ describe("Speech", () => {
[
["UnsupportedOperation", "media.timestamps"],
["UnsupportedOperation", "media.format"],
["UnsupportedOperation", "media.format"],
["UnsupportedOperation", "media.instructions"],
["UnsupportedOperation", "media.format"],
["UnsupportedOperation", "media.format"],
["UnsupportedOperation", "media.voice"],
],
)
expect(errors[1].reason).toMatchObject({ provider: "google", route: "google-speech" })
expect(errors[1].reason).toMatchObject({ provider: "openai", route: "openai-speech" })
expect(errors[2].reason).toMatchObject({ provider: "google", route: "google-speech" })
}).pipe(Effect.provide(layer(() => Effect.die("an unsupported request reached the network")))),
)
it.effect("treats timestamps: false as not asking for timestamps on routes that cannot return them", () =>
Effect.gen(function* () {
const bytes = Uint8Array.from([1, 2, 3])
const gemini = JSON.stringify({
candidates: [
{
content: { parts: [{ inlineData: { mimeType: "audio/L16;codec=pcm;rate=24000", data: "AQID" } }] },
finishReason: "STOP",
},
],
})
const responses = yield* Effect.all([
Speech.generate({ model: openai, text: "Hi", timestamps: false }).pipe(
Effect.provide(respond(new Blob([bytes]).stream(), "audio/mpeg")),
),
Speech.generate({ model: google, text: "Hi", timestamps: false }).pipe(
Effect.provide(respond(gemini, "application/json")),
),
Speech.generate({ model: deepgram, text: "Hi", timestamps: false }).pipe(
Effect.provide(respond(new Blob([bytes]).stream(), "audio/mpeg")),
),
])
for (const response of responses) expect(yield* response.audio.bytes()).toEqual(bytes)
const errors = yield* Effect.all(
[openai, google, deepgram].map((model) =>
Speech.generate({ model, text: "Hi", timestamps: true }).pipe(Effect.flip),
),
).pipe(Effect.provide(layer(() => Effect.die("an unsupported request reached the network"))))
expect(errors.map((error) => [error.reason._tag, "operation" in error.reason && error.reason.operation])).toEqual(
[
["UnsupportedOperation", "media.timestamps"],
["UnsupportedOperation", "media.timestamps"],
["UnsupportedOperation", "media.timestamps"],
],
)
}),
)
it.effect("classifies stream failures and keeps the provider payload and HTTP context", () =>
Effect.gen(function* () {
const badFrame = JSON.stringify({ type: "speech.audio.delta", audio: "not base64!" })
@@ -88,6 +208,38 @@ describe("Speech", () => {
}),
)
it.effect("surfaces Gemini speech that ended without STOP instead of returning it as complete", () =>
Effect.gen(function* () {
const document = (finishReason?: string) =>
JSON.stringify({
candidates: [
{
content: { parts: [{ inlineData: { mimeType: "audio/L16;codec=pcm;rate=24000", data: "AQI=" } }] },
finishReason,
},
],
})
const withheld = JSON.stringify({ candidates: [{ finishReason: "SAFETY" }] })
const generate = (body: string) =>
Speech.generate({ model: google, text: "Hi" }).pipe(Effect.provide(respond(body, "application/json")))
const truncated = yield* generate(document()).pipe(Effect.flip)
const partial = yield* generate(document("MAX_TOKENS"))
const policy = yield* generate(withheld).pipe(Effect.flip)
expect(truncated.reason).toMatchObject({ _tag: "InvalidProviderOutput", classification: "incomplete-stream" })
expect(yield* partial.audio.bytes()).toEqual(Uint8Array.from([1, 2]))
expect(partial.notices).toEqual([
{
type: "other",
message: "Google Speech finished with MAX_TOKENS",
providerMetadata: { google: { finishReason: "MAX_TOKENS" } },
},
])
expect(policy.reason).toMatchObject({ _tag: "ContentPolicy", body: withheld })
}),
)
it.effect("parses ElevenLabs timestamped records split across network chunks", () =>
Effect.gen(function* () {
const record = (bytes: ReadonlyArray<number>, character: string, start: number) =>
+456 -5
View File
@@ -2,10 +2,11 @@ import { describe, expect } from "bun:test"
import { Effect, Fiber, Layer, Stream } from "effect"
import * as TestClock from "effect/testing/TestClock"
import { HttpClientRequest } from "effect/unstable/http"
import { Media, Transcription, TranscriptionClient } from "../src/index.js"
import { AssemblyAI, Deepgram, Google, OpenAI } from "../src/providers.js"
import { Media, Transcription, TranscriptionClient, type TranscriptionEvent } from "../src/index.js"
import { AssemblyAI, Deepgram, ElevenLabs, Google, OpenAI } from "../src/providers.js"
import { it } from "./lib/effect.js"
import { dynamicResponse } from "./lib/http.js"
import { dynamicResponse, json, observe, type Call } from "./lib/http.js"
import { sseEvents } from "./lib/sse.js"
const layer = (handler: Parameters<typeof dynamicResponse>[0]) =>
TranscriptionClient.layer.pipe(Layer.provideMerge(dynamicResponse(handler)))
@@ -16,16 +17,33 @@ const deepgram = Deepgram.configure({ apiKey: "test", baseURL: "https://deepgram
const google = Google.configure({ apiKey: "test", baseURL: "https://google.test/v1beta" }).transcription(
"gemini-3.5-transcribe",
)
/**
* Multipart fields of a recorded request, with repeated names collected in order. The boundary comes from the body:
* each conversion of a FormData request to a web request picks a fresh one, so the recorded headers may not match.
*/
const formFields = (call: Call) =>
Effect.promise(() =>
new Response(call.body, {
headers: { "content-type": `multipart/form-data; boundary=${call.body.slice(2, call.body.indexOf("\r\n"))}` },
}).formData(),
).pipe(
Effect.map((form) =>
Object.fromEntries([...new Set(form.keys())].map((key) => [key, form.getAll(key).map((value) => String(value))])),
),
)
const assemblyai = AssemblyAI.configure({ apiKey: "aai-key", baseURL: "https://assemblyai.test" }).transcription(
"universal-3-5-pro",
)
const elevenlabs = ElevenLabs.configure({ apiKey: "test", baseURL: "https://elevenlabs.test" }).transcription(
"scribe_v2",
)
describe("Transcription", () => {
it.effect("rejects what a route cannot honor before sending anything", () =>
Effect.gen(function* () {
const errors = yield* Effect.all(
[
Stream.runCollect(Transcription.stream({ model: openai.transcription("whisper-1"), audio })),
Transcription.generate({ model: openai.transcription("gpt-4o-mini-transcribe"), audio, diarize: true }),
Transcription.generate({ model: openai.transcription("gpt-4o-mini-transcribe"), audio, timestamps: "word" }),
Transcription.generate({ model: openai.transcription("gpt-4o-transcribe-diarize"), audio, prompt: "Names" }),
@@ -53,7 +71,6 @@ describe("Transcription", () => {
)
expect(errors.map((error) => [error.reason._tag, "operation" in error.reason && error.reason.operation])).toEqual(
[
["UnsupportedOperation", "media.stream"],
["UnsupportedOperation", "media.diarize"],
["UnsupportedOperation", "media.timestamps"],
["UnsupportedOperation", "media.prompt"],
@@ -70,6 +87,259 @@ describe("Transcription", () => {
}).pipe(Effect.provide(layer(() => Effect.die("an unsupported request reached the network")))),
)
it.effect("ignores unknown OpenAI stream events and fails on an error event with the frame", () =>
Effect.gen(function* () {
const sse = (...frames: ReadonlyArray<string>) => frames.map((frame) => `data: ${frame}\n\n`).join("")
const failure = `{"type":"error","error":{"type":"server_error","code":"server_error","message":"The server had an error"}}`
const bodies = [
sse(
`{"type":"transcript.text.delta","delta":"Hi"}`,
`{"type":"transcript.text.future","payload":1}`,
`{"type":"transcript.text.done","text":"Hi"}`,
"[DONE]",
),
sse(`{"type":"transcript.text.delta","delta":"Hi"}`, failure),
]
const model = openai.transcription("gpt-4o-mini-transcribe")
const program = Effect.gen(function* () {
const events = Array.from(yield* Stream.runCollect(Transcription.stream({ model, audio })))
const error = yield* Stream.runCollect(Transcription.stream({ model, audio })).pipe(Effect.flip)
return { events, error }
})
const { events, error } = yield* program.pipe(
Effect.provide(
layer((input) =>
Effect.sync(() =>
input.respond(bodies.shift() ?? "", { headers: { "content-type": "text/event-stream" } }),
),
),
),
)
expect(events.map((event) => event.type)).toEqual(["text-delta", "finish"])
expect(error.reason).toMatchObject({ _tag: "ProviderInternal", body: failure })
expect(error.message).toContain("The server had an error")
}),
)
it.effect("streams whisper-1 as a single finish from a plain request", () =>
Effect.gen(function* () {
const bodies: Array<string> = []
const events = Array.from(
yield* Stream.runCollect(Transcription.stream({ model: openai.transcription("whisper-1"), audio })).pipe(
Effect.provide(
layer((input) =>
Effect.sync(() => {
bodies.push(input.text)
return input.respond(
JSON.stringify({ text: "Hello there.", usage: { type: "duration", seconds: 2 } }),
{ headers: { "content-type": "application/json" } },
)
}),
),
),
),
)
expect(bodies[0]).not.toContain('name="stream"')
expect(events).toEqual([
expect.objectContaining({ type: "finish", text: "Hello there.", usage: { type: "seconds", seconds: 2 } }),
])
}),
)
it.effect("streams diarized segments and finishes with the accumulated segments", () =>
Effect.gen(function* () {
const calls: Array<Call> = []
const body = sseEvents(
{ type: "transcript.text.segment", id: "seg_0", text: " Hello", start: 0.25, end: 0.7, speaker: "A" },
{ type: "transcript.text.segment", id: "seg_1", text: " there.", start: 0.7, end: 1.25, speaker: "B" },
{ type: "transcript.text.done", text: "Hello there.", usage: { type: "duration", seconds: 2 } },
)
const events = Array.from(
yield* Stream.runCollect(
Transcription.stream({ model: openai.transcription("gpt-4o-transcribe-diarize"), audio, diarize: true }),
).pipe(
Effect.provide(
layer((input) =>
observe(calls, input).pipe(
Effect.as(input.respond(body, { headers: { "content-type": "text/event-stream" } })),
),
),
),
),
)
const form = yield* formFields(calls[0])
expect(form).toMatchObject({
model: ["gpt-4o-transcribe-diarize"],
response_format: ["diarized_json"],
chunking_strategy: ["auto"],
stream: ["true"],
})
const segments = [
{ text: "Hello", startSeconds: 0.25, endSeconds: 0.7, speaker: "A" },
{ text: "there.", startSeconds: 0.7, endSeconds: 1.25, speaker: "B" },
]
expect(events).toEqual([
{ type: "segment", segment: segments[0] },
{ type: "segment", segment: segments[1] },
expect.objectContaining({
type: "finish",
text: "Hello there.",
segments,
usage: { type: "seconds", seconds: 2 },
}),
])
}),
)
it.effect("requests whisper-1 segment timestamps as verbose_json", () =>
Effect.gen(function* () {
const calls: Array<Call> = []
const response = yield* Transcription.generate({
model: openai.transcription("whisper-1"),
audio,
timestamps: "segment",
}).pipe(
Effect.provide(
layer((input) =>
observe(calls, input).pipe(
Effect.as(
json(input, {
text: "Hello there.",
language: "English",
duration: 1.25,
segments: [
{ id: 0, text: " Hello", start: 0.25, end: 0.7 },
{ id: 1, text: " there.", start: 0.7, end: 1.25 },
],
usage: { type: "duration", seconds: 2 },
}),
),
),
),
),
)
const form = yield* formFields(calls[0])
expect(form).toMatchObject({
model: ["whisper-1"],
response_format: ["verbose_json"],
"timestamp_granularities[]": ["segment"],
})
expect(form.stream).toBeUndefined()
expect(response).toMatchObject({
text: "Hello there.",
segments: [
{ text: "Hello", startSeconds: 0.25, endSeconds: 0.7 },
{ text: "there.", startSeconds: 0.7, endSeconds: 1.25 },
],
language: "english",
durationSeconds: 1.25,
usage: { type: "seconds", seconds: 2 },
})
}),
)
it.effect("fails an OpenAI stream that ends without transcript.text.done as incomplete", () =>
Effect.gen(function* () {
const events: Array<TranscriptionEvent> = []
const error = yield* Transcription.stream({ model: openai.transcription("gpt-4o-mini-transcribe"), audio }).pipe(
Stream.runForEach((event) => Effect.sync(() => events.push(event))),
Effect.flip,
Effect.provide(
layer((input) =>
Effect.succeed(
input.respond(sseEvents({ type: "transcript.text.delta", delta: "Hel" }), {
headers: { "content-type": "text/event-stream" },
}),
),
),
),
)
expect(events).toEqual([{ type: "text-delta", delta: "Hel" }])
expect(error.reason).toMatchObject({ _tag: "InvalidProviderOutput", classification: "incomplete-stream" })
expect(error.reason.http?.status).toBe(200)
}),
)
it.effect("sends a Deepgram URL source as a JSON body and repeats array query parameters", () =>
Effect.gen(function* () {
const calls: Array<Call> = []
const response = yield* Transcription.generate({
model: deepgram,
audio: Media.url("https://a.test/call.mp3", { mediaType: "audio/mpeg" }),
language: "en",
providerOptions: { keyterm: ["OpenCode", "Effect"] },
}).pipe(
Effect.provide(
layer((input) =>
observe(calls, input).pipe(
Effect.as(
json(input, {
metadata: { request_id: "dg_1", duration: 2 },
results: { channels: [{ alternatives: [{ transcript: "Hello there." }] }] },
}),
),
),
),
),
)
expect(calls).toHaveLength(1)
const url = new URL(calls[0].url)
expect(url.origin + url.pathname).toBe("https://deepgram.test/v1/listen")
expect([...url.searchParams]).toEqual([
["model", "nova-3"],
["smart_format", "true"],
["language", "en"],
["keyterm", "OpenCode"],
["keyterm", "Effect"],
])
expect(calls[0].headers.get("content-type")).toBe("application/json")
expect(JSON.parse(calls[0].body)).toEqual({ url: "https://a.test/call.mp3" })
expect(response).toMatchObject({
text: "Hello there.",
usage: { type: "seconds", seconds: 2 },
providerMetadata: { deepgram: { requestId: "dg_1" } },
})
}),
)
it.effect("transcribes an AssemblyAI URL source without uploading it first", () =>
Effect.gen(function* () {
const calls: Array<Call> = []
const response = yield* Transcription.generate({
model: assemblyai,
audio: Media.url("https://a.test/call.mp3", { mediaType: "audio/mpeg" }),
}).pipe(
Effect.provide(
layer((input) =>
Effect.gen(function* () {
const { call } = yield* observe(calls, input)
if (call.method === "POST") return json(input, { id: "tr_1", status: "queued" })
return json(input, { id: "tr_1", status: "completed", text: "Hello there.", audio_duration: 2 })
}),
),
),
)
expect(calls.map((call) => `${call.method} ${call.url}`)).toEqual([
"POST https://assemblyai.test/v2/transcript",
"GET https://assemblyai.test/v2/transcript/tr_1",
"GET https://assemblyai.test/v2/transcript/tr_1",
])
expect(JSON.parse(calls[0].body)).toEqual({
audio_url: "https://a.test/call.mp3",
speech_models: ["universal-3-5-pro"],
language_detection: true,
})
expect(response).toMatchObject({ text: "Hello there.", usage: { type: "seconds", seconds: 2 } })
}),
)
it.effect(
"uploads inline audio to AssemblyAI, resumes polling from a persisted token, and surfaces failed transcripts",
() =>
@@ -171,4 +441,185 @@ describe("Transcription", () => {
expect(failure.reason).toMatchObject({ _tag: "ProviderInternal", body: failed })
}),
)
it.effect("enables AssemblyAI speaker labels when only an expected speaker count is given", () =>
Effect.gen(function* () {
const calls: Array<Call> = []
yield* Transcription.start({ model: assemblyai, audio: Media.url("https://a.test/call.mp3"), speakers: 2 }).pipe(
Effect.provide(
layer((input) => observe(calls, input).pipe(Effect.as(json(input, { id: "tr_1", status: "queued" })))),
),
)
expect(calls.map((call) => JSON.parse(call.body))).toEqual([
{
audio_url: "https://a.test/call.mp3",
speech_models: ["universal-3-5-pro"],
language_detection: true,
speaker_labels: true,
speakers_expected: 2,
},
])
}),
)
it.effect("rejects ElevenLabs prompts, webhooks, per-channel transcripts, and untimed diarization", () =>
Effect.gen(function* () {
const errors = yield* Effect.all(
[
Transcription.generate({ model: elevenlabs, audio, prompt: "OpenCode" }),
Transcription.generate({ model: elevenlabs, audio, providerOptions: { webhook: true } }),
Transcription.generate({ model: elevenlabs, audio, http: { body: { use_multi_channel: true } } }),
Transcription.generate({
model: elevenlabs,
audio,
diarize: true,
providerOptions: { timestamps_granularity: "none" },
}),
Transcription.generate({
model: elevenlabs,
audio: Media.ref("file_1", { provider: "elevenlabs", mediaType: "audio/mpeg" }),
}),
].map((effect) => Effect.flip(effect)),
)
expect(errors.map((error) => [error.reason._tag, "operation" in error.reason && error.reason.operation])).toEqual(
[
["UnsupportedOperation", "media.prompt"],
["UnsupportedOperation", "transcription.webhook"],
["UnsupportedOperation", "transcription.multichannel"],
["UnsupportedOperation", "media.timestamps"],
["InvalidRequest", false],
],
)
}).pipe(Effect.provide(layer(() => Effect.die("an unsupported request reached the network")))),
)
it.effect("sends ElevenLabs URL audio as source_url and groups diarized words into speaker turns", () =>
Effect.gen(function* () {
const calls: Array<Call> = []
const token = (text: string, type: string, start: number, end: number, speaker_id?: string) => ({
text,
type,
start,
end,
speaker_id,
logprob: 0,
})
const response = yield* Transcription.generate({
model: elevenlabs,
audio: Media.url("https://a.test/call.mp3"),
language: "en",
speakers: 2,
providerOptions: { keyterms: ["OpenCode", "Scribe"], tag_audio_events: true, diarize: false },
}).pipe(
Effect.provide(
layer((input) =>
observe(calls, input).pipe(
Effect.as(
json(input, {
language_code: "ENG",
text: "Ready? (laughs) Yes. Go",
words: [
token("Ready?", "word", 0, 0.5, "speaker_0"),
token(" ", "spacing", 0.5, 0.6, "speaker_0"),
token("(laughs)", "audio_event", 0.6, 1, "speaker_0"),
token(" ", "spacing", 1, 1.1, "speaker_0"),
token("Yes.", "word", 1.2, 1.5, "speaker_1"),
token(" ", "spacing", 1.5, 1.6, "speaker_1"),
token("Go", "word", 1.6, 1.9, "speaker_0"),
],
transcription_id: "tr_1",
audio_duration_secs: 2,
}),
),
),
),
),
)
// `observe` re-encodes the FormData with a new boundary, so read the boundary from the sent body.
const boundary = /^--(\S+)/.exec(calls[0].body)?.[1]
const form = yield* Effect.promise(() =>
new Response(calls[0].body, {
headers: { "content-type": `multipart/form-data; boundary=${boundary}` },
}).formData(),
)
expect(calls[0].url).toBe("https://elevenlabs.test/v1/speech-to-text")
expect(calls[0].headers.get("xi-api-key")).toBe("test")
expect(Array.from(form.entries())).toEqual([
["model_id", "scribe_v2"],
["source_url", "https://a.test/call.mp3"],
["language_code", "en"],
["diarize", "true"],
["num_speakers", "2"],
["keyterms", "OpenCode"],
["keyterms", "Scribe"],
["tag_audio_events", "true"],
])
expect(response.segments).toEqual([
{ text: "Ready?", startSeconds: 0, endSeconds: 0.5, speaker: "speaker_0" },
{ text: "Yes.", startSeconds: 1.2, endSeconds: 1.5, speaker: "speaker_1" },
{ text: "Go", startSeconds: 1.6, endSeconds: 1.9, speaker: "speaker_0" },
])
expect(response.words?.map((word) => [word.text, word.speaker, word.confidence])).toEqual([
["Ready?", "speaker_0", 1],
["Yes.", "speaker_1", 1],
["Go", "speaker_0", 1],
])
expect(response.language).toBe("eng")
expect(response.usage).toEqual({ type: "seconds", seconds: 2 })
expect(response.providerMetadata).toEqual({ elevenlabs: { transcriptionId: "tr_1" } })
}),
)
it.effect("rejects reading an AssemblyAI result before the transcript finishes", () =>
Effect.gen(function* () {
const generation = yield* Transcription.resume(assemblyai, { transcriptID: "tr_1" })
const error = yield* generation.result().pipe(Effect.flip)
expect(error.reason._tag).toBe("InvalidRequest")
expect(error.message).toBe("AssemblyAI generation tr_1 has not finished; await it before reading the result")
expect(error.reason.body).toBe(JSON.stringify({ id: "tr_1", status: "processing" }))
expect(error.reason.http?.status).toBe(200)
}).pipe(
Effect.provide(
layer((input) =>
Effect.succeed(
input.respond(JSON.stringify({ id: "tr_1", status: "processing" }), {
headers: { "content-type": "application/json" },
}),
),
),
),
),
)
it.effect("surfaces Gemini transcripts that ended without STOP instead of returning them as complete", () =>
Effect.gen(function* () {
const document = (finishReason?: string) =>
JSON.stringify({
candidates: [{ content: { parts: [{ audioTranscription: { text: "Hello" } }] }, finishReason }],
})
const withheld = JSON.stringify({ candidates: [{ finishReason: "SAFETY" }] })
const generate = (body: string) =>
Transcription.generate({ model: google, audio }).pipe(
Effect.provide(
layer((input) => Effect.succeed(input.respond(body, { headers: { "content-type": "application/json" } }))),
),
)
const truncated = yield* generate(document()).pipe(Effect.flip)
const partial = yield* generate(document("MAX_TOKENS"))
const policy = yield* generate(withheld).pipe(Effect.flip)
expect(truncated.reason).toMatchObject({ _tag: "InvalidProviderOutput", classification: "incomplete-stream" })
expect(partial.text).toBe("Hello")
expect(partial.notices).toEqual([
{
type: "other",
message: "Google Transcription finished with MAX_TOKENS",
providerMetadata: { google: { finishReason: "MAX_TOKENS" } },
},
])
expect(policy.reason).toMatchObject({ _tag: "ContentPolicy", body: withheld })
}),
)
})
+501 -28
View File
@@ -1,9 +1,10 @@
import { describe, expect } from "bun:test"
import { Effect, Layer, Stream } from "effect"
import { Media, Video, VideoClient, type GenerationEvent } from "../src/index.js"
import { Effect, Fiber, Layer, Stream } from "effect"
import * as TestClock from "effect/testing/TestClock"
import { Media, Video, VideoClient, type GenerationEvent, type VideoEvent } from "../src/index.js"
import { Fal, Google, Runway, XAI } from "../src/providers.js"
import { it } from "./lib/effect.js"
import { dynamicResponse, json, observe, settle, type Call } from "./lib/http.js"
import { dynamicResponse, json, observe, settle, type Call, type HandlerInput } from "./lib/http.js"
const layer = (handler: Parameters<typeof dynamicResponse>[0]) =>
VideoClient.layer.pipe(Layer.provideMerge(dynamicResponse(handler)))
@@ -162,27 +163,39 @@ describe("Video / Google Veo", () => {
),
)
it.effect("surfaces an operation error as a failed generation with the provider body", () =>
Effect.gen(function* () {
const failure = {
name: operation,
done: true,
error: { code: 3, message: "Prompt violates policy", status: "INVALID_ARGUMENT" },
}
const error = yield* Video.generate({ model, prompt: "nope" }).pipe(
Effect.flip,
Effect.provide(
layer((input) =>
Effect.succeed(input.request.method === "POST" ? json(input, { name: operation }) : json(input, failure)),
),
),
)
expect(error.reason._tag).toBe("ProviderInternal")
expect(error.message).toBe("Google Veo operation failed: Prompt violates policy")
expect(error.reason.body).toBe(JSON.stringify(failure))
expect(error.reason.http?.status).toBe(200)
}),
)
for (const terminal of [
{ error: { code: 3, message: "Prompt violates policy", status: "INVALID_ARGUMENT" }, tag: "InvalidRequest" },
{ error: { code: 9, message: "Unsupported resolution", status: "FAILED_PRECONDITION" }, tag: "InvalidRequest" },
{ error: { code: 11, message: "Duration out of range", status: "OUT_OF_RANGE" }, tag: "InvalidRequest" },
{ error: { code: 7, message: "Permission denied", status: "PERMISSION_DENIED" }, tag: "Authentication" },
{ error: { code: 16, message: "Invalid credentials", status: "UNAUTHENTICATED" }, tag: "Authentication" },
{ error: { code: 8, message: "Quota exceeded", status: "RESOURCE_EXHAUSTED" }, tag: "RateLimit" },
{ error: { code: 13, message: "Internal error", status: "INTERNAL" }, tag: "ProviderInternal" },
{ error: { code: 14, message: "Service unavailable", status: "UNAVAILABLE" }, tag: "ProviderInternal" },
{ error: { message: "Something broke" }, tag: "ProviderInternal" },
]) {
it.effect(
`surfaces ${terminal.error.status ?? "an uncoded"} operation error as ${terminal.tag} with the provider body`,
() =>
Effect.gen(function* () {
const failure = { name: operation, done: true, error: terminal.error }
const error = yield* Video.generate({ model, prompt: "nope" }).pipe(
Effect.flip,
Effect.provide(
layer((input) =>
Effect.succeed(
input.request.method === "POST" ? json(input, { name: operation }) : json(input, failure),
),
),
),
)
expect(error.reason._tag).toBe(terminal.tag)
expect(error.message).toBe(`Google Veo operation failed: ${terminal.error.message}`)
expect(error.reason.body).toBe(JSON.stringify(failure))
expect(error.reason.http?.status).toBe(200)
}),
)
}
it.effect("reports fully filtered output as a content policy failure", () =>
Effect.gen(function* () {
@@ -332,12 +345,37 @@ describe("Video / xAI", () => {
for (const terminal of [
{
body: { status: "failed", error: { code: "invalid_argument", message: "Prompt cannot be empty." } },
tag: "ProviderInternal",
tag: "InvalidRequest",
message: "xAI Video generation failed (invalid_argument): Prompt cannot be empty.",
},
{
body: { status: "failed", error: { code: "failed_precondition", message: "Extension is not supported." } },
tag: "InvalidRequest",
message: "xAI Video generation failed (failed_precondition): Extension is not supported.",
},
{
body: { status: "failed", error: { code: "permission_denied", message: "Team lacks access." } },
tag: "Authentication",
message: "xAI Video generation failed (permission_denied): Team lacks access.",
},
{
body: { status: "failed", error: { code: "service_unavailable", message: "Overloaded." } },
tag: "ProviderInternal",
message: "xAI Video generation failed (service_unavailable): Overloaded.",
},
{
body: { status: "failed", error: { code: "internal_error", message: "Generation failed." } },
tag: "ProviderInternal",
message: "xAI Video generation failed (internal_error): Generation failed.",
},
{
body: { status: "failed", error: { code: "constructor", message: "Future code." } },
tag: "ProviderInternal",
message: "xAI Video generation failed (constructor): Future code.",
},
{ body: { status: "expired" }, tag: "InvalidRequest", message: "xAI Video request req_1 expired" },
]) {
it.effect(`surfaces ${terminal.body.status} generations with the provider body`, () =>
it.effect(`surfaces ${terminal.body.error?.code ?? terminal.body.status} generations with the provider body`, () =>
Effect.gen(function* () {
const error = yield* Video.generate({ model, prompt: "x" }).pipe(Effect.flip)
expect(error.reason._tag).toBe(terminal.tag)
@@ -542,6 +580,45 @@ describe("Video / fal", () => {
),
)
for (const failure of [
{
name: "a COMPLETED status carrying an error",
status: { status: "COMPLETED", error: "Invalid input", error_type: "ValidationError" },
result: { status: 422, body: { detail: [{ loc: ["body", "prompt"], msg: "Invalid input" }] } },
tag: "InvalidRequest",
},
{
name: "a failing response_url",
status: { status: "COMPLETED" },
result: { status: 500, body: { detail: "Internal error" } },
tag: "ProviderInternal",
},
]) {
it.effect(`fails await for ${failure.name} with the response_url body and HTTP context`, () =>
Effect.gen(function* () {
// A transient 500 on the result fetch is retried first; the body and HTTP context survive the final failure.
const fiber = yield* Effect.forkChild(Video.generate({ model, prompt: "x" }).pipe(Effect.flip))
yield* TestClock.adjust("5 minutes")
const error = yield* Fiber.join(fiber)
expect(error.reason._tag).toBe(failure.tag)
expect(error.reason.body).toBe(JSON.stringify(failure.result.body))
expect(error.reason.http).toMatchObject({ url: urls.response, status: failure.result.status })
}).pipe(
Effect.provide(
layer((input) =>
Effect.succeed(
input.request.method === "POST"
? json(input, submitted)
: input.request.url === urls.response
? json(input, failure.result.body, { status: failure.result.status })
: json(input, failure.status),
),
),
),
),
)
}
it.effect("rejects model-specific common fields and points at providerOptions", () =>
Effect.gen(function* () {
const errors = yield* Effect.forEach(
@@ -576,7 +653,7 @@ describe("Video / Runway", () => {
const model = runway.video("gen4.5")
const taskUrl = "https://runway.test/v1/tasks/task_1"
it.effect("submits image_to_video with the API version header, polls the task, and reports credits", () =>
it.effect("submits image_to_video, polls the task, reports credits, and keeps the finished task on cancel", () =>
Effect.gen(function* () {
const calls: Array<Call> = []
const program = Effect.gen(function* () {
@@ -624,7 +701,7 @@ describe("Video / Runway", () => {
return json(input, { id: "task_1", estimatedCost: { credits: 25 } })
}
expect(call.url).toBe(taskUrl)
if (call.method === "DELETE") return input.respond(null, { status: 204 })
if (call.method === "DELETE") return yield* Effect.die("cancel deleted a finished Runway task")
if (nth === 1) return json(input, { id: "task_1", status: "PENDING", estimatedCost: { credits: 25 } })
if (nth === 2) return json(input, { id: "task_1", status: "THROTTLED", estimatedCost: { credits: 25 } })
if (nth === 3) return json(input, { id: "task_1", status: "RUNNING", progress: 0.5 })
@@ -653,6 +730,32 @@ describe("Video / Runway", () => {
`GET ${taskUrl}`,
`GET ${taskUrl}`,
`GET ${taskUrl}`,
`GET ${taskUrl}`,
])
}),
)
it.effect("cancels a task that is still running", () =>
Effect.gen(function* () {
const calls: Array<Call> = []
yield* Effect.gen(function* () {
const generation = yield* Video.start({ model, prompt: "x" })
yield* generation.cancel()
}).pipe(
Effect.provide(
layer((input) =>
Effect.gen(function* () {
const { call } = yield* observe(calls, input)
if (call.method === "POST") return json(input, { id: "task_1" })
if (call.method === "DELETE") return input.respond(null, { status: 204 })
return json(input, { id: "task_1", status: "RUNNING", progress: 0.2 })
}),
),
),
)
expect(calls.map((call) => `${call.method} ${call.url}`)).toEqual([
"POST https://runway.test/v1/text_to_video",
`GET ${taskUrl}`,
`DELETE ${taskUrl}`,
])
}),
@@ -703,6 +806,11 @@ describe("Video / Runway", () => {
tag: "ProviderInternal",
message: "Runway task failed (INTERNAL.BAD_OUTPUT.CODE01): Something broke",
},
{
body: { status: "FAILED", failure: "Unsupported dimensions", failureCode: "ASSET.INVALID" },
tag: "InvalidRequest",
message: "Runway task failed (ASSET.INVALID): Unsupported dimensions",
},
{ body: { status: "CANCELLED" }, tag: "InvalidRequest", message: "Runway task task_1 was cancelled" },
]) {
it.effect(`surfaces ${terminal.body.failureCode ?? terminal.body.status} with the task body`, () =>
@@ -792,6 +900,39 @@ describe("Video / Runway", () => {
}),
)
it.effect("streams the observations of a failed task and then fails with the task body", () =>
Effect.gen(function* () {
const calls: Array<Call> = []
const events: Array<VideoEvent> = []
const failed = { status: "FAILED", failure: "Something broke", failureCode: "INTERNAL.BAD_OUTPUT.CODE01" }
const program = Video.stream({ model, prompt: "x" }, { poll: { interval: "1 second" } }).pipe(
Stream.runForEach((event) => Effect.sync(() => events.push(event))),
Effect.flip,
)
const error = yield* settle(program, 3).pipe(
Effect.provide(
layer((input) =>
Effect.gen(function* () {
const { call, nth } = yield* observe(calls, input)
if (call.method === "POST") return json(input, { id: "task_1" })
if (nth === 1) return json(input, { status: "PENDING" })
if (nth === 2) return json(input, { status: "RUNNING", progress: 0.5 })
return json(input, failed)
}),
),
),
)
expect(events).toEqual([
{ type: "generation-queued", id: "task_1", position: undefined },
{ type: "generation-progress", id: "task_1", progress: 0.5 },
])
expect(error.reason._tag).toBe("ProviderInternal")
expect(error.message).toBe("Runway task failed (INTERNAL.BAD_OUTPUT.CODE01): Something broke")
expect(error.reason.body).toBe(JSON.stringify(failed))
expect(error.reason.http?.status).toBe(200)
}),
)
it.effect("fails a stream with a Timeout reason once polling passes the poll deadline", () =>
Effect.gen(function* () {
const program = Video.stream(
@@ -812,3 +953,335 @@ describe("Video / Runway", () => {
}),
)
})
// ---------------------------------------------------------------------------
// Transient read failures
// ---------------------------------------------------------------------------
describe("Video / transient read failures", () => {
const model = Runway.configure({ apiKey: "test", baseURL: "https://runway.test/v1" }).video("gen4.5")
const succeeded = { id: "task_1", status: "SUCCEEDED", output: ["https://runway.test/out.mp4"] }
const failure = (input: HandlerInput, status: number, headers?: Record<string, string>) =>
json(input, { error: `HTTP ${status}` }, { status, headers })
const methods = (calls: ReadonlyArray<Call>) => calls.map((call) => call.method)
it.effect("retries a 503 status poll and a 503 result read, then returns the result", () =>
Effect.gen(function* () {
const calls: Array<Call> = []
const response = yield* settle(
Video.generate({ model, prompt: "x" }, { poll: { interval: "1 second" } }),
5,
).pipe(
Effect.provide(
layer((input) =>
Effect.gen(function* () {
const { call, nth } = yield* observe(calls, input)
if (call.method === "POST") return json(input, { id: "task_1" })
// 1: status fails, 2: status succeeds, 3: result fails, 4: result succeeds.
if (nth === 1 || nth === 3) return failure(input, 503)
return json(input, succeeded)
}),
),
),
)
expect(response.video.source).toMatchObject({ type: "url", url: "https://runway.test/out.mp4" })
expect(methods(calls)).toEqual(["POST", "GET", "GET", "GET", "GET"])
}),
)
it.effect("waits for a 429 retry-after before polling again", () =>
Effect.gen(function* () {
const calls: Array<Call> = []
const fiber = yield* Effect.forkChild(
Video.generate({ model, prompt: "x" }, { poll: { interval: "1 second" } }).pipe(
Effect.provide(
layer((input) =>
Effect.gen(function* () {
const { call, nth } = yield* observe(calls, input)
if (call.method === "POST") return json(input, { id: "task_1" })
if (nth === 1) return failure(input, 429, { "retry-after": "10" })
return json(input, succeeded)
}),
),
),
),
)
yield* TestClock.adjust("9 seconds")
expect(methods(calls)).toEqual(["POST", "GET"])
yield* TestClock.adjust("1 second")
yield* Fiber.join(fiber)
expect(methods(calls)).toEqual(["POST", "GET", "GET", "GET"])
}),
)
it.effect("fails a 400 status poll without retrying", () =>
Effect.gen(function* () {
const calls: Array<Call> = []
const error = yield* Video.generate({ model, prompt: "x" }).pipe(
Effect.flip,
Effect.provide(
layer((input) =>
Effect.gen(function* () {
const { call } = yield* observe(calls, input)
return call.method === "POST" ? json(input, { id: "task_1" }) : failure(input, 400)
}),
),
),
)
expect(error.reason._tag).toBe("InvalidRequest")
expect(methods(calls)).toEqual(["POST", "GET"])
}),
)
it.effect("stops retrying at poll.timeout with a Timeout reason", () =>
Effect.gen(function* () {
const calls: Array<Call> = []
const error = yield* settle(
Video.generate({ model, prompt: "x" }, { poll: { interval: "1 second", timeout: "5 seconds" } }).pipe(
Effect.flip,
),
6,
).pipe(
Effect.provide(
layer((input) =>
Effect.gen(function* () {
const { call } = yield* observe(calls, input)
return call.method === "POST" ? json(input, { id: "task_1" }) : failure(input, 503)
}),
),
),
)
expect(error.reason._tag).toBe("Timeout")
expect(calls.filter((call) => call.method === "GET").length).toBeGreaterThan(1)
}),
)
it.effect("bounds a streamed result read's retries by poll.timeout", () =>
Effect.gen(function* () {
const calls: Array<Call> = []
const error = yield* settle(
Video.stream({ model, prompt: "x" }, { poll: { interval: "1 second", timeout: "5 seconds" } }).pipe(
Stream.runCollect,
Effect.flip,
),
6,
).pipe(
Effect.provide(
layer((input) =>
Effect.gen(function* () {
const { call, nth } = yield* observe(calls, input)
if (call.method === "POST") return json(input, { id: "task_1" })
return nth === 1 ? json(input, succeeded) : failure(input, 503)
}),
),
),
)
expect(error.reason._tag).toBe("Timeout")
expect(calls.filter((call) => call.method === "GET").length).toBeGreaterThan(2)
}),
)
it.effect("never retries a failed submit", () =>
Effect.gen(function* () {
const calls: Array<Call> = []
const error = yield* Video.generate({ model, prompt: "x" }).pipe(
Effect.flip,
Effect.provide(layer((input) => observe(calls, input).pipe(Effect.map(() => failure(input, 503))))),
)
expect(error.reason._tag).toBe("ProviderInternal")
expect(methods(calls)).toEqual(["POST"])
}),
)
it.effect("never retries a failed cancel", () =>
Effect.gen(function* () {
const calls: Array<Call> = []
const error = yield* Effect.gen(function* () {
const generation = yield* Video.start({ model, prompt: "x" })
return yield* generation.cancel().pipe(Effect.flip)
}).pipe(
Effect.provide(
layer((input) =>
Effect.gen(function* () {
const { call } = yield* observe(calls, input)
if (call.method === "POST") return json(input, { id: "task_1" })
if (call.method === "DELETE") return failure(input, 503)
return json(input, { id: "task_1", status: "RUNNING" })
}),
),
),
)
expect(error.reason._tag).toBe("ProviderInternal")
expect(methods(calls)).toEqual(["POST", "GET", "DELETE"])
}),
)
})
// ---------------------------------------------------------------------------
// Shared queued behavior
// ---------------------------------------------------------------------------
describe("Video / queued result", () => {
for (const pending of [
{
model: Google.configure({ apiKey: "test", baseURL: "https://google.test/v1beta" }).video("veo-3.1"),
token: { operation: "models/veo-3.1/operations/op_1" },
body: { name: "models/veo-3.1/operations/op_1", done: false },
name: "Google Veo",
},
{
model: XAI.configure({ apiKey: "test", baseURL: "https://xai.test/v1" }).video("grok-imagine-video-1.5"),
token: { requestID: "req_1" },
body: { status: "pending", progress: 40 },
name: "xAI Video",
},
{
model: Runway.configure({ apiKey: "test", baseURL: "https://runway.test/v1" }).video("gen4.5"),
token: { taskID: "task_1" },
body: { status: "RUNNING", progress: 0.5 },
name: "Runway",
},
]) {
it.effect(`rejects reading a ${pending.model.provider} result before the generation finishes`, () =>
Effect.gen(function* () {
const generation = yield* Video.resume(pending.model, pending.token)
const error = yield* generation.result().pipe(Effect.flip)
expect(error.reason._tag).toBe("InvalidRequest")
expect(error.message).toBe(
`${pending.name} generation ${generation.id} has not finished; await it before reading the result`,
)
expect(error.reason.body).toBe(JSON.stringify(pending.body))
expect(error.reason.http?.status).toBe(200)
}).pipe(Effect.provide(layer((input) => Effect.succeed(json(input, pending.body))))),
)
}
const veoOperation = "models/veo-3.1/operations/op_1"
const falURLs = {
status: "https://queue.fal.test/fal-ai/veo3.1/requests/r1/status",
response: "https://queue.fal.test/fal-ai/veo3.1/requests/r1",
cancel: "https://queue.fal.test/fal-ai/veo3.1/requests/r1/cancel",
}
for (const queued of [
{
model: Google.configure({ apiKey: "test", baseURL: "https://google.test/v1beta" }).video("veo-3.1"),
submitted: { name: veoOperation },
token: { operation: veoOperation },
submitURL: "https://google.test/v1beta/models/veo-3.1:predictLongRunning",
statusURL: `https://google.test/v1beta/${veoOperation}`,
resultURL: `https://google.test/v1beta/${veoOperation}`,
running: { name: veoOperation, done: false },
done: {
name: veoOperation,
done: true,
response: { generateVideoResponse: { generatedSamples: [{ video: { uri: "https://google.test/out.mp4" } }] } },
},
result: undefined,
url: "https://google.test/out.mp4",
},
{
model: XAI.configure({ apiKey: "test", baseURL: "https://xai.test/v1" }).video("grok-imagine-video-1.5"),
submitted: { request_id: "req_1" },
token: { requestID: "req_1" },
submitURL: "https://xai.test/v1/videos/generations",
statusURL: "https://xai.test/v1/videos/req_1",
resultURL: "https://xai.test/v1/videos/req_1",
running: { status: "pending", progress: 40 },
done: { status: "done", video: { url: "https://vidgen.x.ai/out.mp4", respect_moderation: true } },
result: undefined,
url: "https://vidgen.x.ai/out.mp4",
},
{
model: Fal.configure({ apiKey: "test", baseURL: "https://queue.fal.test" }).video("fal-ai/veo3.1"),
submitted: {
request_id: "r1",
status_url: falURLs.status,
response_url: falURLs.response,
cancel_url: falURLs.cancel,
},
token: { requestID: "r1", statusURL: falURLs.status, responseURL: falURLs.response, cancelURL: falURLs.cancel },
submitURL: "https://queue.fal.test/fal-ai/veo3.1",
statusURL: falURLs.status,
resultURL: falURLs.response,
running: { status: "IN_PROGRESS" },
done: { status: "COMPLETED" },
result: { video: { url: "https://v3.fal.media/out.mp4" } },
url: "https://v3.fal.media/out.mp4",
},
]) {
it.effect(`resumes a ${queued.model.provider} generation from a JSON round-tripped token`, () =>
Effect.gen(function* () {
const calls: Array<Call> = []
const response = yield* Effect.gen(function* () {
const started = yield* Video.start({ model: queued.model, prompt: "x" })
const resumed = yield* Video.resume(queued.model, JSON.parse(JSON.stringify(started.token)))
expect(resumed.status).toBe("running")
expect(resumed.token).toEqual(queued.token)
return yield* resumed.await()
}).pipe(
Effect.provide(
layer((input) =>
Effect.gen(function* () {
const { call, nth } = yield* observe(calls, input)
if (call.method === "POST") return json(input, queued.submitted)
if (call.url === queued.resultURL && queued.result !== undefined) return json(input, queued.result)
return json(input, nth === 1 ? queued.running : queued.done)
}),
),
),
)
expect(response.video.source).toEqual(expect.objectContaining({ type: "url", url: queued.url }))
expect(calls.map((call) => `${call.method} ${call.url}`)).toEqual([
`POST ${queued.submitURL}`,
`GET ${queued.statusURL}`,
`GET ${queued.statusURL}`,
`GET ${queued.resultURL}`,
])
}),
)
}
for (const queued of [
{
model: Google.configure({ apiKey: "test", baseURL: "https://google.test/v1beta" }).video("veo-3.1"),
submitted: { name: veoOperation },
},
{
model: XAI.configure({ apiKey: "test", baseURL: "https://xai.test/v1" }).video("grok-imagine-video-1.5"),
submitted: { request_id: "req_1" },
},
]) {
it.effect(`cancels a ${queued.model.provider} generation without sending a request`, () =>
Effect.gen(function* () {
const calls: Array<Call> = []
yield* Effect.gen(function* () {
const generation = yield* Video.start({ model: queued.model, prompt: "x" })
yield* generation.cancel()
}).pipe(
Effect.provide(
layer((input) =>
Effect.gen(function* () {
const { call } = yield* observe(calls, input)
if (call.method !== "POST") return yield* Effect.die(`cancel sent ${call.method} ${call.url}`)
return json(input, queued.submitted)
}),
),
),
)
expect(calls.map((call) => call.method)).toEqual(["POST"])
}),
)
}
it.effect("rejects a status that only matches an inherited property", () =>
Effect.gen(function* () {
const error = yield* Video.resume(
XAI.configure({ apiKey: "test", baseURL: "https://xai.test/v1" }).video("grok-imagine-video-1.5"),
{ requestID: "req_1" },
).pipe(Effect.flip)
expect(error.reason._tag).toBe("InvalidProviderOutput")
expect(error.message).toBe('Unknown generation status "constructor"')
expect(error.reason.body).toBe(JSON.stringify({ status: "constructor" }))
}).pipe(Effect.provide(layer((input) => Effect.succeed(json(input, { status: "constructor" }))))),
)
})
@@ -1,5 +1,5 @@
import { expect, test, type Page } from "@playwright/test"
import type { OpenCodeEvent, SessionMessageInfo } from "@opencode/client/promise"
import type { OpenCodeEvent, SessionInboxInfo, SessionMessageInfo } from "@opencode/client/promise"
import { base64Encode } from "@opencode/util/encode"
import { mockOpenCodeServer } from "../utils/mock-server"
import { expectAppVisible } from "../utils/waits"
@@ -14,7 +14,12 @@ type InboxRow = {
sessionID: string
time: { created: number }
type: "user"
payload: { text: string; metadata?: Record<string, unknown> }
payload: {
text: string
metadata?: Record<string, unknown>
files?: Extract<SessionInboxInfo, { type: "user" }>["payload"]["files"]
agents?: Extract<SessionInboxInfo, { type: "user" }>["payload"]["agents"]
}
delivery: "steer" | "queue"
}
@@ -29,7 +34,7 @@ function createQueueMock(seed: string[], messages: SessionMessageInfo[] = []) {
}))
const events: OpenCodeEvent[] = []
const prompts: Record<string, unknown>[] = []
const changes: { inboxID: string; action: "cancel" | "steer" }[] = []
const changes: { inboxID: string; action: "cancel" | "steer" | "queue" }[] = []
const log: string[] = []
let sequence = 0
const emit = <Type extends OpenCodeEvent["type"]>(
@@ -234,6 +239,108 @@ test("editing restores the existing draft and replaces only the original queue p
expect(mock.log[0]).toBe("prompt:queue")
})
test("Undo cancels only the selected queued prompt and focuses the restored input", async ({ page }) => {
const mock = createQueueMock(["first queued prompt", "second queued prompt", "third queued prompt"])
const view = await openSession(page, mock)
await expect(view.rows).toHaveCount(3)
const row = view.rows.filter({ hasText: "second queued prompt" })
const actions = row.locator('[data-slot="session-queue-actions"] button')
await expect(actions).toHaveCount(3)
expect(
await actions.evaluateAll((buttons) =>
buttons.map((button) => button.getAttribute("aria-label") ?? button.textContent?.trim()),
),
).toEqual(["Steer", "Undo", "Remove"])
const undo = row.getByRole("button", { name: "Undo" })
await expect(undo).toHaveText("")
await expect(undo.locator("svg use")).toHaveAttribute("href", "#opencode-v2-icon-arrow-down-to-line")
await undo.hover()
await expect(page.getByRole("tooltip")).toHaveText("Undo")
await undo.click()
await expect(view.rows.locator('[data-action="session-queue-edit"]')).toHaveText([
"first queued prompt",
"third queued prompt",
])
await expect(view.input).toHaveText("second queued prompt")
await expect(view.input).toBeFocused()
expect(mock.changes).toEqual([{ inboxID: "inb_seed_2", action: "cancel" }])
expect(mock.prompts).toEqual([])
})
test("Undo appends to an existing draft and restores inline attachments", async ({ page }) => {
const mock = createQueueMock(["queued with image"])
mock.rows[0].payload.files = [
{
data: "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVQIHWP4z8DwHwAFgAI/ScL/nwAAAABJRU5ErkJggg==",
mime: "image/png",
source: { type: "inline" },
name: "shot.png",
},
]
const view = await openSession(page, mock)
await view.input.fill("my draft")
await view.rows.getByRole("button", { name: "Undo" }).click()
await expect(view.rows).toHaveCount(0)
await expect(view.input).toHaveText("my draft\n\nqueued with image")
await expect(view.input).toBeFocused()
await expect(view.composer.getByRole("img", { name: "shot.png" })).toBeVisible()
expect(mock.changes).toEqual([{ inboxID: "inb_seed_1", action: "cancel" }])
})
test("Undo stays usable with a long queue on a narrow screen", async ({ page }, testInfo) => {
await page.setViewportSize({ width: 390, height: 844 })
const text = "Review the detailed error report and check every step of the retry path ".repeat(4)
const mock = createQueueMock([text, ...Array.from({ length: 6 }, (_, index) => `queued follow-up ${index + 1}`)])
const view = await openSession(page, mock)
await expect(view.rows).toHaveCount(7)
const row = view.rows.filter({ hasText: text })
await row.getByRole("button", { name: "Undo" }).hover()
await expect(page.getByRole("tooltip")).toHaveText("Undo")
await page.screenshot({ path: testInfo.outputPath("undo-narrow-queue.png") })
await row.getByRole("button", { name: "Undo" }).click()
await expect(view.rows).toHaveCount(6)
await expect(view.input).toHaveText(text)
await expect(view.input).toBeFocused()
expect(mock.changes).toEqual([{ inboxID: "inb_seed_1", action: "cancel" }])
})
test("Undo preserves mentioned file and agent references on resubmission", async ({ page }) => {
const mock = createQueueMock(["inspect @main.ts with @build"])
mock.rows[0].payload.files = [
{
data: "aGk=",
mime: "text/plain",
source: { type: "uri", uri: "file:///repo/main.ts" },
name: "main.ts",
mention: { start: 8, end: 16, text: "@main.ts" },
},
]
mock.rows[0].payload.agents = [{ name: "build", mention: { start: 22, end: 28, text: "@build" } }]
const view = await openSession(page, mock)
await view.rows.getByRole("button", { name: "Undo" }).click()
await expect(view.input).toHaveText("inspect @main.ts with @build")
await view.input.press("Enter")
await expect.poll(() => mock.prompts.length).toBe(1)
expect(mock.prompts[0].files).toMatchObject([
{ uri: "data:text/plain;base64,aGk=", mention: { text: "@main.ts", start: 8, end: 16 } },
])
expect(mock.prompts[0].agents).toMatchObject([{ name: "build", mention: { text: "@build" } }])
})
test("Undo does not discard hidden file context", async ({ page }) => {
const mock = createQueueMock(["inspect this file"])
mock.rows[0].payload.files = [
{ data: "aGk=", mime: "text/plain", source: { type: "uri", uri: "file:///repo/main.ts" }, name: "main.ts" },
]
const view = await openSession(page, mock)
await view.rows.getByRole("button", { name: "Undo" }).click()
await expect(page.getByText("Edit this prompt in the queue to preserve its file context")).toBeVisible()
await expect(view.rows).toHaveCount(1)
await expect(view.input).toHaveText("")
expect(mock.changes).toEqual([])
})
for (const delivery of ["steer", "queue"] as const) {
test(`keeps finished tools above a pending ${delivery === "queue" ? "queue-to-steer" : "steer"} follow-up`, async ({
page,
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@opencode/app",
"version": "2.0.16",
"version": "2.0.18",
"description": "",
"type": "module",
"exports": {
+1
View File
@@ -47,6 +47,7 @@ export type ComposerDelivery = "steer" | "queue"
// is loaded in the editor.
export type ComposerQueue = {
count: Accessor<number>
undoing: Accessor<boolean>
// Delivery a plain submit uses right now.
delivery: Accessor<ComposerDelivery>
// Delivery offered on Mod+Enter and the toolbar hint button; undefined hides the hint.
@@ -168,6 +168,7 @@ function ComposerStory(props: {
alternate: () => props.alternate,
editing: () => undefined,
confirmEdit() {},
undoing: () => false,
cancelEdit() {},
editFirst: () => false,
}
+2
View File
@@ -16,6 +16,7 @@ export function Composer(props: {
class?: string
model: ComposerModel
borderUnderlay?: boolean
readOnly?: boolean
suggestionBoundary?: () => HTMLElement | undefined
}) {
const dialog = useDialog()
@@ -27,6 +28,7 @@ export function Composer(props: {
<ComposerEditor
controller={props.model}
borderUnderlay={props.borderUnderlay}
readOnly={props.readOnly}
class={props.class}
modelControlsVisible={!props.model.model.loading}
attachKeybind={command.keybindParts("file.attach")}
+1
View File
@@ -371,6 +371,7 @@ export function createComposerModel(adapter: ComposerAdapter, options?: { queue?
onSubmit: (submitOptions) => {
if (!available()) return
const queue = options?.queue
if (queue?.undoing()) return
// Confirming an edit re-admits the queued prompt instead of sending
// the composer value as a new prompt. Enter keeps it queued in
// place; the alternate action sends it as a steer.
+3
View File
@@ -874,6 +874,9 @@ export const dict = {
"session.queue.send": "Send",
"session.queue.steerTooltip": "Send without interrupting",
"session.queue.remove": "Remove",
"session.queue.undo": "Undo",
"session.queue.undoShell": "Leave shell mode before undoing a queued prompt",
"session.queue.undoUnavailable": "Edit this prompt in the queue to preserve its file context",
"session.queue.reorder": "Reorder queued prompt",
"session.queue.attachments.one": "{{count}} attachment",
"session.queue.attachments.other": "{{count}} attachments",
@@ -175,6 +175,18 @@ function SessionQueueRow(props: { queue: SessionQueueView; id: string; index: nu
{props.queue.working() ? language.t("session.queue.steer") : language.t("session.queue.send")}
</Button>
</Tooltip>
<Tooltip placement="top" value={language.t("session.queue.undo")}>
<IconButton
data-action="session-queue-undo"
type="button"
size="small"
variant="ghost-muted"
icon={<Icon name="arrow-down-to-line" />}
disabled={props.queue.busy()}
aria-label={language.t("session.queue.undo")}
onClick={() => props.queue.undo(props.id)}
/>
</Tooltip>
</Show>
<Tooltip placement="top" value={language.t("session.queue.remove")}>
<IconButton
@@ -1,6 +1,6 @@
import { describe, expect, test } from "bun:test"
import type { SessionInboxInfo } from "@opencode/client/promise"
import { queuedPromptAttachments, queuedPromptRows } from "./queue"
import { queuedPromptAttachments, queuedPromptUndoDraft, queuedPromptRows } from "./queue"
const queued = [
{
@@ -104,3 +104,43 @@ describe("queuedPromptAttachments", () => {
expect(queuedPromptAttachments(item)).toEqual([])
})
})
describe("queuedPromptUndoDraft", () => {
test("keeps full text, structured mentions, and inline images", () => {
const item = {
...queued[0],
payload: {
text: "inspect @main.ts with @build",
files: [
{
data: "aGk=",
mime: "text/plain",
source: { type: "uri" as const, uri: "file:///repo/main.ts" },
name: "main.ts",
mention: { start: 8, end: 16, text: "@main.ts" },
},
{ data: "aGk=", mime: "image/png", source: { type: "inline" as const }, name: "shot.png" },
],
agents: [{ name: "build", mention: { start: 22, end: 28, text: "@build" } }],
},
} satisfies SessionInboxInfo
expect(queuedPromptUndoDraft(item)).toMatchObject([
{ type: "text", content: "inspect " },
{ type: "file", content: "@main.ts", url: "data:text/plain;base64,aGk=" },
{ type: "text", content: " with " },
{ type: "agent", content: "@build", name: "build" },
{ type: "image", filename: "shot.png" },
])
})
test("does not drop hidden file context", () => {
const item = {
...queued[0],
payload: {
text: "inspect this",
files: [{ data: "aGk=", mime: "text/plain", source: { type: "uri" as const, uri: "file:///repo/main.ts" } }],
},
} satisfies SessionInboxInfo
expect(queuedPromptUndoDraft(item)).toBeUndefined()
})
})
+115 -3
View File
@@ -3,10 +3,11 @@ import { createStore } from "solid-js/store"
import { useMutation } from "@tanstack/solid-query"
import type { SessionInboxInfo } from "@opencode/client/promise"
import { SessionMessage } from "@opencode/schema/session-message"
import { Skill } from "@opencode/schema/skill"
import type { ComposerDelivery } from "@/composer/adapter"
import type { ComposerStateTarget } from "@/composer/submission-state"
import type { ImageAttachmentPart, PathAttachmentPart, Prompt } from "@/composer/state"
import { clonePrompt, isAttachment, promptLength } from "@/composer/prompt-parts"
import { appendPrompt, clonePrompt, isAttachment, promptLength } from "@/composer/prompt-parts"
import { buildPromptRequest } from "@/composer/request"
import { blobDataUrl, createLegacyBlobReference } from "@/runtime/persistence/drafts"
import { readPromptPresentation } from "@/composer/comment-note"
@@ -42,6 +43,7 @@ export function createSessionQueue(input: {
mutationFn: async (
change:
| { type: "reorder"; inboxIDs: string[] }
| { type: "undo"; item: QueuedPrompt; prompt: Prompt }
| {
type: "edit"
inboxIDs: string[]
@@ -54,6 +56,16 @@ export function createSessionQueue(input: {
},
) => {
if (change.type === "reorder") return rewrite(change.inboxIDs)
if (change.type === "undo") {
await server.api.session.inbox.cancel({ sessionID: input.sessionID, inboxID: change.item.id })
const draft = input.draft.current()
const prompt = promptLength(draft)
? appendPrompt(draft, change.prompt)
: [...change.prompt, ...draft.filter(isAttachment)]
input.draft.set(prompt, promptLength(prompt))
input.restoreFocus(promptLength(prompt))
return
}
const replacement = await editedPromptInput(
input.sessionID,
location().directory,
@@ -139,6 +151,21 @@ export function createSessionQueue(input: {
if (state.editing?.id === id) cancelEdit()
return server.api.session.inbox.cancel({ sessionID: input.sessionID, inboxID: id }).catch(() => notify())
}
const undo = (id: string) => {
if (mutation.isPending || state.editing) return
const item = queued().find((entry) => entry.id === id)
if (!item) return
if (input.draft.mode.current() !== "normal") {
showToast({ title: language.t("session.queue.undoShell") })
return
}
const prompt = queuedPromptUndoDraft(item)
if (!prompt) {
showToast({ title: language.t("session.queue.undoUnavailable") })
return
}
mutation.mutate({ type: "undo", item, prompt })
}
const reorder = (inboxIDs: string[]) => {
if (mutation.isPending) return Promise.resolve()
return mutation.mutateAsync({ type: "reorder", inboxIDs }).catch(() => undefined)
@@ -226,9 +253,11 @@ export function createSessionQueue(input: {
editFirst,
rows,
busy: () => mutation.isPending,
undoing: () => mutation.isPending && mutation.variables?.type === "undo",
working: input.working,
steer,
remove,
undo,
edit,
reorder,
}
@@ -239,7 +268,7 @@ export type SessionQueue = ReturnType<typeof createSessionQueue>
// The slice of the queue the panel renders and drives.
export type SessionQueueView = Pick<
SessionQueue,
"rows" | "editing" | "working" | "busy" | "steer" | "remove" | "edit" | "reorder"
"rows" | "editing" | "working" | "busy" | "steer" | "remove" | "undo" | "edit" | "reorder"
>
export function queuedPromptRows(items: QueuedPrompt[], replacement?: { original: string; replacement: string }) {
@@ -249,7 +278,8 @@ export function queuedPromptRows(items: QueuedPrompt[], replacement?: { original
.map((item) => ({
id: item.id,
text: queuedPromptText(item),
attachments: (item.payload.files?.length ?? 0) + (readPromptPresentation(item.payload.metadata)?.attachments.length ?? 0),
attachments:
(item.payload.files?.length ?? 0) + (readPromptPresentation(item.payload.metadata)?.attachments.length ?? 0),
}))
}
@@ -287,6 +317,88 @@ export function queuedPromptAttachments(item: QueuedPrompt): (ImageAttachmentPar
]
}
// Use the full model-visible text so comment notes and path references remain
// in the draft. Convert mentioned files, agents, and skills back into editor
// parts; a detached draft cannot represent non-mentioned file context.
export function queuedPromptUndoDraft(item: QueuedPrompt): Prompt | undefined {
if (
item.payload.files?.some((file) => !isComposerAttachment(file) && !file.mention) ||
item.payload.agents?.some((agent) => !agent.mention) ||
item.payload.skills?.some((skill) => !skill.mention)
)
return
const text = item.payload.text
const references = [
...(item.payload.files ?? []).flatMap((file) =>
file.mention
? [
{
type: "file" as const,
content: file.mention.text,
start: file.mention.start,
end: file.mention.end,
path: file.name ?? file.mention.text.replace(/^@/, ""),
filename: file.name,
mime: file.mime,
url: `data:${file.mime};base64,${file.data}`,
},
]
: [],
),
...(item.payload.agents ?? []).flatMap((agent) =>
agent.mention
? [
{
type: "agent" as const,
content: agent.mention.text,
start: agent.mention.start,
end: agent.mention.end,
name: agent.name,
},
]
: [],
),
...(item.payload.skills ?? []).flatMap((skill) =>
skill.mention
? [
{
type: "skill" as const,
content: skill.mention.text,
start: skill.mention.start,
end: skill.mention.end,
id: Skill.ID.make(skill.id),
name: Skill.Name.make(skill.name),
},
]
: [],
),
].sort((left, right) => left.start - right.start)
if (
references.some(
(part, index) =>
part.start < (references[index - 1]?.end ?? 0) || text.slice(part.start, part.end) !== part.content,
)
)
return
const parts: Prompt = references.flatMap((part, index) => {
const start = references[index - 1]?.end ?? 0
return [
...(part.start > start
? [{ type: "text" as const, content: text.slice(start, part.start), start, end: part.start }]
: []),
part,
]
})
const start = references.at(-1)?.end ?? 0
return [
...parts,
...(text.length > start || !parts.length
? [{ type: "text" as const, content: text.slice(start), start, end: text.length }]
: []),
...queuedPromptAttachments(item).filter((part) => part.type === "image"),
]
}
function isComposerAttachment(file: NonNullable<QueuedPrompt["payload"]["files"]>[number]) {
return !file.mention && file.source.type === "inline"
}
+6 -1
View File
@@ -224,7 +224,12 @@ export function ActiveSessionComposerRegion(props: {
<div class="relative">
<SessionQueuePanel queue={props.model.queue} />
<div class="relative z-10">
<Composer model={props.model.composer} borderUnderlay suggestionBoundary={props.suggestionBoundary} />
<Composer
model={props.model.composer}
borderUnderlay
readOnly={props.model.queue.undoing()}
suggestionBoundary={props.suggestionBoundary}
/>
</div>
</div>
}
+3 -2
View File
@@ -1,7 +1,7 @@
{
"$schema": "https://json.schemastore.org/package.json",
"name": "@opencode/cli",
"version": "2.0.16",
"version": "2.0.18",
"type": "module",
"license": "MIT",
"bin": {
@@ -25,6 +25,7 @@
},
"dependencies": {
"@agentclientprotocol/sdk": "1.2.1",
"@clack/core": "1.0.0-alpha.1",
"@clack/prompts": "1.0.0-alpha.1",
"@effect/platform-node": "catalog:",
"@opencode/client": "workspace:*",
@@ -41,7 +42,7 @@
"effect": "catalog:",
"immer": "11.1.4",
"jsonc-parser": "3.3.1",
"open": "10.1.2",
"picocolors": "1.1.1",
"solid-js": "catalog:",
"tree-sitter-bash": "0.25.0",
"tree-sitter-powershell": "0.25.10",
+6 -4
View File
@@ -142,10 +142,10 @@ const Root = Spec.make(typeof OPENCODE_CLI_NAME === "string" ? OPENCODE_CLI_NAME
],
}),
Spec.make("auth", {
description: "manage AI providers and credentials",
description: "manage integrations and credentials",
commands: [
Spec.make("list", {
description: "list providers and credentials",
description: "list integrations and credentials",
params: {
...ServerParams,
format: Flag.choice("format", ["default", "json"]).pipe(
@@ -155,7 +155,7 @@ const Root = Spec.make(typeof OPENCODE_CLI_NAME === "string" ? OPENCODE_CLI_NAME
},
}),
Spec.make("login", {
description: "log in to a provider",
description: "connect an integration",
params: {
...ServerParams,
target: Argument.string("target").pipe(
@@ -228,7 +228,9 @@ const Root = Spec.make(typeof OPENCODE_CLI_NAME === "string" ? OPENCODE_CLI_NAME
}),
Spec.make("auth", {
description: "Authenticate with an OAuth-capable remote MCP server",
params: { name: Argument.string("name").pipe(Argument.withDescription("Name of the MCP server")) },
params: {
name: Argument.string("name").pipe(Argument.withDescription("Name of the MCP server"), Argument.optional),
},
}),
Spec.make("logout", {
description: "Remove stored OAuth credentials for an MCP server",
@@ -1,8 +1,9 @@
import { autocomplete, intro, log, outro, select, spinner, text } from "@clack/prompts"
import { intro, log, outro, select, spinner, text } from "@clack/prompts"
import { Effect, Option } from "effect"
import type { FormAnswer, IntegrationInfo, OpenCodeClient } from "@opencode/client"
import { Commands } from "../../commands"
import { Runtime } from "../../../framework/runtime"
import { selectIntegration, type IntegrationChoice } from "../../../ui/integration-picker"
import { handlePromptErrors, openUrl, prompt, requireInteractive } from "../../../ui/prompt"
import { answerForm, secret } from "./form"
import {
@@ -21,10 +22,8 @@ const integrationPriority = new Map([
["opencode", 1],
["openai", 2],
["github-copilot", 3],
["google", 4],
["anthropic", 5],
["openrouter", 6],
["vercel", 7],
["anthropic", 4],
["google", 5],
])
export default Runtime.handler(
@@ -74,30 +73,35 @@ const findIntegration = Effect.fn("cli.auth.login.integration")(function* (clien
}
const integrations = yield* loadIntegrations(client)
if (target) return yield* resolveIntegration(integrations, target)
const available = integrations
const choices = loginChoices(integrations)
if (choices.length === 0) return yield* Effect.fail(new Error("No authentication integrations are available"))
const id = yield* prompt<string>(() => selectIntegration(choices))
return yield* resolveIntegration(integrations, id)
})
export function loginChoices(integrations: IntegrationInfo[]): IntegrationChoice[] {
return integrations
.filter((integration) => connectMethods(integration).length > 0)
.toSorted(
(a, b) =>
Number(b.metadata?.source === "mcp") - Number(a.metadata?.source === "mcp") ||
(integrationPriority.get(a.id) ?? integrationPriority.size) -
(integrationPriority.get(b.id) ?? integrationPriority.size) ||
a.name.localeCompare(b.name) ||
a.id.localeCompare(b.id),
)
if (available.length === 0) return yield* Effect.fail(new Error("No authentication integrations are available"))
const id = yield* prompt<string>(() =>
autocomplete({
message: "Select integration",
maxItems: 8,
options: available.map((integration) => {
const option = { value: integration.id, label: integration.name, hint: integration.id }
if (integration.connections.length > 0) return { ...option, hint: "connected" }
if (integration.id === "opencode") return { ...option, hint: "recommended" }
return option
}),
}),
)
return yield* resolveIntegration(available, id)
})
.map((integration) => ({
value: integration.id,
label: integration.name,
category:
integration.metadata?.source === "mcp"
? "MCP"
: integrationPriority.has(integration.id)
? "Popular"
: "Services",
connected: integration.connections.length > 0,
}))
}
const chooseMethod = Effect.fn("cli.auth.login.method")(function* (methods: ConnectMethod[], target?: string) {
if (target) return yield* resolveMethod(methods, target)
@@ -143,17 +147,18 @@ const keyLogin = Effect.fn("cli.auth.login.key")(function* (
)
})
const oauthLogin = Effect.fn("cli.auth.login.oauth")(function* (
export const oauthLogin = Effect.fn("cli.auth.login.oauth")(function* (
client: OpenCodeClient,
integration: IntegrationInfo,
method: Extract<ConnectMethod, { type: "oauth" }>,
answer?: FormAnswer,
label?: string,
) {
const progress = spinner()
progress.start("Starting authorization...")
const started = yield* request((signal) =>
client.integration.oauth.connect(
{ integrationID: integration.id, methodID: method.id, answer, location },
{ integrationID: integration.id, methodID: method.id, answer, label, location },
{ signal },
),
).pipe(Effect.tapCause(() => Effect.sync(() => progress.stop("Authentication failed", 1))))
@@ -190,16 +195,14 @@ const oauthLogin = Effect.fn("cli.auth.login.oauth")(function* (
return
}
const waiting = spinner()
waiting.start("Waiting for authorization...")
const status = yield* waitForOAuth(client, integration.id, attempt.attemptID).pipe(
Effect.tapCause(() => Effect.sync(() => waiting.stop("Authentication failed", 1))),
)
// Clack's spinner captures Ctrl+C and exits the process directly, which would skip the finalizer that
// cancels the attempt. Waits that can last minutes use plain log lines so Ctrl+C interrupts normally.
log.step("Waiting for authorization...")
const status = yield* waitForOAuth(client, integration.id, attempt.attemptID)
if (status.status === "complete") {
waiting.stop(`Connected to ${integration.name}`)
log.success(`Connected to ${integration.name}`)
return
}
waiting.stop("Authentication failed", 1)
if (status.status === "failed") yield* Effect.fail(new Error(status.message))
yield* Effect.fail(new Error("Authorization expired"))
})
@@ -226,14 +229,21 @@ const commandLogin = Effect.fn("cli.auth.login.command")(function* (
),
).pipe(Effect.ignore),
)
const status = yield* waitForCommand(client, integration.id, started.data.attemptID, (message) =>
progress.message(message.trim() || "Waiting for authentication command..."),
).pipe(Effect.tapCause(() => Effect.sync(() => progress.stop("Authentication failed", 1))))
progress.stop("Authentication command started")
// The status message accumulates the command's stderr; print each completed line once.
let printed = 0
log.step("Waiting for authentication command...")
const status = yield* waitForCommand(client, integration.id, started.data.attemptID, (message) => {
const end = message.lastIndexOf("\n") + 1
if (end <= printed) return
const output = message.slice(printed, end).trim()
printed = end
if (output) log.message(output)
})
if (status.status === "complete") {
progress.stop(`Connected to ${integration.name}`)
log.success(`Connected to ${integration.name}`)
return
}
progress.stop("Authentication failed", 1)
if (status.status === "failed") yield* Effect.fail(new Error(status.message))
yield* Effect.fail(new Error("Authentication expired"))
})
+97 -53
View File
@@ -1,65 +1,109 @@
import { EOL } from "node:os"
import { Effect } from "effect"
import {
OpenCode,
type IntegrationAttemptStatus,
type IntegrationOAuthMethod,
type OpenCodeClient,
} from "@opencode/client"
import { confirm, intro, log, outro } from "@clack/prompts"
import { Effect, Option } from "effect"
import { OpenCode, type IntegrationInfo, type IntegrationOAuthMethod, type McpServer } from "@opencode/client"
import { Commands } from "../../commands"
import { Runtime } from "../../../framework/runtime"
import { Service } from "@opencode/client/effect/service"
import { ServiceConfig } from "../../../services/service-config"
import { resolveIntegration } from "./resolve"
import { selectIntegration, type IntegrationChoice } from "../../../ui/integration-picker"
import { handlePromptErrors, prompt, requireInteractive } from "../../../ui/prompt"
import { answerForm } from "../auth/form"
import { oauthLogin } from "../auth/login"
import { loadIntegrations, request } from "../auth/shared"
const location = { directory: process.cwd() }
export default Runtime.handler(
Commands.commands.mcp.commands.auth,
Effect.fn("cli.mcp.auth")(function* (input) {
const endpoint = yield* Service.ensure(yield* ServiceConfig.options())
const client = OpenCode.make({ baseUrl: endpoint.url, headers: Service.headers(endpoint) })
const integration = yield* resolveIntegration(client, input.name, location)
if (!integration)
return yield* Effect.fail(new Error(`MCP server "${input.name}" is not an OAuth-capable remote server`))
const method = integration.methods.find(
(candidate): candidate is IntegrationOAuthMethod => candidate.type === "oauth",
)
if (!method)
return yield* Effect.fail(new Error(`MCP server "${input.name}" is not an OAuth-capable remote server`))
const started = yield* Effect.promise(() =>
client.integration.oauth.connect({ integrationID: integration.id, methodID: method.id, location }),
)
const attempt = started.data
if (attempt.mode === "code")
return yield* Effect.fail(new Error("This server requires manual code entry, which the CLI does not support"))
process.stdout.write(attempt.instructions + EOL + attempt.url + EOL)
const result = yield* poll(client, integration.id, attempt.attemptID)
if (result.status === "complete") {
process.stdout.write(`Authenticated with ${input.name}` + EOL)
return
}
const reason = result.status === "failed" ? `: ${result.message}` : ""
return yield* Effect.fail(new Error(`Authentication ${result.status}${reason}`))
}),
Effect.fn("cli.mcp.auth")((input) => authenticate(Option.getOrUndefined(input.name)).pipe(handlePromptErrors)),
)
const poll = (
client: OpenCodeClient,
integrationID: string,
attemptID: string,
): Effect.Effect<Exclude<IntegrationAttemptStatus, { status: "pending" }>> =>
Effect.gen(function* () {
const status = yield* Effect.promise(() =>
client.integration.oauth.status({ integrationID, attemptID, location }),
).pipe(Effect.map((result) => result.data))
if (status.status === "pending") {
yield* Effect.sleep("1 second")
return yield* poll(client, integrationID, attemptID)
const authenticate = Effect.fn("cli.mcp.auth.run")(function* (name?: string) {
if (!name) yield* requireInteractive("Pass an MCP server name when running without an interactive terminal")
intro("Authenticate an MCP server")
const endpoint = yield* Service.ensure(yield* ServiceConfig.options())
const client = OpenCode.make({ baseUrl: endpoint.url, headers: Service.headers(endpoint) })
const integrations = yield* loadIntegrations(client)
const servers = yield* request((signal) => client.mcp.list({ location }, { signal }))
const choices = mcpAuthChoices(servers.data, integrations)
if (!name && choices.length === 0) {
log.warn("No OAuth-capable MCP servers configured")
log.info(
`Remote MCP servers support OAuth by default. Add one with \`opencode mcp add\` or in opencode.json:\n${exampleConfig}`,
)
outro("Done")
return
}
const server = name ? servers.data.find((item) => item.name === name) : undefined
if (name && !server) return yield* Effect.fail(new Error(`MCP server not found: ${name}`))
const integrationID = server
? server.integrationID
: yield* prompt<string>(() => selectIntegration(choices, "MCP server"))
const integration = integrations.find((item) => item.id === integrationID)
const method = integration?.methods.find(
(candidate): candidate is IntegrationOAuthMethod => candidate.type === "oauth",
)
if (!integration || !method)
return yield* Effect.fail(new Error(`MCP server "${name}" is not an OAuth-capable remote server`))
if (integration.connections.length > 0) {
const status = servers.data.find((item) => item.integrationID === integration.id)?.status.status
if (status === "needs_auth") log.warn(`${integration.name} has expired credentials. Re-authenticating...`)
if (status !== "needs_auth" && process.stdin.isTTY && process.stdout.isTTY) {
const again = yield* prompt<boolean>(() =>
confirm({ message: `${integration.name} already has valid credentials. Re-authenticate?` }),
)
if (!again) {
outro("Cancelled")
return
}
}
return status
})
}
// Re-authenticating replaces the previous sign-in rather than adding an account. The new credential
// keeps the active one's label, and the old ones are only removed once it is stored, so a failed
// attempt keeps them.
const previous = integration.connections.filter((connection) => connection.type === "credential")
yield* oauthLogin(client, integration, method, yield* answerForm(method.form), previous[0]?.label)
yield* Effect.forEach(
previous,
(connection) => request((signal) => client.credential.remove({ credentialID: connection.id }, { signal })),
{ discard: true },
)
outro("Done")
})
const exampleConfig = `
"mcp": {
"my-server": {
"type": "remote",
"url": "https://example.com/mcp"
}
}`
// Choices carry the server-owned integration ID so provider integrations with colliding names never match.
export function mcpAuthChoices(servers: McpServer[], integrations: IntegrationInfo[]): IntegrationChoice[] {
const byID = new Map(integrations.map((integration) => [integration.id, integration]))
return servers
.flatMap((server) => {
const integration = server.integrationID ? byID.get(server.integrationID) : undefined
if (!integration?.methods.some((method) => method.type === "oauth")) return []
return [
{
value: integration.id,
label: server.name,
category: "MCP" as const,
connected: integration.connections.length > 0,
hint: statusHint(server.status),
},
]
})
.toSorted((a, b) => a.label.localeCompare(b.label) || a.value.localeCompare(b.value))
}
function statusHint(status: McpServer["status"]) {
if (status.status === "needs_auth") return "needs authentication"
if (status.status === "failed" || status.status === "disabled") return status.status
return undefined
}
+67
View File
@@ -0,0 +1,67 @@
import { AutocompletePrompt } from "@clack/core"
import { S_BAR, S_BAR_END, S_RADIO_ACTIVE, S_RADIO_INACTIVE, symbol } from "@clack/prompts"
import color from "picocolors"
export type IntegrationChoice = {
value: string
label: string
category: "MCP" | "Popular" | "Services"
connected: boolean
hint?: string
}
export async function selectIntegration(choices: IntegrationChoice[], kind = "integration") {
const result = await new AutocompletePrompt<IntegrationChoice>({
options: choices,
filter: (search, choice) =>
[choice.label, choice.value, choice.category].some((value) => value.toLowerCase().includes(search.toLowerCase())),
validate: (value) => (value ? undefined : `Select an ${kind}`),
render() {
const title = `${color.gray(S_BAR)}\n${symbol(this.state)} Select ${kind}`
if (this.state === "submit") {
const choice = choices.find((item) => item.value === this.value)
return `${title}\n${color.gray(S_BAR)} ${color.dim(choice?.label ?? "")}`
}
if (this.state === "cancel")
return `${title}\n${color.gray(S_BAR)} ${color.strikethrough(color.dim(this.userInput))}`
// Leave room for the category headings as well as Clack's title and footer.
const maxItems = Math.min(8, Math.max(2, (process.stdout.rows ?? 24) - 14 - Number(this.state === "error")))
const compact = (process.stdout.rows ?? 24) < 18
const start = Math.min(
Math.max(0, this.cursor - Math.min(2, maxItems - 1)),
Math.max(0, this.filteredOptions.length - maxItems),
)
const visible = this.filteredOptions.slice(start, start + maxItems)
const rows = visible.flatMap((choice, index) => [
...(index === 0 || visible[index - 1].category !== choice.category
? [...(compact ? [] : [`${color.cyan(S_BAR)} `]), `${color.cyan(S_BAR)} ${color.bold(choice.category)}`]
: []),
`${color.cyan(S_BAR)} ${start + index === this.cursor ? color.green(S_RADIO_ACTIVE) : color.dim(S_RADIO_INACTIVE)} ${
start + index === this.cursor ? choice.label : color.dim(choice.label)
}${choice.connected ? ` ${color.green("✓")}` : ""}${choice.hint ? ` ${color.dim(`(${choice.hint})`)}` : ""}`,
])
return [
title,
`${color.cyan(S_BAR)} ${color.dim("Search:")} ${this.isNavigating ? color.dim(this.userInput) : this.userInputWithCursor}`,
...(visible.length === 0 && this.userInput
? [`${color.cyan(S_BAR)} ${color.yellow(`No ${kind}s found`)}`]
: []),
...(this.state === "error" && visible.length > 0
? [`${color.yellow(S_BAR)} ${color.yellow(this.error)}`]
: []),
...(start > 0 ? [`${color.cyan(S_BAR)} ${color.dim("…")}`] : []),
...rows,
...(start + maxItems < this.filteredOptions.length ? [`${color.cyan(S_BAR)} ${color.dim("…")}`] : []),
`${color.cyan(S_BAR)} ${color.dim(
(process.stdout.columns ?? 80) < 50
? "↑/↓ navigate • Enter select"
: "↑/↓ to select • Enter: confirm • Type: to search",
)}`,
color.cyan(S_BAR_END),
].join("\n")
},
}).prompt()
if (typeof result === "string" || typeof result === "symbol") return result
throw new Error(`No ${kind} selected`)
}
+3 -2
View File
@@ -16,12 +16,13 @@ export function requireInteractive(message: string) {
}
export const openUrl = Effect.fn("cli.prompt.open-url")(function* (url: string) {
const { default: open } = yield* Effect.promise(() => import("open"))
yield* Effect.promise(() => open(url)).pipe(Effect.ignore)
const browser = yield* Effect.promise(() => import("@opencode/util/open"))
yield* Effect.promise(() => browser.openUrl(url)).pipe(Effect.ignore)
})
export function handlePromptErrors<A, E, R>(effect: Effect.Effect<A, E, R>) {
return effect.pipe(
Effect.onInterrupt(() => Effect.sync(() => cancel("Cancelled"))),
Effect.catchIf(
(error) => error === cancelled,
() =>
@@ -0,0 +1,30 @@
import { expect, test } from "bun:test"
import type { IntegrationInfo } from "@opencode/client"
import { loginChoices } from "../src/commands/handlers/auth/login"
const integration = (value: Partial<IntegrationInfo> & Pick<IntegrationInfo, "id" | "name">): IntegrationInfo => ({
methods: [{ type: "key" }],
connections: [],
...value,
})
test("groups the CLI choices like /connect while keeping stable login IDs", () => {
expect(
loginChoices([
integration({ id: "mistral", name: "Mistral" }),
integration({ id: "openai", name: "OpenAI" }),
integration({ id: "linear", name: "Linear", metadata: { source: "mcp" } }),
integration({ id: "github", name: "GitHub", metadata: { source: "mcp" } }),
integration({ id: "opencode", name: "OpenCode Console" }),
integration({ id: "opencode-go", name: "OpenCode Go", connections: [{ type: "env", name: "GO_KEY" }] }),
integration({ id: "unused", name: "Unused", methods: [{ type: "env", names: ["UNUSED_KEY"] }] }),
]),
).toEqual([
{ value: "github", label: "GitHub", category: "MCP", connected: false },
{ value: "linear", label: "Linear", category: "MCP", connected: false },
{ value: "opencode-go", label: "OpenCode Go", category: "Popular", connected: true },
{ value: "opencode", label: "OpenCode Console", category: "Popular", connected: false },
{ value: "openai", label: "OpenAI", category: "Popular", connected: false },
{ value: "mistral", label: "Mistral", category: "Services", connected: false },
])
})
+10 -6
View File
@@ -20,12 +20,12 @@ describe("auth command", () => {
expect(auth.stdout).toContain("list")
expect(auth.stdout).toContain("login")
expect(auth.stdout).toContain("logout")
expect(auth.stdout).toContain("manage AI providers and credentials")
expect(auth.stdout).toContain("list providers and credentials")
expect(auth.stdout).toContain("log in to a provider")
expect(auth.stdout).toContain("manage integrations and credentials")
expect(auth.stdout).toContain("list integrations and credentials")
expect(auth.stdout).toContain("connect an integration")
expect(auth.stdout).toContain("log out of a saved account")
expect(auth.stdout).toContain("switch the active account for an integration")
expect(auth.stdout).not.toContain("connect")
expect(auth.stdout).not.toMatch(/^ connect\s/m)
expect(list.exitCode).toBe(0)
expect(list.stdout).toContain("opencode auth list [flags]")
expect(list.stdout).toContain("--format")
@@ -216,7 +216,8 @@ describe("auth command", () => {
expect(requests).toContainEqual({ method: "DELETE", path: `${endpoint}/con_oauth` })
})
test("settles the OAuth spinner when status polling fails", async () => {
test("reports OAuth status polling failures and cancels the attempt", async () => {
let cancelled = false
using server = authServer((request, url) => {
if (url.pathname === "/api/integration") {
return Response.json(
@@ -245,6 +246,7 @@ describe("auth command", () => {
return new Response("Unavailable", { status: 500 })
}
if (url.pathname === "/api/integration/openai/connect/oauth/con_oauth" && request.method === "DELETE") {
cancelled = true
return new Response(null, { status: 204 })
}
return new Response("Not found", { status: 404 })
@@ -252,8 +254,10 @@ describe("auth command", () => {
const result = await cli(["auth", "login", "openai", "--server", server.url.toString()])
expect(result.exitCode).toBe(1)
expect(result.stdout).toContain("Authentication failed")
expect(result.stdout).toContain("Waiting for authorization...")
expect(result.stdout).toContain("UnexpectedStatus: 500")
expect(result.stdout).toContain("Failed")
expect(cancelled).toBe(true)
expect(result.stdout).not.toContain("\n at ")
})
@@ -0,0 +1,62 @@
import { expect, test } from "bun:test"
import path from "node:path"
import type { IntegrationInfo, McpServer } from "@opencode/client"
import { mcpAuthChoices } from "../src/commands/handlers/mcp/auth"
const server = (
name: string,
integrationID?: string,
status: McpServer["status"] = { status: "pending" },
): McpServer => ({ name, integrationID, status })
const integration = (id: string, methods: IntegrationInfo["methods"], connected = false): IntegrationInfo => ({
id,
name: id,
methods,
connections: connected ? [{ type: "credential", method: "oauth", id: "cred_1", label: "Work" }] : [],
})
test("offers only OAuth-capable MCP servers by their server identity", () => {
expect(
mcpAuthChoices(
[
server("Linear", "mcp_linear", { status: "needs_auth", error: "expired" }),
server("Local"),
server("API key only", "mcp_key"),
server("GitHub", "mcp_github"),
server("Sentry", "mcp_sentry", { status: "failed", error: "boom" }),
server("Unresolved", "mcp_missing"),
],
[
integration("mcp_linear", [{ type: "oauth", id: "login", label: "Linear" }], true),
integration("mcp_github", [{ type: "oauth", id: "login", label: "GitHub" }]),
integration("mcp_sentry", [{ type: "oauth", id: "login", label: "Sentry" }]),
integration("mcp_key", [{ type: "key" }]),
integration("Linear", [{ type: "oauth", id: "login", label: "A provider with a colliding name" }]),
],
),
).toEqual([
{ value: "mcp_github", label: "GitHub", category: "MCP", connected: false, hint: undefined },
{ value: "mcp_linear", label: "Linear", category: "MCP", connected: true, hint: "needs authentication" },
{ value: "mcp_sentry", label: "Sentry", category: "MCP", connected: false, hint: "failed" },
])
})
test("mcp auth accepts an optional server name and rejects no-name noninteractive calls before connecting", async () => {
const cli = (args: string[]) =>
Bun.spawn([process.execPath, "run", "src/index.ts", "mcp", "auth", ...args], {
cwd: path.join(import.meta.dir, ".."),
stdout: "pipe",
stderr: "pipe",
})
const help = cli(["--help"])
expect(await new Response(help.stdout).text()).toContain("opencode mcp auth [flags] [<name>]")
expect(await help.exited).toBe(0)
const missing = cli([])
expect(await new Response(missing.stdout).text()).toContain(
"Pass an MCP server name when running without an interactive terminal",
)
expect(await new Response(missing.stderr).text()).toBe("")
expect(await missing.exited).toBe(1)
})
+1 -1
View File
@@ -1,7 +1,7 @@
{
"$schema": "https://json.schemastore.org/package.json",
"name": "@opencode/client",
"version": "2.0.16",
"version": "2.0.18",
"type": "module",
"license": "MIT",
"repository": {
-33
View File
@@ -235,12 +235,6 @@ export type SessionForkInput = { readonly sessionID: Session.ID; readonly before
export type SessionForkOutput = Session.Info
export type SessionForkOperation<E = never> = (input: SessionForkInput) => Effect.Effect<SessionForkOutput, E>
export type SessionCompanionInput = { readonly sessionID: Session.ID }
export type SessionCompanionOutput = Session.Info
export type SessionCompanionOperation<E = never> = (
input: SessionCompanionInput,
) => Effect.Effect<SessionCompanionOutput, E>
export type SessionSwitchAgentInput = { readonly sessionID: Session.ID; readonly agent: Agent.ID }
export type SessionSwitchAgentOutput = void
export type SessionSwitchAgentOperation<E = never> = (
@@ -451,7 +445,6 @@ export type SessionLogOutput =
}
readonly subpath?: RelativePath | undefined
readonly parentID?: Session.ID | undefined
readonly kind?: Session.Kind | undefined
readonly slug: string
readonly title?: string | undefined
readonly agent?: Agent.ID | undefined
@@ -1425,7 +1418,6 @@ export interface SessionApi<E = never> {
readonly get: SessionGetOperation<E>
readonly remove: SessionRemoveOperation<E>
readonly fork: SessionForkOperation<E>
readonly companion: SessionCompanionOperation<E>
readonly switchAgent: SessionSwitchAgentOperation<E>
readonly switchModel: SessionSwitchModelOperation<E>
readonly update: SessionUpdateOperation<E>
@@ -1521,30 +1513,6 @@ export interface GenerateApi<E = never> {
readonly text: GenerateTextOperation<E>
}
export type VoiceTranscribeInput = { readonly mediaType: string; readonly payload: globalThis.Uint8Array }
export type VoiceTranscribeOutput = { readonly text: string }
export type VoiceTranscribeOperation<E = never> = (
input: VoiceTranscribeInput,
) => Effect.Effect<VoiceTranscribeOutput, E>
export type VoiceSpeechInput = { readonly text: string }
export type VoiceSpeechOutput =
| {
readonly type: "format"
readonly format:
| { readonly type: "mp3" }
| { readonly type: "pcm"; readonly sampleRate: number; readonly channels: number }
}
| { readonly type: "audio"; readonly data: string }
| { readonly type: "done" }
| { readonly type: "error"; readonly message: string }
export type VoiceSpeechOperation<E = never> = (input: VoiceSpeechInput) => Stream.Stream<VoiceSpeechOutput, E>
export interface VoiceApi<E = never> {
readonly transcribe: VoiceTranscribeOperation<E>
readonly speech: VoiceSpeechOperation<E>
}
export type ProviderListInput = { readonly location?: { readonly directory?: string | undefined } | undefined }
export type ProviderListOutput = { readonly location: Location.PublicRef; readonly data: ReadonlyArray<Provider.Info> }
export type ProviderListOperation<E = never> = (input?: ProviderListInput) => Effect.Effect<ProviderListOutput, E>
@@ -2374,7 +2342,6 @@ export interface AppApi<E = never> {
readonly message: MessageApi<E>
readonly model: ModelApi<E>
readonly generate: GenerateApi<E>
readonly voice: VoiceApi<E>
readonly provider: ProviderApi<E>
readonly integration: IntegrationApi<E>
readonly mcp: McpApi<E>
@@ -39,8 +39,6 @@ import type {
SessionRemoveOutput,
SessionForkInput,
SessionForkOutput,
SessionCompanionInput,
SessionCompanionOutput,
SessionSwitchAgentInput,
SessionSwitchAgentOutput,
SessionSwitchModelInput,
@@ -117,10 +115,6 @@ import type {
ModelDefaultOutput,
GenerateTextInput,
GenerateTextOutput,
VoiceTranscribeInput,
VoiceTranscribeOutput,
VoiceSpeechInput,
VoiceSpeechOutput,
ProviderListInput,
ProviderListOutput,
ProviderGetInput,
@@ -456,14 +450,6 @@ const EndpointSessionFork = (raw: RawClient["server.session"]) => (input: Sessio
),
)
const EndpointSessionCompanion = (raw: RawClient["server.session"]) => (input: SessionCompanionInput) =>
preserveEffect<SessionCompanionOutput>()(
raw["session.companion"]({ params: { sessionID: input["sessionID"] } }).pipe(
Effect.mapError(mapClientError),
Effect.map((value) => value.data),
),
)
const EndpointSessionSwitchAgent = (raw: RawClient["server.session"]) => (input: SessionSwitchAgentInput) =>
preserveEffect<SessionSwitchAgentOutput>()(
raw["session.switchAgent"]({ params: { sessionID: input["sessionID"] }, payload: { agent: input["agent"] } }).pipe(
@@ -776,7 +762,6 @@ const adaptGroupSession = (raw: RawClient["server.session"]) => ({
get: EndpointSessionGet(raw),
remove: EndpointSessionRemove(raw),
fork: EndpointSessionFork(raw),
companion: EndpointSessionCompanion(raw),
switchAgent: EndpointSessionSwitchAgent(raw),
switchModel: EndpointSessionSwitchModel(raw),
update: EndpointSessionUpdate(raw),
@@ -858,33 +843,6 @@ const EndpointGenerateText = (raw: RawClient["server.generate"]) => (input: Gene
const adaptGroupGenerate = (raw: RawClient["server.generate"]) => ({ text: EndpointGenerateText(raw) })
type VoiceTranscribeRequest = Parameters<RawClient["server.voice"]["voice.transcribe"]>[0]
const EndpointVoiceTranscribe = (raw: RawClient["server.voice"]) => (input: VoiceTranscribeInput) =>
preserveEffect<VoiceTranscribeOutput>()(
raw["voice.transcribe"]({
query: { mediaType: input["mediaType"] },
payload: input["payload"],
} as VoiceTranscribeRequest).pipe(
Effect.mapError(mapClientError),
Effect.map((value) => value.data),
),
)
const EndpointVoiceSpeech = (raw: RawClient["server.voice"]) => (input: VoiceSpeechInput) =>
preserveStream<VoiceSpeechOutput>()(
Stream.unwrap(
raw["voice.speech"]({ payload: { text: input["text"] } }).pipe(
Effect.mapError(mapClientError),
Effect.map((stream) => stream.pipe(Stream.mapError(mapClientError))),
),
),
)
const adaptGroupVoice = (raw: RawClient["server.voice"]) => ({
transcribe: EndpointVoiceTranscribe(raw),
speech: EndpointVoiceSpeech(raw),
})
const EndpointProviderList = (raw: RawClient["server.provider"]) => (input?: ProviderListInput) =>
preserveEffect<ProviderListOutput>()(
raw["provider.list"]({ query: { location: input?.["location"] } }).pipe(Effect.mapError(mapClientError)),
@@ -1592,7 +1550,6 @@ const adaptClient = (raw: RawClient) => ({
message: adaptGroupMessage(raw["server.message"]),
model: adaptGroupModel(raw["server.model"]),
generate: adaptGroupGenerate(raw["server.generate"]),
voice: adaptGroupVoice(raw["server.voice"]),
provider: adaptGroupProvider(raw["server.provider"]),
integration: adaptGroupIntegration(raw["server.integration"]),
mcp: adaptGroupMcp(raw["server.mcp"]),
@@ -33,8 +33,6 @@ import type {
SessionRemoveOutput,
SessionForkInput,
SessionForkOutput,
SessionCompanionInput,
SessionCompanionOutput,
SessionSwitchAgentInput,
SessionSwitchAgentOutput,
SessionSwitchModelInput,
@@ -111,10 +109,6 @@ import type {
ModelDefaultOutput,
GenerateTextInput,
GenerateTextOutput,
VoiceTranscribeInput,
VoiceTranscribeOutput,
VoiceSpeechInput,
VoiceSpeechOutput,
ProviderListInput,
ProviderListOutput,
ProviderGetInput,
@@ -662,17 +656,6 @@ export function make(options: ClientOptions) {
},
requestOptions,
).then((value) => value.data),
companion: (input: SessionCompanionInput, requestOptions?: RequestOptions) =>
request<{ readonly data: SessionCompanionOutput }>(
{
method: "POST",
path: `/api/experimental/session/${encodeURIComponent(input.sessionID)}/companion`,
successStatus: 200,
declaredStatuses: [400, 401, 404],
empty: false,
},
requestOptions,
).then((value) => value.data),
switchAgent: (input: SessionSwitchAgentInput, requestOptions?: RequestOptions) =>
request<SessionSwitchAgentOutput>(
{
@@ -1158,34 +1141,6 @@ export function make(options: ClientOptions) {
requestOptions,
).then((value) => value.data),
},
voice: {
transcribe: (input: VoiceTranscribeInput, requestOptions?: RequestOptions) =>
request<{ readonly data: VoiceTranscribeOutput }>(
{
method: "POST",
path: `/api/experimental/voice/transcribe`,
query: { mediaType: input["mediaType"] },
body: input["payload"],
successStatus: 200,
declaredStatuses: [400, 401, 503],
empty: false,
binaryBody: true,
},
requestOptions,
).then((value) => value.data),
speech: (input: VoiceSpeechInput, requestOptions?: RequestOptions): AsyncIterable<VoiceSpeechOutput> =>
sse<VoiceSpeechOutput>(
{
method: "POST",
path: `/api/experimental/voice/speech`,
body: { text: input["text"] },
successStatus: 200,
declaredStatuses: [400, 401, 503],
empty: false,
},
requestOptions,
),
},
provider: {
list: (input?: ProviderListInput, requestOptions?: RequestOptions) =>
request<ProviderListOutput>(
@@ -30,8 +30,6 @@ export type PluginFeatures = { server?: true; tui?: true; rpc?: true }
export type PluginState = { status: "active" } | { status: "failed"; error: string; ref?: string }
export type SessionKind = "companion"
export type SessionForkBoundary = { type: "before"; messageID: string } | { type: "through"; messageID: string }
export type MoneyUSD = number
@@ -233,10 +231,6 @@ export type MoneyUSDPerMillionTokens = number
export type GenerateTextResponse = { data: { text: string } }
export type VoiceTranscribeResponse = { data: { text: string } }
export type VoiceSpeechFormat = { type: "mp3" } | { type: "pcm"; sampleRate: number; channels: number }
export type IntegrationCommandMethod = { id: string; type: "command"; label: string; command: Array<string> }
export type IntegrationEnvMethod = { type: "env"; names: Array<string> }
@@ -1475,12 +1469,6 @@ export type ModelCost = {
cache: { read: MoneyUSDPerMillionTokens; write: MoneyUSDPerMillionTokens }
}
export type VoiceSpeechEvent =
| { type: "format"; format: VoiceSpeechFormat }
| { type: "audio"; data: string }
| { type: "done" }
| { type: "error"; message: string }
export type ConnectionInfo = ConnectionCredentialInfo | ConnectionEnvInfo
export type McpServer = {
@@ -1969,7 +1957,6 @@ export type SessionPermissions = {
export type SessionInfo = {
id: string
parentID?: string
kind?: SessionKind
fork?: { sessionID: string; boundary: SessionForkBoundary }
projectID: string
agent?: string
@@ -1999,7 +1986,6 @@ export type SessionCreated = {
location: LocationRef
subpath?: string
parentID?: string
kind?: SessionKind
slug: string
title?: string
agent?: string
@@ -2066,16 +2052,6 @@ export type ConfigEntry =
media?: {
image?: { auto_resize?: boolean; max_width?: number; max_height?: number; max_base64_bytes?: number }
}
voice?: {
transcription?: { model: string | { providerID: string; model: string; variant?: string }; language?: string }
speech?: {
model: string | { providerID: string; model: string; variant?: string }
voice?: string
language?: string
speed?: number
instructions?: string
}
}
tool_output?: { max_lines?: number; max_bytes?: number }
mcp?: {
timeout?: { startup?: number; catalog?: number; execution?: number }
@@ -2999,7 +2975,6 @@ export type SessionImportInput = {
readonly info: {
readonly id: string
readonly parentID?: string
readonly kind?: "companion"
readonly fork?: {
readonly sessionID: string
readonly boundary:
@@ -3317,7 +3292,6 @@ export type SessionImportInput = {
readonly info: {
readonly id: string
readonly parentID?: string
readonly kind?: "companion"
readonly fork?: {
readonly sessionID: string
readonly boundary:
@@ -3635,7 +3609,6 @@ export type SessionImportInput = {
readonly info: {
readonly id: string
readonly parentID?: string
readonly kind?: "companion"
readonly fork?: {
readonly sessionID: string
readonly boundary:
@@ -3977,10 +3950,6 @@ export type SessionForkInput = {
export type SessionForkOutput = { data: SessionInfo }["data"]
export type SessionCompanionInput = { readonly sessionID: { readonly sessionID: string }["sessionID"] }
export type SessionCompanionOutput = { data: SessionInfo }["data"]
export type SessionSwitchAgentInput = {
readonly sessionID: { readonly sessionID: string }["sessionID"]
readonly agent: { readonly agent: string }["agent"]
@@ -5514,17 +5483,6 @@ export type GenerateTextInput = {
export type GenerateTextOutput = GenerateTextResponse["data"]
export type VoiceTranscribeInput = {
readonly mediaType: { readonly mediaType: string }["mediaType"]
readonly payload: globalThis.Uint8Array
}
export type VoiceTranscribeOutput = VoiceTranscribeResponse["data"]
export type VoiceSpeechInput = { readonly text: { readonly text: string }["text"] }
export type VoiceSpeechOutput = VoiceSpeechEvent
export type ProviderListInput = {
readonly location?: { readonly location?: { readonly directory?: string | undefined } | undefined }["location"]
}
-1
View File
@@ -13,7 +13,6 @@ test("exposes every standard HTTP API group", () => {
"message",
"model",
"generate",
"voice",
"provider",
"integration",
"mcp",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"$schema": "https://json.schemastore.org/package.json",
"name": "@opencode/codemode",
"version": "2.0.16",
"version": "2.0.18",
"description": "Effect-native confined code execution over schema-described tools",
"type": "module",
"license": "MIT",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@opencode/console-app",
"version": "2.0.16",
"version": "2.0.18",
"type": "module",
"license": "MIT",
"scripts": {
+1 -1
View File
@@ -1,7 +1,7 @@
{
"$schema": "https://json.schemastore.org/package.json",
"name": "@opencode/console-core",
"version": "2.0.16",
"version": "2.0.18",
"private": true,
"type": "module",
"license": "MIT",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@opencode/console-function",
"version": "2.0.16",
"version": "2.0.18",
"$schema": "https://json.schemastore.org/package.json",
"private": true,
"type": "module",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@opencode/console-mail",
"version": "2.0.16",
"version": "2.0.18",
"dependencies": {
"@jsx-email/all": "2.2.3",
"@jsx-email/cli": "1.4.3",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@opencode/console-support",
"version": "2.0.16",
"version": "2.0.18",
"type": "module",
"license": "MIT",
"scripts": {

Some files were not shown because too many files have changed in this diff Show More