mirror of
https://github.com/anomalyco/opencode.git
synced 2026-07-24 19:56:14 +00:00
Compare commits
60
Commits
tooltip-delay
..
v2
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d1d97014b4 | ||
|
|
b31747124b | ||
|
|
49bec25ae5 | ||
|
|
d66d0cb904 | ||
|
|
3193f3aa95 | ||
|
|
ee5460a152 | ||
|
|
b09a066fb5 | ||
|
|
423fad730c | ||
|
|
5ae2d6d3f6 | ||
|
|
993f046dd9 | ||
|
|
9b640cf97d | ||
|
|
2200d100d0 | ||
|
|
68ef893818 | ||
|
|
7ab3dd04ad | ||
|
|
4f201f87a9 | ||
|
|
5aa276c117 | ||
|
|
4605308be2 | ||
|
|
c64d813347 | ||
|
|
6e4a972bb9 | ||
|
|
835149e42b | ||
|
|
0f3c30118c | ||
|
|
35d31d8ec1 | ||
|
|
0374d29232 | ||
|
|
d90da82be2 | ||
|
|
c06186a9d9 | ||
|
|
c5680a206e | ||
|
|
edaee143d9 | ||
|
|
4184149b90 | ||
|
|
00f063b381 | ||
|
|
ea010ab3a4 | ||
|
|
65c5c7e3f6 | ||
|
|
ee69a91f26 | ||
|
|
7456598cde | ||
|
|
c228fc4886 | ||
|
|
bb3f4cc3c7 | ||
|
|
8d9727be9f | ||
|
|
6401eeaea0 | ||
|
|
e7ecee5df2 | ||
|
+30 |
02f2725154 | ||
|
|
79c1544072 | ||
|
|
8cac010bac | ||
|
|
193f6be99c | ||
|
|
2a9f8e3a2c | ||
|
|
2c814120c7 | ||
|
|
18fccac6ff | ||
|
|
360e7b412d | ||
|
|
ad596fb42b | ||
|
|
74e92f73e0 | ||
|
|
8f3465c951 | ||
|
|
466b75b19d | ||
|
|
833dd2ed7f | ||
|
|
bbe985b4d0 | ||
|
|
5b1321a8ca | ||
|
|
52c98a4eeb | ||
|
|
b6f85c2250 | ||
|
|
6e8aefcfa0 | ||
|
|
f1f0f47ee2 | ||
|
|
203b9f59b7 | ||
|
|
48bcbd09ef | ||
|
|
d86f732df3 |
@@ -0,0 +1,5 @@
|
||||
---
|
||||
"@opencode-ai/ai": patch
|
||||
---
|
||||
|
||||
Report OpenAI prompt cache write tokens in normalized usage.
|
||||
@@ -0,0 +1,8 @@
|
||||
---
|
||||
"@opencode-ai/plugin": minor
|
||||
"@opencode-ai/sdk": minor
|
||||
"@opencode-ai/client": minor
|
||||
"@opencode-ai/protocol": minor
|
||||
---
|
||||
|
||||
Replace the V2 tool result model with one canonical representation per fact. Tools lose `structured`, projection callbacks, the `Structured` generic, and the exported `Tool.settle` interpreter; tool responses carry schema-validated `output`, model-visible `content`, and optional compact JSON `metadata`. Code Mode receives the validated encoded output. Durable tool success stores non-empty model content plus optional metadata; failure stores one error plus the final bounded partial snapshot. Progress carries metadata only, while `execute.after` hooks receive the canonical terminal outcome and managed `outputPaths`. A one-time migration rewrites existing projected tool rows and moves provider-hosted result payloads into provider-owned result state.
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
"@opencode-ai/ai": patch
|
||||
---
|
||||
|
||||
Improve Anthropic and Bedrock prompt reuse with layered cache breakpoints that roll through long tool loops.
|
||||
@@ -0,0 +1,68 @@
|
||||
---
|
||||
name: ideal-pseudocode
|
||||
description: Function-by-function refactoring loop driven by ideal pseudocode. Use when the user says "ideal pseudocode", asks to make a function read like its pseudocode, or wants a dense module cleaned up one function at a time.
|
||||
---
|
||||
|
||||
# Ideal Pseudocode
|
||||
|
||||
Clean up one function at a time by writing the pseudocode it _should_ read as, naming every delta between that and the real code, and closing only the gaps the user approves.
|
||||
|
||||
## Loop
|
||||
|
||||
One function per round. Never touch code before the user picks a direction.
|
||||
|
||||
1. **Pick the target** with the user — usually the next function up or down the call chain from the last round.
|
||||
2. **Read the current code** fresh from disk. It may have unsaved or parallel edits; ask before overwriting anything unexpected.
|
||||
3. **Distill.** Write the function's ideal pseudocode in a `ts`-fenced code block — TypeScript-flavored for syntax highlighting, but pseudocode: comments over mechanics, one line per idea, every arm of a loop visible as an arm. For a dense or unfamiliar function, first show the _current_ structure as pseudocode, then the ideal.
|
||||
4. **Name the deltas.** A numbered list; each delta is one concrete gap: control flow smuggled through combinators, a flag-typed parameter, colliding names, side-effect plumbing drowning a loop body, twin functions differing only by a filter. For each, give an honest cost/benefit and a decisive recommendation — including "leave it" when the density has reasons (state the reasons).
|
||||
5. **Verify before proposing.** Any API the proposal builds on (a combinator, schedule, matcher) gets checked against the installed version's source, not memory.
|
||||
6. **Get buy-in, then apply.** Minimal edits. Quarantine irreducible ugliness inside a small helper whose name is the pseudocode line it implements. Collapse shallow sibling functions into one deep function only when the distinguishing argument is a domain noun, not a behavior switch.
|
||||
7. **Validate.** Typecheck and the focused test suites from the package directory; rerun timing-sensitive tests repeatedly. Flag any semantic change the restructure introduced, even when tests pass.
|
||||
8. **Offer the next target** and stop.
|
||||
|
||||
## Style
|
||||
|
||||
- Keep each round tight: pseudocode block, deltas, recommendation, one question.
|
||||
- State mutation belongs in one visible place (usually the loop tail), never spread across error channels or callback closures.
|
||||
- Prefer early return and `if` chains for control flow; matchers are for producing values.
|
||||
- Compare names against the question each caller is actually asking; rename toward the domain noun.
|
||||
|
||||
## Example round
|
||||
|
||||
Every round should read like this (abridged from a real one, on an Effect step-runner):
|
||||
|
||||
> `runStep` it is. The ideal pseudocode:
|
||||
>
|
||||
> ```ts
|
||||
> // One logical step: one settled model call, however many attempts it takes.
|
||||
> // An attempt can end without settling in two ways:
|
||||
> // transient provider failure -> retry the same call, same assistant message
|
||||
> // compaction rewrote history -> rebuild the request and call again
|
||||
> function runStep(session, promotable, step) {
|
||||
> while (true) {
|
||||
> const result = callModel(session, promotable, step)
|
||||
>
|
||||
> if (result.completed) return result
|
||||
>
|
||||
> if (result.retryable) {
|
||||
> wait(retrySchedule) // exhausted -> publish Step.Failed, fail drain
|
||||
> promotable = none // never re-promote on a second attempt
|
||||
> continue
|
||||
> }
|
||||
>
|
||||
> // compaction restarted the step: fresh request from rewritten history
|
||||
> promotable = none
|
||||
> step = result.step
|
||||
> }
|
||||
> }
|
||||
> ```
|
||||
>
|
||||
> Comparing against the real thing, three deltas:
|
||||
>
|
||||
> 1. **The retry arm doesn't live in the loop.** In the pseudocode, retry is just another loop arm. In the real code it's smuggled through the error channel: `Effect.tapError` mutating loop variables via `Effect.sync` closures, then `Effect.retryOrElse` re-running a suspended call. Four mutable variables mutated from three different places — the state flow is invisible unless you already know the trick.
|
||||
> 2. **`recoverOverflow: typeof compaction.compact | undefined`** — a function-typed parameter used as a one-shot armed/disarmed flag. The callee already closes over `compaction`; a boolean says what it is.
|
||||
> 3. **`assistantMessageID`** exists solely to resume the same durable message across retries — fine, but only legible once the retry arm is visible in the loop.
|
||||
>
|
||||
> The unifying move for (1): a schedule driver, so a retryable failure becomes a caught value handled in the loop body exactly like the compaction arm. One honest trade: it's more manual than the blessed combinator — but the manual version is the loop the combinator was hiding, and the loop already exists for restarts. Two restart mechanisms, one control structure.
|
||||
>
|
||||
> Want me to apply it — unified loop, simplified schedule input, boolean `recoverOverflow`?
|
||||
@@ -61,6 +61,7 @@
|
||||
"@dnd-kit/helpers": "0.5.0",
|
||||
"@dnd-kit/solid": "0.5.0",
|
||||
"@kobalte/core": "catalog:",
|
||||
"@opencode-ai/client": "file:vendor/opencode-ai-client-1.17.13.tgz",
|
||||
"@opencode-ai/core": "workspace:*",
|
||||
"@opencode-ai/schema": "workspace:*",
|
||||
"@opencode-ai/sdk": "workspace:*",
|
||||
@@ -124,7 +125,7 @@
|
||||
"opencode2": "./bin/opencode2.cjs",
|
||||
},
|
||||
"dependencies": {
|
||||
"@agentclientprotocol/sdk": "0.21.0",
|
||||
"@agentclientprotocol/sdk": "1.2.1",
|
||||
"@effect/platform-node": "catalog:",
|
||||
"@opencode-ai/client": "workspace:*",
|
||||
"@opencode-ai/plugin": "workspace:*",
|
||||
@@ -350,7 +351,7 @@
|
||||
"@ai-sdk/google": "3.0.73",
|
||||
"@ai-sdk/google-vertex": "4.0.128",
|
||||
"@ai-sdk/groq": "3.0.31",
|
||||
"@ai-sdk/mistral": "3.0.27",
|
||||
"@ai-sdk/mistral": "3.0.51",
|
||||
"@ai-sdk/openai": "3.0.84",
|
||||
"@ai-sdk/openai-compatible": "2.0.41",
|
||||
"@ai-sdk/perplexity": "3.0.26",
|
||||
@@ -362,7 +363,7 @@
|
||||
"@aws-sdk/credential-providers": "3.1057.0",
|
||||
"@effect/platform-node": "catalog:",
|
||||
"@effect/sql-sqlite-bun": "catalog:",
|
||||
"@ff-labs/fff-bun": "0.9.4",
|
||||
"@ff-labs/fff-bun": "0.10.1",
|
||||
"@lydell/node-pty": "catalog:",
|
||||
"@modelcontextprotocol/sdk": "1.29.0",
|
||||
"@opencode-ai/ai": "workspace:*",
|
||||
@@ -470,7 +471,9 @@
|
||||
"packages/docs": {
|
||||
"name": "@opencode-ai/docs",
|
||||
"devDependencies": {
|
||||
"effect": "catalog:",
|
||||
"mint": "4.2.666",
|
||||
"prettier": "3.6.2",
|
||||
},
|
||||
},
|
||||
"packages/effect-drizzle-sqlite": {
|
||||
@@ -706,6 +709,7 @@
|
||||
"version": "1.18.4",
|
||||
"dependencies": {
|
||||
"@kobalte/core": "catalog:",
|
||||
"@opencode-ai/client": "file:../app/vendor/opencode-ai-client-1.17.13.tgz",
|
||||
"@opencode-ai/core": "workspace:*",
|
||||
"@opencode-ai/sdk": "workspace:*",
|
||||
"@opencode-ai/ui": "workspace:*",
|
||||
@@ -1079,6 +1083,7 @@
|
||||
"@ai-sdk/xai@3.0.102": "patches/@ai-sdk%2Fxai@3.0.102.patch",
|
||||
"@modelcontextprotocol/sdk@1.29.0": "patches/@modelcontextprotocol%2Fsdk@1.29.0.patch",
|
||||
"@standard-community/standard-openapi@0.2.9": "patches/@standard-community%2Fstandard-openapi@0.2.9.patch",
|
||||
"@ai-sdk/mistral@3.0.51": "patches/@ai-sdk%2Fmistral@3.0.51.patch",
|
||||
"@npmcli/agent@4.0.2": "patches/@npmcli%2Fagent@4.0.2.patch",
|
||||
"@silvia-odwyer/photon-node@0.3.4": "patches/@silvia-odwyer%2Fphoton-node@0.3.4.patch",
|
||||
"solid-js@1.9.10": "patches/solid-js@1.9.10.patch",
|
||||
@@ -1173,7 +1178,7 @@
|
||||
|
||||
"@adobe/css-tools": ["@adobe/css-tools@4.5.0", "", {}, "sha512-6OzddxPio9UiWTCemp4N8cYLV2ZN1ncRnV1cVGtve7dhPOtRkleRyx32GQCYSwDYgaHU3USMm84tNsvKzRCa1Q=="],
|
||||
|
||||
"@agentclientprotocol/sdk": ["@agentclientprotocol/sdk@0.21.0", "", { "peerDependencies": { "zod": "^3.25.0 || ^4.0.0" } }, "sha512-ONj+Q8qOdNQp5XbH5jnMwzT9IKZJsSN0p0lkceS4GtUtNOPVLpNzSS8gqQdGMKfBvA0ESbkL8BTaSN1Rc9miEw=="],
|
||||
"@agentclientprotocol/sdk": ["@agentclientprotocol/sdk@1.2.1", "", { "peerDependencies": { "zod": "^3.25.0 || ^4.0.0" } }, "sha512-jwYUdOQR7tc+Zfch53VL4JJyUNK/46q03uUTYb+PjECsmnNl94XFXOfYLJ8RBpMNidXd1rpOAVgb0vqD98xImA=="],
|
||||
|
||||
"@ai-sdk/alibaba": ["@ai-sdk/alibaba@1.0.17", "", { "dependencies": { "@ai-sdk/openai-compatible": "2.0.41", "@ai-sdk/provider": "3.0.8", "@ai-sdk/provider-utils": "4.0.23" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-ZbE+U5bWz2JBc5DERLowx5+TKbjGBE93LqKZAWvuEn7HOSQMraxFMZuc0ST335QZJAyfBOzh7m1mPQ+y7EaaoA=="],
|
||||
|
||||
@@ -1205,7 +1210,7 @@
|
||||
|
||||
"@ai-sdk/groq": ["@ai-sdk/groq@3.0.31", "", { "dependencies": { "@ai-sdk/provider": "3.0.8", "@ai-sdk/provider-utils": "4.0.21" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-XbbugpnFmXGu2TlXiq8KUJskP6/VVbuFcnFIGDzDIB/Chg6XHsNnqrTF80Zxkh0Pd3+NvbM+2Uqrtsndk6bDAg=="],
|
||||
|
||||
"@ai-sdk/mistral": ["@ai-sdk/mistral@3.0.27", "", { "dependencies": { "@ai-sdk/provider": "3.0.8", "@ai-sdk/provider-utils": "4.0.21" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-ZXe7nZQgliDdjz5ufH5RKpHWxbN72AzmzzKGbF/z+0K9GN5tUCnftrQRvTRFHA5jAzTapcm2BEevmGLVbMkW+A=="],
|
||||
"@ai-sdk/mistral": ["@ai-sdk/mistral@3.0.51", "", { "dependencies": { "@ai-sdk/provider": "3.0.14", "@ai-sdk/provider-utils": "4.0.40" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-83eXY6p0lUFhSuMvNDmTKDuMciK5XDAWDlNh5c0L80tKjmtCFRItA1MZHp4IKe1r7eK8Rb5nN7qtxqMLUFRIRw=="],
|
||||
|
||||
"@ai-sdk/openai": ["@ai-sdk/openai@3.0.48", "", { "dependencies": { "@ai-sdk/provider": "3.0.8", "@ai-sdk/provider-utils": "4.0.21" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-ALmj/53EXpcRqMbGpPJPP4UOSWw0q4VGpnDo7YctvsynjkrKDmoneDG/1a7VQnSPYHnJp6tTRMf5ZdxZ5whulg=="],
|
||||
|
||||
@@ -1641,23 +1646,25 @@
|
||||
|
||||
"@fastify/busboy": ["@fastify/busboy@2.1.1", "", {}, "sha512-vBZP4NlzfOlerQTnba4aqZoMhE/a9HY7HRqoOPaETQcSQuWEIyZMHGfVu6w9wGtGK5fED5qRs2DteVCjOH60sA=="],
|
||||
|
||||
"@ff-labs/fff-bin-darwin-arm64": ["@ff-labs/fff-bin-darwin-arm64@0.9.4", "", { "os": "darwin", "cpu": "arm64" }, "sha512-xyivu2xB++O5xXDx5Qm50JsU2aXt8YgXlGVhH/HE7UMYDrE6L6f1RYdYs8Y0bn0D3D0+bFBrN5ELPszK9E4Wbw=="],
|
||||
"@ff-labs/fff-bin-android-arm64": ["@ff-labs/fff-bin-android-arm64@0.10.1", "", { "os": "android", "cpu": "arm64" }, "sha512-6Bsaa6yKEd2HV1M2WtqSYhoZucKYffIUms6GYoPN48QHP8hZgO8GXJ85/JDxKlkCJsN2hw1ROiwQK0+xUHYhFQ=="],
|
||||
|
||||
"@ff-labs/fff-bin-darwin-x64": ["@ff-labs/fff-bin-darwin-x64@0.9.4", "", { "os": "darwin", "cpu": "x64" }, "sha512-xLooAhCnTDCipPSMMZz7kGF3lhRHx6aP5fb6DJ0Ipyw/w/UWJb+xITJFszUl/QnIBoJ/qjDc93/FZMo1dk6gVA=="],
|
||||
"@ff-labs/fff-bin-darwin-arm64": ["@ff-labs/fff-bin-darwin-arm64@0.10.1", "", { "os": "darwin", "cpu": "arm64" }, "sha512-7yUP+56sG3UTrLg7eepOD16yM2dgiD7g6Ase2XWQB7oXwLV7mBMylPKIli2pP3kHzfw9K+BS3WAwHKHJ2QhxYw=="],
|
||||
|
||||
"@ff-labs/fff-bin-linux-arm64-gnu": ["@ff-labs/fff-bin-linux-arm64-gnu@0.9.4", "", { "os": "linux", "cpu": "arm64" }, "sha512-m5+8vA+1veaUUWonwva1WsU6m1HRm8CpYUzr06KDB65mewlmPbqz7+Fh7hjEfiD8C4mHVHe6RysULvAH1yhsdw=="],
|
||||
"@ff-labs/fff-bin-darwin-x64": ["@ff-labs/fff-bin-darwin-x64@0.10.1", "", { "os": "darwin", "cpu": "x64" }, "sha512-9zb+P1xtyqu/jVklm5RKF2zm9doRRsBNbkF/a8S4aSqSJoNlQR8ZF7C129fzOLfffVAjjgcO2l8oJgxOzHYiwQ=="],
|
||||
|
||||
"@ff-labs/fff-bin-linux-arm64-musl": ["@ff-labs/fff-bin-linux-arm64-musl@0.9.4", "", { "os": "linux", "cpu": "arm64" }, "sha512-EMeWm7CSTVkizy4ZEzUkLDP024tVcbCUthduuIhekFQRDsiaAze0YboIylWb9HBHJCZlCCoZrWAl4nnJbsX7AA=="],
|
||||
"@ff-labs/fff-bin-linux-arm64-gnu": ["@ff-labs/fff-bin-linux-arm64-gnu@0.10.1", "", { "os": "linux", "cpu": "arm64" }, "sha512-KTwr9CUfCJv0vtG2xG+nMxDae/2NJY2/oVmtgSvTnH9baI6JprqsFGphbukx7mdW/QMPwBiYtoO8ZGoC7i92jA=="],
|
||||
|
||||
"@ff-labs/fff-bin-linux-x64-gnu": ["@ff-labs/fff-bin-linux-x64-gnu@0.9.4", "", { "os": "linux", "cpu": "x64" }, "sha512-pglE0uLkhnlE6bStXqfgUjYTSj+2sVwXaPfoA0QksidAsQor6NRt8004mygzC9DPubgHq5B9QezPfEwigKaP9Q=="],
|
||||
"@ff-labs/fff-bin-linux-arm64-musl": ["@ff-labs/fff-bin-linux-arm64-musl@0.10.1", "", { "os": "linux", "cpu": "arm64" }, "sha512-oznmSpV+zjiAPiwqbA4y+SAsMaYvDhGllHiT9L4ebdQnSOfcQzlUwvo45OhBPwHBt47tIois4bkF/MITlDk54A=="],
|
||||
|
||||
"@ff-labs/fff-bin-linux-x64-musl": ["@ff-labs/fff-bin-linux-x64-musl@0.9.4", "", { "os": "linux", "cpu": "x64" }, "sha512-VNKxgl8qs3aTfXViX7lqRK1aLu311h8dtBFqG4Scv+9Oi7WprybUp5L7IZ8sxKERaDAaiJMXHodXa1c90QdK8w=="],
|
||||
"@ff-labs/fff-bin-linux-x64-gnu": ["@ff-labs/fff-bin-linux-x64-gnu@0.10.1", "", { "os": "linux", "cpu": "x64" }, "sha512-KDpl8lwSEOauP/6FSJIvnARzE+ILm2rVIRQAio9dc5nn56EvlsryBWry0c5V0Tbw1PV0lNrZ+bzcIiPuTacvzw=="],
|
||||
|
||||
"@ff-labs/fff-bin-win32-arm64": ["@ff-labs/fff-bin-win32-arm64@0.9.4", "", { "os": "win32", "cpu": "arm64" }, "sha512-uFEt0aNL54vQxq1ivjxRuo+thnhS4wLqa4INl4VXnXJUmwB42XXxD+gsj7vzhBLLx4cFf0aWgy/+TVDR8yjZtQ=="],
|
||||
"@ff-labs/fff-bin-linux-x64-musl": ["@ff-labs/fff-bin-linux-x64-musl@0.10.1", "", { "os": "linux", "cpu": "x64" }, "sha512-mZCpojVtGNDr/wdCUEAHdfrl6qORvKHv3Pw35TaOdshnG4wocoA5bBKTGj/zmAU11o0GMK4V+wUhdNrgqoE10w=="],
|
||||
|
||||
"@ff-labs/fff-bin-win32-x64": ["@ff-labs/fff-bin-win32-x64@0.9.4", "", { "os": "win32", "cpu": "x64" }, "sha512-Yd2Eyxj+slWv+0QDW9/xBpu9FXq+hwD0rXQD5184/88d+xwWCLKhEP2w8I6OO9XCg+kLT79UJb+k0WwXUtBtMw=="],
|
||||
"@ff-labs/fff-bin-win32-arm64": ["@ff-labs/fff-bin-win32-arm64@0.10.1", "", { "os": "win32", "cpu": "arm64" }, "sha512-SbT76ETXC5AgV9J8sVDozKG8wzrrxsOn8lbBprlOtx+O5V5EYmS1W7BlsatTHy6ddQ3uU5oOYO04PWZBuP6wXg=="],
|
||||
|
||||
"@ff-labs/fff-bun": ["@ff-labs/fff-bun@0.9.4", "", { "optionalDependencies": { "@ff-labs/fff-bin-darwin-arm64": "0.9.4", "@ff-labs/fff-bin-darwin-x64": "0.9.4", "@ff-labs/fff-bin-linux-arm64-gnu": "0.9.4", "@ff-labs/fff-bin-linux-arm64-musl": "0.9.4", "@ff-labs/fff-bin-linux-x64-gnu": "0.9.4", "@ff-labs/fff-bin-linux-x64-musl": "0.9.4", "@ff-labs/fff-bin-win32-arm64": "0.9.4", "@ff-labs/fff-bin-win32-x64": "0.9.4" }, "os": [ "linux", "win32", "darwin", ], "cpu": [ "x64", "arm64", ] }, "sha512-7HUraaK/g5dStAnuKAuzsXVOQvqqX0ylo5G+DxYwsCjCDc42bjoEAAHqz/3Sn3raUNw97KMoz87XR9QyrLEfVw=="],
|
||||
"@ff-labs/fff-bin-win32-x64": ["@ff-labs/fff-bin-win32-x64@0.10.1", "", { "os": "win32", "cpu": "x64" }, "sha512-dcpHUCBEZoXKQCdKa3bADfgcNyeyfM8tatXrILqIFYi9GL8E4GnzKx1rGkgP5ukVtuj0WuoJyyqSvGjpJPIT8w=="],
|
||||
|
||||
"@ff-labs/fff-bun": ["@ff-labs/fff-bun@0.10.1", "", { "optionalDependencies": { "@ff-labs/fff-bin-android-arm64": "0.10.1", "@ff-labs/fff-bin-darwin-arm64": "0.10.1", "@ff-labs/fff-bin-darwin-x64": "0.10.1", "@ff-labs/fff-bin-linux-arm64-gnu": "0.10.1", "@ff-labs/fff-bin-linux-arm64-musl": "0.10.1", "@ff-labs/fff-bin-linux-x64-gnu": "0.10.1", "@ff-labs/fff-bin-linux-x64-musl": "0.10.1", "@ff-labs/fff-bin-win32-arm64": "0.10.1", "@ff-labs/fff-bin-win32-x64": "0.10.1" }, "os": [ "!aix", "!sunos", "!freebsd", "!openbsd", ], "cpu": [ "x64", "arm64", ] }, "sha512-9oUCxypGbf2q3vNfKZ31wdzt5KqjhA9S6TwQaFol/j1lkSHVGvtIa3RvdGHPs1UUuvR/MO7b6pHj054BR9bXPQ=="],
|
||||
|
||||
"@floating-ui/core": ["@floating-ui/core@1.8.0", "", { "dependencies": { "@floating-ui/utils": "^0.2.12" } }, "sha512-0CIZ5itps/8x7BG8dEIhs53BvCUH2PCoogtakwRTut+Arm58sJooJ0AuZhLw2HJYIR5cMLNPBSS728sPho2khQ=="],
|
||||
|
||||
@@ -6149,7 +6156,9 @@
|
||||
|
||||
"@ai-sdk/groq/@ai-sdk/provider-utils": ["@ai-sdk/provider-utils@4.0.21", "", { "dependencies": { "@ai-sdk/provider": "3.0.8", "@standard-schema/spec": "^1.1.0", "eventsource-parser": "^3.0.6" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-MtFUYI1/8mgDvRmaBDjbLJPFFrMG777AvSgyIFQtZHIMzm88R/12vYBBpnk7pfiWLFE1DSZzY4WDYzGbKAcmiw=="],
|
||||
|
||||
"@ai-sdk/mistral/@ai-sdk/provider-utils": ["@ai-sdk/provider-utils@4.0.21", "", { "dependencies": { "@ai-sdk/provider": "3.0.8", "@standard-schema/spec": "^1.1.0", "eventsource-parser": "^3.0.6" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-MtFUYI1/8mgDvRmaBDjbLJPFFrMG777AvSgyIFQtZHIMzm88R/12vYBBpnk7pfiWLFE1DSZzY4WDYzGbKAcmiw=="],
|
||||
"@ai-sdk/mistral/@ai-sdk/provider": ["@ai-sdk/provider@3.0.14", "", { "dependencies": { "json-schema": "^0.4.0" } }, "sha512-5X1k57JBJ4H7H1QjX7CnJYAB1I19r/trVZTMcSms7/kLNZ8RaU4Nt2agcwZzv82Hfx6Q7/TOLU7agAKeFfc8cA=="],
|
||||
|
||||
"@ai-sdk/mistral/@ai-sdk/provider-utils": ["@ai-sdk/provider-utils@4.0.40", "", { "dependencies": { "@ai-sdk/provider": "3.0.14", "@standard-schema/spec": "^1.1.0", "eventsource-parser": "^3.0.8" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-OL5IrpUm9Y8Dwy+w/vvFwPotS6m52O9W0op2oXgXdCROMJIBalBI0oro6OIBYkPxvm5Xg02GSkoQN25RlR0bnw=="],
|
||||
|
||||
"@ai-sdk/openai/@ai-sdk/provider-utils": ["@ai-sdk/provider-utils@4.0.21", "", { "dependencies": { "@ai-sdk/provider": "3.0.8", "@standard-schema/spec": "^1.1.0", "eventsource-parser": "^3.0.6" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-MtFUYI1/8mgDvRmaBDjbLJPFFrMG777AvSgyIFQtZHIMzm88R/12vYBBpnk7pfiWLFE1DSZzY4WDYzGbKAcmiw=="],
|
||||
|
||||
@@ -6579,6 +6588,8 @@
|
||||
|
||||
"@openauthjs/openauth/jose": ["jose@5.9.6", "", {}, "sha512-AMlnetc9+CV9asI19zHmrgS/WYsWUwCn2R7RzlbJWD7F9eWYUTGyBmU9o6PxngtLGOiDGPRu+Uc4fhKzbpteZQ=="],
|
||||
|
||||
"@opencode-ai/app/@opencode-ai/client": ["@opencode-ai/client@vendor/opencode-ai-client-1.17.13.tgz", {}, "sha512-1cYJikTCrXNhnS2qQ3P3rtdbGqhvJKksswG0amJNqgaeUfz3xDlKEDx+YoIosT7Cqk/AtO18jtip0lYL3TVdHQ=="],
|
||||
|
||||
"@opencode-ai/cli/semver": ["semver@7.7.4", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-vFKC2IEtQnVhpT78h1Yp8wzwrf8CM+MzKMHGJZfBtzhZNycRFnXsHk6E5TxIkkMsgNS7mdX3AGB7x2QM2di4lA=="],
|
||||
|
||||
"@opencode-ai/console-app/@smithy/eventstream-codec": ["@smithy/eventstream-codec@4.2.7", "", { "dependencies": { "@aws-crypto/crc32": "5.2.0", "@smithy/types": "^4.11.0", "@smithy/util-hex-encoding": "^4.2.0", "tslib": "^2.6.2" } }, "sha512-DrpkEoM3j9cBBWhufqBwnbbn+3nf1N9FP6xuVJ+e220jbactKuQgaZwjwP5CP1t+O94brm2JgVMD2atMGX3xIQ=="],
|
||||
@@ -6597,6 +6608,8 @@
|
||||
|
||||
"@opencode-ai/desktop/typescript": ["typescript@5.6.3", "", { "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" } }, "sha512-hjcS1mhfuyi4WW8IWtjP7brDrG2cuDZukyrYrSauoXGNgx0S7zceP07adYkJycEr56BOUTNPzbInooiN3fn1qw=="],
|
||||
|
||||
"@opencode-ai/session-ui/@opencode-ai/client": ["@opencode-ai/client@../app/vendor/opencode-ai-client-1.17.13.tgz", {}, "sha512-1cYJikTCrXNhnS2qQ3P3rtdbGqhvJKksswG0amJNqgaeUfz3xDlKEDx+YoIosT7Cqk/AtO18jtip0lYL3TVdHQ=="],
|
||||
|
||||
"@opencode-ai/session-ui/@solid-primitives/resize-observer": ["@solid-primitives/resize-observer@2.1.3", "", { "dependencies": { "@solid-primitives/event-listener": "^2.4.3", "@solid-primitives/rootless": "^1.5.2", "@solid-primitives/static-store": "^0.1.2", "@solid-primitives/utils": "^6.3.2" }, "peerDependencies": { "solid-js": "^1.6.12" } }, "sha512-zBLje5E06TgOg93S7rGPldmhDnouNGhvfZVKOp+oG2XU8snA+GoCSSCz1M+jpNAg5Ek2EakU5UVQqL152WmdXQ=="],
|
||||
|
||||
"@opencode-ai/storybook/@types/react": ["@types/react@18.0.25", "", { "dependencies": { "@types/prop-types": "*", "@types/scheduler": "*", "csstype": "^3.0.2" } }, "sha512-xD6c0KDT4m7n9uD4ZHi02lzskaiqcBxf4zi+tXZY98a04wvc0hi/TcCPC2FOESZi51Nd7tlUeOJY8RofL799/g=="],
|
||||
@@ -6925,6 +6938,8 @@
|
||||
|
||||
"aggregate-error/indent-string": ["indent-string@5.0.0", "", {}, "sha512-m6FAo/spmsW2Ab2fU35JTYwtOKa2yAwXSwgjSv1TJzh4Mh7mC3lzAOVLBprb72XsTrgkEIsl7YrFNAiDiRhIGg=="],
|
||||
|
||||
"ai-gateway-provider/@ai-sdk/mistral": ["@ai-sdk/mistral@3.0.27", "", { "dependencies": { "@ai-sdk/provider": "3.0.8", "@ai-sdk/provider-utils": "4.0.21" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-ZXe7nZQgliDdjz5ufH5RKpHWxbN72AzmzzKGbF/z+0K9GN5tUCnftrQRvTRFHA5jAzTapcm2BEevmGLVbMkW+A=="],
|
||||
|
||||
"ai-gateway-provider/@ai-sdk/openai": ["@ai-sdk/openai@3.0.84", "", { "dependencies": { "@ai-sdk/provider": "3.0.14", "@ai-sdk/provider-utils": "4.0.38" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-cmgbeJL0bbY0yTJH4/AdmP5E7MjWRL9G8UdhIi0JlV/So03o82ORJofW8OzwCZPTORVQblFbpZXYGDcUd9NdUQ=="],
|
||||
|
||||
"ansi-align/string-width": ["string-width@4.2.3", "", { "dependencies": { "emoji-regex": "^8.0.0", "is-fullwidth-code-point": "^3.0.0", "strip-ansi": "^6.0.1" } }, "sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g=="],
|
||||
@@ -8045,6 +8060,8 @@
|
||||
|
||||
"@vitest/expect/@vitest/utils/@vitest/pretty-format": ["@vitest/pretty-format@3.2.4", "", { "dependencies": { "tinyrainbow": "^2.0.0" } }, "sha512-IVNZik8IVRJRTr9fxlitMKeJeXFFFN0JaB9PHPGQ8NKQbGpfjlTx9zO4RefN8gp7eqjNy8nyK3NZmBzOPeIxtA=="],
|
||||
|
||||
"ai-gateway-provider/@ai-sdk/mistral/@ai-sdk/provider-utils": ["@ai-sdk/provider-utils@4.0.21", "", { "dependencies": { "@ai-sdk/provider": "3.0.8", "@standard-schema/spec": "^1.1.0", "eventsource-parser": "^3.0.6" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-MtFUYI1/8mgDvRmaBDjbLJPFFrMG777AvSgyIFQtZHIMzm88R/12vYBBpnk7pfiWLFE1DSZzY4WDYzGbKAcmiw=="],
|
||||
|
||||
"ai-gateway-provider/@ai-sdk/openai/@ai-sdk/provider": ["@ai-sdk/provider@3.0.14", "", { "dependencies": { "json-schema": "^0.4.0" } }, "sha512-5X1k57JBJ4H7H1QjX7CnJYAB1I19r/trVZTMcSms7/kLNZ8RaU4Nt2agcwZzv82Hfx6Q7/TOLU7agAKeFfc8cA=="],
|
||||
|
||||
"ai-gateway-provider/@ai-sdk/openai/@ai-sdk/provider-utils": ["@ai-sdk/provider-utils@4.0.38", "", { "dependencies": { "@ai-sdk/provider": "3.0.14", "@standard-schema/spec": "^1.1.0", "eventsource-parser": "^3.0.8" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-/HHGmtKllqjg1OLc023v9w9kK3laW7Z6TzfZukYQWCsGBbzB9p60zTvvpXFVcs44NZBVXL3viOa1HRKUbeee8g=="],
|
||||
|
||||
+4
-4
@@ -1,8 +1,8 @@
|
||||
{
|
||||
"nodeModules": {
|
||||
"x86_64-linux": "sha256-qt11SKmOjq0KU542QFbs+u7YyJicn4drCcwCdg325yk=",
|
||||
"aarch64-linux": "sha256-z68doReXTrWS7HeiAjc0btIjAsvzeZZ7hXAlHr0c77Q=",
|
||||
"aarch64-darwin": "sha256-PILYH1Pi8XBvSkuZ+1sNnUTao5kba+m5Z8iJKx6YXPo=",
|
||||
"x86_64-darwin": "sha256-KpcJzP4m0SUavu/WaSffgzOxrHq8ljdy0GOzs9p16lo="
|
||||
"x86_64-linux": "sha256-0kcwV34P2C3yKg2eG9W2nW+OedrSBb+1TdpuUeYtauY=",
|
||||
"aarch64-linux": "sha256-yHVygApQchAB34wrtFR4GU0CkmZOlLsl3wsp15u0xzs=",
|
||||
"aarch64-darwin": "sha256-DyalcwyK2Wn5R6249keFcNVECbgtjYNjscOFqTi88FI=",
|
||||
"x86_64-darwin": "sha256-BkGw0GWN9W9q+/g4FYR0MqxUuFP80BPoERO+ypz/arQ="
|
||||
}
|
||||
}
|
||||
|
||||
+1
-1
@@ -155,12 +155,12 @@
|
||||
"@types/node": "catalog:"
|
||||
},
|
||||
"patchedDependencies": {
|
||||
"@ff-labs/fff-bun@0.9.3": "patches/@ff-labs%2Ffff-bun@0.9.3.patch",
|
||||
"@npmcli/agent@4.0.2": "patches/@npmcli%2Fagent@4.0.2.patch",
|
||||
"@silvia-odwyer/photon-node@0.3.4": "patches/@silvia-odwyer%2Fphoton-node@0.3.4.patch",
|
||||
"@standard-community/standard-openapi@0.2.9": "patches/@standard-community%2Fstandard-openapi@0.2.9.patch",
|
||||
"solid-js@1.9.10": "patches/solid-js@1.9.10.patch",
|
||||
"@ai-sdk/xai@3.0.102": "patches/@ai-sdk%2Fxai@3.0.102.patch",
|
||||
"@ai-sdk/mistral@3.0.51": "patches/@ai-sdk%2Fmistral@3.0.51.patch",
|
||||
"gcp-metadata@8.1.2": "patches/gcp-metadata@8.1.2.patch",
|
||||
"pacote@21.5.0": "patches/pacote@21.5.0.patch",
|
||||
"@ai-sdk/google@3.0.73": "patches/@ai-sdk%2Fgoogle@3.0.73.patch",
|
||||
|
||||
@@ -54,7 +54,7 @@ Filter or narrow `LLMEvent` streams with `LLMEvent.is.*` (camelCase guards, e.g.
|
||||
|
||||
A route is the registered, runnable composition of four orthogonal pieces:
|
||||
|
||||
- **`Protocol`** (`src/route/protocol.ts`) — semantic API contract. Owns request body construction (`body.from`), the body schema (`body.schema`), the streaming-event schema (`stream.event`), and the event-to-`LLMEvent` state machine (`stream.step`). `Route.make(...)` validates and JSON-encodes the body from `body.schema` and decodes frames with `stream.event`. Examples: `OpenAIChat.protocol`, `OpenAIResponses.protocol`, `AnthropicMessages.protocol`, `Gemini.protocol`, `BedrockConverse.protocol`.
|
||||
- **`Protocol`** (`src/route/protocol.ts`) — semantic API contract. Owns request body construction (`body.from`), the body schema (`body.schema`), the streaming-event schema (`stream.event`), and the event-to-`LLMEvent` state machine (`stream.step`). `Route.make(...)` validates and JSON-encodes the body from `body.schema` and decodes frames with `stream.event`. Examples: `OpenAIChat.protocol`, `OpenResponses.protocol`, `OpenAIResponses.protocol`, `AnthropicMessages.protocol`, `Gemini.protocol`, `BedrockConverse.protocol`.
|
||||
- **`Endpoint`** (`src/route/endpoint.ts`) — URL construction. The host, path, and route query live on the endpoint. `Endpoint.path("/chat/completions", { baseURL })` is the common case; pass a function for paths that embed the model id or a body field (e.g. `Endpoint.path(({ body }) => `/model/${body.modelId}/converse-stream`)`).
|
||||
- **`Auth`** (`src/route/auth.ts`) — per-request transport authentication. Provider facades configure credentials onto the route before model selection, usually via `Auth.bearer(apiKey)` or `Auth.header(name, apiKey)`. Routes that need per-request signing (Bedrock SigV4, future Vertex IAM, Azure AAD) implement `Auth` as a function that signs the body and merges signed headers into the result.
|
||||
- **`Framing`** (`src/route/framing.ts`) — bytes → frames. SSE (`Framing.sse`) is shared; Bedrock keeps its AWS event-stream framing as a typed `Framing<object>` value alongside its protocol.
|
||||
@@ -158,13 +158,14 @@ packages/ai/src/
|
||||
protocols/
|
||||
shared.ts ProviderShared toolkit used inside protocol impls
|
||||
openai-chat.ts protocol + route (compose OpenAIChat.protocol)
|
||||
openai-responses.ts
|
||||
open-responses.ts provider-neutral Responses protocol baseline
|
||||
openai-responses.ts OpenAI tools/events/transports composed over OpenResponses
|
||||
anthropic-messages.ts
|
||||
gemini.ts
|
||||
bedrock-converse.ts
|
||||
bedrock-event-stream.ts framing for AWS event-stream binary frames
|
||||
openai-compatible-chat.ts route that reuses OpenAIChat.protocol, no canonical URL
|
||||
openai-compatible-responses.ts route that reuses OpenAIResponses.protocol, no canonical URL
|
||||
openai-compatible-responses.ts deployment adapter that reuses OpenResponses.protocol, no canonical URL
|
||||
utils/ per-protocol helpers (auth, cache, media, tool-stream, ...)
|
||||
providers/
|
||||
openai-compatible.ts generic Chat helper + family model helpers
|
||||
@@ -175,7 +176,7 @@ packages/ai/src/
|
||||
tool-runtime.ts narrow one-call typed tool dispatcher
|
||||
```
|
||||
|
||||
The dependency arrow points down: `providers/*.ts` files import protocol routes and auth-option utilities; protocol modules import `endpoint`, `auth`, `framing`, and transport pieces. Protocols do not import provider facades. Lower-level modules know nothing about provider catalog metadata.
|
||||
The dependency arrow points down: `providers/*.ts` files import protocol routes and auth-option utilities; protocol modules import `endpoint`, `auth`, `framing`, and transport pieces. Protocols do not import provider facades. Lower-level modules know nothing about provider catalog metadata. `OpenAIResponses` composes the provider-neutral `OpenResponses` protocol; the baseline never imports the OpenAI extension.
|
||||
|
||||
### Shared protocol helpers
|
||||
|
||||
@@ -240,7 +241,7 @@ const get_weather = tool({
|
||||
|
||||
const tools = { get_weather, get_time, ... }
|
||||
const events = yield* LLM.stream(
|
||||
LLM.updateRequest(request, { tools: Tool.toDefinitions(tools) }),
|
||||
LLMRequest.update(request, { tools: Tool.toDefinitions(tools) }),
|
||||
).pipe(Stream.runCollect)
|
||||
|
||||
const call = Array.from(events).find(LLMEvent.is.toolCall)
|
||||
|
||||
@@ -315,7 +315,8 @@ const longer = {
|
||||
}
|
||||
```
|
||||
|
||||
There is no `LLM.updateRequest(...)` helper and no request Schema class.
|
||||
There is no `LLM.updateRequest(...)` helper. The current Schema-backed implementation
|
||||
uses `LLMRequest.update(...)` when canonical request data must be derived.
|
||||
|
||||
### Conversation history
|
||||
|
||||
@@ -436,7 +437,7 @@ const call = Array.from(events).find(LLMEvent.is.toolCall)
|
||||
|
||||
if (call && !call.providerExecuted) {
|
||||
const dispatched = yield * ToolRuntime.dispatch(tools, call)
|
||||
const followUp = LLM.updateRequest(request, {
|
||||
const followUp = LLMRequest.update(request, {
|
||||
messages: [...request.messages, Message.assistant([call]), Message.tool({ ...call, result: dispatched.result })],
|
||||
})
|
||||
// Caller must invoke the provider again and repeat the loop.
|
||||
|
||||
@@ -207,7 +207,9 @@ Prompt caching is **on by default**. Every `LLMRequest` resolves to `cache: "aut
|
||||
|
||||
### Auto placement
|
||||
|
||||
`"auto"` places three breakpoints — last tool definition, last system part, latest user message. The last-user-message boundary is the load-bearing detail: in a tool-use loop, a single user turn expands into many assistant/tool round-trips, all sharing that prefix. Caching at that boundary lets every intra-turn API call hit.
|
||||
`"auto"` places up to four breakpoints — the last tool definition, the first system part, the last system part when distinct, and the final message boundary. These expose successively larger reusable prefixes for tools, the base agent, project instructions, and the active conversation. The rolling final-message boundary is the load-bearing detail in tool loops: it advances on every request so the previous cache entry stays within Anthropic's 20-block lookback.
|
||||
|
||||
Tools precede every system and conversation block in the provider prefix, so tool definitions must remain byte-stable and deterministically ordered for downstream breakpoints to remain reusable.
|
||||
|
||||
The math justifies the default: Anthropic's 5-minute cache write is 1.25× base, read is 0.1×, so a single reuse within 5 minutes already wins. One-shot completions below the per-model minimum-cacheable-token threshold silently no-op on the wire, so the worst case is harmless.
|
||||
|
||||
@@ -235,7 +237,7 @@ cache: {
|
||||
|
||||
### Manual hints
|
||||
|
||||
Inline `CacheHint` on any text / system / tool / tool-result part overrides automatic placement. The auto policy preserves manual hints; it only fills gaps.
|
||||
Inline `CacheHint` on any text / system / tool / tool-result part overrides automatic placement. The auto policy preserves manual hints, counts them against Anthropic and Bedrock's four-breakpoint limit, and only fills the remaining slots.
|
||||
|
||||
```ts
|
||||
LLM.request({
|
||||
@@ -251,8 +253,8 @@ LLM.request({
|
||||
|
||||
| Protocol | `cache: "auto"` |
|
||||
| ----------------------- | ------------------------------------------------------------------------- |
|
||||
| Anthropic Messages | emits up to 3 `cache_control` markers (4-breakpoint cap enforced) |
|
||||
| Bedrock Converse | emits up to 3 `cachePoint` blocks (4-breakpoint cap enforced) |
|
||||
| Anthropic Messages | emits up to 4 `cache_control` markers (4-breakpoint cap enforced) |
|
||||
| Bedrock Converse | emits up to 4 `cachePoint` blocks (4-breakpoint cap enforced) |
|
||||
| OpenAI Chat / Responses | no-op (implicit caching above 1024 tokens) |
|
||||
| Gemini | no-op (implicit caching on 2.5+; explicit `CachedContent` is out-of-band) |
|
||||
|
||||
@@ -300,7 +302,7 @@ OpenAI Chat and OpenAI Responses are separate semantic entrypoints:
|
||||
- `@opencode-ai/ai/providers/google-vertex/responses`
|
||||
- `@opencode-ai/ai/providers/google-vertex/messages`
|
||||
|
||||
Responses HTTP versus WebSocket is a scoped `transport` setting on the OpenAI Responses entrypoint, not another entrypoint. Azure follows the same Chat/Responses split at `providers/azure/chat` and `providers/azure/responses`. Generic OpenAI-compatible Chat remains at `providers/openai-compatible`; compatible Responses is separate at `providers/openai-compatible/responses`. Generic Anthropic Messages-compatible providers use `providers/anthropic-compatible`, which the named Anthropic provider composes. Google Gemini and Amazon Bedrock expose their single native API through their existing provider paths.
|
||||
Responses HTTP versus WebSocket is a scoped `transport` setting on the OpenAI Responses entrypoint, not another entrypoint. Azure follows the same Chat/Responses split at `providers/azure/chat` and `providers/azure/responses`. Generic OpenAI-compatible Chat remains at `providers/openai-compatible`; the Responses adapter at `providers/openai-compatible/responses` uses the provider-neutral Open Responses protocol. OpenAI Responses extends that baseline with OpenAI tools, event variants, metadata, defaults, and transports. Generic Anthropic Messages-compatible providers use `providers/anthropic-compatible`, which the named Anthropic provider composes. Google Gemini and Amazon Bedrock expose their single native API through their existing provider paths.
|
||||
|
||||
Vertex Gemini, Vertex Chat, Vertex Responses, and Vertex Messages are separate API entrypoints. All accept `project`, `location`, and an optional `accessToken`; when no explicit token or auth override is supplied they lazily use Google Application Default Credentials. Vertex Gemini instead selects express mode when `apiKey` or `GOOGLE_VERTEX_API_KEY` is present. Vertex Chat targets MaaS models through the OpenAI-compatible Chat Completions endpoint, while Vertex Responses targets Grok models and defaults `store` to `false` as required by Vertex. `providers/google-vertex` remains the default alias for `providers/google-vertex/gemini`.
|
||||
|
||||
|
||||
+24
-24
@@ -1,6 +1,6 @@
|
||||
# LLM Provider Parity Status
|
||||
|
||||
Last reviewed: 2026-07-17
|
||||
Last reviewed: 2026-07-24
|
||||
|
||||
This file tracks the gap between the native `@opencode-ai/ai` package and the AI SDK provider packages that opencode still depends on for many catalog/runtime paths.
|
||||
|
||||
@@ -13,26 +13,26 @@ This file tracks the gap between the native `@opencode-ai/ai` package and the AI
|
||||
|
||||
## Current Implementation Snapshot
|
||||
|
||||
| Native slice | Source | Current state | Main gaps |
|
||||
| ---------------------------------- | ---------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| OpenAI Chat | `src/protocols/openai-chat.ts`, `src/providers/openai.ts` | Usable. Streams text, reasoning deltas, tool calls, usage, images, and common generation controls. | No typed structured-output / `response_format` path. Limited typed OpenAI option surface compared with SDK escape hatches. |
|
||||
| OpenAI Responses HTTP | `src/protocols/openai-responses.ts`, `src/providers/openai.ts` | Usable. Supports hosted-tool event surfacing, reasoning replay metadata, GPT-5 defaults, and cache usage. | No explicit `previous_response_id` path. Typed options cover only a subset of Responses fields. Structured output is still mostly synthetic-tool based. |
|
||||
| OpenAI Responses WebSocket | `src/protocols/openai-responses.ts`, `src/route/transport/websocket.ts` | Present as `OpenAI.responsesWebSocket(...)`. | Runner/catalog support explicitly must not downgrade WebSocket routes; broader runtime selection is not complete. |
|
||||
| OpenAI-compatible Chat | `src/protocols/openai-compatible-chat.ts`, `src/providers/openai-compatible.ts` | Usable for generic Chat and several profiles: Baseten, Cerebras, DeepInfra, DeepSeek, Fireworks, Groq, TogetherAI. | Family quirks are mostly endpoint defaults, not full typed behavior. |
|
||||
| OpenAI-compatible Responses | `src/protocols/openai-compatible-responses.ts`, `src/providers/openai-compatible-responses.ts` | Usable for deployments that implement the OpenAI Responses wire protocol. | No named family profiles or recorded deployment coverage yet. |
|
||||
| Anthropic-compatible Messages | `src/protocols/anthropic-messages.ts`, `src/providers/anthropic-compatible.ts` | Usable for deployments that implement the Anthropic Messages wire protocol. Named Anthropic composes this base; MiniMax M3 has recorded text and tool-loop coverage. | No named compatible family profiles yet. |
|
||||
| Anthropic Messages | `src/protocols/anthropic-messages.ts`, `src/providers/anthropic.ts` | Usable. Supports tools, thinking, cache control, images, server-hosted tool events, and usage. | Provider option surface is small. Beta/header handling, metadata, and newer Messages fields need a typed parity pass. |
|
||||
| Gemini Developer API | `src/protocols/gemini.ts`, `src/providers/google.ts` | Usable for Google API key flow. Supports text, images, tools, thinking signatures, and cache usage. | This is not Vertex. Typed provider options are narrow; many Gemini request fields currently require raw `http.body` overlays. |
|
||||
| Vertex Gemini | `src/protocols/gemini.ts`, `src/providers/google-vertex.ts` | Usable through API-key express mode, explicit OAuth tokens, or ADC with project/location endpoint derivation, including tuned `endpoints/...` deployments. | Core runner/catalog mapping and recorded provider coverage are missing. |
|
||||
| Vertex Chat | `src/protocols/openai-chat.ts`, `src/providers/google-vertex-chat.ts` | Usable for MaaS models through OpenAI-compatible Chat Completions with explicit OAuth tokens or ADC and project/location endpoint derivation. | Core runner/catalog mapping and recorded provider coverage are missing; MaaS family-specific request parity needs review. |
|
||||
| Vertex Responses | `src/protocols/openai-responses.ts`, `src/providers/google-vertex-responses.ts` | Usable for Grok models through OpenAI-compatible Responses with explicit OAuth tokens or ADC, project/location endpoint derivation, and storage disabled by default. | Core runner/catalog mapping and recorded provider coverage are missing; stateful continuation is not supported by Vertex. |
|
||||
| Vertex Messages | `src/protocols/anthropic-messages.ts`, `src/providers/google-vertex-messages.ts` | Usable through explicit OAuth tokens or ADC, including global, regional, and `eu`/`us` multi-region endpoints. | Core runner/catalog mapping and recorded provider coverage are missing; Vertex-specific hosted-tool parity needs review. |
|
||||
| Bedrock Converse | `src/protocols/bedrock-converse.ts`, `src/providers/amazon-bedrock.ts` | Partial but real. Supports AWS event-stream framing, SigV4 with supplied credentials, bearer auth, tools, reasoning signatures, media, cache points, and recorded tests. | Native facade does not mirror the AI SDK plugin's default AWS credential chain/profile behavior. Runner/catalog mapping is missing. Guardrails, inference profiles, region-specific model ID fixes, and model-specific request fields need a parity pass. |
|
||||
| Azure OpenAI | `src/providers/azure.ts` using OpenAI Chat/Responses protocols | Partial. Supports resource/base URL setup, API key auth, API version query, Chat, and Responses selectors. | Core runner does not map `@ai-sdk/azure` to this native facade. AAD/token auth and Azure-specific endpoint variants need review. |
|
||||
| Cloudflare AI Gateway / Workers AI | `src/providers/cloudflare.ts` | Present via OpenAI-compatible Chat routes. | Useful but not part of the critical AI SDK replacement set yet. Needs per-product recorded coverage before relying on it broadly. |
|
||||
| OpenRouter | `src/providers/openrouter.ts` | Present with OpenRouter-specific usage/reasoning/prompt-cache options over Chat. | Responses-style OpenRouter support is absent. |
|
||||
| xAI | `src/providers/xai.ts` | Present with Responses and Chat selectors. | Needs package-parity review against the AI SDK xAI provider. |
|
||||
| GitHub Copilot | `src/providers/github-copilot.ts` | Present as explicit-base-URL OpenAI Chat/Responses facade. | Runtime/catalog integration remains specialized and should stay separate from public OpenAI-compatible defaults. |
|
||||
| Native slice | Source | Current state | Main gaps |
|
||||
| ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| OpenAI Chat | `src/protocols/openai-chat.ts`, `src/providers/openai.ts` | Usable. Streams text, reasoning deltas, tool calls, usage, images, and common generation controls. | No typed structured-output / `response_format` path. Limited typed OpenAI option surface compared with SDK escape hatches. |
|
||||
| OpenAI Responses HTTP | `src/protocols/open-responses.ts`, `src/protocols/openai-responses.ts`, `src/providers/openai.ts` | Usable. Extends the Open Responses baseline with hosted-tool event surfacing, reasoning replay metadata, GPT-5 defaults, and cache usage. | No explicit `previous_response_id` path. Typed options cover only a subset of Responses fields. Structured output is still mostly synthetic-tool based. |
|
||||
| OpenAI Responses WebSocket | `src/protocols/openai-responses.ts`, `src/route/transport/websocket.ts` | Present as `OpenAI.responsesWebSocket(...)`. | Runner/catalog support explicitly must not downgrade WebSocket routes; broader runtime selection is not complete. |
|
||||
| OpenAI-compatible Chat | `src/protocols/openai-compatible-chat.ts`, `src/providers/openai-compatible.ts` | Usable for generic Chat and several profiles: Baseten, Cerebras, DeepInfra, DeepSeek, Fireworks, Groq, TogetherAI. | Family quirks are mostly endpoint defaults, not full typed behavior. |
|
||||
| Open Responses-compatible | `src/protocols/open-responses.ts`, `src/protocols/openai-compatible-responses.ts`, `src/providers/openai-compatible-responses.ts` | Usable for deployments that implement the provider-neutral Open Responses protocol. The deployment adapter does not inherit OpenAI tools, events, metadata, or defaults. | No named family profiles or recorded deployment coverage yet. |
|
||||
| Anthropic-compatible Messages | `src/protocols/anthropic-messages.ts`, `src/providers/anthropic-compatible.ts` | Usable for deployments that implement the Anthropic Messages wire protocol. Named Anthropic composes this base; MiniMax M3 has recorded text and tool-loop coverage. | No named compatible family profiles yet. |
|
||||
| Anthropic Messages | `src/protocols/anthropic-messages.ts`, `src/providers/anthropic.ts` | Usable. Supports tools, thinking, cache control, images, server-hosted tool events, and usage. | Provider option surface is small. Beta/header handling, metadata, and newer Messages fields need a typed parity pass. |
|
||||
| Gemini Developer API | `src/protocols/gemini.ts`, `src/providers/google.ts` | Usable for Google API key flow. Supports text, images, tools, thinking signatures, and cache usage. | This is not Vertex. Typed provider options are narrow; many Gemini request fields currently require raw `http.body` overlays. |
|
||||
| Vertex Gemini | `src/protocols/gemini.ts`, `src/providers/google-vertex.ts` | Usable through API-key express mode, explicit OAuth tokens, or ADC with project/location endpoint derivation, including tuned `endpoints/...` deployments. | Core runner/catalog mapping and recorded provider coverage are missing. |
|
||||
| Vertex Chat | `src/protocols/openai-chat.ts`, `src/providers/google-vertex-chat.ts` | Usable for MaaS models through OpenAI-compatible Chat Completions with explicit OAuth tokens or ADC and project/location endpoint derivation. | Core runner/catalog mapping and recorded provider coverage are missing; MaaS family-specific request parity needs review. |
|
||||
| Vertex Responses | `src/protocols/open-responses.ts`, `src/providers/google-vertex-responses.ts` | Usable for Grok models through Open Responses with explicit OAuth tokens or ADC, project/location endpoint derivation, and an explicit `store: false` Vertex default. | Core runner/catalog mapping and recorded provider coverage are missing; stateful continuation is not supported by Vertex. |
|
||||
| Vertex Messages | `src/protocols/anthropic-messages.ts`, `src/providers/google-vertex-messages.ts` | Usable through explicit OAuth tokens or ADC, including global, regional, and `eu`/`us` multi-region endpoints. | Core runner/catalog mapping and recorded provider coverage are missing; Vertex-specific hosted-tool parity needs review. |
|
||||
| Bedrock Converse | `src/protocols/bedrock-converse.ts`, `src/providers/amazon-bedrock.ts` | Partial but real. Supports AWS event-stream framing, SigV4 with supplied credentials, bearer auth, tools, reasoning signatures, media, cache points, and recorded tests. | Native facade does not mirror the AI SDK plugin's default AWS credential chain/profile behavior. Runner/catalog mapping is missing. Guardrails, inference profiles, region-specific model ID fixes, and model-specific request fields need a parity pass. |
|
||||
| Azure OpenAI | `src/providers/azure.ts` using OpenAI Chat/Responses protocols | Partial. Supports resource/base URL setup, API key auth, API version query, Chat, and Responses selectors. | Core runner does not map `@ai-sdk/azure` to this native facade. AAD/token auth and Azure-specific endpoint variants need review. |
|
||||
| Cloudflare AI Gateway / Workers AI | `src/providers/cloudflare.ts` | Present via OpenAI-compatible Chat routes. | Useful but not part of the critical AI SDK replacement set yet. Needs per-product recorded coverage before relying on it broadly. |
|
||||
| OpenRouter | `src/providers/openrouter.ts` | Present with OpenRouter-specific usage/reasoning/prompt-cache options over Chat. | Responses-style OpenRouter support is absent. |
|
||||
| xAI | `src/providers/xai.ts` | Present with Responses and Chat selectors. | Needs package-parity review against the AI SDK xAI provider. |
|
||||
| GitHub Copilot | `src/providers/github-copilot.ts` | Present as explicit-base-URL OpenAI Chat/Responses facade. | Runtime/catalog integration remains specialized and should stay separate from public OpenAI-compatible defaults. |
|
||||
|
||||
## V2 Runner Status
|
||||
|
||||
@@ -65,7 +65,7 @@ Other `aisdk:` packages, including Google Vertex, Azure, and Bedrock, currently
|
||||
## Highest-Risk Gaps
|
||||
|
||||
1. Runner support is narrower than the LLM package. The package has native provider facades for Google, Azure, and Bedrock, but the V2 Session runner only maps OpenAI, Anthropic, and explicit OpenAI-compatible Chat from `aisdk` catalog metadata.
|
||||
2. OpenAI-compatible Responses is available as a separate package entrypoint, but the V2 runner still maps `@ai-sdk/openai-compatible` to Chat only. Catalog selection must become API-aware before Responses deployments can use it.
|
||||
2. The Open Responses adapter is available through a separate package entrypoint, but the V2 runner still maps `@ai-sdk/openai-compatible` to Chat only. Catalog selection must become API-aware before Responses deployments can use it.
|
||||
3. Bedrock native auth is not AI SDK parity. The AI SDK plugin uses the default AWS provider chain, profile, container credentials, and Bedrock bearer token env behavior. Native Bedrock currently expects explicit credentials or bearer auth on the facade.
|
||||
4. Vertex Gemini, Vertex Chat, Vertex Responses, and Vertex Messages now have native package entrypoints, but the core runner does not map catalog metadata to them yet and recorded provider coverage is still missing.
|
||||
5. Azure is only a provider facade, not a full runtime replacement. Native Azure exists, but the catalog runner does not select it, and token auth/resource variants need review.
|
||||
@@ -83,13 +83,13 @@ These are implementation/API slices, not separate npm packages.
|
||||
| OpenAI Chat | `@opencode-ai/ai/providers/openai/chat` | OpenAI `/chat/completions` semantics. |
|
||||
| OpenAI Responses | `@opencode-ai/ai/providers/openai/responses` | OpenAI `/responses` semantics with HTTP/WebSocket selected through settings. |
|
||||
| OpenAI-compatible Chat | `@opencode-ai/ai/providers/openai-compatible` | Generic OpenAI-compatible `/chat/completions`. |
|
||||
| OpenAI-compatible Responses | `@opencode-ai/ai/providers/openai-compatible/responses` | Generic OpenAI-compatible `/responses`. |
|
||||
| Open Responses-compatible | `@opencode-ai/ai/providers/openai-compatible/responses` | Generic provider-neutral `/responses`. |
|
||||
| Anthropic-compatible Messages | `@opencode-ai/ai/providers/anthropic-compatible` | Generic Anthropic-compatible `/messages`. |
|
||||
| Anthropic Messages | `@opencode-ai/ai/providers/anthropic` | Anthropic Messages API. |
|
||||
| Gemini Developer API | `@opencode-ai/ai/providers/google` | Google AI Studio Gemini API. |
|
||||
| Vertex Gemini | `@opencode-ai/ai/providers/google-vertex/gemini` | Vertex Gemini API; `providers/google-vertex` is the default alias. |
|
||||
| Vertex Chat | `@opencode-ai/ai/providers/google-vertex/chat` | Vertex OpenAI-compatible Chat Completions for MaaS models. |
|
||||
| Vertex Responses | `@opencode-ai/ai/providers/google-vertex/responses` | Vertex OpenAI-compatible Responses for Grok models. |
|
||||
| Vertex Responses | `@opencode-ai/ai/providers/google-vertex/responses` | Vertex Open Responses for Grok models. |
|
||||
| Vertex Messages | `@opencode-ai/ai/providers/google-vertex/messages` | Vertex-hosted Anthropic Messages API. |
|
||||
| Bedrock Converse | `@opencode-ai/ai/providers/amazon-bedrock` | AWS Bedrock Converse API. |
|
||||
| Bedrock Mantle | Missing | AWS Bedrock Mantle OpenAI-compatible APIs. |
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { Config, Effect, Formatter, Layer, Schema, Stream } from "effect"
|
||||
import { LLM, LLMClient, Message, ProviderID, Tool, ToolRuntime } from "@opencode-ai/ai"
|
||||
import { LLM, LLMClient, LLMRequest, Message, ProviderID, Tool, ToolRuntime } from "@opencode-ai/ai"
|
||||
import { Route, Auth, Endpoint, Framing, Protocol, RequestExecutor, WebSocketExecutor } from "@opencode-ai/ai/route"
|
||||
import { OpenAI } from "@opencode-ai/ai/providers"
|
||||
|
||||
@@ -78,7 +78,10 @@ const streamText = LLM.stream(request).pipe(
|
||||
Stream.tap((event) =>
|
||||
Effect.sync(() => {
|
||||
if (event.type === "text-delta") process.stdout.write(`\ntext: ${event.text}`)
|
||||
if (event.type === "finish") process.stdout.write(`\nfinish: ${event.reason}\n`)
|
||||
if (event.type === "finish")
|
||||
process.stdout.write(
|
||||
`\nfinish: ${event.reason.normalized}${event.reason.raw ? ` (${event.reason.raw})` : ""}\n`,
|
||||
)
|
||||
}),
|
||||
),
|
||||
Stream.runDrain,
|
||||
@@ -113,7 +116,7 @@ const streamWithTools = Effect.gen(function* () {
|
||||
|
||||
// A durable agent would persist these messages before starting another
|
||||
// raw model turn. This tutorial keeps the boundary visible instead.
|
||||
const followUp = LLM.updateRequest(request, {
|
||||
const followUp = LLMRequest.update(request, {
|
||||
messages: [
|
||||
...request.messages,
|
||||
Message.assistant([event]),
|
||||
@@ -194,7 +197,7 @@ const FakeProtocol = Protocol.make<FakeBody, string, string, void>({
|
||||
event: Schema.String,
|
||||
initial: () => undefined,
|
||||
step: (_, frame) => Effect.succeed([undefined, [{ type: "text-delta", id: "text-0", text: frame }]] as const),
|
||||
onHalt: () => [{ type: "finish", reason: "stop" }],
|
||||
onHalt: () => [{ type: "finish", reason: { normalized: "stop" } }],
|
||||
},
|
||||
})
|
||||
|
||||
|
||||
@@ -2,32 +2,31 @@
|
||||
// the policy designates. Runs once at compile time, before the per-protocol
|
||||
// body builder, so the existing inline-hint lowering path handles the rest.
|
||||
//
|
||||
// The default `"auto"` shape places one breakpoint at the last tool definition,
|
||||
// one at the last system part, and one at the latest user message. This
|
||||
// matches what production agent harnesses (LangChain's caching middleware,
|
||||
// kern-ai's 10x cost-reduction playbook) converge on for tool-use loops: the
|
||||
// latest user message stays put while a single turn explodes into many
|
||||
// assistant/tool round-trips, so caching at that boundary lets every
|
||||
// intra-turn API call hit the prefix.
|
||||
// The default `"auto"` shape places breakpoints at the last tool definition,
|
||||
// the first and last distinct system parts, and the conversation tail. This
|
||||
// exposes reusable tool, base-agent, project, and session prefixes while
|
||||
// advancing the tail after each tool result keeps the previous cache entry
|
||||
// within Anthropic's 20-block lookback during long agent turns.
|
||||
//
|
||||
// Manual `cache: CacheHint` placements on individual parts are preserved —
|
||||
// this function only fills gaps the caller left empty.
|
||||
// Manual `cache: CacheHint` placements on individual parts are preserved and
|
||||
// count against the four-breakpoint budget; auto only fills remaining slots.
|
||||
import { CacheHint, type CachePolicy, type CachePolicyObject } from "./schema/options"
|
||||
import { LLMRequest, Message, ToolDefinition, type ContentPart } from "./schema/messages"
|
||||
|
||||
const AUTO: CachePolicyObject = {
|
||||
tools: true,
|
||||
system: true,
|
||||
messages: "latest-user-message",
|
||||
messages: { tail: 1 },
|
||||
}
|
||||
|
||||
const NONE: CachePolicyObject = {}
|
||||
const BREAKPOINT_CAP = 4
|
||||
|
||||
// Resolution rules:
|
||||
// - undefined → "auto" — caching is on by default. The math favors it:
|
||||
// Anthropic 5m-cache write is 1.25x base, read is 0.1x,
|
||||
// so a single reuse within 5 minutes already wins.
|
||||
// - "auto" → tools + system + latest user msg.
|
||||
// - "auto" → tools + first/last system + final message boundary.
|
||||
// - "none" → no auto placement; manual `CacheHint`s still flow.
|
||||
// - object form → exactly what the caller asked for.
|
||||
const resolve = (policy: CachePolicy | undefined): CachePolicyObject => {
|
||||
@@ -44,18 +43,32 @@ const RESPECTS_INLINE_HINTS = new Set(["anthropic-messages", "bedrock-converse"]
|
||||
const makeHint = (ttlSeconds: number | undefined): CacheHint =>
|
||||
ttlSeconds !== undefined ? new CacheHint({ type: "ephemeral", ttlSeconds }) : new CacheHint({ type: "ephemeral" })
|
||||
|
||||
const markLastTool = (tools: ReadonlyArray<ToolDefinition>, hint: CacheHint): ReadonlyArray<ToolDefinition> => {
|
||||
interface Budget {
|
||||
remaining: number
|
||||
}
|
||||
|
||||
const markLastTool = (
|
||||
tools: ReadonlyArray<ToolDefinition>,
|
||||
hint: CacheHint,
|
||||
budget: Budget,
|
||||
): ReadonlyArray<ToolDefinition> => {
|
||||
if (tools.length === 0) return tools
|
||||
const last = tools.length - 1
|
||||
if (tools[last]!.cache) return tools
|
||||
if (tools[last]!.cache || budget.remaining === 0) return tools
|
||||
budget.remaining -= 1
|
||||
return tools.map((tool, i) => (i === last ? new ToolDefinition({ ...tool, cache: hint }) : tool))
|
||||
}
|
||||
|
||||
const markLastSystem = (system: LLMRequest["system"], hint: CacheHint): LLMRequest["system"] => {
|
||||
const markSystemBoundaries = (system: LLMRequest["system"], hint: CacheHint, budget: Budget): LLMRequest["system"] => {
|
||||
if (system.length === 0) return system
|
||||
const last = system.length - 1
|
||||
if (system[last]!.cache) return system
|
||||
return system.map((part, i) => (i === last ? { ...part, cache: hint } : part))
|
||||
let changed = false
|
||||
const next = system.map((part, index) => {
|
||||
if ((index !== 0 && index !== system.length - 1) || part.cache || budget.remaining === 0) return part
|
||||
budget.remaining -= 1
|
||||
changed = true
|
||||
return { ...part, cache: hint }
|
||||
})
|
||||
return changed ? next : system
|
||||
}
|
||||
|
||||
const lastIndexOfRole = (messages: ReadonlyArray<Message>, role: Message["role"]): number =>
|
||||
@@ -64,14 +77,20 @@ const lastIndexOfRole = (messages: ReadonlyArray<Message>, role: Message["role"]
|
||||
// Mark the last text part of `messages[index]`. If no text part exists, mark
|
||||
// the last content part regardless of type — that's the breakpoint position
|
||||
// in tool-result-only messages too.
|
||||
const markMessageAt = (messages: ReadonlyArray<Message>, index: number, hint: CacheHint): ReadonlyArray<Message> => {
|
||||
const markMessageAt = (
|
||||
messages: ReadonlyArray<Message>,
|
||||
index: number,
|
||||
hint: CacheHint,
|
||||
budget: Budget,
|
||||
): ReadonlyArray<Message> => {
|
||||
if (index < 0 || index >= messages.length) return messages
|
||||
const target = messages[index]!
|
||||
if (target.content.length === 0) return messages
|
||||
const lastTextIndex = target.content.findLastIndex((part) => part.type === "text")
|
||||
const markAt = lastTextIndex >= 0 ? lastTextIndex : target.content.length - 1
|
||||
const existing = target.content[markAt]!
|
||||
if ("cache" in existing && existing.cache) return messages
|
||||
if (("cache" in existing && existing.cache) || budget.remaining === 0) return messages
|
||||
budget.remaining -= 1
|
||||
const nextContent = target.content.map((part, i) => (i === markAt ? ({ ...part, cache: hint } as ContentPart) : part))
|
||||
const next = new Message({ ...target, content: nextContent })
|
||||
// Single pass over `messages`, substituting the one updated entry. Long
|
||||
@@ -86,25 +105,42 @@ const markMessages = (
|
||||
messages: ReadonlyArray<Message>,
|
||||
strategy: NonNullable<CachePolicyObject["messages"]>,
|
||||
hint: CacheHint,
|
||||
budget: Budget,
|
||||
): ReadonlyArray<Message> => {
|
||||
if (messages.length === 0) return messages
|
||||
if (strategy === "latest-user-message") return markMessageAt(messages, lastIndexOfRole(messages, "user"), hint)
|
||||
if (strategy === "latest-assistant") return markMessageAt(messages, lastIndexOfRole(messages, "assistant"), hint)
|
||||
if (strategy === "latest-user-message")
|
||||
return markMessageAt(messages, lastIndexOfRole(messages, "user"), hint, budget)
|
||||
if (strategy === "latest-assistant")
|
||||
return markMessageAt(messages, lastIndexOfRole(messages, "assistant"), hint, budget)
|
||||
const start = Math.max(0, messages.length - strategy.tail)
|
||||
let next = messages
|
||||
for (let i = start; i < messages.length; i++) next = markMessageAt(next, i, hint)
|
||||
for (let i = start; i < messages.length; i++) next = markMessageAt(next, i, hint, budget)
|
||||
return next
|
||||
}
|
||||
|
||||
const countHints = (request: LLMRequest) =>
|
||||
request.tools.reduce((count, tool) => count + (tool.cache === undefined ? 0 : 1), 0) +
|
||||
request.system.reduce((count, part) => count + (part.cache === undefined ? 0 : 1), 0) +
|
||||
request.messages.reduce(
|
||||
(count, message) =>
|
||||
count +
|
||||
message.content.reduce(
|
||||
(contentCount, part) => contentCount + ("cache" in part && part.cache !== undefined ? 1 : 0),
|
||||
0,
|
||||
),
|
||||
0,
|
||||
)
|
||||
|
||||
export const applyCachePolicy = (request: LLMRequest): LLMRequest => {
|
||||
if (!RESPECTS_INLINE_HINTS.has(request.model.route.id)) return request
|
||||
const policy = resolve(request.cache)
|
||||
if (!policy.tools && !policy.system && !policy.messages) return request
|
||||
|
||||
const hint = makeHint(policy.ttlSeconds)
|
||||
const tools = policy.tools ? markLastTool(request.tools, hint) : request.tools
|
||||
const system = policy.system ? markLastSystem(request.system, hint) : request.system
|
||||
const messages = policy.messages ? markMessages(request.messages, policy.messages, hint) : request.messages
|
||||
const budget = { remaining: Math.max(0, BREAKPOINT_CAP - countHints(request)) }
|
||||
const tools = policy.tools ? markLastTool(request.tools, hint, budget) : request.tools
|
||||
const system = policy.system ? markSystemBoundaries(request.system, hint, budget) : request.system
|
||||
const messages = policy.messages ? markMessages(request.messages, policy.messages, hint, budget) : request.messages
|
||||
|
||||
if (tools === request.tools && system === request.system && messages === request.messages) return request
|
||||
return LLMRequest.update(request, { tools, system, messages })
|
||||
|
||||
+3
-21
@@ -9,24 +9,13 @@ import {
|
||||
LLMRequest,
|
||||
LLMResponse,
|
||||
Message,
|
||||
type ModelInput as SchemaModelInput,
|
||||
SystemPart,
|
||||
ToolChoice,
|
||||
ToolDefinition,
|
||||
type ContentPart,
|
||||
ToolResultPart,
|
||||
} from "./schema"
|
||||
import { make as makeTool, toDefinitions, type ToolSchema } from "./tool"
|
||||
|
||||
export type ModelInput = SchemaModelInput
|
||||
|
||||
export type MessageInput = Message.Input
|
||||
|
||||
export type ToolChoiceInput = ToolChoice.Input
|
||||
export type ToolChoiceMode = ToolChoice.Mode
|
||||
|
||||
export type ToolResultInput = Parameters<typeof ToolResultPart.make>[0]
|
||||
|
||||
/** Input accepted by `LLM.request`, normalized into the canonical `LLMRequest` class. */
|
||||
export type RequestInput = Omit<
|
||||
ConstructorParameters<typeof LLMRequest>[0],
|
||||
@@ -34,9 +23,9 @@ export type RequestInput = Omit<
|
||||
> & {
|
||||
readonly system?: string | SystemPart | ReadonlyArray<SystemPart>
|
||||
readonly prompt?: string | ContentPart | ReadonlyArray<ContentPart>
|
||||
readonly messages?: ReadonlyArray<Message | MessageInput>
|
||||
readonly messages?: ReadonlyArray<Message | Message.Input>
|
||||
readonly tools?: ReadonlyArray<ToolDefinition.Input>
|
||||
readonly toolChoice?: ToolChoiceInput
|
||||
readonly toolChoice?: ToolChoice.Input
|
||||
readonly generation?: GenerationOptions.Input
|
||||
readonly providerOptions?: ConstructorParameters<typeof LLMRequest>[0]["providerOptions"]
|
||||
readonly http?: HttpOptions.Input
|
||||
@@ -46,10 +35,6 @@ export const generate = LLMClient.generate
|
||||
|
||||
export const stream = LLMClient.stream
|
||||
|
||||
export const requestInput = (input: LLMRequest): RequestInput => ({
|
||||
...LLMRequest.input(input),
|
||||
})
|
||||
|
||||
export const request = (input: RequestInput) => {
|
||||
const {
|
||||
system: requestSystem,
|
||||
@@ -74,14 +59,11 @@ export const request = (input: RequestInput) => {
|
||||
})
|
||||
}
|
||||
|
||||
export const updateRequest = (input: LLMRequest, patch: Partial<RequestInput>) =>
|
||||
request({ ...requestInput(input), ...patch })
|
||||
|
||||
const GENERATE_OBJECT_TOOL_NAME = "generate_object"
|
||||
|
||||
const GENERATE_OBJECT_TOOL_DESCRIPTION = "Return the structured result by calling this tool."
|
||||
|
||||
type GenerateObjectBase = Omit<RequestInput, "tools" | "toolChoice" | "responseFormat">
|
||||
type GenerateObjectBase = Omit<RequestInput, "tools" | "toolChoice">
|
||||
|
||||
export class GenerateObjectResponse<T> {
|
||||
constructor(
|
||||
|
||||
@@ -9,10 +9,12 @@ import {
|
||||
LLMEvent,
|
||||
Usage,
|
||||
type CacheHint,
|
||||
type FinishReasonDetails,
|
||||
type FinishReason,
|
||||
type JsonSchema,
|
||||
type LLMRequest,
|
||||
type MediaPart,
|
||||
type ProviderOptions,
|
||||
type ProviderMetadata,
|
||||
type ToolCallPart,
|
||||
type ToolDefinition,
|
||||
@@ -31,6 +33,29 @@ const MEDIA_MIMES = new Set<string>([...ProviderShared.IMAGE_MIMES, ...ProviderS
|
||||
export const DEFAULT_BASE_URL = "https://api.anthropic.com/v1"
|
||||
export const PATH = "/messages"
|
||||
|
||||
export type ThinkingInput =
|
||||
| {
|
||||
readonly type: "adaptive"
|
||||
readonly display?: "summarized" | "omitted"
|
||||
}
|
||||
| {
|
||||
readonly type: "disabled"
|
||||
}
|
||||
| ({ readonly type: "enabled" } & (
|
||||
| { readonly budgetTokens: number; readonly budget_tokens?: number }
|
||||
| { readonly budgetTokens?: number; readonly budget_tokens: number }
|
||||
))
|
||||
|
||||
export interface OptionsInput {
|
||||
readonly [key: string]: unknown
|
||||
readonly thinking?: ThinkingInput
|
||||
readonly effort?: string
|
||||
}
|
||||
|
||||
export type ProviderOptionsInput = ProviderOptions & {
|
||||
readonly anthropic?: OptionsInput
|
||||
}
|
||||
|
||||
// =============================================================================
|
||||
// Request Body Schema
|
||||
// =============================================================================
|
||||
@@ -75,6 +100,15 @@ const AnthropicThinkingBlock = Schema.Struct({
|
||||
cache_control: Schema.optional(AnthropicCacheControl),
|
||||
})
|
||||
|
||||
// Safety-filtered thinking arrives as an opaque encrypted `data` payload with
|
||||
// no visible text. It must round-trip verbatim so multi-turn thinking + tool
|
||||
// use conversations keep their reasoning continuity.
|
||||
const AnthropicRedactedThinkingBlock = Schema.Struct({
|
||||
type: Schema.tag("redacted_thinking"),
|
||||
data: Schema.String,
|
||||
cache_control: Schema.optional(AnthropicCacheControl),
|
||||
})
|
||||
|
||||
const AnthropicToolUseBlock = Schema.Struct({
|
||||
type: Schema.tag("tool_use"),
|
||||
id: Schema.String,
|
||||
@@ -136,6 +170,7 @@ type AnthropicUserBlock = Schema.Schema.Type<typeof AnthropicUserBlock>
|
||||
const AnthropicAssistantBlock = Schema.Union([
|
||||
AnthropicTextBlock,
|
||||
AnthropicThinkingBlock,
|
||||
AnthropicRedactedThinkingBlock,
|
||||
AnthropicToolUseBlock,
|
||||
AnthropicServerToolUseBlock,
|
||||
AnthropicServerToolResultBlock,
|
||||
@@ -159,7 +194,7 @@ const AnthropicTool = Schema.Struct({
|
||||
type AnthropicTool = Schema.Schema.Type<typeof AnthropicTool>
|
||||
|
||||
const AnthropicToolChoice = Schema.Union([
|
||||
Schema.Struct({ type: Schema.Literals(["auto", "any"]) }),
|
||||
Schema.Struct({ type: Schema.Literals(["auto", "any", "none"]) }),
|
||||
Schema.Struct({ type: Schema.tag("tool"), name: Schema.String }),
|
||||
])
|
||||
|
||||
@@ -214,6 +249,9 @@ const AnthropicStreamBlock = Schema.Struct({
|
||||
text: Schema.optional(Schema.String),
|
||||
thinking: Schema.optional(Schema.String),
|
||||
signature: Schema.optional(Schema.String),
|
||||
// redacted_thinking blocks arrive whole in content_block_start with the
|
||||
// encrypted payload in `data`; there is no streaming delta sequence.
|
||||
data: Schema.optional(Schema.String),
|
||||
input: Schema.optional(Schema.Unknown),
|
||||
// *_tool_result blocks arrive whole as content_block_start (no streaming
|
||||
// delta) with the structured payload in `content` and the originating
|
||||
@@ -251,7 +289,12 @@ type AnthropicEvent = Schema.Schema.Type<typeof AnthropicEvent>
|
||||
|
||||
interface ParserState {
|
||||
readonly tools: ToolStream.State<number>
|
||||
readonly reasoningSignatures: Readonly<Record<number, string>>
|
||||
readonly usage?: Usage
|
||||
readonly pendingFinish?: {
|
||||
readonly reason: FinishReasonDetails
|
||||
readonly providerMetadata?: ProviderMetadata
|
||||
}
|
||||
readonly lifecycle: Lifecycle.State
|
||||
}
|
||||
|
||||
@@ -287,6 +330,12 @@ const signatureFromMetadata = (metadata: ProviderMetadata | undefined): string |
|
||||
return typeof anthropic.signature === "string" ? anthropic.signature : undefined
|
||||
}
|
||||
|
||||
const redactedDataFromMetadata = (metadata: ProviderMetadata | undefined): string | undefined => {
|
||||
const anthropic = metadata?.anthropic
|
||||
if (!ProviderShared.isRecord(anthropic)) return undefined
|
||||
return typeof anthropic.redactedData === "string" ? anthropic.redactedData : undefined
|
||||
}
|
||||
|
||||
const lowerTool = (breakpoints: Cache.Breakpoints, tool: ToolDefinition, inputSchema: JsonSchema): AnthropicTool => ({
|
||||
name: tool.name,
|
||||
description: tool.description,
|
||||
@@ -297,7 +346,7 @@ const lowerTool = (breakpoints: Cache.Breakpoints, tool: ToolDefinition, inputSc
|
||||
const lowerToolChoice = (toolChoice: NonNullable<LLMRequest["toolChoice"]>) =>
|
||||
ProviderShared.matchToolChoice("Anthropic Messages", toolChoice, {
|
||||
auto: () => ({ type: "auto" as const }),
|
||||
none: () => undefined,
|
||||
none: () => ({ type: "none" as const }),
|
||||
required: () => ({ type: "any" as const }),
|
||||
tool: (name) => ({ type: "tool" as const, name }),
|
||||
})
|
||||
@@ -330,7 +379,10 @@ const lowerServerToolResult = Effect.fn("AnthropicMessages.lowerServerToolResult
|
||||
const wireType = serverToolResultType(part.name)
|
||||
if (!wireType)
|
||||
return yield* invalid(`Anthropic Messages does not know how to round-trip server tool result for ${part.name}`)
|
||||
return { type: wireType, tool_use_id: part.id, content: part.result.value } satisfies AnthropicServerToolResultBlock
|
||||
// Prefer the provider-owned replay payload; fall back to the result value for
|
||||
// histories constructed directly from provider events.
|
||||
const payload = part.providerMetadata?.anthropic?.["result"] ?? part.result.value
|
||||
return { type: wireType, tool_use_id: part.id, content: payload } satisfies AnthropicServerToolResultBlock
|
||||
})
|
||||
|
||||
const lowerMedia = Effect.fn("AnthropicMessages.lowerMedia")(function* (part: MediaPart) {
|
||||
@@ -469,11 +521,16 @@ const lowerMessages = Effect.fn("AnthropicMessages.lowerMessages")(function* (
|
||||
continue
|
||||
}
|
||||
if (part.type === "reasoning") {
|
||||
content.push({
|
||||
type: "thinking",
|
||||
thinking: part.text,
|
||||
signature: part.encrypted ?? signatureFromMetadata(part.providerMetadata),
|
||||
})
|
||||
// Mirrors Vercel's @ai-sdk/anthropic: a signature marks visible
|
||||
// thinking; only signature-less parts carrying redactedData
|
||||
// round-trip as opaque redacted_thinking blocks.
|
||||
const signature = part.encrypted ?? signatureFromMetadata(part.providerMetadata)
|
||||
const redactedData = redactedDataFromMetadata(part.providerMetadata)
|
||||
if (signature === undefined && redactedData !== undefined) {
|
||||
content.push({ type: "redacted_thinking", data: redactedData })
|
||||
continue
|
||||
}
|
||||
content.push({ type: "thinking", thinking: part.text, signature })
|
||||
continue
|
||||
}
|
||||
if (part.type === "tool-call") {
|
||||
@@ -510,39 +567,39 @@ const lowerMessages = Effect.fn("AnthropicMessages.lowerMessages")(function* (
|
||||
return messages
|
||||
})
|
||||
|
||||
const anthropicOptions = (request: LLMRequest) => request.providerOptions?.anthropic
|
||||
const resolveOptions = Effect.fn("AnthropicMessages.resolveOptions")(function* (request: LLMRequest) {
|
||||
const input = request.providerOptions?.anthropic
|
||||
return {
|
||||
thinking: yield* resolveThinking(input?.thinking),
|
||||
effort: typeof input?.effort === "string" ? input.effort : undefined,
|
||||
}
|
||||
})
|
||||
|
||||
const lowerThinking = Effect.fn("AnthropicMessages.lowerThinking")(function* (request: LLMRequest) {
|
||||
const thinking = anthropicOptions(request)?.thinking
|
||||
if (!ProviderShared.isRecord(thinking)) return undefined
|
||||
if (thinking.type === "adaptive") {
|
||||
const resolveThinking = Effect.fn("AnthropicMessages.resolveThinking")(function* (input: unknown) {
|
||||
if (!ProviderShared.isRecord(input)) return undefined
|
||||
if (input.type === "adaptive") {
|
||||
const display =
|
||||
thinking.display === "summarized"
|
||||
input.display === "summarized"
|
||||
? ("summarized" as const)
|
||||
: thinking.display === "omitted"
|
||||
: input.display === "omitted"
|
||||
? ("omitted" as const)
|
||||
: undefined
|
||||
return { type: "adaptive" as const, ...(display === undefined ? {} : { display }) }
|
||||
}
|
||||
if (thinking.type === "disabled") return { type: "disabled" as const }
|
||||
if (thinking.type !== "enabled") return undefined
|
||||
if (input.type === "disabled") return { type: "disabled" as const }
|
||||
if (input.type !== "enabled") return undefined
|
||||
const budget =
|
||||
typeof thinking.budgetTokens === "number"
|
||||
? thinking.budgetTokens
|
||||
: typeof thinking.budget_tokens === "number"
|
||||
? thinking.budget_tokens
|
||||
typeof input.budgetTokens === "number"
|
||||
? input.budgetTokens
|
||||
: typeof input.budget_tokens === "number"
|
||||
? input.budget_tokens
|
||||
: undefined
|
||||
if (budget === undefined) return yield* invalid("Anthropic thinking provider option requires budgetTokens")
|
||||
if (budget === undefined)
|
||||
return yield* ProviderShared.invalidRequest("Anthropic thinking provider option requires budgetTokens")
|
||||
return { type: "enabled" as const, budget_tokens: budget }
|
||||
})
|
||||
|
||||
const outputConfig = (request: LLMRequest) => {
|
||||
const effort = anthropicOptions(request)?.effort
|
||||
return typeof effort === "string" ? { effort } : undefined
|
||||
}
|
||||
|
||||
const fromRequest = Effect.fn("AnthropicMessages.fromRequest")(function* (request: LLMRequest) {
|
||||
const toolChoice = request.toolChoice ? yield* lowerToolChoice(request.toolChoice) : undefined
|
||||
const generation = request.generation
|
||||
const toolSchemaCompatibility = request.model.compatibility?.toolSchema
|
||||
const outputLimit = request.model.defaults?.limits?.output ?? request.model.route.defaults.limits?.output ?? 4096
|
||||
@@ -551,7 +608,7 @@ const fromRequest = Effect.fn("AnthropicMessages.fromRequest")(function* (reques
|
||||
// over-mark we keep their tool hints and shed the message-tail ones first.
|
||||
const breakpoints = Cache.newBreakpoints(ANTHROPIC_BREAKPOINT_CAP)
|
||||
const tools =
|
||||
request.tools.length === 0 || request.toolChoice?.type === "none"
|
||||
request.tools.length === 0
|
||||
? undefined
|
||||
: request.tools.map((tool) =>
|
||||
lowerTool(
|
||||
@@ -560,6 +617,8 @@ const fromRequest = Effect.fn("AnthropicMessages.fromRequest")(function* (reques
|
||||
ToolSchemaProjection.modelCompatibility(tool.inputSchema, toolSchemaCompatibility),
|
||||
),
|
||||
)
|
||||
// Anthropic rejects tool_choice when tools are absent; "none" is only meaningful with tools present.
|
||||
const toolChoice = tools === undefined || !request.toolChoice ? undefined : yield* lowerToolChoice(request.toolChoice)
|
||||
const system =
|
||||
request.system.length === 0
|
||||
? undefined
|
||||
@@ -574,6 +633,7 @@ const fromRequest = Effect.fn("AnthropicMessages.fromRequest")(function* (reques
|
||||
`Anthropic Messages: dropped ${breakpoints.dropped} cache breakpoint(s); the API allows at most ${ANTHROPIC_BREAKPOINT_CAP} per request.`,
|
||||
)
|
||||
}
|
||||
const options = yield* resolveOptions(request)
|
||||
return {
|
||||
model: request.model.id,
|
||||
system,
|
||||
@@ -586,8 +646,8 @@ const fromRequest = Effect.fn("AnthropicMessages.fromRequest")(function* (reques
|
||||
top_p: generation?.topP,
|
||||
top_k: generation?.topK,
|
||||
stop_sequences: generation?.stop,
|
||||
thinking: yield* lowerThinking(request),
|
||||
output_config: outputConfig(request),
|
||||
thinking: options.thinking,
|
||||
output_config: options.effort === undefined ? undefined : { effort: options.effort },
|
||||
}
|
||||
})
|
||||
|
||||
@@ -596,7 +656,7 @@ const fromRequest = Effect.fn("AnthropicMessages.fromRequest")(function* (reques
|
||||
// =============================================================================
|
||||
const mapFinishReason = (reason: string | null | undefined): FinishReason => {
|
||||
if (reason === "end_turn" || reason === "stop_sequence" || reason === "pause_turn") return "stop"
|
||||
if (reason === "max_tokens") return "length"
|
||||
if (reason === "max_tokens" || reason === "model_context_window_exceeded") return "length"
|
||||
if (reason === "tool_use") return "tool-calls"
|
||||
if (reason === "refusal") return "content-filter"
|
||||
return "unknown"
|
||||
@@ -680,7 +740,9 @@ const serverToolResultEvent = (block: NonNullable<AnthropicEvent["content_block"
|
||||
name: SERVER_TOOL_RESULT_NAMES[block.type],
|
||||
result: isError ? { type: "error", value: block.content } : { type: "json", value: block.content },
|
||||
providerExecuted: true,
|
||||
providerMetadata: anthropicMetadata({ blockType: block.type }),
|
||||
// The complete payload is irreducible provider replay state: subsequent
|
||||
// stateless requests must round-trip the typed result block verbatim.
|
||||
providerMetadata: anthropicMetadata({ blockType: block.type, result: block.content }),
|
||||
})
|
||||
}
|
||||
|
||||
@@ -707,6 +769,10 @@ const onContentBlockStart = (state: ParserState, event: AnthropicEvent): StepRes
|
||||
tools: ToolStream.start(state.tools, event.index, {
|
||||
id: block.id ?? String(event.index),
|
||||
name: block.name ?? "",
|
||||
input:
|
||||
block.input !== undefined && (!ProviderShared.isRecord(block.input) || Object.keys(block.input).length > 0)
|
||||
? ProviderShared.encodeJson(block.input)
|
||||
: undefined,
|
||||
providerExecuted: block.type === "server_tool_use",
|
||||
}),
|
||||
},
|
||||
@@ -721,20 +787,50 @@ const onContentBlockStart = (state: ParserState, event: AnthropicEvent): StepRes
|
||||
]
|
||||
}
|
||||
|
||||
if (block.type === "text" && block.text) {
|
||||
if (block.type === "text" && block.text !== undefined) {
|
||||
const events: LLMEvent[] = []
|
||||
const id = `text-${event.index ?? 0}`
|
||||
const lifecycle = Lifecycle.textStart(state.lifecycle, events, id)
|
||||
return [
|
||||
{ ...state, lifecycle: Lifecycle.textDelta(state.lifecycle, events, `text-${event.index ?? 0}`, block.text) },
|
||||
{ ...state, lifecycle: block.text ? Lifecycle.textDelta(lifecycle, events, id, block.text) : lifecycle },
|
||||
events,
|
||||
]
|
||||
}
|
||||
|
||||
if (block.type === "thinking" && block.thinking) {
|
||||
if (block.type === "thinking" && block.thinking !== undefined) {
|
||||
const events: LLMEvent[] = []
|
||||
const id = `reasoning-${event.index ?? 0}`
|
||||
const providerMetadata = block.signature === undefined ? undefined : anthropicMetadata({ signature: block.signature })
|
||||
const lifecycle = Lifecycle.reasoningStart(state.lifecycle, events, id, providerMetadata)
|
||||
return [
|
||||
{
|
||||
...state,
|
||||
lifecycle: block.thinking
|
||||
? Lifecycle.reasoningDelta(lifecycle, events, id, block.thinking, providerMetadata)
|
||||
: lifecycle,
|
||||
reasoningSignatures:
|
||||
event.index === undefined || block.signature === undefined
|
||||
? state.reasoningSignatures
|
||||
: { ...state.reasoningSignatures, [event.index]: block.signature },
|
||||
},
|
||||
events,
|
||||
]
|
||||
}
|
||||
|
||||
// Redacted thinking surfaces as an empty reasoning part carrying the opaque
|
||||
// payload as `redactedData` metadata (same model as Vercel's
|
||||
// @ai-sdk/anthropic). The existing content_block_stop closes the part.
|
||||
if (block.type === "redacted_thinking" && block.data !== undefined) {
|
||||
const events: LLMEvent[] = []
|
||||
return [
|
||||
{
|
||||
...state,
|
||||
lifecycle: Lifecycle.reasoningDelta(state.lifecycle, events, `reasoning-${event.index ?? 0}`, block.thinking),
|
||||
lifecycle: Lifecycle.reasoningStart(
|
||||
state.lifecycle,
|
||||
events,
|
||||
`reasoning-${event.index ?? 0}`,
|
||||
anthropicMetadata({ redactedData: block.data }),
|
||||
),
|
||||
},
|
||||
events,
|
||||
]
|
||||
@@ -772,18 +868,13 @@ const onContentBlockDelta = Effect.fn("AnthropicMessages.onContentBlockDelta")(f
|
||||
}
|
||||
|
||||
if (delta?.type === "signature_delta" && delta.signature) {
|
||||
const events: LLMEvent[] = []
|
||||
const index = event.index ?? 0
|
||||
return [
|
||||
{
|
||||
...state,
|
||||
lifecycle: Lifecycle.reasoningEnd(
|
||||
state.lifecycle,
|
||||
events,
|
||||
`reasoning-${event.index ?? 0}`,
|
||||
anthropicMetadata({ signature: delta.signature }),
|
||||
),
|
||||
reasoningSignatures: { ...state.reasoningSignatures, [index]: delta.signature },
|
||||
},
|
||||
events,
|
||||
NO_EVENTS,
|
||||
] satisfies StepResult
|
||||
}
|
||||
|
||||
@@ -814,28 +905,53 @@ const onContentBlockStop = Effect.fn("AnthropicMessages.onContentBlockStop")(fun
|
||||
const result = yield* ToolStream.finish(ADAPTER, state.tools, event.index)
|
||||
const events: LLMEvent[] = []
|
||||
const resultEvents = result.events ?? []
|
||||
const signature = state.reasoningSignatures[event.index]
|
||||
const lifecycle = resultEvents.length
|
||||
? Lifecycle.stepStart(state.lifecycle, events)
|
||||
: Lifecycle.reasoningEnd(
|
||||
Lifecycle.textEnd(state.lifecycle, events, `text-${event.index}`),
|
||||
events,
|
||||
`reasoning-${event.index}`,
|
||||
signature === undefined ? undefined : anthropicMetadata({ signature }),
|
||||
)
|
||||
events.push(...resultEvents)
|
||||
return [{ ...state, lifecycle, tools: result.tools }, events] satisfies StepResult
|
||||
const reasoningSignatures = { ...state.reasoningSignatures }
|
||||
delete reasoningSignatures[event.index]
|
||||
return [{ ...state, lifecycle, tools: result.tools, reasoningSignatures }, events] satisfies StepResult
|
||||
})
|
||||
|
||||
const onMessageDelta = (state: ParserState, event: AnthropicEvent): StepResult => {
|
||||
const usage = mergeUsage(state.usage, mapUsage(event.usage))
|
||||
return [
|
||||
{
|
||||
...state,
|
||||
usage,
|
||||
pendingFinish: {
|
||||
reason: {
|
||||
normalized: mapFinishReason(event.delta?.stop_reason),
|
||||
raw: event.delta?.stop_reason ?? undefined,
|
||||
},
|
||||
providerMetadata:
|
||||
event.delta?.stop_sequence === null || event.delta?.stop_sequence === undefined
|
||||
? undefined
|
||||
: anthropicMetadata({ stopSequence: event.delta.stop_sequence }),
|
||||
},
|
||||
},
|
||||
NO_EVENTS,
|
||||
]
|
||||
}
|
||||
|
||||
const onMessageStop = (state: ParserState): StepResult => {
|
||||
const events: LLMEvent[] = []
|
||||
const lifecycle = Lifecycle.finish(state.lifecycle, events, {
|
||||
reason: mapFinishReason(event.delta?.stop_reason),
|
||||
usage,
|
||||
providerMetadata: event.delta?.stop_sequence
|
||||
? anthropicMetadata({ stopSequence: event.delta.stop_sequence })
|
||||
: undefined,
|
||||
reason: state.pendingFinish?.reason ?? {
|
||||
normalized: "unknown",
|
||||
raw: undefined,
|
||||
},
|
||||
usage: state.usage,
|
||||
providerMetadata: state.pendingFinish?.providerMetadata,
|
||||
})
|
||||
return [{ ...state, lifecycle, usage }, events]
|
||||
return [{ ...state, lifecycle }, events]
|
||||
}
|
||||
|
||||
// Prefix `error.type` so overloads, rate limits, and quota errors are visible
|
||||
@@ -860,6 +976,7 @@ const step = (state: ParserState, event: AnthropicEvent) => {
|
||||
if (event.type === "content_block_delta") return onContentBlockDelta(state, event)
|
||||
if (event.type === "content_block_stop") return onContentBlockStop(state, event)
|
||||
if (event.type === "message_delta") return Effect.succeed(onMessageDelta(state, event))
|
||||
if (event.type === "message_stop") return Effect.succeed(onMessageStop(state))
|
||||
if (event.type === "error") return onError(event)
|
||||
return Effect.succeed<StepResult>([state, NO_EVENTS])
|
||||
}
|
||||
@@ -880,7 +997,11 @@ export const protocol = Protocol.make({
|
||||
},
|
||||
stream: {
|
||||
event: Protocol.jsonEvent(AnthropicEvent),
|
||||
initial: () => ({ tools: ToolStream.empty<number>(), lifecycle: Lifecycle.initial() }),
|
||||
initial: () => ({
|
||||
tools: ToolStream.empty<number>(),
|
||||
reasoningSignatures: {},
|
||||
lifecycle: Lifecycle.initial(),
|
||||
}),
|
||||
step,
|
||||
},
|
||||
})
|
||||
|
||||
@@ -8,6 +8,7 @@ import {
|
||||
Usage,
|
||||
type CacheHint,
|
||||
type FinishReason,
|
||||
type FinishReasonDetails,
|
||||
type JsonSchema,
|
||||
type LLMRequest,
|
||||
type ModelToolSchemaCompatibility,
|
||||
@@ -65,14 +66,15 @@ const BedrockToolResultBlock = Schema.Struct({
|
||||
type BedrockToolResultBlock = Schema.Schema.Type<typeof BedrockToolResultBlock>
|
||||
|
||||
const BedrockReasoningBlock = Schema.Struct({
|
||||
reasoningContent: Schema.Struct({
|
||||
reasoningText: Schema.optional(
|
||||
Schema.Struct({
|
||||
reasoningContent: Schema.Union([
|
||||
Schema.Struct({
|
||||
reasoningText: Schema.Struct({
|
||||
text: Schema.String,
|
||||
signature: Schema.optional(Schema.String),
|
||||
}),
|
||||
),
|
||||
}),
|
||||
}),
|
||||
Schema.Struct({ redactedContent: Schema.String }),
|
||||
]),
|
||||
})
|
||||
|
||||
const BedrockUserBlock = Schema.Union([
|
||||
@@ -153,6 +155,12 @@ const BedrockUsageSchema = Schema.Struct({
|
||||
})
|
||||
type BedrockUsageSchema = Schema.Schema.Type<typeof BedrockUsageSchema>
|
||||
|
||||
const BedrockStreamException = Schema.Struct({
|
||||
message: Schema.optional(Schema.String),
|
||||
originalMessage: Schema.optional(Schema.String),
|
||||
originalStatusCode: Schema.optional(Schema.Number),
|
||||
})
|
||||
|
||||
// Streaming event shape — the AWS event stream wraps each JSON payload by its
|
||||
// `:event-type` header (e.g. `messageStart`, `contentBlockDelta`). We
|
||||
// reconstruct that wrapping in `decodeFrames` below so the event schema can
|
||||
@@ -180,6 +188,11 @@ const BedrockEvent = Schema.Struct({
|
||||
Schema.Struct({
|
||||
text: Schema.optional(Schema.String),
|
||||
signature: Schema.optional(Schema.String),
|
||||
// Blob fields in Bedrock's JSON event stream are base64 strings.
|
||||
redactedContent: Schema.optional(Schema.String),
|
||||
// Vercel's Bedrock provider exposes the same delta under
|
||||
// Anthropic's shorter `data` spelling.
|
||||
data: Schema.optional(Schema.String),
|
||||
}),
|
||||
),
|
||||
}),
|
||||
@@ -199,11 +212,11 @@ const BedrockEvent = Schema.Struct({
|
||||
metrics: Schema.optional(Schema.Unknown),
|
||||
}),
|
||||
),
|
||||
internalServerException: Schema.optional(Schema.Struct({ message: Schema.String })),
|
||||
modelStreamErrorException: Schema.optional(Schema.Struct({ message: Schema.String })),
|
||||
validationException: Schema.optional(Schema.Struct({ message: Schema.String })),
|
||||
throttlingException: Schema.optional(Schema.Struct({ message: Schema.String })),
|
||||
serviceUnavailableException: Schema.optional(Schema.Struct({ message: Schema.String })),
|
||||
internalServerException: Schema.optional(BedrockStreamException),
|
||||
modelStreamErrorException: Schema.optional(BedrockStreamException),
|
||||
validationException: Schema.optional(BedrockStreamException),
|
||||
throttlingException: Schema.optional(BedrockStreamException),
|
||||
serviceUnavailableException: Schema.optional(BedrockStreamException),
|
||||
})
|
||||
type BedrockEvent = Schema.Schema.Type<typeof BedrockEvent>
|
||||
|
||||
@@ -259,6 +272,13 @@ const reasoningSignature = (part: ReasoningPart) => {
|
||||
)
|
||||
}
|
||||
|
||||
const reasoningRedactedData = (part: ReasoningPart) => {
|
||||
const bedrock = part.providerMetadata?.bedrock
|
||||
return ProviderShared.isRecord(bedrock) && typeof bedrock.redactedData === "string"
|
||||
? bedrock.redactedData
|
||||
: undefined
|
||||
}
|
||||
|
||||
const lowerToolCall = (part: ToolCallPart): BedrockToolUseBlock => ({
|
||||
toolUse: {
|
||||
toolUseId: part.id,
|
||||
@@ -348,11 +368,13 @@ const lowerMessages = Effect.fn("BedrockConverse.lowerMessages")(function* (
|
||||
continue
|
||||
}
|
||||
if (part.type === "reasoning") {
|
||||
content.push({
|
||||
reasoningContent: {
|
||||
reasoningText: { text: part.text, signature: reasoningSignature(part) },
|
||||
},
|
||||
})
|
||||
const signature = reasoningSignature(part)
|
||||
const redactedData = reasoningRedactedData(part)
|
||||
if (signature === undefined && redactedData !== undefined) {
|
||||
content.push({ reasoningContent: { redactedContent: redactedData } })
|
||||
continue
|
||||
}
|
||||
content.push({ reasoningContent: { reasoningText: { text: part.text, signature } } })
|
||||
continue
|
||||
}
|
||||
if (part.type === "tool-call") {
|
||||
@@ -392,8 +414,13 @@ const fromRequest = Effect.fn("BedrockConverse.fromRequest")(function* (request:
|
||||
// tools → system → messages order to favour the highest-impact prefixes.
|
||||
const breakpoints = BedrockCache.breakpoints()
|
||||
const toolConfig =
|
||||
request.tools.length > 0 && request.toolChoice?.type !== "none"
|
||||
? { tools: lowerTools(request.model.compatibility?.toolSchema, breakpoints, request.tools), toolChoice }
|
||||
request.tools.length > 0
|
||||
? {
|
||||
tools: lowerTools(request.model.compatibility?.toolSchema, breakpoints, request.tools),
|
||||
// Converse has no native "none". Keep definitions stable for prompt
|
||||
// caching and omit only the unsupported choice.
|
||||
toolChoice,
|
||||
}
|
||||
: undefined
|
||||
const system = request.system.length === 0 ? undefined : lowerSystem(breakpoints, request.system)
|
||||
const messages = yield* lowerMessages(request, breakpoints)
|
||||
@@ -430,27 +457,29 @@ const fromRequest = Effect.fn("BedrockConverse.fromRequest")(function* (request:
|
||||
// =============================================================================
|
||||
const mapFinishReason = (reason: string): FinishReason => {
|
||||
if (reason === "end_turn" || reason === "stop_sequence") return "stop"
|
||||
if (reason === "max_tokens") return "length"
|
||||
if (reason === "max_tokens" || reason === "model_context_window_exceeded") return "length"
|
||||
if (reason === "tool_use") return "tool-calls"
|
||||
if (reason === "content_filtered" || reason === "guardrail_intervened") return "content-filter"
|
||||
if (reason === "malformed_model_output" || reason === "malformed_tool_use") return "error"
|
||||
return "unknown"
|
||||
}
|
||||
|
||||
// AWS Bedrock Converse reports `inputTokens` (inclusive total) with
|
||||
// `cacheReadInputTokens` and `cacheWriteInputTokens` as subsets. Pass
|
||||
// the total through and derive the non-cached breakdown. Bedrock does
|
||||
// not break reasoning out of `outputTokens` for any current model.
|
||||
// AWS reports inputTokens separately from cache reads and writes.
|
||||
// Bedrock does not break reasoning out of outputTokens for current models.
|
||||
const mapUsage = (usage: BedrockUsageSchema | undefined): Usage | undefined => {
|
||||
if (!usage) return undefined
|
||||
const cacheTotal = (usage.cacheReadInputTokens ?? 0) + (usage.cacheWriteInputTokens ?? 0)
|
||||
const nonCached = ProviderShared.subtractTokens(usage.inputTokens, cacheTotal)
|
||||
const inputTokens = ProviderShared.sumTokens(
|
||||
usage.inputTokens,
|
||||
usage.cacheReadInputTokens,
|
||||
usage.cacheWriteInputTokens,
|
||||
)
|
||||
return new Usage({
|
||||
inputTokens: usage.inputTokens,
|
||||
inputTokens,
|
||||
outputTokens: usage.outputTokens,
|
||||
nonCachedInputTokens: nonCached,
|
||||
nonCachedInputTokens: usage.inputTokens,
|
||||
cacheReadInputTokens: usage.cacheReadInputTokens,
|
||||
cacheWriteInputTokens: usage.cacheWriteInputTokens,
|
||||
totalTokens: ProviderShared.totalTokens(usage.inputTokens, usage.outputTokens, usage.totalTokens),
|
||||
totalTokens: ProviderShared.totalTokens(inputTokens, usage.outputTokens, usage.totalTokens),
|
||||
providerMetadata: { bedrock: usage },
|
||||
})
|
||||
}
|
||||
@@ -460,7 +489,7 @@ interface ParserState {
|
||||
// Bedrock splits the finish into `messageStop` (carries `stopReason`) and
|
||||
// `metadata` (carries usage). Hold the terminal event in state so `onHalt`
|
||||
// can emit exactly one finish after both chunks have had a chance to arrive.
|
||||
readonly pendingFinish: { readonly reason: FinishReason; readonly usage?: Usage } | undefined
|
||||
readonly pendingFinish: { readonly reason: FinishReasonDetails; readonly usage?: Usage } | undefined
|
||||
readonly hasToolCalls: boolean
|
||||
readonly lifecycle: Lifecycle.State
|
||||
readonly reasoningSignatures: Readonly<Record<number, string>>
|
||||
@@ -511,12 +540,26 @@ const step = (state: ParserState, event: BedrockEvent) =>
|
||||
const index = event.contentBlockDelta.contentBlockIndex
|
||||
const reasoning = event.contentBlockDelta.delta.reasoningContent
|
||||
const events: LLMEvent[] = []
|
||||
const redactedData = reasoning.redactedContent ?? reasoning.data
|
||||
const providerMetadata = reasoning.signature
|
||||
? bedrockMetadata({ signature: reasoning.signature })
|
||||
: redactedData !== undefined
|
||||
? bedrockMetadata({ redactedData })
|
||||
: undefined
|
||||
const lifecycle =
|
||||
reasoning.text !== undefined || providerMetadata !== undefined
|
||||
? Lifecycle.reasoningDelta(
|
||||
state.lifecycle,
|
||||
events,
|
||||
`reasoning-${index}`,
|
||||
reasoning.text ?? "",
|
||||
providerMetadata,
|
||||
)
|
||||
: state.lifecycle
|
||||
return [
|
||||
{
|
||||
...state,
|
||||
lifecycle: reasoning.text
|
||||
? Lifecycle.reasoningDelta(state.lifecycle, events, `reasoning-${index}`, reasoning.text)
|
||||
: state.lifecycle,
|
||||
lifecycle,
|
||||
reasoningSignatures: reasoning.signature
|
||||
? { ...state.reasoningSignatures, [index]: reasoning.signature }
|
||||
: state.reasoningSignatures,
|
||||
@@ -577,15 +620,30 @@ const step = (state: ParserState, event: BedrockEvent) =>
|
||||
return [
|
||||
{
|
||||
...state,
|
||||
pendingFinish: { reason: mapFinishReason(event.messageStop.stopReason), usage: state.pendingFinish?.usage },
|
||||
pendingFinish: {
|
||||
reason: {
|
||||
normalized: mapFinishReason(event.messageStop.stopReason),
|
||||
raw: event.messageStop.stopReason,
|
||||
},
|
||||
usage: state.pendingFinish?.usage,
|
||||
},
|
||||
},
|
||||
[],
|
||||
] as const
|
||||
}
|
||||
|
||||
if (event.metadata) {
|
||||
const usage = mapUsage(event.metadata.usage)
|
||||
return [{ ...state, pendingFinish: { reason: state.pendingFinish?.reason ?? "stop", usage } }, []] as const
|
||||
const usage = mapUsage(event.metadata.usage) ?? state.pendingFinish?.usage
|
||||
return [
|
||||
{
|
||||
...state,
|
||||
pendingFinish: {
|
||||
reason: state.pendingFinish?.reason ?? { normalized: "stop" },
|
||||
usage,
|
||||
},
|
||||
},
|
||||
[],
|
||||
] as const
|
||||
}
|
||||
|
||||
const exception = (
|
||||
@@ -602,7 +660,7 @@ const step = (state: ParserState, event: BedrockEvent) =>
|
||||
module: ADAPTER,
|
||||
method: "stream",
|
||||
reason: classifyProviderFailure({
|
||||
message: exception[1]?.message ?? "Bedrock Converse stream error",
|
||||
message: exception[1]?.message ?? exception[1]?.originalMessage ?? "Bedrock Converse stream error",
|
||||
code: exception[0],
|
||||
}),
|
||||
})
|
||||
@@ -618,8 +676,13 @@ const onHalt = (state: ParserState): ReadonlyArray<LLMEvent> =>
|
||||
? (() => {
|
||||
const events: LLMEvent[] = []
|
||||
Lifecycle.finish(state.lifecycle, events, {
|
||||
reason:
|
||||
state.pendingFinish.reason === "stop" && state.hasToolCalls ? "tool-calls" : state.pendingFinish.reason,
|
||||
reason: {
|
||||
...state.pendingFinish.reason,
|
||||
normalized:
|
||||
state.pendingFinish.reason.normalized === "stop" && state.hasToolCalls
|
||||
? "tool-calls"
|
||||
: state.pendingFinish.reason.normalized,
|
||||
},
|
||||
usage: state.pendingFinish.usage,
|
||||
})
|
||||
return events
|
||||
|
||||
@@ -53,8 +53,22 @@ const consumeFrames = (route: string) => (state: FrameBufferState, chunk: Uint8A
|
||||
})
|
||||
cursor = { buffer: cursor.buffer, offset: cursor.offset + totalLength }
|
||||
|
||||
if (decoded.headers[":message-type"]?.value !== "event") continue
|
||||
const eventType = decoded.headers[":event-type"]?.value
|
||||
const messageType = decoded.headers[":message-type"]?.value
|
||||
if (messageType === "error") {
|
||||
const code = decoded.headers[":error-code"]?.value
|
||||
const message = decoded.headers[":error-message"]?.value
|
||||
return yield* ProviderShared.eventError(
|
||||
route,
|
||||
[code, message].filter((value): value is string => typeof value === "string").join(": ") ||
|
||||
"Bedrock Converse event-stream error",
|
||||
)
|
||||
}
|
||||
const eventType =
|
||||
messageType === "event"
|
||||
? decoded.headers[":event-type"]?.value
|
||||
: messageType === "exception"
|
||||
? decoded.headers[":exception-type"]?.value
|
||||
: undefined
|
||||
if (typeof eventType !== "string") continue
|
||||
const payload = utf8.decode(decoded.body)
|
||||
if (!payload) continue
|
||||
|
||||
@@ -11,6 +11,7 @@ import {
|
||||
type JsonSchema,
|
||||
type LLMRequest,
|
||||
type MediaPart,
|
||||
type ProviderOptions,
|
||||
type ProviderMetadata,
|
||||
type TextPart,
|
||||
type ToolCallPart,
|
||||
@@ -26,6 +27,18 @@ const ADAPTER = "gemini"
|
||||
const MEDIA_MIMES = new Set<string>(ProviderShared.MEDIA_MIMES)
|
||||
export const DEFAULT_BASE_URL = "https://generativelanguage.googleapis.com/v1beta"
|
||||
|
||||
export interface OptionsInput {
|
||||
readonly [key: string]: unknown
|
||||
readonly thinkingConfig?: {
|
||||
readonly thinkingBudget?: number
|
||||
readonly includeThoughts?: boolean
|
||||
}
|
||||
}
|
||||
|
||||
export type ProviderOptionsInput = ProviderOptions & {
|
||||
readonly gemini?: OptionsInput
|
||||
}
|
||||
|
||||
// =============================================================================
|
||||
// Request Body Schema
|
||||
// =============================================================================
|
||||
@@ -203,7 +216,9 @@ const thoughtSignature = (providerMetadata: ProviderMetadata | undefined) => {
|
||||
|
||||
const functionCallId = (providerMetadata: ProviderMetadata | undefined) => {
|
||||
const google = providerMetadata?.google
|
||||
return ProviderShared.isRecord(google) && typeof google.functionCallId === "string" ? google.functionCallId : undefined
|
||||
return ProviderShared.isRecord(google) && typeof google.functionCallId === "string"
|
||||
? google.functionCallId
|
||||
: undefined
|
||||
}
|
||||
|
||||
const lowerToolCall = (part: ToolCallPart) => ({
|
||||
@@ -300,21 +315,22 @@ const lowerMessages = Effect.fn("Gemini.lowerMessages")(function* (request: LLMR
|
||||
return contents
|
||||
})
|
||||
|
||||
const geminiOptions = (request: LLMRequest) => request.providerOptions?.gemini
|
||||
|
||||
const thinkingConfig = (request: LLMRequest) => {
|
||||
const value = geminiOptions(request)?.thinkingConfig
|
||||
if (!ProviderShared.isRecord(value)) return undefined
|
||||
const result = {
|
||||
const resolveOptions = (request: LLMRequest) => {
|
||||
const value = request.providerOptions?.gemini?.thinkingConfig
|
||||
if (!ProviderShared.isRecord(value)) return {}
|
||||
const thinkingConfig = {
|
||||
thinkingBudget: typeof value.thinkingBudget === "number" ? value.thinkingBudget : undefined,
|
||||
includeThoughts: typeof value.includeThoughts === "boolean" ? value.includeThoughts : undefined,
|
||||
}
|
||||
return Object.values(result).some((item) => item !== undefined) ? result : undefined
|
||||
return {
|
||||
thinkingConfig: Object.values(thinkingConfig).some((item) => item !== undefined) ? thinkingConfig : undefined,
|
||||
}
|
||||
}
|
||||
|
||||
const fromRequest = Effect.fn("Gemini.fromRequest")(function* (request: LLMRequest) {
|
||||
const toolsEnabled = request.tools.length > 0 && request.toolChoice?.type !== "none"
|
||||
const hasTools = request.tools.length > 0
|
||||
const generation = request.generation
|
||||
const options = resolveOptions(request)
|
||||
const toolSchemaCompatibility = request.model.compatibility?.toolSchema
|
||||
const generationConfig = {
|
||||
maxOutputTokens: generation?.maxTokens,
|
||||
@@ -322,14 +338,14 @@ const fromRequest = Effect.fn("Gemini.fromRequest")(function* (request: LLMReque
|
||||
topP: generation?.topP,
|
||||
topK: generation?.topK,
|
||||
stopSequences: generation?.stop,
|
||||
thinkingConfig: thinkingConfig(request),
|
||||
thinkingConfig: options.thinkingConfig,
|
||||
}
|
||||
|
||||
return {
|
||||
contents: yield* lowerMessages(request),
|
||||
systemInstruction:
|
||||
request.system.length === 0 ? undefined : { parts: [{ text: ProviderShared.joinText(request.system) }] },
|
||||
tools: toolsEnabled
|
||||
tools: hasTools
|
||||
? [
|
||||
{
|
||||
functionDeclarations: request.tools.map((tool) =>
|
||||
@@ -338,7 +354,7 @@ const fromRequest = Effect.fn("Gemini.fromRequest")(function* (request: LLMReque
|
||||
},
|
||||
]
|
||||
: undefined,
|
||||
toolConfig: toolsEnabled && request.toolChoice ? yield* lowerToolConfig(request.toolChoice) : undefined,
|
||||
toolConfig: hasTools && request.toolChoice ? yield* lowerToolConfig(request.toolChoice) : undefined,
|
||||
generationConfig: Object.values(generationConfig).some((value) => value !== undefined)
|
||||
? generationConfig
|
||||
: undefined,
|
||||
@@ -382,10 +398,22 @@ const mapFinishReason = (finishReason: string | undefined, hasToolCalls: boolean
|
||||
finishReason === "SAFETY" ||
|
||||
finishReason === "BLOCKLIST" ||
|
||||
finishReason === "PROHIBITED_CONTENT" ||
|
||||
finishReason === "SPII"
|
||||
finishReason === "SPII" ||
|
||||
finishReason === "MODEL_ARMOR" ||
|
||||
finishReason === "IMAGE_PROHIBITED_CONTENT" ||
|
||||
finishReason === "IMAGE_RECITATION" ||
|
||||
finishReason === "LANGUAGE"
|
||||
)
|
||||
return "content-filter"
|
||||
if (finishReason === "MALFORMED_FUNCTION_CALL") return "error"
|
||||
if (
|
||||
finishReason === "MALFORMED_FUNCTION_CALL" ||
|
||||
finishReason === "UNEXPECTED_TOOL_CALL" ||
|
||||
finishReason === "NO_IMAGE" ||
|
||||
finishReason === "TOO_MANY_TOOL_CALLS" ||
|
||||
finishReason === "MISSING_THOUGHT_SIGNATURE" ||
|
||||
finishReason === "MALFORMED_RESPONSE"
|
||||
)
|
||||
return "error"
|
||||
return "unknown"
|
||||
}
|
||||
|
||||
@@ -402,7 +430,10 @@ const finish = (state: ParserState): ReadonlyArray<LLMEvent> =>
|
||||
)
|
||||
: state.lifecycle
|
||||
Lifecycle.finish(lifecycle, events, {
|
||||
reason: mapFinishReason(state.finishReason, state.hasToolCalls),
|
||||
reason: {
|
||||
normalized: mapFinishReason(state.finishReason, state.hasToolCalls),
|
||||
raw: state.finishReason,
|
||||
},
|
||||
usage: state.usage,
|
||||
})
|
||||
return events
|
||||
|
||||
@@ -6,3 +6,4 @@ export * as OpenAIImages from "./openai-images"
|
||||
export * as OpenAICompatibleChat from "./openai-compatible-chat"
|
||||
export * as OpenAICompatibleResponses from "./openai-compatible-responses"
|
||||
export * as OpenAIResponses from "./openai-responses"
|
||||
export * as OpenResponses from "./open-responses"
|
||||
|
||||
@@ -0,0 +1,956 @@
|
||||
import { Effect, Schema } from "effect"
|
||||
import { HttpTransport } from "../route/transport"
|
||||
import { Protocol } from "../route/protocol"
|
||||
import {
|
||||
LLMError,
|
||||
LLMEvent,
|
||||
Usage,
|
||||
type FinishReason,
|
||||
type JsonSchema,
|
||||
type LLMRequest,
|
||||
type MediaPart,
|
||||
type ProviderMetadata,
|
||||
type ReasoningPart,
|
||||
type TextPart,
|
||||
type ToolCallPart,
|
||||
type ToolDefinition,
|
||||
type ToolContent,
|
||||
type ToolResultPart,
|
||||
} from "../schema"
|
||||
import { JsonObject, optionalArray, optionalNull, ProviderShared } from "./shared"
|
||||
import { classifyProviderFailure } from "../provider-error"
|
||||
import { OpenResponsesOptions } from "./utils/open-responses-options"
|
||||
import { Lifecycle } from "./utils/lifecycle"
|
||||
import { ToolSchemaProjection } from "./utils/tool-schema"
|
||||
import { ToolStream } from "./utils/tool-stream"
|
||||
|
||||
const ADAPTER = "open-responses"
|
||||
const NAME = "Open Responses"
|
||||
const MEDIA_MIMES = new Set<string>([...ProviderShared.IMAGE_MIMES, ...ProviderShared.PDF_MIMES])
|
||||
export const PATH = "/responses"
|
||||
|
||||
// =============================================================================
|
||||
// Request Body Schema
|
||||
// =============================================================================
|
||||
const OpenResponsesInputText = Schema.Struct({
|
||||
type: Schema.tag("input_text"),
|
||||
text: Schema.String,
|
||||
})
|
||||
const OpenResponsesInputImage = Schema.Struct({
|
||||
type: Schema.tag("input_image"),
|
||||
image_url: Schema.String,
|
||||
})
|
||||
const OpenResponsesInputFile = Schema.Struct({
|
||||
type: Schema.tag("input_file"),
|
||||
filename: Schema.String,
|
||||
file_data: Schema.String,
|
||||
mime_type: Schema.optional(Schema.String),
|
||||
})
|
||||
const MediaInput = Schema.Union([OpenResponsesInputImage, OpenResponsesInputFile])
|
||||
export type MediaInput = Schema.Schema.Type<typeof MediaInput>
|
||||
const OpenResponsesInputContent = Schema.Union([OpenResponsesInputText, MediaInput])
|
||||
|
||||
const OpenResponsesOutputText = Schema.Struct({
|
||||
type: Schema.tag("output_text"),
|
||||
text: Schema.String,
|
||||
})
|
||||
|
||||
const OpenResponsesReasoningSummaryText = Schema.Struct({
|
||||
type: Schema.tag("summary_text"),
|
||||
text: Schema.String,
|
||||
})
|
||||
|
||||
const OpenResponsesReasoningItem = Schema.Struct({
|
||||
type: Schema.tag("reasoning"),
|
||||
id: Schema.optionalKey(Schema.String),
|
||||
summary: Schema.Array(OpenResponsesReasoningSummaryText),
|
||||
encrypted_content: optionalNull(Schema.String),
|
||||
})
|
||||
|
||||
const OpenResponsesItemReference = Schema.Struct({
|
||||
type: Schema.tag("item_reference"),
|
||||
id: Schema.String,
|
||||
})
|
||||
|
||||
// `function_call_output.output` accepts either a plain string or an ordered
|
||||
// array of content items so tools can return images and files in addition to text.
|
||||
// https://www.openresponses.org/reference
|
||||
const OpenResponsesFunctionCallOutputContent = Schema.Union([
|
||||
OpenResponsesInputText,
|
||||
OpenResponsesInputImage,
|
||||
OpenResponsesInputFile,
|
||||
])
|
||||
|
||||
const OpenResponsesFunctionCallOutput = Schema.Union([
|
||||
Schema.String,
|
||||
Schema.Array(OpenResponsesFunctionCallOutputContent),
|
||||
])
|
||||
|
||||
const OpenResponsesInputItem = Schema.Union([
|
||||
Schema.Struct({ role: Schema.tag("system"), content: Schema.String }),
|
||||
Schema.Struct({ role: Schema.tag("user"), content: Schema.Array(OpenResponsesInputContent) }),
|
||||
Schema.Struct({ role: Schema.tag("assistant"), content: Schema.Array(OpenResponsesOutputText) }),
|
||||
OpenResponsesReasoningItem,
|
||||
OpenResponsesItemReference,
|
||||
Schema.Struct({
|
||||
type: Schema.tag("function_call"),
|
||||
call_id: Schema.String,
|
||||
name: Schema.String,
|
||||
arguments: Schema.String,
|
||||
}),
|
||||
Schema.Struct({
|
||||
type: Schema.tag("function_call_output"),
|
||||
call_id: Schema.String,
|
||||
output: OpenResponsesFunctionCallOutput,
|
||||
}),
|
||||
])
|
||||
type OpenResponsesInputItem = Schema.Schema.Type<typeof OpenResponsesInputItem>
|
||||
|
||||
// Mutable counterpart of the schema reasoning item so `lowerMessages` can fold
|
||||
// multiple streamed summary parts into the same item before flushing.
|
||||
type OpenResponsesReasoningInput = {
|
||||
type: "reasoning"
|
||||
id: string
|
||||
summary: Array<{ type: "summary_text"; text: string }>
|
||||
encrypted_content?: string | null
|
||||
}
|
||||
type OpenResponsesReasoningReplay = Omit<OpenResponsesReasoningInput, "id">
|
||||
|
||||
export const Tool = Schema.Struct({
|
||||
type: Schema.tag("function"),
|
||||
name: Schema.String,
|
||||
description: Schema.String,
|
||||
parameters: JsonObject,
|
||||
strict: Schema.optional(Schema.Boolean),
|
||||
})
|
||||
|
||||
export const ToolChoice = Schema.Union([
|
||||
Schema.Literals(["auto", "none", "required"]),
|
||||
Schema.Struct({ type: Schema.tag("function"), name: Schema.String }),
|
||||
])
|
||||
|
||||
// Fields shared between the HTTP body and the WebSocket `response.create`
|
||||
// message. The HTTP body adds `stream: true`; the WebSocket message adds
|
||||
// `type: "response.create"`. Defining the shared shape once keeps the two
|
||||
// transports in sync without a destructure-and-strip dance.
|
||||
export const coreFields = {
|
||||
model: Schema.String,
|
||||
input: Schema.Array(OpenResponsesInputItem),
|
||||
instructions: Schema.optional(Schema.String),
|
||||
tools: optionalArray(Tool),
|
||||
tool_choice: Schema.optional(ToolChoice),
|
||||
store: Schema.optional(Schema.Boolean),
|
||||
service_tier: Schema.optional(OpenResponsesOptions.ServiceTierSchema),
|
||||
prompt_cache_key: Schema.optional(Schema.String),
|
||||
include: optionalArray(OpenResponsesOptions.ResponseIncludableSchema),
|
||||
reasoning: Schema.optional(
|
||||
Schema.Struct({
|
||||
effort: Schema.optional(OpenResponsesOptions.ReasoningEffort),
|
||||
summary: Schema.optional(Schema.Literals(["auto", "concise", "detailed"])),
|
||||
}),
|
||||
),
|
||||
text: Schema.optional(
|
||||
Schema.Struct({
|
||||
verbosity: Schema.optional(OpenResponsesOptions.TextVerbositySchema),
|
||||
}),
|
||||
),
|
||||
max_output_tokens: Schema.optional(Schema.Number),
|
||||
temperature: Schema.optional(Schema.Number),
|
||||
top_p: Schema.optional(Schema.Number),
|
||||
}
|
||||
|
||||
const OpenResponsesBody = Schema.Struct({
|
||||
...coreFields,
|
||||
stream: Schema.Literal(true),
|
||||
})
|
||||
export type OpenResponsesBody = Schema.Schema.Type<typeof OpenResponsesBody>
|
||||
|
||||
const OpenResponsesUsage = Schema.Struct({
|
||||
input_tokens: Schema.optional(Schema.Number),
|
||||
input_tokens_details: optionalNull(
|
||||
Schema.Struct({
|
||||
cached_tokens: Schema.optional(Schema.Number),
|
||||
cache_write_tokens: Schema.optional(Schema.Number),
|
||||
}),
|
||||
),
|
||||
output_tokens: Schema.optional(Schema.Number),
|
||||
output_tokens_details: optionalNull(Schema.Struct({ reasoning_tokens: Schema.optional(Schema.Number) })),
|
||||
total_tokens: Schema.optional(Schema.Number),
|
||||
})
|
||||
type OpenResponsesUsage = Schema.Schema.Type<typeof OpenResponsesUsage>
|
||||
|
||||
export const StreamItem = Schema.StructWithRest(
|
||||
Schema.Struct({
|
||||
type: Schema.String,
|
||||
id: Schema.optional(Schema.String),
|
||||
call_id: Schema.optional(Schema.String),
|
||||
name: Schema.optional(Schema.String),
|
||||
arguments: Schema.optional(Schema.String),
|
||||
encrypted_content: optionalNull(Schema.String),
|
||||
}),
|
||||
[Schema.Record(Schema.String, Schema.Unknown)],
|
||||
)
|
||||
export type StreamItem = Schema.Schema.Type<typeof StreamItem>
|
||||
|
||||
// The Responses schema puts streaming error details at the top level and
|
||||
// response failures under `response.error`. WebSocket failures use an
|
||||
// event-level `error` envelope, so accept all three shapes here.
|
||||
// https://www.openresponses.org/specification
|
||||
const OpenResponsesErrorPayload = Schema.Struct({
|
||||
code: optionalNull(Schema.String),
|
||||
message: optionalNull(Schema.String),
|
||||
param: optionalNull(Schema.String),
|
||||
})
|
||||
|
||||
export const Event = Schema.StructWithRest(
|
||||
Schema.Struct({
|
||||
type: Schema.String,
|
||||
delta: Schema.optional(Schema.String),
|
||||
item_id: Schema.optional(Schema.String),
|
||||
summary_index: Schema.optional(Schema.Number),
|
||||
item: Schema.optional(StreamItem),
|
||||
response: Schema.optional(
|
||||
Schema.StructWithRest(
|
||||
Schema.Struct({
|
||||
id: Schema.optional(Schema.String),
|
||||
service_tier: optionalNull(Schema.String),
|
||||
incomplete_details: optionalNull(Schema.Struct({ reason: Schema.optional(Schema.String) })),
|
||||
usage: optionalNull(OpenResponsesUsage),
|
||||
error: optionalNull(OpenResponsesErrorPayload),
|
||||
}),
|
||||
[Schema.Record(Schema.String, Schema.Unknown)],
|
||||
),
|
||||
),
|
||||
code: optionalNull(Schema.String),
|
||||
message: Schema.optional(Schema.String),
|
||||
param: optionalNull(Schema.String),
|
||||
error: optionalNull(OpenResponsesErrorPayload),
|
||||
}),
|
||||
[Schema.Record(Schema.String, Schema.Unknown)],
|
||||
)
|
||||
export type Event = Schema.Schema.Type<typeof Event>
|
||||
|
||||
export interface Extension {
|
||||
readonly id: string
|
||||
readonly name: string
|
||||
readonly lowerMedia?: (input: {
|
||||
readonly part: MediaPart
|
||||
readonly media: ProviderShared.ValidatedMedia
|
||||
readonly request: LLMRequest
|
||||
}) => MediaInput | undefined
|
||||
}
|
||||
|
||||
const BASE: Extension = { id: ADAPTER, name: NAME }
|
||||
|
||||
export interface ParserState {
|
||||
readonly id: string
|
||||
readonly name: string
|
||||
readonly providerMetadataKey: string
|
||||
readonly tools: ToolStream.State<string>
|
||||
readonly hasFunctionCall: boolean
|
||||
readonly lifecycle: Lifecycle.State
|
||||
readonly reasoningItems: Readonly<Record<string, ReasoningStreamItem>>
|
||||
readonly store: boolean | undefined
|
||||
}
|
||||
|
||||
type ReasoningSummaryStatus = "active" | "can-conclude" | "concluded"
|
||||
|
||||
interface ReasoningStreamItem {
|
||||
readonly encryptedContent: string | null | undefined
|
||||
// Keyed by the wire protocol's numeric `summary_index`. JS object keys coerce to
|
||||
// strings, but typing the map as `Record<number, ...>` documents intent
|
||||
// and matches the wire field.
|
||||
readonly summaryParts: Readonly<Record<number, ReasoningSummaryStatus>>
|
||||
}
|
||||
|
||||
// =============================================================================
|
||||
// Request Lowering
|
||||
// =============================================================================
|
||||
export const lowerTool = Effect.fn("OpenResponses.lowerTool")(function* (
|
||||
protocolName: string,
|
||||
tool: ToolDefinition,
|
||||
inputSchema: JsonSchema,
|
||||
) {
|
||||
if (tool.native !== undefined)
|
||||
return yield* ProviderShared.invalidRequest(`${protocolName} does not support provider-native tool ${tool.name}`)
|
||||
return {
|
||||
type: "function" as const,
|
||||
name: tool.name,
|
||||
description: tool.description,
|
||||
parameters: ToolSchemaProjection.responses(inputSchema),
|
||||
// TODO: Read this from Responses tool options so direct LLM callers can opt into strict schemas.
|
||||
strict: false,
|
||||
}
|
||||
})
|
||||
|
||||
export const lowerToolChoice = (protocolName: string, toolChoice: NonNullable<LLMRequest["toolChoice"]>) =>
|
||||
ProviderShared.matchToolChoice(protocolName, toolChoice, {
|
||||
auto: () => "auto" as const,
|
||||
none: () => "none" as const,
|
||||
required: () => "required" as const,
|
||||
tool: (toolName) => ({ type: "function" as const, name: toolName }),
|
||||
})
|
||||
|
||||
const lowerToolCall = (part: ToolCallPart): OpenResponsesInputItem => ({
|
||||
type: "function_call",
|
||||
call_id: part.id,
|
||||
name: part.name,
|
||||
arguments: ProviderShared.encodeJson(part.input),
|
||||
})
|
||||
|
||||
const lowerReasoning = (part: ReasoningPart, providerMetadataKey: string): OpenResponsesReasoningInput | undefined => {
|
||||
const metadata = part.providerMetadata?.[providerMetadataKey]
|
||||
if (!ProviderShared.isRecord(metadata) || typeof metadata.itemId !== "string" || metadata.itemId.length === 0)
|
||||
return undefined
|
||||
const encryptedContent =
|
||||
typeof metadata.reasoningEncryptedContent === "string" || metadata.reasoningEncryptedContent === null
|
||||
? metadata.reasoningEncryptedContent
|
||||
: undefined
|
||||
return {
|
||||
type: "reasoning",
|
||||
id: metadata.itemId,
|
||||
summary: part.text.length > 0 ? [{ type: "summary_text", text: part.text }] : [],
|
||||
encrypted_content: encryptedContent,
|
||||
}
|
||||
}
|
||||
|
||||
const hostedToolItemID = (part: ToolResultPart, providerMetadataKey: string) => {
|
||||
const metadata = part.providerMetadata?.[providerMetadataKey]
|
||||
return ProviderShared.isRecord(metadata) && typeof metadata.itemId === "string" && metadata.itemId.length > 0
|
||||
? metadata.itemId
|
||||
: undefined
|
||||
}
|
||||
|
||||
const lowerMedia = Effect.fn("OpenResponses.lowerMedia")(function* (
|
||||
part: MediaPart,
|
||||
request: LLMRequest,
|
||||
extension: Extension,
|
||||
) {
|
||||
const media = yield* ProviderShared.validateMedia(extension.name, part, MEDIA_MIMES)
|
||||
const extended = extension.lowerMedia?.({ part, media, request })
|
||||
if (extended) return extended
|
||||
if (media.mime === "application/pdf") {
|
||||
return {
|
||||
type: "input_file" as const,
|
||||
filename: part.filename ?? "document.pdf",
|
||||
file_data: media.dataUrl,
|
||||
}
|
||||
}
|
||||
return { type: "input_image" as const, image_url: media.dataUrl }
|
||||
})
|
||||
|
||||
const lowerUserContent = Effect.fn("OpenResponses.lowerUserContent")(function* (
|
||||
part: LLMRequest["messages"][number]["content"][number],
|
||||
request: LLMRequest,
|
||||
extension: Extension,
|
||||
) {
|
||||
if (part.type === "text") return { type: "input_text" as const, text: part.text }
|
||||
if (part.type === "media") return yield* lowerMedia(part, request, extension)
|
||||
return yield* ProviderShared.unsupportedContent(extension.name, "user", ["text", "media"])
|
||||
})
|
||||
|
||||
// Tool results may carry structured text, images, and files. Keep media as provider-native
|
||||
// content instead of JSON-stringifying base64 into a prompt string.
|
||||
const lowerToolResultContentItem = Effect.fn("OpenResponses.lowerToolResultContentItem")(function* (
|
||||
item: ToolContent,
|
||||
request: LLMRequest,
|
||||
extension: Extension,
|
||||
) {
|
||||
if (item.type === "text") return { type: "input_text" as const, text: item.text }
|
||||
return yield* lowerMedia(
|
||||
{ type: "media", mediaType: item.mime, data: item.uri, filename: item.name },
|
||||
request,
|
||||
extension,
|
||||
)
|
||||
})
|
||||
|
||||
const lowerToolResultOutput = Effect.fn("OpenResponses.lowerToolResultOutput")(function* (
|
||||
part: ToolResultPart,
|
||||
request: LLMRequest,
|
||||
extension: Extension,
|
||||
) {
|
||||
// Text/json/error results are encoded as a plain string for backward
|
||||
// compatibility with existing cassettes and provider expectations.
|
||||
if (part.result.type !== "content") return ProviderShared.toolResultText(part)
|
||||
// Preserve the narrowed array element type when compiled through a consumer package.
|
||||
const content: ReadonlyArray<ToolContent> = part.result.value
|
||||
return yield* Effect.forEach(content, (item) => lowerToolResultContentItem(item, request, extension))
|
||||
})
|
||||
|
||||
const lowerMessages = Effect.fn("OpenResponses.lowerMessages")(function* (request: LLMRequest, extension: Extension) {
|
||||
const system: OpenResponsesInputItem[] =
|
||||
request.system.length === 0 ? [] : [{ role: "system", content: ProviderShared.joinText(request.system) }]
|
||||
const input: OpenResponsesInputItem[] = [...system]
|
||||
const store = OpenResponsesOptions.resolve(request).store
|
||||
const providerMetadataKey = request.model.route.providerMetadataKey ?? "openresponses"
|
||||
|
||||
for (const message of request.messages) {
|
||||
if (message.role === "system") {
|
||||
const part = yield* ProviderShared.wrappedSystemUpdate(extension.name, message)
|
||||
const previous = input.at(-1)
|
||||
if (previous && "role" in previous && previous.role === "user")
|
||||
input[input.length - 1] = {
|
||||
role: "user",
|
||||
content: [...previous.content, { type: "input_text", text: part.text }],
|
||||
}
|
||||
else input.push({ role: "user", content: [{ type: "input_text", text: part.text }] })
|
||||
continue
|
||||
}
|
||||
|
||||
if (message.role === "user") {
|
||||
input.push({
|
||||
role: "user",
|
||||
content: yield* Effect.forEach(message.content, (part) => lowerUserContent(part, request, extension)),
|
||||
})
|
||||
continue
|
||||
}
|
||||
|
||||
if (message.role === "assistant") {
|
||||
const content: TextPart[] = []
|
||||
const reasoningItems: Record<string, OpenResponsesReasoningReplay> = {}
|
||||
const reasoningReferences = new Set<string>()
|
||||
const hostedToolReferences = new Set<string>()
|
||||
const flushText = () => {
|
||||
if (content.length === 0) return
|
||||
input.push({ role: "assistant", content: content.map((part) => ({ type: "output_text", text: part.text })) })
|
||||
content.splice(0, content.length)
|
||||
}
|
||||
for (const part of message.content) {
|
||||
if (part.type === "text") {
|
||||
content.push(part)
|
||||
continue
|
||||
}
|
||||
if (part.type === "reasoning") {
|
||||
flushText()
|
||||
const reasoning = lowerReasoning(part, providerMetadataKey)
|
||||
if (!reasoning) continue
|
||||
if (store !== false) {
|
||||
if (!reasoningReferences.has(reasoning.id)) input.push({ type: "item_reference", id: reasoning.id })
|
||||
reasoningReferences.add(reasoning.id)
|
||||
continue
|
||||
}
|
||||
const existing = reasoningItems[reasoning.id]
|
||||
if (existing) {
|
||||
existing.summary.push(...reasoning.summary)
|
||||
if (typeof reasoning.encrypted_content === "string")
|
||||
existing.encrypted_content = reasoning.encrypted_content
|
||||
continue
|
||||
}
|
||||
const replay = {
|
||||
type: reasoning.type,
|
||||
summary: reasoning.summary,
|
||||
encrypted_content: reasoning.encrypted_content,
|
||||
}
|
||||
reasoningItems[reasoning.id] = replay
|
||||
input.push(replay)
|
||||
continue
|
||||
}
|
||||
if (part.type === "tool-call") {
|
||||
flushText()
|
||||
if (part.providerExecuted === true) continue
|
||||
input.push(lowerToolCall(part))
|
||||
continue
|
||||
}
|
||||
if (part.type === "tool-result" && part.providerExecuted === true) {
|
||||
flushText()
|
||||
const itemID = hostedToolItemID(part, providerMetadataKey)
|
||||
if (store !== false && itemID && !hostedToolReferences.has(itemID))
|
||||
input.push({ type: "item_reference", id: itemID })
|
||||
if (store === false && part.result.type === "content") {
|
||||
const content: ReadonlyArray<ToolContent> = part.result.value
|
||||
input.push({
|
||||
role: "user",
|
||||
content: yield* Effect.forEach(content, (item) => lowerToolResultContentItem(item, request, extension)),
|
||||
})
|
||||
}
|
||||
if (itemID) hostedToolReferences.add(itemID)
|
||||
continue
|
||||
}
|
||||
return yield* ProviderShared.unsupportedContent(extension.name, "assistant", [
|
||||
"text",
|
||||
"reasoning",
|
||||
"tool-call",
|
||||
"tool-result",
|
||||
])
|
||||
}
|
||||
flushText()
|
||||
continue
|
||||
}
|
||||
|
||||
for (const part of message.content) {
|
||||
if (!ProviderShared.supportsContent(part, ["tool-result"]))
|
||||
return yield* ProviderShared.unsupportedContent(extension.name, "tool", ["tool-result"])
|
||||
input.push({
|
||||
type: "function_call_output",
|
||||
call_id: part.id,
|
||||
output: yield* lowerToolResultOutput(part, request, extension),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// With store:false, Responses APIs only accept previous reasoning items when the
|
||||
// complete item has encrypted state. Summary blocks for one item may carry
|
||||
// that state only on the last block, so filter after they have been joined.
|
||||
return store === false
|
||||
? input.filter(
|
||||
(item) => !("type" in item) || item.type !== "reasoning" || typeof item.encrypted_content === "string",
|
||||
)
|
||||
: input
|
||||
})
|
||||
|
||||
const lowerOptions = (request: LLMRequest) => {
|
||||
const options = OpenResponsesOptions.resolve(request)
|
||||
return {
|
||||
...(options.instructions ? { instructions: options.instructions } : {}),
|
||||
...(options.store !== undefined ? { store: options.store } : {}),
|
||||
...(options.promptCacheKey ? { prompt_cache_key: options.promptCacheKey } : {}),
|
||||
...(options.include ? { include: options.include } : {}),
|
||||
...(options.reasoningEffort || options.reasoningSummary
|
||||
? { reasoning: { effort: options.reasoningEffort, summary: options.reasoningSummary } }
|
||||
: {}),
|
||||
...(options.textVerbosity ? { text: { verbosity: options.textVerbosity } } : {}),
|
||||
...(options.serviceTier ? { service_tier: options.serviceTier } : {}),
|
||||
}
|
||||
}
|
||||
|
||||
export const fromRequest = Effect.fn("OpenResponses.fromRequest")(function* (
|
||||
request: LLMRequest,
|
||||
extension: Extension = BASE,
|
||||
) {
|
||||
const generation = request.generation
|
||||
const toolSchemaCompatibility = request.model.compatibility?.toolSchema
|
||||
return {
|
||||
model: request.model.id,
|
||||
input: yield* lowerMessages(request, extension),
|
||||
tools:
|
||||
request.tools.length === 0
|
||||
? undefined
|
||||
: yield* Effect.forEach(request.tools, (tool) =>
|
||||
lowerTool(
|
||||
extension.name,
|
||||
tool,
|
||||
ToolSchemaProjection.modelCompatibility(tool.inputSchema, toolSchemaCompatibility),
|
||||
),
|
||||
),
|
||||
tool_choice: request.toolChoice ? yield* lowerToolChoice(extension.name, request.toolChoice) : undefined,
|
||||
stream: true as const,
|
||||
max_output_tokens: generation?.maxTokens,
|
||||
temperature: generation?.temperature,
|
||||
top_p: generation?.topP,
|
||||
...lowerOptions(request),
|
||||
}
|
||||
})
|
||||
|
||||
// =============================================================================
|
||||
// Stream Parsing
|
||||
// =============================================================================
|
||||
// Responses APIs report `input_tokens` (inclusive total) with a
|
||||
// cached-read and cache-write subsets, and `output_tokens` (inclusive total)
|
||||
// with a `reasoning_tokens` subset. Pass the totals through and derive the
|
||||
// non-cached breakdown.
|
||||
const mapUsage = (usage: OpenResponsesUsage | null | undefined, providerMetadataKey: string) => {
|
||||
if (!usage) return undefined
|
||||
const cached = usage.input_tokens_details?.cached_tokens
|
||||
const cacheWrite = usage.input_tokens_details?.cache_write_tokens
|
||||
const reasoning = usage.output_tokens_details?.reasoning_tokens
|
||||
const nonCached = ProviderShared.subtractTokens(usage.input_tokens, ProviderShared.sumTokens(cached, cacheWrite))
|
||||
return new Usage({
|
||||
inputTokens: usage.input_tokens,
|
||||
outputTokens: usage.output_tokens,
|
||||
nonCachedInputTokens: nonCached,
|
||||
cacheReadInputTokens: cached,
|
||||
cacheWriteInputTokens: cacheWrite,
|
||||
reasoningTokens: reasoning,
|
||||
totalTokens: ProviderShared.totalTokens(usage.input_tokens, usage.output_tokens, usage.total_tokens),
|
||||
providerMetadata: { [providerMetadataKey]: usage },
|
||||
})
|
||||
}
|
||||
|
||||
const mapFinishReason = (event: Event, hasFunctionCall: boolean): FinishReason => {
|
||||
const reason = event.response?.incomplete_details?.reason
|
||||
if (reason === undefined || reason === null) {
|
||||
if (hasFunctionCall) return "tool-calls"
|
||||
if (event.type === "response.incomplete") return "unknown"
|
||||
return "stop"
|
||||
}
|
||||
if (reason === "max_output_tokens") return "length"
|
||||
if (reason === "content_filter") return "content-filter"
|
||||
return hasFunctionCall ? "tool-calls" : "unknown"
|
||||
}
|
||||
|
||||
export const providerMetadata = (state: ParserState, metadata: Record<string, unknown>): ProviderMetadata => ({
|
||||
[state.providerMetadataKey]: metadata,
|
||||
})
|
||||
|
||||
const isReasoningItem = (item: StreamItem): item is StreamItem & { type: "reasoning"; id: string } =>
|
||||
item.type === "reasoning" && typeof item.id === "string" && item.id.length > 0
|
||||
|
||||
export type StepResult = readonly [ParserState, ReadonlyArray<LLMEvent>]
|
||||
|
||||
const NO_EVENTS: StepResult["1"] = []
|
||||
|
||||
// `response.completed` / `response.incomplete` are clean finishes that emit a
|
||||
// `finish` event; `response.failed` is a hard failure. All three end the stream,
|
||||
// so keep this set aligned with `step` and the protocol's terminal predicate.
|
||||
const TERMINAL_TYPES = new Set(["response.completed", "response.incomplete", "response.failed"])
|
||||
export const terminal = (event: Event) => TERMINAL_TYPES.has(event.type)
|
||||
|
||||
const onOutputTextDelta = (state: ParserState, event: Event): StepResult => {
|
||||
if (!event.delta) return [state, NO_EVENTS]
|
||||
const events: LLMEvent[] = []
|
||||
return [
|
||||
{ ...state, lifecycle: Lifecycle.textDelta(state.lifecycle, events, event.item_id ?? "text-0", event.delta) },
|
||||
events,
|
||||
]
|
||||
}
|
||||
|
||||
const onOutputTextDone = (state: ParserState, event: Event): StepResult => {
|
||||
const events: LLMEvent[] = []
|
||||
return [{ ...state, lifecycle: Lifecycle.textEnd(state.lifecycle, events, event.item_id ?? "text-0") }, events]
|
||||
}
|
||||
|
||||
export const onReasoningDelta = (state: ParserState, event: Event): StepResult => {
|
||||
if (!event.delta) return [state, NO_EVENTS]
|
||||
const events: LLMEvent[] = []
|
||||
const itemID = event.item_id ?? "reasoning-0"
|
||||
const id =
|
||||
event.summary_index !== undefined || state.reasoningItems[itemID] ? `${itemID}:${event.summary_index ?? 0}` : itemID
|
||||
return [
|
||||
{
|
||||
...state,
|
||||
lifecycle: Lifecycle.reasoningDelta(state.lifecycle, events, id, event.delta),
|
||||
},
|
||||
events,
|
||||
]
|
||||
}
|
||||
|
||||
export const onReasoningDone = (state: ParserState, _event: Event): StepResult => [state, NO_EVENTS]
|
||||
|
||||
const reasoningMetadata = (state: ParserState, item: StreamItem & { id: string }) =>
|
||||
providerMetadata(state, { itemId: item.id, reasoningEncryptedContent: item.encrypted_content ?? null })
|
||||
|
||||
// Responses APIs stream reasoning items in a stable order:
|
||||
// `output_item.added` (reasoning) →
|
||||
// `reasoning_summary_part.added` (index=0) →
|
||||
// `reasoning_summary_text.delta` →
|
||||
// `reasoning_summary_part.done` (index=0) →
|
||||
// (repeat for index>0) →
|
||||
// `output_item.done` (reasoning).
|
||||
// The handlers below rely on this ordering: `onOutputItemAdded` seeds the
|
||||
// per-item entry, `onReasoningSummaryPartAdded` for `summary_index === 0`
|
||||
// short-circuits when the entry already exists, and higher-index handlers
|
||||
// fold against the same entry. Behaviour for out-of-order events is
|
||||
// best-effort, not guaranteed.
|
||||
const onOutputItemAdded = (state: ParserState, event: Event): StepResult => {
|
||||
const item = event.item
|
||||
if (item && isReasoningItem(item)) {
|
||||
const events: LLMEvent[] = []
|
||||
return [
|
||||
{
|
||||
...state,
|
||||
lifecycle: Lifecycle.reasoningStart(state.lifecycle, events, `${item.id}:0`, reasoningMetadata(state, item)),
|
||||
reasoningItems: {
|
||||
...state.reasoningItems,
|
||||
[item.id]: { encryptedContent: item.encrypted_content, summaryParts: { 0: "active" } },
|
||||
},
|
||||
},
|
||||
events,
|
||||
]
|
||||
}
|
||||
if (item?.type !== "function_call" || !item.id) return [state, NO_EVENTS]
|
||||
const metadata = providerMetadata(state, { itemId: item.id })
|
||||
const events: LLMEvent[] = []
|
||||
const lifecycle = Lifecycle.stepStart(state.lifecycle, events)
|
||||
return [
|
||||
{
|
||||
...state,
|
||||
lifecycle,
|
||||
tools: ToolStream.start(state.tools, item.id, {
|
||||
id: item.call_id ?? item.id,
|
||||
name: item.name ?? "",
|
||||
input: item.arguments ?? "",
|
||||
providerMetadata: metadata,
|
||||
}),
|
||||
},
|
||||
[
|
||||
...events,
|
||||
LLMEvent.toolInputStart({ id: item.call_id ?? item.id, name: item.name ?? "", providerMetadata: metadata }),
|
||||
],
|
||||
]
|
||||
}
|
||||
|
||||
const onReasoningSummaryPartAdded = (state: ParserState, event: Event): StepResult => {
|
||||
if (!event.item_id || event.summary_index === undefined) return [state, NO_EVENTS]
|
||||
const item = state.reasoningItems[event.item_id] ?? { encryptedContent: undefined, summaryParts: {} }
|
||||
if (event.summary_index === 0) {
|
||||
if (state.reasoningItems[event.item_id]) return [state, NO_EVENTS]
|
||||
const events: LLMEvent[] = []
|
||||
return [
|
||||
{
|
||||
...state,
|
||||
lifecycle: Lifecycle.reasoningStart(
|
||||
state.lifecycle,
|
||||
events,
|
||||
`${event.item_id}:0`,
|
||||
providerMetadata(state, { itemId: event.item_id, reasoningEncryptedContent: null }),
|
||||
),
|
||||
reasoningItems: {
|
||||
...state.reasoningItems,
|
||||
[event.item_id]: { ...item, summaryParts: { 0: "active" } },
|
||||
},
|
||||
},
|
||||
events,
|
||||
]
|
||||
}
|
||||
|
||||
const events: LLMEvent[] = []
|
||||
const closed = Object.entries(item.summaryParts)
|
||||
.filter((entry) => entry[1] === "can-conclude")
|
||||
.reduce(
|
||||
(lifecycle, entry) =>
|
||||
Lifecycle.reasoningEnd(
|
||||
lifecycle,
|
||||
events,
|
||||
`${event.item_id}:${entry[0]}`,
|
||||
providerMetadata(state, { itemId: event.item_id }),
|
||||
),
|
||||
state.lifecycle,
|
||||
)
|
||||
return [
|
||||
{
|
||||
...state,
|
||||
lifecycle: Lifecycle.reasoningStart(
|
||||
closed,
|
||||
events,
|
||||
`${event.item_id}:${event.summary_index}`,
|
||||
providerMetadata(state, { itemId: event.item_id, reasoningEncryptedContent: item.encryptedContent ?? null }),
|
||||
),
|
||||
reasoningItems: {
|
||||
...state.reasoningItems,
|
||||
[event.item_id]: {
|
||||
...item,
|
||||
summaryParts: {
|
||||
...Object.fromEntries(
|
||||
Object.entries(item.summaryParts).map((entry) =>
|
||||
entry[1] === "can-conclude" ? [entry[0], "concluded" as const] : entry,
|
||||
),
|
||||
),
|
||||
[event.summary_index]: "active",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
events,
|
||||
]
|
||||
}
|
||||
|
||||
const onReasoningSummaryPartDone = (state: ParserState, event: Event): StepResult => {
|
||||
if (!event.item_id || event.summary_index === undefined) return [state, NO_EVENTS]
|
||||
const item = state.reasoningItems[event.item_id]
|
||||
if (!item) return [state, NO_EVENTS]
|
||||
const events: LLMEvent[] = []
|
||||
return [
|
||||
{
|
||||
...state,
|
||||
lifecycle:
|
||||
state.store !== false
|
||||
? Lifecycle.reasoningEnd(
|
||||
state.lifecycle,
|
||||
events,
|
||||
`${event.item_id}:${event.summary_index}`,
|
||||
providerMetadata(state, { itemId: event.item_id }),
|
||||
)
|
||||
: state.lifecycle,
|
||||
reasoningItems: {
|
||||
...state.reasoningItems,
|
||||
[event.item_id]: {
|
||||
...item,
|
||||
summaryParts: {
|
||||
...item.summaryParts,
|
||||
[event.summary_index]: state.store !== false ? "concluded" : "can-conclude",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
events,
|
||||
]
|
||||
}
|
||||
|
||||
const onFunctionCallArgumentsDelta = Effect.fn("OpenResponses.onFunctionCallArgumentsDelta")(function* (
|
||||
state: ParserState,
|
||||
event: Event,
|
||||
) {
|
||||
if (!event.item_id || !event.delta) return [state, NO_EVENTS] satisfies StepResult
|
||||
const result = ToolStream.appendExisting(
|
||||
state.id,
|
||||
state.tools,
|
||||
event.item_id,
|
||||
event.delta,
|
||||
`${state.name} tool argument delta is missing its tool call`,
|
||||
)
|
||||
if (ToolStream.isError(result)) return yield* result
|
||||
const events: LLMEvent[] = []
|
||||
const lifecycle = result.events.length ? Lifecycle.stepStart(state.lifecycle, events) : state.lifecycle
|
||||
events.push(...result.events)
|
||||
return [{ ...state, lifecycle, tools: result.tools }, events] satisfies StepResult
|
||||
})
|
||||
|
||||
const onOutputItemDone = Effect.fn("OpenResponses.onOutputItemDone")(function* (state: ParserState, event: Event) {
|
||||
const item = event.item
|
||||
if (!item) return [state, NO_EVENTS] satisfies StepResult
|
||||
|
||||
if (item.type === "message" && item.id) return onOutputTextDone(state, { ...event, item_id: item.id })
|
||||
|
||||
if (item.type === "function_call") {
|
||||
if (!item.id || !item.call_id || !item.name) return [state, NO_EVENTS] satisfies StepResult
|
||||
const tools = state.tools[item.id]
|
||||
? state.tools
|
||||
: ToolStream.start(state.tools, item.id, { id: item.call_id, name: item.name })
|
||||
const result =
|
||||
item.arguments === undefined
|
||||
? yield* ToolStream.finish(state.id, tools, item.id)
|
||||
: yield* ToolStream.finishWithInput(state.id, tools, item.id, item.arguments)
|
||||
const events: LLMEvent[] = []
|
||||
const resultEvents = result.events ?? []
|
||||
const lifecycle = resultEvents.length ? Lifecycle.stepStart(state.lifecycle, events) : state.lifecycle
|
||||
events.push(...resultEvents)
|
||||
return [
|
||||
{
|
||||
...state,
|
||||
lifecycle,
|
||||
hasFunctionCall:
|
||||
resultEvents.some((event) => LLMEvent.is.toolCall(event) || LLMEvent.is.toolInputError(event)) ||
|
||||
state.hasFunctionCall,
|
||||
tools: result.tools,
|
||||
},
|
||||
events,
|
||||
] satisfies StepResult
|
||||
}
|
||||
|
||||
if (isReasoningItem(item)) {
|
||||
const events: LLMEvent[] = []
|
||||
const metadata = reasoningMetadata(state, item)
|
||||
const reasoningItem = state.reasoningItems[item.id]
|
||||
if (reasoningItem) {
|
||||
const lifecycle = Object.entries(reasoningItem.summaryParts)
|
||||
.filter((entry) => entry[1] === "active" || entry[1] === "can-conclude")
|
||||
.reduce(
|
||||
(lifecycle, entry) => Lifecycle.reasoningEnd(lifecycle, events, `${item.id}:${entry[0]}`, metadata),
|
||||
state.lifecycle,
|
||||
)
|
||||
const { [item.id]: _removed, ...reasoningItems } = state.reasoningItems
|
||||
return [{ ...state, lifecycle, reasoningItems }, events] satisfies StepResult
|
||||
}
|
||||
if (!state.lifecycle.reasoning.has(item.id)) {
|
||||
const lifecycle = Lifecycle.stepStart(state.lifecycle, events)
|
||||
events.push(LLMEvent.reasoningStart({ id: item.id, providerMetadata: metadata }))
|
||||
events.push(LLMEvent.reasoningEnd({ id: item.id, providerMetadata: metadata }))
|
||||
return [{ ...state, lifecycle }, events] satisfies StepResult
|
||||
}
|
||||
return [
|
||||
{ ...state, lifecycle: Lifecycle.reasoningEnd(state.lifecycle, events, item.id, metadata) },
|
||||
events,
|
||||
] satisfies StepResult
|
||||
}
|
||||
|
||||
return [state, NO_EVENTS] satisfies StepResult
|
||||
})
|
||||
|
||||
const onResponseFinish = (state: ParserState, event: Event): StepResult => {
|
||||
const events: LLMEvent[] = []
|
||||
const lifecycle = Lifecycle.finish(state.lifecycle, events, {
|
||||
reason: {
|
||||
normalized: mapFinishReason(event, state.hasFunctionCall),
|
||||
raw: event.response?.incomplete_details?.reason,
|
||||
},
|
||||
usage: mapUsage(event.response?.usage, state.providerMetadataKey),
|
||||
providerMetadata:
|
||||
event.response?.id || event.response?.service_tier
|
||||
? providerMetadata(state, {
|
||||
responseId: event.response.id,
|
||||
serviceTier: event.response.service_tier,
|
||||
})
|
||||
: undefined,
|
||||
})
|
||||
return [{ ...state, lifecycle }, events]
|
||||
}
|
||||
|
||||
// Build a single human-readable message from whatever the provider supplied.
|
||||
// When both code and message are present, prefix the code so consumers see
|
||||
// the failure mode (e.g. `rate_limit_exceeded: Slow down`) instead of just
|
||||
// the bare message — production rate limits and context-length failures used
|
||||
// to be indistinguishable from generic stream drops.
|
||||
const providerErrorMessage = (event: Event, fallback: string): string => {
|
||||
const nested = event.error ?? event.response?.error ?? undefined
|
||||
const message = event.message || nested?.message || undefined
|
||||
const code = event.code || nested?.code || undefined
|
||||
if (message && code) return `${code}: ${message}`
|
||||
return message || code || fallback
|
||||
}
|
||||
|
||||
const providerError = (state: ParserState, event: Event, fallback: string) => {
|
||||
const code = event.code || event.error?.code || event.response?.error?.code || undefined
|
||||
const message = providerErrorMessage(event, fallback)
|
||||
return new LLMError({
|
||||
module: state.id,
|
||||
method: "stream",
|
||||
reason: classifyProviderFailure({ message, code }),
|
||||
})
|
||||
}
|
||||
|
||||
export const step = (state: ParserState, event: Event) => {
|
||||
if (event.type === "response.output_text.delta") return Effect.succeed(onOutputTextDelta(state, event))
|
||||
if (event.type === "response.output_text.done") return Effect.succeed(onOutputTextDone(state, event))
|
||||
if (event.type === "response.reasoning.delta" || event.type === "response.reasoning_summary_text.delta")
|
||||
return Effect.succeed(onReasoningDelta(state, event))
|
||||
if (event.type === "response.reasoning.done" || event.type === "response.reasoning_summary_text.done")
|
||||
return Effect.succeed(onReasoningDone(state, event))
|
||||
if (event.type === "response.reasoning_summary_part.added")
|
||||
return Effect.succeed(onReasoningSummaryPartAdded(state, event))
|
||||
if (event.type === "response.reasoning_summary_part.done")
|
||||
return Effect.succeed(onReasoningSummaryPartDone(state, event))
|
||||
if (event.type === "response.output_item.added") return Effect.succeed(onOutputItemAdded(state, event))
|
||||
if (event.type === "response.function_call_arguments.delta") return onFunctionCallArgumentsDelta(state, event)
|
||||
if (event.type === "response.output_item.done") return onOutputItemDone(state, event)
|
||||
if (event.type === "response.completed" || event.type === "response.incomplete")
|
||||
return Effect.succeed(onResponseFinish(state, event))
|
||||
if (event.type === "response.failed") return providerError(state, event, `${state.name} response failed`)
|
||||
if (event.type === "error") return providerError(state, event, `${state.name} stream error`)
|
||||
return Effect.succeed<StepResult>([state, NO_EVENTS])
|
||||
}
|
||||
|
||||
// =============================================================================
|
||||
// Protocol
|
||||
// =============================================================================
|
||||
/**
|
||||
* The provider-neutral Open Responses protocol. Provider-specific Responses
|
||||
* implementations compose this baseline with their own tools and event variants.
|
||||
*/
|
||||
export const initial = (request: LLMRequest, extension: Extension = BASE): ParserState => ({
|
||||
id: extension.id,
|
||||
name: extension.name,
|
||||
providerMetadataKey: request.model.route.providerMetadataKey ?? "openresponses",
|
||||
hasFunctionCall: false,
|
||||
tools: ToolStream.empty<string>(),
|
||||
lifecycle: Lifecycle.initial(),
|
||||
reasoningItems: {},
|
||||
store: OpenResponsesOptions.resolve(request).store,
|
||||
})
|
||||
|
||||
export const protocol = Protocol.make({
|
||||
id: ADAPTER,
|
||||
body: {
|
||||
schema: OpenResponsesBody,
|
||||
from: fromRequest,
|
||||
},
|
||||
stream: {
|
||||
event: Protocol.jsonEvent(Event),
|
||||
initial,
|
||||
step,
|
||||
terminal,
|
||||
},
|
||||
})
|
||||
|
||||
export const httpTransport = HttpTransport.sseJson.with<OpenResponsesBody>()
|
||||
|
||||
export * as OpenResponses from "./open-responses"
|
||||
@@ -5,9 +5,11 @@ import { Endpoint } from "../route/endpoint"
|
||||
import { HttpTransport } from "../route/transport"
|
||||
import { Protocol } from "../route/protocol"
|
||||
import {
|
||||
LLMError,
|
||||
LLMEvent,
|
||||
Usage,
|
||||
type FinishReason,
|
||||
type FinishReasonDetails,
|
||||
type JsonSchema,
|
||||
type LLMRequest,
|
||||
type MediaPart,
|
||||
@@ -17,6 +19,7 @@ import {
|
||||
type ToolDefinition,
|
||||
type ToolContent,
|
||||
} from "../schema"
|
||||
import { classifyProviderFailure } from "../provider-error"
|
||||
import { isRecord, JsonObject, optionalArray, optionalNull, ProviderShared } from "./shared"
|
||||
import { OpenAIOptions } from "./utils/openai-options"
|
||||
import { Lifecycle } from "./utils/lifecycle"
|
||||
@@ -128,6 +131,7 @@ const OpenAIChatUsage = Schema.Struct({
|
||||
prompt_tokens_details: optionalNull(
|
||||
Schema.Struct({
|
||||
cached_tokens: Schema.optional(Schema.Number),
|
||||
cache_write_tokens: Schema.optional(Schema.Number),
|
||||
}),
|
||||
),
|
||||
completion_tokens_details: optionalNull(
|
||||
@@ -164,11 +168,18 @@ const OpenAIChatDelta = Schema.StructWithRest(
|
||||
const OpenAIChatChoice = Schema.Struct({
|
||||
delta: optionalNull(OpenAIChatDelta),
|
||||
finish_reason: optionalNull(Schema.String),
|
||||
native_finish_reason: optionalNull(Schema.String),
|
||||
})
|
||||
|
||||
const OpenAIChatError = Schema.Struct({
|
||||
code: optionalNull(Schema.Union([Schema.String, Schema.Number])),
|
||||
message: Schema.String,
|
||||
})
|
||||
|
||||
export const OpenAIChatEvent = Schema.Struct({
|
||||
choices: Schema.Array(OpenAIChatChoice),
|
||||
choices: optionalNull(Schema.Array(OpenAIChatChoice)),
|
||||
usage: optionalNull(OpenAIChatUsage),
|
||||
error: optionalNull(OpenAIChatError),
|
||||
})
|
||||
export type OpenAIChatEvent = Schema.Schema.Type<typeof OpenAIChatEvent>
|
||||
type OpenAIChatRequestMessage = LLMRequest["messages"][number]
|
||||
@@ -184,7 +195,7 @@ export interface ParserState {
|
||||
readonly pendingTools: Partial<Record<number, PendingToolDelta>>
|
||||
readonly toolCallEvents: ReadonlyArray<LLMEvent>
|
||||
readonly usage?: Usage
|
||||
readonly finishReason?: FinishReason
|
||||
readonly finishReason?: FinishReasonDetails
|
||||
readonly lifecycle: Lifecycle.State
|
||||
readonly reasoningField?: string
|
||||
readonly reasoningDetails: Array<unknown>
|
||||
@@ -386,14 +397,13 @@ const lowerMessages = Effect.fn("OpenAIChat.lowerMessages")(function* (request:
|
||||
return messages
|
||||
})
|
||||
|
||||
const lowerOptions = Effect.fn("OpenAIChat.lowerOptions")(function* (request: LLMRequest) {
|
||||
const store = OpenAIOptions.store(request)
|
||||
const reasoningEffort = OpenAIOptions.reasoningEffort(request)
|
||||
const lowerOptions = (request: LLMRequest) => {
|
||||
const options = OpenAIOptions.resolve(request)
|
||||
return {
|
||||
...(store !== undefined ? { store } : {}),
|
||||
...(reasoningEffort ? { reasoning_effort: reasoningEffort } : {}),
|
||||
...(options.store !== undefined ? { store: options.store } : {}),
|
||||
...(options.reasoningEffort ? { reasoning_effort: options.reasoningEffort } : {}),
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
const fromRequest = Effect.fn("OpenAIChat.fromRequest")(function* (request: LLMRequest) {
|
||||
// `fromRequest` returns the provider body only. Endpoint, auth, framing,
|
||||
@@ -424,7 +434,7 @@ const fromRequest = Effect.fn("OpenAIChat.fromRequest")(function* (request: LLMR
|
||||
presence_penalty: generation?.presencePenalty,
|
||||
seed: generation?.seed,
|
||||
stop: generation?.stop,
|
||||
...(yield* lowerOptions(request)),
|
||||
...lowerOptions(request),
|
||||
}
|
||||
})
|
||||
|
||||
@@ -439,24 +449,27 @@ const mapFinishReason = (reason: string | null | undefined): FinishReason => {
|
||||
if (reason === "length") return "length"
|
||||
if (reason === "content_filter") return "content-filter"
|
||||
if (reason === "function_call" || reason === "tool_calls") return "tool-calls"
|
||||
if (reason === "error") return "error"
|
||||
return "unknown"
|
||||
}
|
||||
|
||||
// OpenAI Chat reports `prompt_tokens` (inclusive total) with a
|
||||
// `cached_tokens` subset, and `completion_tokens` (inclusive total) with
|
||||
// a `reasoning_tokens` subset. We pass the inclusive totals through and
|
||||
// derive the non-cached breakdown so the `LLM.Usage` contract is
|
||||
// cached-read and cache-write subsets, and `completion_tokens` (inclusive
|
||||
// total) with a `reasoning_tokens` subset. We pass the inclusive totals
|
||||
// through and derive the non-cached breakdown so the `LLM.Usage` contract is
|
||||
// satisfied on both sides.
|
||||
const mapUsage = (usage: OpenAIChatEvent["usage"]): Usage | undefined => {
|
||||
if (!usage) return undefined
|
||||
const cached = usage.prompt_tokens_details?.cached_tokens
|
||||
const cacheWrite = usage.prompt_tokens_details?.cache_write_tokens
|
||||
const reasoning = usage.completion_tokens_details?.reasoning_tokens
|
||||
const nonCached = ProviderShared.subtractTokens(usage.prompt_tokens, cached)
|
||||
const nonCached = ProviderShared.subtractTokens(usage.prompt_tokens, ProviderShared.sumTokens(cached, cacheWrite))
|
||||
return new Usage({
|
||||
inputTokens: usage.prompt_tokens,
|
||||
outputTokens: usage.completion_tokens,
|
||||
nonCachedInputTokens: nonCached,
|
||||
cacheReadInputTokens: cached,
|
||||
cacheWriteInputTokens: cacheWrite,
|
||||
reasoningTokens: reasoning,
|
||||
totalTokens: ProviderShared.totalTokens(usage.prompt_tokens, usage.completion_tokens, usage.total_tokens),
|
||||
providerMetadata: { openai: usage },
|
||||
@@ -532,10 +545,22 @@ const reasoningMetadata = (field: ParserState["reasoningField"], details?: Reado
|
||||
|
||||
const step = (state: ParserState, event: OpenAIChatEvent) =>
|
||||
Effect.gen(function* () {
|
||||
if (event.error)
|
||||
return yield* new LLMError({
|
||||
module: ADAPTER,
|
||||
method: "stream",
|
||||
reason: classifyProviderFailure({
|
||||
message: event.error.message,
|
||||
code: event.error.code === undefined || event.error.code === null ? undefined : String(event.error.code),
|
||||
status: typeof event.error.code === "number" ? event.error.code : undefined,
|
||||
}),
|
||||
})
|
||||
const events: LLMEvent[] = []
|
||||
const usage = mapUsage(event.usage) ?? state.usage
|
||||
const choice = event.choices[0]
|
||||
const finishReason = choice?.finish_reason ? mapFinishReason(choice.finish_reason) : state.finishReason
|
||||
const choice = event.choices?.[0]
|
||||
const finishReason = choice?.finish_reason
|
||||
? { normalized: mapFinishReason(choice.finish_reason), raw: choice.native_finish_reason ?? choice.finish_reason }
|
||||
: state.finishReason
|
||||
const delta = choice?.delta
|
||||
const toolDeltas = delta?.tool_calls ?? []
|
||||
let tools = state.tools
|
||||
@@ -627,7 +652,13 @@ const step = (state: ParserState, event: OpenAIChatEvent) =>
|
||||
const finishEvents = (state: ParserState): ReadonlyArray<LLMEvent> => {
|
||||
const events: LLMEvent[] = []
|
||||
const hasToolCalls = state.toolCallEvents.length > 0
|
||||
const reason = state.finishReason === "stop" && hasToolCalls ? "tool-calls" : state.finishReason
|
||||
const reason = state.finishReason
|
||||
? {
|
||||
...state.finishReason,
|
||||
normalized:
|
||||
state.finishReason.normalized === "stop" && hasToolCalls ? "tool-calls" : state.finishReason.normalized,
|
||||
}
|
||||
: undefined
|
||||
const metadata = reasoningMetadata(
|
||||
state.reasoningField,
|
||||
state.reasoningDetailsObserved ? state.reasoningDetails : undefined,
|
||||
|
||||
@@ -1,23 +1,22 @@
|
||||
import { Route, type RouteRoutedModelInput } from "../route/client"
|
||||
import { Endpoint } from "../route/endpoint"
|
||||
import { OpenAIResponses } from "./openai-responses"
|
||||
import { OpenResponses } from "./open-responses"
|
||||
|
||||
const ADAPTER = "openai-compatible-responses"
|
||||
|
||||
export type OpenAICompatibleResponsesModelInput = RouteRoutedModelInput
|
||||
|
||||
/**
|
||||
* Route for providers that expose an OpenAI Responses-compatible `/responses`
|
||||
* endpoint. Provider helpers configure identity, endpoint, and auth before
|
||||
* model selection while this route reuses the OpenAI Responses protocol.
|
||||
* Deployment adapter for providers that expose an Open Responses-compatible
|
||||
* `/responses` endpoint. Provider helpers configure identity, endpoint, and
|
||||
* auth while the semantic protocol remains provider-neutral.
|
||||
*/
|
||||
export const route = Route.make({
|
||||
id: ADAPTER,
|
||||
providerMetadataKey: "openai",
|
||||
protocol: OpenAIResponses.protocol,
|
||||
endpoint: Endpoint.path(OpenAIResponses.PATH),
|
||||
transport: OpenAIResponses.httpTransport,
|
||||
defaults: { providerOptions: { openai: { store: false } } },
|
||||
providerMetadataKey: "openresponses",
|
||||
protocol: OpenResponses.protocol,
|
||||
endpoint: Endpoint.path(OpenResponses.PATH),
|
||||
transport: OpenResponses.httpTransport,
|
||||
})
|
||||
|
||||
export * as OpenAICompatibleResponses from "./openai-compatible-responses"
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,4 +1,4 @@
|
||||
import { LLMEvent, type FinishReason, type ProviderMetadata, type Usage } from "../../schema"
|
||||
import { LLMEvent, type FinishReasonDetails, type ProviderMetadata, type Usage } from "../../schema"
|
||||
|
||||
export interface State {
|
||||
readonly stepStarted: boolean
|
||||
@@ -14,16 +14,19 @@ export const stepStart = (state: State, events: LLMEvent[]): State => {
|
||||
return { ...state, stepStarted: true }
|
||||
}
|
||||
|
||||
export const textDelta = (state: State, events: LLMEvent[], id: string, text: string): State => {
|
||||
export const textStart = (state: State, events: LLMEvent[], id: string, providerMetadata?: ProviderMetadata): State => {
|
||||
if (state.text.has(id)) return state
|
||||
const stepped = stepStart(state, events)
|
||||
if (stepped.text.has(id)) {
|
||||
events.push(LLMEvent.textDelta({ id, text }))
|
||||
return stepped
|
||||
}
|
||||
events.push(LLMEvent.textStart({ id }), LLMEvent.textDelta({ id, text }))
|
||||
events.push(LLMEvent.textStart({ id, providerMetadata }))
|
||||
return { ...stepped, text: new Set([...stepped.text, id]) }
|
||||
}
|
||||
|
||||
export const textDelta = (state: State, events: LLMEvent[], id: string, text: string): State => {
|
||||
const started = textStart(state, events, id)
|
||||
events.push(LLMEvent.textDelta({ id, text }))
|
||||
return started
|
||||
}
|
||||
|
||||
export const reasoningStart = (
|
||||
state: State,
|
||||
events: LLMEvent[],
|
||||
@@ -81,7 +84,7 @@ export const finish = (
|
||||
state: State,
|
||||
events: LLMEvent[],
|
||||
input: {
|
||||
readonly reason: FinishReason
|
||||
readonly reason: FinishReasonDetails
|
||||
readonly usage?: Usage
|
||||
readonly providerMetadata?: ProviderMetadata
|
||||
},
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
import { Schema } from "effect"
|
||||
import { TextVerbosity, type LLMRequest } from "../../schema"
|
||||
|
||||
export const ResponseIncludables = [
|
||||
"file_search_call.results",
|
||||
"web_search_call.results",
|
||||
"web_search_call.action.sources",
|
||||
"message.input_image.image_url",
|
||||
"computer_call_output.output.image_url",
|
||||
"code_interpreter_call.outputs",
|
||||
"reasoning.encrypted_content",
|
||||
"message.output_text.logprobs",
|
||||
] as const
|
||||
export type ResponseIncludable = (typeof ResponseIncludables)[number]
|
||||
|
||||
export const ServiceTiers = ["auto", "default", "flex", "priority"] as const
|
||||
export type ServiceTier = (typeof ServiceTiers)[number]
|
||||
|
||||
const TEXT_VERBOSITY = new Set<string>(["low", "medium", "high"])
|
||||
const INCLUDABLES = new Set<string>(ResponseIncludables)
|
||||
const SERVICE_TIERS = new Set<string>(ServiceTiers)
|
||||
|
||||
const isTextVerbosity = (value: unknown): value is Schema.Schema.Type<typeof TextVerbosity> =>
|
||||
typeof value === "string" && TEXT_VERBOSITY.has(value)
|
||||
|
||||
const isServiceTier = (value: unknown): value is ServiceTier => typeof value === "string" && SERVICE_TIERS.has(value)
|
||||
|
||||
export const ReasoningEffort = Schema.String
|
||||
export const TextVerbositySchema = TextVerbosity
|
||||
export const ResponseIncludableSchema = Schema.Literals(ResponseIncludables)
|
||||
export const ServiceTierSchema = Schema.Literals(ServiceTiers)
|
||||
|
||||
export interface Resolved {
|
||||
readonly instructions?: string
|
||||
readonly store?: boolean
|
||||
readonly promptCacheKey?: string
|
||||
readonly reasoningEffort?: string
|
||||
readonly reasoningSummary?: "auto" | "concise" | "detailed"
|
||||
readonly include?: ReadonlyArray<ResponseIncludable>
|
||||
readonly textVerbosity?: Schema.Schema.Type<typeof TextVerbosity>
|
||||
readonly serviceTier?: ServiceTier
|
||||
}
|
||||
|
||||
export const resolve = (request: LLMRequest): Resolved => {
|
||||
const input = request.providerOptions?.[request.model.route.providerMetadataKey ?? "openresponses"]
|
||||
const include = Array.isArray(input?.include)
|
||||
? input.include.filter((entry): entry is ResponseIncludable => INCLUDABLES.has(entry))
|
||||
: []
|
||||
const reasoningSummary = input?.reasoningSummary
|
||||
return {
|
||||
instructions: typeof input?.instructions === "string" ? input.instructions : undefined,
|
||||
store: typeof input?.store === "boolean" ? input.store : undefined,
|
||||
promptCacheKey: typeof input?.promptCacheKey === "string" ? input.promptCacheKey : undefined,
|
||||
reasoningEffort: typeof input?.reasoningEffort === "string" ? input.reasoningEffort : undefined,
|
||||
reasoningSummary:
|
||||
reasoningSummary === "auto" || reasoningSummary === "concise" || reasoningSummary === "detailed"
|
||||
? reasoningSummary
|
||||
: undefined,
|
||||
include: include.length > 0 ? include : undefined,
|
||||
textVerbosity: isTextVerbosity(input?.textVerbosity) ? input.textVerbosity : undefined,
|
||||
serviceTier: isServiceTier(input?.serviceTier) ? input.serviceTier : undefined,
|
||||
}
|
||||
}
|
||||
|
||||
export * as OpenResponsesOptions from "./open-responses-options"
|
||||
@@ -1,85 +1,23 @@
|
||||
import { Schema } from "effect"
|
||||
import type { LLMRequest, TextVerbosity as TextVerbosityValue } from "../../schema"
|
||||
import { ReasoningEfforts, TextVerbosity } from "../../schema"
|
||||
import { ReasoningEfforts } from "../../schema"
|
||||
import { OpenResponsesOptions } from "./open-responses-options"
|
||||
|
||||
export const OpenAIReasoningEfforts = ReasoningEfforts
|
||||
export type OpenAIReasoningEffort = string
|
||||
|
||||
// Mirrors OpenAI's `ResponseIncludable` union from the official SDK. Keep this
|
||||
// in lockstep with `openai-node/src/resources/responses/responses.ts`.
|
||||
export const OpenAIResponseIncludables = [
|
||||
"file_search_call.results",
|
||||
"web_search_call.results",
|
||||
"web_search_call.action.sources",
|
||||
"message.input_image.image_url",
|
||||
"computer_call_output.output.image_url",
|
||||
"code_interpreter_call.outputs",
|
||||
"reasoning.encrypted_content",
|
||||
"message.output_text.logprobs",
|
||||
] as const
|
||||
export type OpenAIResponseIncludable = (typeof OpenAIResponseIncludables)[number]
|
||||
export const OpenAIServiceTiers = ["auto", "default", "flex", "priority"] as const
|
||||
export type OpenAIServiceTier = (typeof OpenAIServiceTiers)[number]
|
||||
export const OpenAIResponseIncludables = OpenResponsesOptions.ResponseIncludables
|
||||
export type OpenAIResponseIncludable = OpenResponsesOptions.ResponseIncludable
|
||||
export const OpenAIServiceTiers = OpenResponsesOptions.ServiceTiers
|
||||
export type OpenAIServiceTier = OpenResponsesOptions.ServiceTier
|
||||
|
||||
const TEXT_VERBOSITY = new Set<string>(["low", "medium", "high"])
|
||||
const INCLUDABLES = new Set<string>(OpenAIResponseIncludables)
|
||||
const SERVICE_TIERS = new Set<string>(OpenAIServiceTiers)
|
||||
|
||||
export const OpenAIReasoningEffort = Schema.String
|
||||
export const OpenAITextVerbosity = TextVerbosity
|
||||
export const OpenAIResponseIncludable = Schema.Literals(OpenAIResponseIncludables)
|
||||
export const OpenAIServiceTier = Schema.Literals(OpenAIServiceTiers)
|
||||
export const OpenAIReasoningEffort = OpenResponsesOptions.ReasoningEffort
|
||||
export const OpenAITextVerbosity = OpenResponsesOptions.TextVerbositySchema
|
||||
export const OpenAIResponseIncludable = OpenResponsesOptions.ResponseIncludableSchema
|
||||
export const OpenAIServiceTier = OpenResponsesOptions.ServiceTierSchema
|
||||
|
||||
export const isReasoningEffort = (effort: unknown): effort is OpenAIReasoningEffort => typeof effort === "string"
|
||||
|
||||
const isTextVerbosity = (value: unknown): value is TextVerbosityValue =>
|
||||
typeof value === "string" && TEXT_VERBOSITY.has(value)
|
||||
|
||||
const options = (request: LLMRequest) => request.providerOptions?.openai
|
||||
|
||||
export const store = (request: LLMRequest): boolean | undefined => {
|
||||
const value = options(request)?.store
|
||||
return typeof value === "boolean" ? value : undefined
|
||||
}
|
||||
|
||||
export const reasoningEffort = (request: LLMRequest): string | undefined => {
|
||||
const value = options(request)?.reasoningEffort
|
||||
return typeof value === "string" ? value : undefined
|
||||
}
|
||||
|
||||
export const reasoningSummary = (request: LLMRequest): "auto" | undefined =>
|
||||
options(request)?.reasoningSummary === "auto" ? "auto" : undefined
|
||||
|
||||
// Resolve the OpenAI Responses `include` field. Filters out unknown
|
||||
// includable values defensively so a typo in upstream config drops the
|
||||
// invalid entry instead of poisoning the wire body. An empty array (either
|
||||
// passed directly or produced by filtering) is treated as "no include" and
|
||||
// returns undefined so the request body omits the field entirely.
|
||||
export const include = (request: LLMRequest): ReadonlyArray<OpenAIResponseIncludable> | undefined => {
|
||||
const value = options(request)?.include
|
||||
if (!Array.isArray(value)) return undefined
|
||||
const filtered = value.filter((entry): entry is OpenAIResponseIncludable => INCLUDABLES.has(entry))
|
||||
return filtered.length > 0 ? filtered : undefined
|
||||
}
|
||||
|
||||
export const promptCacheKey = (request: LLMRequest) => {
|
||||
const value = options(request)?.promptCacheKey
|
||||
return typeof value === "string" ? value : undefined
|
||||
}
|
||||
|
||||
export const textVerbosity = (request: LLMRequest) => {
|
||||
const value = options(request)?.textVerbosity
|
||||
return isTextVerbosity(value) ? value : undefined
|
||||
}
|
||||
|
||||
export const serviceTier = (request: LLMRequest) => {
|
||||
const value = options(request)?.serviceTier
|
||||
return typeof value === "string" && SERVICE_TIERS.has(value) ? (value as OpenAIServiceTier) : undefined
|
||||
}
|
||||
|
||||
export const instructions = (request: LLMRequest) => {
|
||||
const value = options(request)?.instructions
|
||||
return typeof value === "string" ? value : undefined
|
||||
}
|
||||
export const resolve = OpenResponsesOptions.resolve
|
||||
|
||||
export * as OpenAIOptions from "./openai-options"
|
||||
|
||||
@@ -63,6 +63,8 @@ const openAI = (schema: JsonSchema): JsonSchema => {
|
||||
return isRecord(normalized) ? normalized : { type: "object" }
|
||||
}
|
||||
|
||||
const responses = openAI
|
||||
|
||||
const gemini = (schema: JsonSchema): JsonSchema => GeminiToolSchema.convert(schema) ?? {}
|
||||
|
||||
const modelCompatibility = (
|
||||
@@ -83,4 +85,5 @@ export const ToolSchemaProjection = {
|
||||
modelCompatibility,
|
||||
moonshot,
|
||||
openAI,
|
||||
responses,
|
||||
} as const
|
||||
|
||||
@@ -5,12 +5,17 @@ import type { ProviderAuthOption } from "../route/auth-options"
|
||||
import type { RouteDefaultsInput } from "../route/client"
|
||||
import { ProviderID, type ModelID } from "../schema"
|
||||
|
||||
export type AnthropicOptionsInput = AnthropicMessages.OptionsInput
|
||||
export type AnthropicProviderOptionsInput = AnthropicMessages.ProviderOptionsInput
|
||||
export type AnthropicThinkingInput = AnthropicMessages.ThinkingInput
|
||||
|
||||
export const id = ProviderID.make("anthropic-compatible")
|
||||
|
||||
export type Config = RouteDefaultsInput &
|
||||
ProviderAuthOption<"optional"> & {
|
||||
readonly provider?: string
|
||||
readonly baseURL: string
|
||||
readonly providerOptions?: AnthropicMessages.ProviderOptionsInput
|
||||
}
|
||||
|
||||
export type Settings = ProviderPackage.Settings &
|
||||
@@ -20,6 +25,7 @@ export type Settings = ProviderPackage.Settings &
|
||||
) & {
|
||||
readonly baseURL: string
|
||||
readonly provider?: string
|
||||
readonly providerOptions?: AnthropicMessages.ProviderOptionsInput
|
||||
}
|
||||
|
||||
export const routes = [AnthropicMessages.route]
|
||||
@@ -61,6 +67,7 @@ export const model: ProviderPackage.Definition<Settings>["model"] = (modelID, se
|
||||
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
|
||||
limits: settings.limits,
|
||||
provider: settings.provider,
|
||||
providerOptions: settings.providerOptions,
|
||||
}).model(modelID)
|
||||
}
|
||||
|
||||
|
||||
@@ -6,11 +6,19 @@ import { ProviderID, type ModelID } from "../schema"
|
||||
import { AnthropicMessages } from "../protocols/anthropic-messages"
|
||||
import { AnthropicCompatible } from "./anthropic-compatible"
|
||||
|
||||
export type AnthropicOptionsInput = AnthropicMessages.OptionsInput
|
||||
export type AnthropicProviderOptionsInput = AnthropicMessages.ProviderOptionsInput
|
||||
export type AnthropicThinkingInput = AnthropicMessages.ThinkingInput
|
||||
|
||||
export const id = ProviderID.make("anthropic")
|
||||
|
||||
export const routes = [AnthropicMessages.route]
|
||||
|
||||
export type Config = RouteDefaultsInput & ProviderAuthOption<"optional"> & { readonly baseURL?: string }
|
||||
export type Config = RouteDefaultsInput &
|
||||
ProviderAuthOption<"optional"> & {
|
||||
readonly baseURL?: string
|
||||
readonly providerOptions?: AnthropicMessages.ProviderOptionsInput
|
||||
}
|
||||
|
||||
export type Settings = ProviderPackage.Settings &
|
||||
(
|
||||
@@ -18,6 +26,7 @@ export type Settings = ProviderPackage.Settings &
|
||||
| { readonly apiKey?: never; readonly authToken?: string }
|
||||
) & {
|
||||
readonly baseURL?: string
|
||||
readonly providerOptions?: AnthropicMessages.ProviderOptionsInput
|
||||
}
|
||||
|
||||
const auth = (options: ProviderAuthOption<"optional">) => {
|
||||
@@ -52,5 +61,6 @@ export const model: ProviderPackage.Definition<Settings>["model"] = (modelID, se
|
||||
headers: settings.headers === undefined ? undefined : { ...settings.headers },
|
||||
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
|
||||
limits: settings.limits,
|
||||
providerOptions: settings.providerOptions,
|
||||
}).model(modelID)
|
||||
}
|
||||
|
||||
@@ -6,9 +6,13 @@ import { Route, type RouteDefaultsInput } from "../route/client"
|
||||
import { Endpoint } from "../route/endpoint"
|
||||
import { Framing } from "../route/framing"
|
||||
import { Protocol } from "../route/protocol"
|
||||
import { ProviderID, type ModelID, type ProviderOptions } from "../schema"
|
||||
import { ProviderID, type ModelID } from "../schema"
|
||||
import { GoogleVertexShared } from "./google-vertex-shared"
|
||||
|
||||
export type AnthropicOptionsInput = AnthropicMessages.OptionsInput
|
||||
export type AnthropicProviderOptionsInput = AnthropicMessages.ProviderOptionsInput
|
||||
export type AnthropicThinkingInput = AnthropicMessages.ThinkingInput
|
||||
|
||||
const VERSION = "vertex-2023-10-16" as const
|
||||
|
||||
// models.dev uses this provider id even though the API contract is Anthropic Messages.
|
||||
@@ -19,6 +23,7 @@ export type Config = RouteDefaultsInput &
|
||||
readonly baseURL?: string
|
||||
readonly location?: string
|
||||
readonly project?: string
|
||||
readonly providerOptions?: AnthropicMessages.ProviderOptionsInput
|
||||
}
|
||||
|
||||
export interface Settings extends ProviderPackage.Settings {
|
||||
@@ -27,7 +32,7 @@ export interface Settings extends ProviderPackage.Settings {
|
||||
readonly baseURL?: string
|
||||
readonly location?: string
|
||||
readonly project?: string
|
||||
readonly providerOptions?: ProviderOptions
|
||||
readonly providerOptions?: AnthropicMessages.ProviderOptionsInput
|
||||
}
|
||||
|
||||
const route = Route.make({
|
||||
|
||||
@@ -25,6 +25,7 @@ export interface Settings extends ProviderPackage.Settings {
|
||||
const route = OpenAICompatibleResponses.route.with({
|
||||
id: "google-vertex-responses",
|
||||
provider: id,
|
||||
providerOptions: { openresponses: { store: false } },
|
||||
})
|
||||
|
||||
export const routes = [route]
|
||||
|
||||
@@ -4,9 +4,12 @@ import { Auth } from "../route/auth"
|
||||
import { Route, type RouteDefaultsInput } from "../route/client"
|
||||
import { Endpoint } from "../route/endpoint"
|
||||
import { Framing } from "../route/framing"
|
||||
import { ProviderID, type ModelID, type ProviderOptions } from "../schema"
|
||||
import { ProviderID, type ModelID } from "../schema"
|
||||
import { GoogleVertexShared } from "./google-vertex-shared"
|
||||
|
||||
export type GeminiOptionsInput = Gemini.OptionsInput
|
||||
export type GeminiProviderOptionsInput = Gemini.ProviderOptionsInput
|
||||
|
||||
export const id = ProviderID.make("google-vertex")
|
||||
|
||||
export type Config = RouteDefaultsInput &
|
||||
@@ -14,6 +17,7 @@ export type Config = RouteDefaultsInput &
|
||||
readonly baseURL?: string
|
||||
readonly location?: string
|
||||
readonly project?: string
|
||||
readonly providerOptions?: Gemini.ProviderOptionsInput
|
||||
}
|
||||
|
||||
export type Settings = ProviderPackage.Settings &
|
||||
@@ -24,7 +28,7 @@ export type Settings = ProviderPackage.Settings &
|
||||
readonly baseURL?: string
|
||||
readonly location?: string
|
||||
readonly project?: string
|
||||
readonly providerOptions?: ProviderOptions
|
||||
readonly providerOptions?: Gemini.ProviderOptionsInput
|
||||
}
|
||||
|
||||
const route = Route.make({
|
||||
|
||||
@@ -2,11 +2,13 @@ import type { RouteDefaultsInput } from "../route/client"
|
||||
import { Auth } from "../route/auth"
|
||||
import type { ProviderAuthOption } from "../route/auth-options"
|
||||
import type { ProviderPackage } from "../provider-package"
|
||||
import { HttpOptions, ProviderID, mergeHttpOptions, type ModelID, type ProviderOptions } from "../schema"
|
||||
import { HttpOptions, ProviderID, mergeHttpOptions, type ModelID } from "../schema"
|
||||
import { Gemini } from "../protocols/gemini"
|
||||
import { GoogleImages } from "../protocols/google-images"
|
||||
|
||||
export type { GoogleImageOptions } from "../protocols/google-images"
|
||||
export type GeminiOptionsInput = Gemini.OptionsInput
|
||||
export type GeminiProviderOptionsInput = Gemini.ProviderOptionsInput
|
||||
|
||||
export const id = ProviderID.make("google")
|
||||
|
||||
@@ -15,12 +17,13 @@ export const routes = [Gemini.route]
|
||||
export type Config = RouteDefaultsInput &
|
||||
ProviderAuthOption<"optional"> & {
|
||||
readonly baseURL?: string
|
||||
readonly providerOptions?: Gemini.ProviderOptionsInput
|
||||
}
|
||||
|
||||
export interface Settings extends ProviderPackage.Settings {
|
||||
readonly apiKey?: string
|
||||
readonly baseURL?: string
|
||||
readonly providerOptions?: ProviderOptions
|
||||
readonly providerOptions?: Gemini.ProviderOptionsInput
|
||||
}
|
||||
|
||||
const auth = (options: ProviderAuthOption<"optional">) => {
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
import type { ResponseIncludable, ServiceTier } from "../protocols/utils/open-responses-options"
|
||||
import type { ProviderOptions, ReasoningEffort, TextVerbosity } from "../schema"
|
||||
|
||||
export interface OpenResponsesOptionsInput {
|
||||
readonly [key: string]: unknown
|
||||
readonly instructions?: string
|
||||
readonly store?: boolean
|
||||
readonly promptCacheKey?: string
|
||||
readonly reasoningEffort?: ReasoningEffort
|
||||
readonly reasoningSummary?: "auto" | "concise" | "detailed"
|
||||
readonly include?: ReadonlyArray<ResponseIncludable>
|
||||
readonly textVerbosity?: TextVerbosity
|
||||
readonly serviceTier?: ServiceTier
|
||||
}
|
||||
|
||||
export type OpenResponsesProviderOptionsInput = ProviderOptions & {
|
||||
readonly openresponses?: OpenResponsesOptionsInput
|
||||
}
|
||||
|
||||
export * as OpenResponsesProviderOptions from "./open-responses-options"
|
||||
@@ -3,7 +3,9 @@ import { OpenAICompatibleResponses } from "../protocols/openai-compatible-respon
|
||||
import { AuthOptions, type ProviderAuthOption } from "../route/auth-options"
|
||||
import type { RouteDefaultsInput } from "../route/client"
|
||||
import { ProviderID, type ModelID } from "../schema"
|
||||
import type { OpenAIProviderOptionsInput } from "./openai-options"
|
||||
import type { OpenResponsesProviderOptionsInput } from "./open-responses-options"
|
||||
|
||||
export type { OpenResponsesOptionsInput, OpenResponsesProviderOptionsInput } from "./open-responses-options"
|
||||
|
||||
export const id = ProviderID.make("openai-compatible")
|
||||
|
||||
@@ -11,13 +13,14 @@ export type Config = RouteDefaultsInput &
|
||||
ProviderAuthOption<"optional"> & {
|
||||
readonly provider?: string
|
||||
readonly baseURL: string
|
||||
readonly providerOptions?: OpenResponsesProviderOptionsInput
|
||||
}
|
||||
|
||||
export interface Settings extends ProviderPackage.Settings {
|
||||
readonly apiKey?: string
|
||||
readonly baseURL: string
|
||||
readonly provider?: string
|
||||
readonly providerOptions?: OpenAIProviderOptionsInput
|
||||
readonly providerOptions?: OpenResponsesProviderOptionsInput
|
||||
}
|
||||
|
||||
export const routes = [OpenAICompatibleResponses.route]
|
||||
|
||||
@@ -1,22 +1,10 @@
|
||||
import type { ProviderOptions, ReasoningEffort, TextVerbosity } from "../schema"
|
||||
import type { ProviderOptions } from "../schema"
|
||||
import { mergeProviderOptions } from "../schema"
|
||||
import type { OpenAIResponseIncludable, OpenAIServiceTier } from "../protocols/utils/openai-options"
|
||||
import type { OpenResponsesOptionsInput } from "./open-responses-options"
|
||||
|
||||
export type { OpenAIResponseIncludable, OpenAIServiceTier } from "../protocols/utils/openai-options"
|
||||
|
||||
export interface OpenAIOptionsInput {
|
||||
readonly [key: string]: unknown
|
||||
readonly store?: boolean
|
||||
readonly promptCacheKey?: string
|
||||
readonly reasoningEffort?: ReasoningEffort
|
||||
readonly reasoningSummary?: "auto"
|
||||
// OpenAI Responses `include` wire field. Mirrors the official SDK's
|
||||
// `ResponseIncludable[]` union exactly so AI SDK callers and direct
|
||||
// native-SDK callers share one shape and no translation is required.
|
||||
readonly include?: ReadonlyArray<OpenAIResponseIncludable>
|
||||
readonly textVerbosity?: TextVerbosity
|
||||
readonly serviceTier?: OpenAIServiceTier
|
||||
}
|
||||
export type OpenAIOptionsInput = OpenResponsesOptionsInput
|
||||
|
||||
export type OpenAIProviderOptionsInput = ProviderOptions & {
|
||||
readonly openai?: OpenAIOptionsInput
|
||||
|
||||
@@ -12,7 +12,8 @@ import type { LLMError, LLMEvent, LLMRequest, ProtocolID } from "../schema"
|
||||
* Examples:
|
||||
*
|
||||
* - `OpenAIChat.protocol` — chat completions style
|
||||
* - `OpenAIResponses.protocol` — responses API
|
||||
* - `OpenResponses.protocol` — provider-neutral Responses API baseline
|
||||
* - `OpenAIResponses.protocol` — OpenAI extensions to that baseline
|
||||
* - `AnthropicMessages.protocol` — messages API with content blocks
|
||||
* - `Gemini.protocol` — generateContent
|
||||
* - `BedrockConverse.protocol` — Converse with binary event-stream framing
|
||||
|
||||
@@ -34,11 +34,12 @@ import { ProviderFailureClassification } from "./errors"
|
||||
*
|
||||
* **Semantics by provider**:
|
||||
*
|
||||
* - OpenAI Chat / Responses / Gemini / Bedrock: provider reports inclusive
|
||||
* - OpenAI Chat / Responses / Gemini: provider reports inclusive
|
||||
* `inputTokens` and an inclusive `outputTokens`; mapper subtracts to
|
||||
* derive the breakdown.
|
||||
* - Anthropic: provider reports the breakdown natively (`input_tokens` is
|
||||
* non-cached only); mapper sums to derive the inclusive `inputTokens`.
|
||||
* - Anthropic and Bedrock report the input breakdown natively: Anthropic's
|
||||
* `input_tokens` and Bedrock's `inputTokens` are non-cached only. Their
|
||||
* mappers sum the breakdown to derive the inclusive `inputTokens`.
|
||||
* Anthropic does *not* break extended-thinking out of `output_tokens`, so
|
||||
* `reasoningTokens` is `undefined` and `outputTokens` carries the
|
||||
* combined total — a documented limitation of the Anthropic API.
|
||||
@@ -190,10 +191,16 @@ export const ToolError = Schema.Struct({
|
||||
}).annotate({ identifier: "LLM.Event.ToolError" })
|
||||
export type ToolError = Schema.Schema.Type<typeof ToolError>
|
||||
|
||||
export const FinishReasonDetails = Schema.Struct({
|
||||
normalized: FinishReason,
|
||||
raw: Schema.optional(Schema.String),
|
||||
}).annotate({ identifier: "LLM.FinishReasonDetails" })
|
||||
export type FinishReasonDetails = Schema.Schema.Type<typeof FinishReasonDetails>
|
||||
|
||||
export const StepFinish = Schema.Struct({
|
||||
type: Schema.tag("step-finish"),
|
||||
index: Schema.Number,
|
||||
reason: FinishReason,
|
||||
reason: FinishReasonDetails,
|
||||
usage: Schema.optional(Usage),
|
||||
providerMetadata: Schema.optional(ProviderMetadata),
|
||||
}).annotate({ identifier: "LLM.Event.StepFinish" })
|
||||
@@ -201,7 +208,7 @@ export type StepFinish = Schema.Schema.Type<typeof StepFinish>
|
||||
|
||||
export const Finish = Schema.Struct({
|
||||
type: Schema.tag("finish"),
|
||||
reason: FinishReason,
|
||||
reason: FinishReasonDetails,
|
||||
usage: Schema.optional(Usage),
|
||||
providerMetadata: Schema.optional(ProviderMetadata),
|
||||
}).annotate({ identifier: "LLM.Event.Finish" })
|
||||
@@ -364,7 +371,7 @@ interface ResponseState {
|
||||
readonly events: ReadonlyArray<LLMEvent>
|
||||
readonly message: Message
|
||||
readonly usage?: Usage
|
||||
readonly finishReason?: FinishReason
|
||||
readonly finishReason?: FinishReasonDetails
|
||||
readonly textParts: Readonly<Record<string, ContentAssembly>>
|
||||
readonly reasoningParts: Readonly<Record<string, ContentAssembly>>
|
||||
readonly toolInputs: Readonly<Record<string, ToolInputAssembly>>
|
||||
@@ -392,7 +399,7 @@ const appendEvent = (state: ResponseState, event: LLMEvent): ResponseState => {
|
||||
return {
|
||||
...state,
|
||||
events,
|
||||
finishReason: state.finishReason ?? "error",
|
||||
finishReason: state.finishReason ?? { normalized: "error" },
|
||||
}
|
||||
}
|
||||
return {
|
||||
@@ -579,7 +586,7 @@ export class LLMResponse extends Schema.Class<LLMResponse>("LLM.Response")({
|
||||
message: Message,
|
||||
events: Schema.Array(LLMEvent),
|
||||
usage: Schema.optional(Usage),
|
||||
finishReason: FinishReason,
|
||||
finishReason: FinishReasonDetails,
|
||||
}) {
|
||||
/** Concatenated assistant text assembled from streamed `text-delta` events. */
|
||||
get text() {
|
||||
|
||||
@@ -261,13 +261,6 @@ export namespace ToolChoice {
|
||||
}
|
||||
}
|
||||
|
||||
export const ResponseFormat = Schema.Union([
|
||||
Schema.Struct({ type: Schema.Literal("text") }),
|
||||
Schema.Struct({ type: Schema.Literal("json"), schema: JsonSchema }),
|
||||
Schema.Struct({ type: Schema.Literal("tool"), tool: ToolDefinition }),
|
||||
]).pipe(Schema.toTaggedUnion("type"))
|
||||
export type ResponseFormat = Schema.Schema.Type<typeof ResponseFormat>
|
||||
|
||||
export class LLMRequest extends Schema.Class<LLMRequest>("LLM.Request")({
|
||||
id: Schema.optional(Schema.String),
|
||||
model: ModelSchema,
|
||||
@@ -278,7 +271,6 @@ export class LLMRequest extends Schema.Class<LLMRequest>("LLM.Request")({
|
||||
generation: Schema.optional(GenerationOptions),
|
||||
providerOptions: Schema.optional(ProviderOptions),
|
||||
http: Schema.optional(HttpOptions),
|
||||
responseFormat: Schema.optional(ResponseFormat),
|
||||
cache: Schema.optional(CachePolicy),
|
||||
metadata: Schema.optional(Schema.Record(Schema.String, Schema.Unknown)),
|
||||
}) {}
|
||||
@@ -296,7 +288,6 @@ export namespace LLMRequest {
|
||||
generation: request.generation,
|
||||
providerOptions: request.providerOptions,
|
||||
http: request.http,
|
||||
responseFormat: request.responseFormat,
|
||||
cache: request.cache,
|
||||
metadata: request.metadata,
|
||||
})
|
||||
|
||||
@@ -251,11 +251,11 @@ export class CacheHint extends Schema.Class<CacheHint>("LLM.CacheHint")({
|
||||
// Auto-placement policy for prompt caching. The protocol-neutral lowering step
|
||||
// reads this and injects `CacheHint`s at the configured boundaries; the
|
||||
// per-protocol body builders then translate those hints into wire markers as
|
||||
// usual. `"auto"` is the recommended default for agent loops — it places one
|
||||
// breakpoint at the last tool definition, one at the last system part, and one
|
||||
// at the latest user message. The combination of provider invalidation
|
||||
// hierarchy (tools → system → messages) and Anthropic/Bedrock's 20-block
|
||||
// lookback means three trailing breakpoints reliably cover the static prefix.
|
||||
// usual. `"auto"` is the recommended default for agent loops — it places
|
||||
// breakpoints at the last tool definition, the first and last distinct system
|
||||
// parts, and the conversation tail. The rolling message breakpoint keeps a
|
||||
// prior cache entry within Anthropic/Bedrock's 20-block lookback during long
|
||||
// tool loops.
|
||||
//
|
||||
// Pass `"none"` to opt out entirely (the legacy behavior). Pass the granular
|
||||
// object form to override individual choices.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect, Schema, Stream } from "effect"
|
||||
import { LLM, LLMResponse } from "../src"
|
||||
import { LLM, LLMRequest, LLMResponse } from "../src"
|
||||
import { Route, Endpoint, LLMClient, Protocol, type FramingDef } from "../src/route"
|
||||
import { Model } from "../src/schema"
|
||||
import { testEffect } from "./lib/effect"
|
||||
@@ -40,7 +40,7 @@ const fakeFraming: FramingDef<FakeEvent> = {
|
||||
|
||||
const raiseEvent = (event: FakeEvent): import("../src/schema").LLMEvent =>
|
||||
event.type === "finish"
|
||||
? { type: "finish", reason: event.reason }
|
||||
? { type: "finish", reason: { normalized: event.reason } }
|
||||
: { type: "text-delta", id: "text-0", text: event.text }
|
||||
|
||||
const fakeProtocol = Protocol.make<FakeBody, FakeEvent, FakeEvent, void>({
|
||||
@@ -141,7 +141,7 @@ describe("llm route", () => {
|
||||
Effect.gen(function* () {
|
||||
const llm = yield* LLMClient.Service
|
||||
const prepared = yield* llm.prepare(
|
||||
LLM.updateRequest(request, { model: updateModel(request.model, { route: configuredGemini }) }),
|
||||
LLMRequest.update(request, { model: updateModel(request.model, { route: configuredGemini }) }),
|
||||
)
|
||||
|
||||
expect(prepared.route).toBe("gemini-fake")
|
||||
@@ -174,7 +174,7 @@ describe("llm route", () => {
|
||||
})
|
||||
|
||||
const prepared = yield* (yield* LLMClient.Service).prepare(
|
||||
LLM.updateRequest(request, { model: updateModel(request.model, { route: duplicate }) }),
|
||||
LLMRequest.update(request, { model: updateModel(request.model, { route: duplicate }) }),
|
||||
)
|
||||
|
||||
expect(prepared.body).toEqual({ body: "late-default" })
|
||||
|
||||
@@ -137,15 +137,26 @@ Azure.configure({ apiKey: "azure-key", resourceName: "resource" }).chat("deploym
|
||||
Azure.configure({ resourceName: "resource", apiKey: "azure-key", auth: Auth.header("api-key", "override") })
|
||||
|
||||
Anthropic.configure({ apiKey: "anthropic-key" }).model("claude-haiku")
|
||||
Anthropic.configure({
|
||||
apiKey: "anthropic-key",
|
||||
providerOptions: {
|
||||
anthropic: { thinking: { type: "enabled", budgetTokens: 1_024 }, effort: "high" },
|
||||
},
|
||||
}).model("claude-haiku")
|
||||
// @ts-expect-error Anthropic model selectors only accept model ids.
|
||||
Anthropic.configure({ apiKey: "anthropic-key" }).model("claude-haiku", {})
|
||||
// @ts-expect-error Anthropic package settings accept only one auth source.
|
||||
Anthropic.model("claude-sonnet-4-6", { apiKey: "anthropic-key", authToken: "anthropic-token" })
|
||||
// @ts-expect-error Enabled Anthropic thinking requires a token budget.
|
||||
Anthropic.configure({ providerOptions: { anthropic: { thinking: { type: "enabled" } } } })
|
||||
// @ts-expect-error Anthropic thinking budgets must be numbers.
|
||||
Anthropic.configure({ providerOptions: { anthropic: { thinking: { type: "enabled", budgetTokens: "large" } } } })
|
||||
|
||||
AnthropicCompatible.configure({
|
||||
apiKey: "messages-key",
|
||||
baseURL: "https://messages.example.com/v1",
|
||||
provider: "example",
|
||||
providerOptions: { anthropic: { thinking: { type: "disabled" } } },
|
||||
}).model("compatible-model")
|
||||
// @ts-expect-error Anthropic-compatible providers require a base URL.
|
||||
AnthropicCompatible.configure({ apiKey: "messages-key" })
|
||||
@@ -159,10 +170,19 @@ AnthropicCompatible.model("compatible-model", {
|
||||
})
|
||||
|
||||
Google.configure({ apiKey: "google-key" }).model("gemini-2.5-flash")
|
||||
Google.configure({
|
||||
apiKey: "google-key",
|
||||
providerOptions: { gemini: { thinkingConfig: { thinkingBudget: 0, includeThoughts: false } } },
|
||||
}).model("gemini-2.5-flash")
|
||||
// @ts-expect-error Google model selectors only accept model ids.
|
||||
Google.configure({ apiKey: "google-key" }).model("gemini-2.5-flash", {})
|
||||
// @ts-expect-error Gemini thinking budgets must be numbers.
|
||||
Google.configure({ providerOptions: { gemini: { thinkingConfig: { thinkingBudget: "large" } } } })
|
||||
|
||||
GoogleVertex.configure({ apiKey: "vertex-key" }).model("gemini-3.5-flash")
|
||||
GoogleVertex.configure({
|
||||
apiKey: "vertex-key",
|
||||
providerOptions: { gemini: { thinkingConfig: { thinkingBudget: 1_024 } } },
|
||||
}).model("gemini-3.5-flash")
|
||||
GoogleVertex.configure({ accessToken: "vertex-token", project: "project" }).model("gemini-3.5-flash")
|
||||
GoogleVertex.configure({ auth: Auth.bearer("vertex-token"), project: "project" }).model("gemini-3.5-flash")
|
||||
// @ts-expect-error Vertex Gemini model selectors only accept model ids.
|
||||
@@ -208,7 +228,11 @@ GoogleVertexResponses.configure({
|
||||
project: "project",
|
||||
})
|
||||
|
||||
GoogleVertexMessages.configure({ accessToken: "vertex-token", project: "project" }).model("claude-sonnet-4-6")
|
||||
GoogleVertexMessages.configure({
|
||||
accessToken: "vertex-token",
|
||||
project: "project",
|
||||
providerOptions: { anthropic: { thinking: { type: "adaptive", display: "omitted" }, effort: "low" } },
|
||||
}).model("claude-sonnet-4-6")
|
||||
// @ts-expect-error Vertex Messages package settings do not accept API keys.
|
||||
GoogleVertexMessages.model("claude-sonnet-4-6", { apiKey: "vertex-key", project: "project" })
|
||||
GoogleVertexMessages.configure({ auth: Auth.bearer("vertex-token"), project: "project" }).model("claude-sonnet-4-6")
|
||||
|
||||
@@ -39,8 +39,8 @@ describe("applyCachePolicy", () => {
|
||||
}),
|
||||
)
|
||||
|
||||
// No explicit cache field → auto policy fires → last system part + latest
|
||||
// user message both get cache_control markers.
|
||||
// A single system block is both the first and last boundary, so the auto
|
||||
// policy deduplicates it and still marks the conversation tail.
|
||||
expect(prepared.body).toMatchObject({
|
||||
system: [{ type: "text", text: "You are concise.", cache_control: { type: "ephemeral" } }],
|
||||
messages: [{ role: "user", content: [{ type: "text", text: "hi", cache_control: { type: "ephemeral" } }] }],
|
||||
@@ -48,12 +48,15 @@ describe("applyCachePolicy", () => {
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("'auto' marks the last tool, last system part, and latest user message on Anthropic", () =>
|
||||
it.effect("'auto' marks the last tool, first and last system parts, and final message boundary on Anthropic", () =>
|
||||
Effect.gen(function* () {
|
||||
const prepared = yield* LLMClient.prepare(
|
||||
LLM.request({
|
||||
model: anthropicModel,
|
||||
system: "Sys A",
|
||||
system: [
|
||||
{ type: "text", text: "Base agent" },
|
||||
{ type: "text", text: "Project instructions" },
|
||||
],
|
||||
tools: [{ name: "t1", description: "t1", inputSchema: { type: "object", properties: {} } }],
|
||||
messages: [
|
||||
Message.user("first user"),
|
||||
@@ -66,7 +69,10 @@ describe("applyCachePolicy", () => {
|
||||
|
||||
expect(prepared.body).toMatchObject({
|
||||
tools: [{ name: "t1", cache_control: { type: "ephemeral" } }],
|
||||
system: [{ type: "text", text: "Sys A", cache_control: { type: "ephemeral" } }],
|
||||
system: [
|
||||
{ type: "text", text: "Base agent", cache_control: { type: "ephemeral" } },
|
||||
{ type: "text", text: "Project instructions", cache_control: { type: "ephemeral" } },
|
||||
],
|
||||
messages: [
|
||||
{ role: "user", content: [{ type: "text", text: "first user" }] },
|
||||
{ role: "assistant", content: [{ type: "text", text: "assistant reply" }] },
|
||||
@@ -120,7 +126,10 @@ describe("applyCachePolicy", () => {
|
||||
const prepared = yield* LLMClient.prepare(
|
||||
LLM.request({
|
||||
model: bedrockModel,
|
||||
system: "Sys",
|
||||
system: [
|
||||
{ type: "text", text: "Base agent" },
|
||||
{ type: "text", text: "Project instructions" },
|
||||
],
|
||||
tools: [{ name: "t1", description: "t1", inputSchema: { type: "object", properties: {} } }],
|
||||
messages: [Message.user("first user"), Message.assistant("reply"), Message.user("latest user")],
|
||||
cache: "auto",
|
||||
@@ -131,7 +140,12 @@ describe("applyCachePolicy", () => {
|
||||
toolConfig: {
|
||||
tools: [{ toolSpec: { name: "t1" } }, { cachePoint: { type: "default" } }],
|
||||
},
|
||||
system: [{ text: "Sys" }, { cachePoint: { type: "default" } }],
|
||||
system: [
|
||||
{ text: "Base agent" },
|
||||
{ cachePoint: { type: "default" } },
|
||||
{ text: "Project instructions" },
|
||||
{ cachePoint: { type: "default" } },
|
||||
],
|
||||
messages: [
|
||||
{ role: "user", content: [{ text: "first user" }] },
|
||||
{ role: "assistant", content: [{ text: "reply" }] },
|
||||
@@ -193,9 +207,55 @@ describe("applyCachePolicy", () => {
|
||||
}),
|
||||
)
|
||||
|
||||
const body = prepared.body as { system: Array<{ text: string; cache_control?: unknown }> }
|
||||
const body = prepared.body as {
|
||||
system: Array<{ text: string; cache_control?: unknown }>
|
||||
messages: Array<{ content: Array<{ cache_control?: unknown }> }>
|
||||
}
|
||||
expect(body.system[0]?.cache_control).toEqual({ type: "ephemeral", ttl: "1h" })
|
||||
expect(body.system[1]?.cache_control).toEqual({ type: "ephemeral" })
|
||||
expect(body.messages[0]?.content[0]?.cache_control).toEqual({ type: "ephemeral" })
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("auto policy stays within the four-breakpoint cap when preserving manual hints", () =>
|
||||
Effect.gen(function* () {
|
||||
const request = LLM.request({
|
||||
model: anthropicModel,
|
||||
system: [
|
||||
{ type: "text", text: "Base agent" },
|
||||
{
|
||||
type: "text",
|
||||
text: "Manual context",
|
||||
cache: new CacheHint({ type: "ephemeral", ttlSeconds: 3600 }),
|
||||
},
|
||||
{ type: "text", text: "Project instructions" },
|
||||
],
|
||||
tools: [{ name: "t1", description: "t1", inputSchema: { type: "object", properties: {} } }],
|
||||
prompt: "hi",
|
||||
cache: "auto",
|
||||
})
|
||||
const applied = applyCachePolicy(request)
|
||||
expect(applied.tools[0]?.cache).toBeDefined()
|
||||
expect(applied.system.map((part) => part.cache !== undefined)).toEqual([true, true, true])
|
||||
const tail = applied.messages[0]!.content[0]!
|
||||
expect("cache" in tail ? tail.cache : undefined).toBeUndefined()
|
||||
expect(applyCachePolicy(applied)).toBe(applied)
|
||||
|
||||
const prepared = yield* LLMClient.prepare(request)
|
||||
|
||||
const body = prepared.body as {
|
||||
tools: Array<{ cache_control?: unknown }>
|
||||
system: Array<{ cache_control?: unknown }>
|
||||
messages: Array<{ content: Array<{ cache_control?: unknown }> }>
|
||||
}
|
||||
const marked = [
|
||||
...body.tools.map((tool) => tool.cache_control),
|
||||
...body.system.map((part) => part.cache_control),
|
||||
...body.messages.flatMap((message) => message.content.map((part) => part.cache_control)),
|
||||
].filter((cache) => cache !== undefined)
|
||||
expect(marked).toHaveLength(4)
|
||||
expect(body.system[1]?.cache_control).toEqual({ type: "ephemeral", ttl: "1h" })
|
||||
expect(body.messages[0]?.content[0]?.cache_control).toBeUndefined()
|
||||
}),
|
||||
)
|
||||
|
||||
|
||||
@@ -11,7 +11,13 @@ import {
|
||||
XAI,
|
||||
} from "@opencode-ai/ai/providers"
|
||||
import * as GitHubCopilot from "@opencode-ai/ai/providers/github-copilot"
|
||||
import { OpenAIChat, OpenAICompatibleChat, OpenAICompatibleResponses, OpenAIResponses } from "@opencode-ai/ai/protocols"
|
||||
import {
|
||||
OpenAIChat,
|
||||
OpenAICompatibleChat,
|
||||
OpenAICompatibleResponses,
|
||||
OpenAIResponses,
|
||||
OpenResponses,
|
||||
} from "@opencode-ai/ai/protocols"
|
||||
import * as AnthropicMessages from "@opencode-ai/ai/protocols/anthropic-messages"
|
||||
|
||||
describe("public exports", () => {
|
||||
@@ -74,7 +80,9 @@ describe("public exports", () => {
|
||||
test("protocol barrels expose supported low-level routes", () => {
|
||||
expect(OpenAIChat.route.id).toBe("openai-chat")
|
||||
expect(OpenAICompatibleChat.route.id).toBe("openai-compatible-chat")
|
||||
expect(OpenResponses.protocol.id).toBe("open-responses")
|
||||
expect(OpenAICompatibleResponses.route.id).toBe("openai-compatible-responses")
|
||||
expect(OpenAICompatibleResponses.route.protocol).toBe("open-responses")
|
||||
expect(OpenAIResponses.route.id).toBe("openai-responses")
|
||||
expect(OpenAIResponses.webSocketRoute.id).toBe("openai-responses-websocket")
|
||||
expect(AnthropicMessages.route.id).toBe("anthropic-messages")
|
||||
|
||||
+54
File diff suppressed because one or more lines are too long
+53
File diff suppressed because one or more lines are too long
@@ -83,7 +83,7 @@ const indexStep = (event: LLMEvent, index: number): LLMEvent => {
|
||||
const stepState = (events: ReadonlyArray<LLMEvent>) => {
|
||||
const assistantContent: ContentPart[] = []
|
||||
const toolCalls: ToolCallPart[] = []
|
||||
let reason: Extract<LLMEvent, { type: "finish" }>["reason"] = "unknown"
|
||||
let reason: Extract<LLMEvent, { type: "finish" }>["reason"] = { normalized: "unknown" }
|
||||
let usage: Usage | undefined
|
||||
let providerMetadata: ProviderMetadata | undefined
|
||||
|
||||
|
||||
@@ -2,7 +2,16 @@ import { describe, expect, test } from "bun:test"
|
||||
import { CacheHint, LLM, LLMResponse } from "../src"
|
||||
import * as OpenAIChat from "../src/protocols/openai-chat"
|
||||
import * as OpenAIResponses from "../src/protocols/openai-responses"
|
||||
import { LLMRequest, Message, Model, ToolCallPart, ToolChoice, ToolDefinition, ToolResultPart } from "../src/schema"
|
||||
import {
|
||||
GenerationOptions,
|
||||
LLMRequest,
|
||||
Message,
|
||||
Model,
|
||||
ToolCallPart,
|
||||
ToolChoice,
|
||||
ToolDefinition,
|
||||
ToolResultPart,
|
||||
} from "../src/schema"
|
||||
|
||||
const chatRoute = OpenAIChat.route
|
||||
const responsesRoute = OpenAIResponses.route
|
||||
@@ -31,8 +40,8 @@ describe("llm constructors", () => {
|
||||
model: Model.make({ id: "fake-model", provider: "fake", route: chatRoute }),
|
||||
prompt: "Say hello.",
|
||||
})
|
||||
const updated = LLM.updateRequest(base, {
|
||||
generation: { maxTokens: 20 },
|
||||
const updated = LLMRequest.update(base, {
|
||||
generation: GenerationOptions.make({ maxTokens: 20 }),
|
||||
messages: [...base.messages, Message.assistant("Hi.")],
|
||||
})
|
||||
|
||||
@@ -191,7 +200,7 @@ describe("llm constructors", () => {
|
||||
LLMResponse.text({
|
||||
events: [
|
||||
{ type: "text-delta", id: "text-0", text: "hi" },
|
||||
{ type: "finish", reason: "stop" },
|
||||
{ type: "finish", reason: { normalized: "stop" } },
|
||||
],
|
||||
}),
|
||||
).toBe("hi")
|
||||
|
||||
@@ -59,7 +59,7 @@ describe("provider package entrypoints", () => {
|
||||
headers: { "x-application": "opencode" },
|
||||
body: { service_tier: "priority" },
|
||||
limits: { context: 200_000, output: 64_000 },
|
||||
providerOptions: { openai: { reasoningEffort: "low", store: true } },
|
||||
providerOptions: { openresponses: { reasoningEffort: "low", store: true } },
|
||||
})
|
||||
|
||||
expect(String(selected.provider)).toBe("example")
|
||||
@@ -72,7 +72,7 @@ describe("provider package entrypoints", () => {
|
||||
expect(selected.route.defaults.http?.body).toEqual({ service_tier: "priority" })
|
||||
expect(selected.route.defaults.limits).toEqual({ context: 200_000, output: 64_000 })
|
||||
expect(selected.route.defaults.providerOptions).toEqual({
|
||||
openai: { reasoningEffort: "low", store: true },
|
||||
openresponses: { reasoningEffort: "low", store: true },
|
||||
})
|
||||
})
|
||||
|
||||
@@ -85,6 +85,7 @@ describe("provider package entrypoints", () => {
|
||||
headers: { "x-application": "opencode" },
|
||||
body: { metadata: { user_id: "user_1" } },
|
||||
limits: { context: 200_000, output: 64_000 },
|
||||
providerOptions: { anthropic: { effort: "low" } },
|
||||
})
|
||||
|
||||
expect(String(selected.provider)).toBe("example")
|
||||
@@ -96,6 +97,19 @@ describe("provider package entrypoints", () => {
|
||||
expect(selected.route.defaults.headers).toEqual({ "x-application": "opencode" })
|
||||
expect(selected.route.defaults.http?.body).toEqual({ metadata: { user_id: "user_1" } })
|
||||
expect(selected.route.defaults.limits).toEqual({ context: 200_000, output: 64_000 })
|
||||
expect(selected.route.defaults.providerOptions).toEqual({ anthropic: { effort: "low" } })
|
||||
})
|
||||
|
||||
test("maps Anthropic provider options onto the executable model", async () => {
|
||||
const Anthropic = await import("@opencode-ai/ai/providers/anthropic")
|
||||
const selected = Anthropic.model("claude-sonnet-4-6", {
|
||||
apiKey: "fixture",
|
||||
providerOptions: { anthropic: { thinking: { type: "adaptive" } } },
|
||||
})
|
||||
|
||||
expect(selected.route.defaults.providerOptions).toEqual({
|
||||
anthropic: { thinking: { type: "adaptive" } },
|
||||
})
|
||||
})
|
||||
|
||||
test("requires an Anthropic-compatible base URL at runtime", async () => {
|
||||
@@ -235,12 +249,12 @@ describe("provider package entrypoints", () => {
|
||||
path: "/chat/completions",
|
||||
})
|
||||
expect(responses.route.id).toBe("google-vertex-responses")
|
||||
expect(responses.route.protocol).toBe("openai-responses")
|
||||
expect(responses.route.protocol).toBe("open-responses")
|
||||
expect(responses.route.endpoint).toMatchObject({
|
||||
baseURL: "https://aiplatform.googleapis.com/v1/projects/vertex-project/locations/global/endpoints/openapi",
|
||||
path: "/responses",
|
||||
})
|
||||
expect(responses.route.defaults.providerOptions).toEqual({ openai: { store: false } })
|
||||
expect(responses.route.defaults.providerOptions).toEqual({ openresponses: { store: false } })
|
||||
})
|
||||
|
||||
test("rejects conflicting Vertex auth settings at runtime", async () => {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect } from "effect"
|
||||
import { CacheHint, LLM } from "../../src"
|
||||
import { CacheHint, LLM, LLMRequest, Message, ToolCallPart, ToolDefinition } from "../../src"
|
||||
import { LLMClient } from "../../src/route"
|
||||
import * as Anthropic from "../../src/providers/anthropic"
|
||||
import { LARGE_CACHEABLE_SYSTEM } from "../recorded-scenarios"
|
||||
@@ -24,6 +24,39 @@ const cacheRequest = LLM.request({
|
||||
generation: { maxTokens: 16, temperature: 0 },
|
||||
})
|
||||
|
||||
const lookup = ToolDefinition.make({
|
||||
name: "lookup",
|
||||
description: "Look up a fixture value.",
|
||||
inputSchema: {
|
||||
type: "object",
|
||||
properties: { index: { type: "number" } },
|
||||
required: ["index"],
|
||||
additionalProperties: false,
|
||||
},
|
||||
})
|
||||
const longToolTurn = [
|
||||
Message.user("Run the fixture lookups."),
|
||||
...Array.from({ length: 11 }, (_, index) => {
|
||||
const id = `lookup_${index}`
|
||||
return [
|
||||
Message.assistant(ToolCallPart.make({ id, name: lookup.name, input: { index } })),
|
||||
Message.tool({
|
||||
id,
|
||||
name: lookup.name,
|
||||
result: `Fixture result ${index}. `.repeat(80),
|
||||
}),
|
||||
]
|
||||
}).flat(),
|
||||
]
|
||||
const longToolTurnRequest = LLM.request({
|
||||
id: "recorded_anthropic_cache_long_tool_turn",
|
||||
model,
|
||||
system: LARGE_CACHEABLE_SYSTEM,
|
||||
messages: longToolTurn,
|
||||
tools: [lookup],
|
||||
generation: { maxTokens: 16, temperature: 0 },
|
||||
})
|
||||
|
||||
const recorded = recordedTests({
|
||||
prefix: "anthropic-messages-cache",
|
||||
provider: "anthropic",
|
||||
@@ -50,4 +83,28 @@ describe("Anthropic Messages cache recorded", () => {
|
||||
expect(second.usage?.cacheReadInputTokens ?? 0).toBeGreaterThan(0)
|
||||
}),
|
||||
)
|
||||
|
||||
recorded.effect.with("keeps a long tool turn inside the cache lookback", { tags: ["cache", "tool"] }, () =>
|
||||
Effect.gen(function* () {
|
||||
const first = yield* LLMClient.generate(longToolTurnRequest)
|
||||
const firstRead = first.usage?.cacheReadInputTokens ?? 0
|
||||
const firstWrite = first.usage?.cacheWriteInputTokens ?? 0
|
||||
const firstCached = firstRead + firstWrite
|
||||
// The prefix may already be warm when recording, so either a read or a
|
||||
// write establishes that Anthropic recognized the cache boundary.
|
||||
expect(firstCached).toBeGreaterThan(0)
|
||||
|
||||
const second = yield* LLMClient.generate(
|
||||
LLMRequest.update(longToolTurnRequest, {
|
||||
messages: [
|
||||
...longToolTurn,
|
||||
Message.assistant("The fixture lookups are complete."),
|
||||
Message.user("Reply exactly: OK"),
|
||||
],
|
||||
}),
|
||||
)
|
||||
expect(second.usage?.cacheReadInputTokens ?? 0).toBeGreaterThanOrEqual(firstCached)
|
||||
expect(second.usage?.cacheWriteInputTokens ?? 0).toBeLessThan(firstCached)
|
||||
}),
|
||||
)
|
||||
})
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect } from "effect"
|
||||
import { HttpClientRequest } from "effect/unstable/http"
|
||||
import { CacheHint, LLM, LLMError, Message, ToolCallPart, Usage } from "../../src"
|
||||
import { CacheHint, LLM, LLMError, LLMRequest, Message, ToolCallPart, ToolDefinition, Usage } from "../../src"
|
||||
import { Auth, LLMClient } from "../../src/route"
|
||||
import * as AnthropicMessages from "../../src/protocols/anthropic-messages"
|
||||
import { continuationRequest, nativeAnthropicMessagesContinuation } from "../continuation-scenarios"
|
||||
@@ -60,7 +60,7 @@ describe("Anthropic Messages route", () => {
|
||||
it.effect("lowers adaptive thinking settings with effort", () =>
|
||||
Effect.gen(function* () {
|
||||
const prepared = yield* LLMClient.prepare<AnthropicMessages.AnthropicMessagesBody>(
|
||||
LLM.updateRequest(request, {
|
||||
LLMRequest.update(request, {
|
||||
providerOptions: {
|
||||
anthropic: { thinking: { type: "adaptive", display: "summarized" }, effort: "low" },
|
||||
},
|
||||
@@ -74,6 +74,42 @@ describe("Anthropic Messages route", () => {
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("normalizes enabled and disabled thinking settings", () =>
|
||||
Effect.gen(function* () {
|
||||
const enabled = yield* LLMClient.prepare<AnthropicMessages.AnthropicMessagesBody>(
|
||||
LLMRequest.update(request, {
|
||||
providerOptions: { anthropic: { thinking: { type: "enabled", budgetTokens: 1_024 } } },
|
||||
}),
|
||||
)
|
||||
const legacy = yield* LLMClient.prepare<AnthropicMessages.AnthropicMessagesBody>(
|
||||
LLMRequest.update(request, {
|
||||
providerOptions: { anthropic: { thinking: { type: "enabled", budget_tokens: 2_048 } } },
|
||||
}),
|
||||
)
|
||||
const disabled = yield* LLMClient.prepare<AnthropicMessages.AnthropicMessagesBody>(
|
||||
LLMRequest.update(request, {
|
||||
providerOptions: { anthropic: { thinking: { type: "disabled" } } },
|
||||
}),
|
||||
)
|
||||
|
||||
expect(enabled.body.thinking).toEqual({ type: "enabled", budget_tokens: 1_024 })
|
||||
expect(legacy.body.thinking).toEqual({ type: "enabled", budget_tokens: 2_048 })
|
||||
expect(disabled.body.thinking).toEqual({ type: "disabled" })
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("rejects enabled thinking without a budget", () =>
|
||||
Effect.gen(function* () {
|
||||
const error = yield* LLMClient.prepare(
|
||||
LLMRequest.update(request, {
|
||||
providerOptions: { anthropic: { thinking: { type: "enabled" } } },
|
||||
}),
|
||||
).pipe(Effect.flip)
|
||||
|
||||
expect(error.message).toContain("Anthropic thinking provider option requires budgetTokens")
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("lowers chronological system updates natively for Claude Opus 4.8 with cache hints", () =>
|
||||
Effect.gen(function* () {
|
||||
const prepared = yield* LLMClient.prepare<AnthropicMessages.AnthropicMessagesBody>(
|
||||
@@ -235,6 +271,34 @@ describe("Anthropic Messages route", () => {
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("keeps tools and sends tool_choice none", () =>
|
||||
Effect.gen(function* () {
|
||||
const prepared = yield* LLMClient.prepare<AnthropicMessages.AnthropicMessagesBody>(
|
||||
LLM.request({
|
||||
id: "req_tool_choice_none",
|
||||
model,
|
||||
tools: [{ name: "lookup", description: "Look things up", inputSchema: { type: "object", properties: {} } }],
|
||||
messages: [
|
||||
Message.user("What is the weather?"),
|
||||
Message.assistant([ToolCallPart.make({ id: "call_1", name: "lookup", input: { query: "weather" } })]),
|
||||
Message.tool({ id: "call_1", name: "lookup", result: { forecast: "sunny" } }),
|
||||
],
|
||||
toolChoice: "none",
|
||||
cache: "none",
|
||||
}),
|
||||
)
|
||||
|
||||
expect(prepared.body.tools).toEqual([
|
||||
{
|
||||
name: "lookup",
|
||||
description: "Look things up",
|
||||
input_schema: { type: "object", properties: {} },
|
||||
},
|
||||
])
|
||||
expect(prepared.body.tool_choice).toEqual({ type: "none" })
|
||||
}),
|
||||
)
|
||||
|
||||
// Regression: read tool results must stay structured so base64 media data is
|
||||
// not JSON-stringified into `tool_result.content`.
|
||||
it.effect("lowers media tool-result content as structured blocks", () =>
|
||||
@@ -381,6 +445,34 @@ describe("Anthropic Messages route", () => {
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("round-trips redacted thinking as redacted_thinking blocks", () =>
|
||||
Effect.gen(function* () {
|
||||
const prepared = yield* LLMClient.prepare(
|
||||
LLM.request({
|
||||
model,
|
||||
messages: [
|
||||
Message.assistant([
|
||||
{ type: "reasoning", text: "", providerMetadata: { anthropic: { redactedData: "opaque_1" } } },
|
||||
{ type: "reasoning", text: "visible", providerMetadata: { anthropic: { signature: "sig_1" } } },
|
||||
]),
|
||||
],
|
||||
}),
|
||||
)
|
||||
|
||||
expect(prepared.body).toMatchObject({
|
||||
messages: [
|
||||
{
|
||||
role: "assistant",
|
||||
content: [
|
||||
{ type: "redacted_thinking", data: "opaque_1" },
|
||||
{ type: "thinking", thinking: "visible", signature: "sig_1" },
|
||||
],
|
||||
},
|
||||
],
|
||||
})
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("parses text, reasoning, and usage stream fixtures", () =>
|
||||
Effect.gen(function* () {
|
||||
const body = sseEvents(
|
||||
@@ -414,18 +506,291 @@ describe("Anthropic Messages route", () => {
|
||||
expect(response.events.find((event) => event.type === "reasoning-end")).toMatchObject({
|
||||
providerMetadata: { anthropic: { signature: "sig_1" } },
|
||||
})
|
||||
expect(response.events.find((event) => event.type === "reasoning-delta" && event.text === "")).toBeUndefined()
|
||||
expect(response.message.content).toEqual([
|
||||
{ type: "text", text: "Hello!" },
|
||||
{ type: "reasoning", text: "thinking", providerMetadata: { anthropic: { signature: "sig_1" } } },
|
||||
])
|
||||
expect(response.events.at(-1)).toMatchObject({
|
||||
type: "finish",
|
||||
reason: "stop",
|
||||
reason: { normalized: "stop", raw: "end_turn" },
|
||||
providerMetadata: { anthropic: { stopSequence: "\n\nHuman:" } },
|
||||
})
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("requires message_stop before completing a streamed message", () =>
|
||||
Effect.gen(function* () {
|
||||
const error = yield* LLMClient.generate(request).pipe(
|
||||
Effect.provide(
|
||||
fixedResponse(
|
||||
sseEvents(
|
||||
{ type: "message_start", message: { usage: { input_tokens: 5 } } },
|
||||
{ type: "content_block_start", index: 0, content_block: { type: "text", text: "" } },
|
||||
{ type: "content_block_delta", index: 0, delta: { type: "text_delta", text: "Hello" } },
|
||||
{ type: "content_block_stop", index: 0 },
|
||||
{ type: "message_delta", delta: { stop_reason: "end_turn" }, usage: { output_tokens: 1 } },
|
||||
),
|
||||
),
|
||||
),
|
||||
Effect.flip,
|
||||
)
|
||||
|
||||
expect(error.reason).toMatchObject({
|
||||
_tag: "InvalidProviderOutput",
|
||||
message: "Provider stream ended without a terminal finish event",
|
||||
})
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("round-trips omitted thinking carried only by a signature delta", () =>
|
||||
Effect.gen(function* () {
|
||||
const response = yield* LLMClient.generate(request).pipe(
|
||||
Effect.provide(
|
||||
fixedResponse(
|
||||
sseEvents(
|
||||
{ type: "message_start", message: { usage: { input_tokens: 5 } } },
|
||||
{
|
||||
type: "content_block_start",
|
||||
index: 0,
|
||||
content_block: { type: "thinking", thinking: "", signature: "" },
|
||||
},
|
||||
{ type: "content_block_delta", index: 0, delta: { type: "signature_delta", signature: "sig_1" } },
|
||||
{ type: "content_block_stop", index: 0 },
|
||||
{ type: "message_delta", delta: { stop_reason: "end_turn" }, usage: { output_tokens: 1 } },
|
||||
{ type: "message_stop" },
|
||||
),
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
expect(response.message.content).toEqual([
|
||||
{ type: "reasoning", text: "", providerMetadata: { anthropic: { signature: "sig_1" } } },
|
||||
])
|
||||
|
||||
const prepared = yield* LLMClient.prepare<AnthropicMessages.AnthropicMessagesBody>(
|
||||
LLM.request({ model, messages: [response.message], cache: "none" }),
|
||||
)
|
||||
expect(prepared.body.messages).toEqual([
|
||||
{ role: "assistant", content: [{ type: "thinking", thinking: "", signature: "sig_1" }] },
|
||||
])
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("retains a thinking signature supplied in content_block_start", () =>
|
||||
Effect.gen(function* () {
|
||||
const response = yield* LLMClient.generate(request).pipe(
|
||||
Effect.provide(
|
||||
fixedResponse(
|
||||
sseEvents(
|
||||
{ type: "message_start", message: { usage: { input_tokens: 5 } } },
|
||||
{
|
||||
type: "content_block_start",
|
||||
index: 0,
|
||||
content_block: { type: "thinking", thinking: "", signature: "sig_1" },
|
||||
},
|
||||
{ type: "content_block_stop", index: 0 },
|
||||
{ type: "message_delta", delta: { stop_reason: "end_turn" }, usage: { output_tokens: 1 } },
|
||||
{ type: "message_stop" },
|
||||
),
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
expect(response.message.content).toEqual([
|
||||
{ type: "reasoning", text: "", providerMetadata: { anthropic: { signature: "sig_1" } } },
|
||||
])
|
||||
expect(response.events.find((event) => event.type === "reasoning-end")).toMatchObject({
|
||||
providerMetadata: { anthropic: { signature: "sig_1" } },
|
||||
})
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("retains complete tool input from content_block_start", () =>
|
||||
Effect.gen(function* () {
|
||||
const response = yield* LLMClient.generate(request).pipe(
|
||||
Effect.provide(
|
||||
fixedResponse(
|
||||
sseEvents(
|
||||
{ type: "message_start", message: { usage: { input_tokens: 5 } } },
|
||||
{
|
||||
type: "content_block_start",
|
||||
index: 0,
|
||||
content_block: { type: "tool_use", id: "call_1", name: "lookup", input: { query: "weather" } },
|
||||
},
|
||||
{ type: "content_block_stop", index: 0 },
|
||||
{ type: "message_delta", delta: { stop_reason: "tool_use" }, usage: { output_tokens: 1 } },
|
||||
{ type: "message_stop" },
|
||||
),
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
expect(response.toolCalls).toMatchObject([
|
||||
{ id: "call_1", name: "lookup", input: { query: "weather" } },
|
||||
])
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("retains empty text blocks", () =>
|
||||
Effect.gen(function* () {
|
||||
const response = yield* LLMClient.generate(request).pipe(
|
||||
Effect.provide(
|
||||
fixedResponse(
|
||||
sseEvents(
|
||||
{ type: "message_start", message: { usage: { input_tokens: 5 } } },
|
||||
{ type: "content_block_start", index: 0, content_block: { type: "text", text: "" } },
|
||||
{ type: "content_block_stop", index: 0 },
|
||||
{ type: "message_delta", delta: { stop_reason: "end_turn" }, usage: { output_tokens: 1 } },
|
||||
{ type: "message_stop" },
|
||||
),
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
expect(response.message.content).toEqual([{ type: "text", text: "" }])
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("parses redacted thinking into empty reasoning with redactedData metadata", () =>
|
||||
Effect.gen(function* () {
|
||||
const body = sseEvents(
|
||||
{ type: "message_start", message: { usage: { input_tokens: 5 } } },
|
||||
{ type: "content_block_start", index: 0, content_block: { type: "redacted_thinking", data: "opaque_1" } },
|
||||
{ type: "content_block_stop", index: 0 },
|
||||
{ type: "content_block_start", index: 1, content_block: { type: "text", text: "" } },
|
||||
{ type: "content_block_delta", index: 1, delta: { type: "text_delta", text: "Hello" } },
|
||||
{ type: "content_block_stop", index: 1 },
|
||||
{ type: "message_delta", delta: { stop_reason: "end_turn" }, usage: { output_tokens: 2 } },
|
||||
{ type: "message_stop" },
|
||||
)
|
||||
const response = yield* LLMClient.generate(request).pipe(Effect.provide(fixedResponse(body)))
|
||||
|
||||
expect(response.events.find((event) => event.type === "reasoning-start")).toMatchObject({
|
||||
providerMetadata: { anthropic: { redactedData: "opaque_1" } },
|
||||
})
|
||||
expect(response.message.content).toEqual([
|
||||
{ type: "reasoning", text: "", providerMetadata: { anthropic: { redactedData: "opaque_1" } } },
|
||||
{ type: "text", text: "Hello" },
|
||||
])
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("round-trips streamed redacted thinking with tool use into a continuation request", () =>
|
||||
Effect.gen(function* () {
|
||||
// Anthropic types `redacted_thinking.data` as an opaque string. Its
|
||||
// contents are provider-owned and must be replayed without inspection.
|
||||
const redactedData = "cmVkYWN0ZWQtdGhpbmtpbmc="
|
||||
const response = yield* LLMClient.generate(
|
||||
LLMRequest.update(request, {
|
||||
tools: [ToolDefinition.make({ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } })],
|
||||
}),
|
||||
).pipe(
|
||||
Effect.provide(
|
||||
fixedResponse(
|
||||
sseEvents(
|
||||
{ type: "message_start", message: { usage: { input_tokens: 5 } } },
|
||||
{
|
||||
type: "content_block_start",
|
||||
index: 0,
|
||||
content_block: { type: "redacted_thinking", data: redactedData },
|
||||
},
|
||||
{ type: "content_block_stop", index: 0 },
|
||||
{
|
||||
type: "content_block_start",
|
||||
index: 1,
|
||||
content_block: { type: "tool_use", id: "call_1", name: "lookup" },
|
||||
},
|
||||
{
|
||||
type: "content_block_delta",
|
||||
index: 1,
|
||||
delta: { type: "input_json_delta", partial_json: '{"query":"weather"}' },
|
||||
},
|
||||
{ type: "content_block_stop", index: 1 },
|
||||
{ type: "message_delta", delta: { stop_reason: "tool_use" }, usage: { output_tokens: 1 } },
|
||||
{ type: "message_stop" },
|
||||
),
|
||||
),
|
||||
),
|
||||
)
|
||||
const prepared = yield* LLMClient.prepare<AnthropicMessages.AnthropicMessagesBody>(
|
||||
LLM.request({
|
||||
model,
|
||||
messages: [
|
||||
Message.user("Say hello."),
|
||||
response.message,
|
||||
Message.tool({ id: "call_1", name: "lookup", result: "sunny", resultType: "text" }),
|
||||
],
|
||||
tools: [ToolDefinition.make({ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } })],
|
||||
cache: "none",
|
||||
}),
|
||||
)
|
||||
|
||||
expect(prepared.body.messages).toEqual([
|
||||
{ role: "user", content: [{ type: "text", text: "Say hello." }] },
|
||||
{
|
||||
role: "assistant",
|
||||
content: [
|
||||
{ type: "redacted_thinking", data: redactedData },
|
||||
{ type: "tool_use", id: "call_1", name: "lookup", input: { query: "weather" } },
|
||||
],
|
||||
},
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{
|
||||
type: "tool_result",
|
||||
tool_use_id: "call_1",
|
||||
content: "sunny",
|
||||
is_error: undefined,
|
||||
cache_control: undefined,
|
||||
},
|
||||
],
|
||||
},
|
||||
])
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("maps context-window truncation to length", () =>
|
||||
Effect.gen(function* () {
|
||||
const response = yield* LLMClient.generate(request).pipe(
|
||||
Effect.provide(
|
||||
fixedResponse(
|
||||
sseEvents(
|
||||
{ type: "message_start", message: { usage: { input_tokens: 5 } } },
|
||||
{
|
||||
type: "message_delta",
|
||||
delta: { stop_reason: "model_context_window_exceeded" },
|
||||
usage: { output_tokens: 1 },
|
||||
},
|
||||
{ type: "message_stop" },
|
||||
),
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
expect(response.finishReason).toEqual({ normalized: "length", raw: "model_context_window_exceeded" })
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("preserves pause_turn while normalizing it to stop", () =>
|
||||
Effect.gen(function* () {
|
||||
const response = yield* LLMClient.generate(request).pipe(
|
||||
Effect.provide(
|
||||
fixedResponse(
|
||||
sseEvents(
|
||||
{ type: "message_start", message: { usage: { input_tokens: 5 } } },
|
||||
{ type: "message_delta", delta: { stop_reason: "pause_turn" }, usage: { output_tokens: 1 } },
|
||||
{ type: "message_stop" },
|
||||
),
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
expect(response.finishReason).toEqual({ normalized: "stop", raw: "pause_turn" })
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("assembles streamed tool call input", () =>
|
||||
Effect.gen(function* () {
|
||||
const body = sseEvents(
|
||||
@@ -435,10 +800,11 @@ describe("Anthropic Messages route", () => {
|
||||
{ type: "content_block_delta", index: 0, delta: { type: "input_json_delta", partial_json: ':"weather"}' } },
|
||||
{ type: "content_block_stop", index: 0 },
|
||||
{ type: "message_delta", delta: { stop_reason: "tool_use" }, usage: { output_tokens: 1 } },
|
||||
{ type: "message_stop" },
|
||||
)
|
||||
const response = yield* LLMClient.generate(
|
||||
LLM.updateRequest(request, {
|
||||
tools: [{ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } }],
|
||||
LLMRequest.update(request, {
|
||||
tools: [ToolDefinition.make({ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } })],
|
||||
}),
|
||||
).pipe(Effect.provide(fixedResponse(body)))
|
||||
const usage = new Usage({
|
||||
@@ -475,10 +841,16 @@ describe("Anthropic Messages route", () => {
|
||||
providerExecuted: undefined,
|
||||
providerMetadata: undefined,
|
||||
},
|
||||
{ type: "step-finish", index: 0, reason: "tool-calls", usage, providerMetadata: undefined },
|
||||
{
|
||||
type: "step-finish",
|
||||
index: 0,
|
||||
reason: { normalized: "tool-calls", raw: "tool_use" },
|
||||
usage,
|
||||
providerMetadata: undefined,
|
||||
},
|
||||
{
|
||||
type: "finish",
|
||||
reason: "tool-calls",
|
||||
reason: { normalized: "tool-calls", raw: "tool_use" },
|
||||
providerMetadata: undefined,
|
||||
usage,
|
||||
},
|
||||
@@ -614,10 +986,13 @@ describe("Anthropic Messages route", () => {
|
||||
{ type: "content_block_delta", index: 2, delta: { type: "text_delta", text: "Found it." } },
|
||||
{ type: "content_block_stop", index: 2 },
|
||||
{ type: "message_delta", delta: { stop_reason: "end_turn" }, usage: { output_tokens: 8 } },
|
||||
{ type: "message_stop" },
|
||||
)
|
||||
const response = yield* LLMClient.generate(
|
||||
LLM.updateRequest(request, {
|
||||
tools: [{ name: "web_search", description: "Web search", inputSchema: { type: "object" } }],
|
||||
LLMRequest.update(request, {
|
||||
tools: [
|
||||
ToolDefinition.make({ name: "web_search", description: "Web search", inputSchema: { type: "object" } }),
|
||||
],
|
||||
}),
|
||||
).pipe(Effect.provide(fixedResponse(body)))
|
||||
|
||||
@@ -636,10 +1011,20 @@ describe("Anthropic Messages route", () => {
|
||||
name: "web_search",
|
||||
result: { type: "json", value: [{ type: "web_search_result", url: "https://example.com", title: "Example" }] },
|
||||
providerExecuted: true,
|
||||
providerMetadata: { anthropic: { blockType: "web_search_tool_result" } },
|
||||
// The complete payload rides in provider metadata as irreducible replay
|
||||
// state for later stateless requests.
|
||||
providerMetadata: {
|
||||
anthropic: {
|
||||
blockType: "web_search_tool_result",
|
||||
result: [{ type: "web_search_result", url: "https://example.com", title: "Example" }],
|
||||
},
|
||||
},
|
||||
})
|
||||
expect(response.text).toBe("Found it.")
|
||||
expect(response.events.at(-1)).toMatchObject({ type: "finish", reason: "stop" })
|
||||
expect(response.events.at(-1)).toMatchObject({
|
||||
type: "finish",
|
||||
reason: { normalized: "stop", raw: "end_turn" },
|
||||
})
|
||||
}),
|
||||
)
|
||||
|
||||
@@ -665,10 +1050,13 @@ describe("Anthropic Messages route", () => {
|
||||
},
|
||||
{ type: "content_block_stop", index: 1 },
|
||||
{ type: "message_delta", delta: { stop_reason: "end_turn" }, usage: { output_tokens: 1 } },
|
||||
{ type: "message_stop" },
|
||||
)
|
||||
const response = yield* LLMClient.generate(
|
||||
LLM.updateRequest(request, {
|
||||
tools: [{ name: "web_search", description: "Web search", inputSchema: { type: "object" } }],
|
||||
LLMRequest.update(request, {
|
||||
tools: [
|
||||
ToolDefinition.make({ name: "web_search", description: "Web search", inputSchema: { type: "object" } }),
|
||||
],
|
||||
}),
|
||||
).pipe(Effect.provide(fixedResponse(body)))
|
||||
|
||||
@@ -784,7 +1172,10 @@ describe("Anthropic Messages route", () => {
|
||||
content: [
|
||||
{ type: "text", text: "What is in this image?" },
|
||||
{ type: "image", source: { type: "base64", media_type: "image/png", data: "AAECAw==" } },
|
||||
{ type: "document", source: { type: "base64", media_type: "application/pdf", data: "JVBERi0xLjQ=" } },
|
||||
{
|
||||
type: "document",
|
||||
source: { type: "base64", media_type: "application/pdf", data: "JVBERi0xLjQ=" },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
|
||||
@@ -13,12 +13,8 @@ const RECORDING_REGION = process.env.BEDROCK_RECORDING_REGION ?? "us-east-1"
|
||||
// call wouldn't deterministically prove cache mapping works. Override with
|
||||
// BEDROCK_CACHE_MODEL_ID if your account has access elsewhere.
|
||||
const model = AmazonBedrock.configure({
|
||||
credentials: {
|
||||
region: RECORDING_REGION,
|
||||
accessKeyId: process.env.AWS_ACCESS_KEY_ID ?? "fixture",
|
||||
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY ?? "fixture",
|
||||
sessionToken: process.env.AWS_SESSION_TOKEN,
|
||||
},
|
||||
apiKey: process.env.AWS_BEARER_TOKEN_BEDROCK ?? "fixture",
|
||||
region: RECORDING_REGION,
|
||||
}).model(process.env.BEDROCK_CACHE_MODEL_ID ?? "us.anthropic.claude-haiku-4-5-20251001-v1:0")
|
||||
|
||||
const cacheRequest = LLM.request({
|
||||
@@ -36,7 +32,7 @@ const recorded = recordedTests({
|
||||
prefix: "bedrock-converse-cache",
|
||||
provider: "amazon-bedrock",
|
||||
protocol: "bedrock-converse",
|
||||
requires: ["AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY"],
|
||||
requires: ["AWS_BEARER_TOKEN_BEDROCK"],
|
||||
// Two identical requests in one cassette — replay walks the cassette in
|
||||
// recording order so the second call replays the cached-hit interaction.
|
||||
})
|
||||
@@ -45,10 +41,20 @@ describe("Bedrock Converse cache recorded", () => {
|
||||
recorded.effect.with("writes then reads cachePoint on identical second call", { tags: ["cache"] }, () =>
|
||||
Effect.gen(function* () {
|
||||
const first = yield* LLMClient.generate(cacheRequest)
|
||||
expect(first.usage?.cacheReadInputTokens ?? 0).toBeGreaterThanOrEqual(0)
|
||||
expect(first.usage?.cacheWriteInputTokens ?? 0).toBeGreaterThan(0)
|
||||
expect(first.usage?.inputTokens).toBe(
|
||||
(first.usage?.nonCachedInputTokens ?? 0) +
|
||||
(first.usage?.cacheReadInputTokens ?? 0) +
|
||||
(first.usage?.cacheWriteInputTokens ?? 0),
|
||||
)
|
||||
|
||||
const second = yield* LLMClient.generate(cacheRequest)
|
||||
expect(second.usage?.cacheReadInputTokens ?? 0).toBeGreaterThan(0)
|
||||
expect(second.usage?.inputTokens).toBe(
|
||||
(second.usage?.nonCachedInputTokens ?? 0) +
|
||||
(second.usage?.cacheReadInputTokens ?? 0) +
|
||||
(second.usage?.cacheWriteInputTokens ?? 0),
|
||||
)
|
||||
}),
|
||||
)
|
||||
})
|
||||
|
||||
@@ -2,7 +2,16 @@ import { EventStreamCodec } from "@smithy/eventstream-codec"
|
||||
import { fromUtf8, toUtf8 } from "@smithy/util-utf8"
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect } from "effect"
|
||||
import { CacheHint, LLM, Message, ToolCallPart, ToolChoice } from "../../src"
|
||||
import {
|
||||
CacheHint,
|
||||
GenerationOptions,
|
||||
LLM,
|
||||
LLMRequest,
|
||||
Message,
|
||||
ToolCallPart,
|
||||
ToolChoice,
|
||||
ToolDefinition,
|
||||
} from "../../src"
|
||||
import { LLMClient } from "../../src/route"
|
||||
import { AmazonBedrock } from "../../src/providers"
|
||||
import * as BedrockConverse from "../../src/protocols/bedrock-converse"
|
||||
@@ -34,6 +43,26 @@ const eventFrame = (type: string, payload: object) =>
|
||||
body: utf8Encoder.encode(JSON.stringify(payload)),
|
||||
})
|
||||
|
||||
const exceptionFrame = (type: string, payload: object) =>
|
||||
codec.encode({
|
||||
headers: {
|
||||
":message-type": { type: "string", value: "exception" },
|
||||
":exception-type": { type: "string", value: type },
|
||||
":content-type": { type: "string", value: "application/json" },
|
||||
},
|
||||
body: utf8Encoder.encode(JSON.stringify(payload)),
|
||||
})
|
||||
|
||||
const errorFrame = (code: string, message: string) =>
|
||||
codec.encode({
|
||||
headers: {
|
||||
":message-type": { type: "string", value: "error" },
|
||||
":error-code": { type: "string", value: code },
|
||||
":error-message": { type: "string", value: message },
|
||||
},
|
||||
body: new Uint8Array(),
|
||||
})
|
||||
|
||||
const concat = (frames: ReadonlyArray<Uint8Array>) => {
|
||||
const total = frames.reduce((sum, frame) => sum + frame.length, 0)
|
||||
const out = new Uint8Array(total)
|
||||
@@ -86,7 +115,9 @@ describe("Bedrock Converse route", () => {
|
||||
it.effect("passes topK through additionalModelRequestFields as top_k", () =>
|
||||
Effect.gen(function* () {
|
||||
const prepared = yield* LLMClient.prepare<BedrockConverse.BedrockConverseBody>(
|
||||
LLM.updateRequest(baseRequest, { generation: { maxTokens: 64, temperature: 0, topK: 40 } }),
|
||||
LLMRequest.update(baseRequest, {
|
||||
generation: GenerationOptions.make({ maxTokens: 64, temperature: 0, topK: 40 }),
|
||||
}),
|
||||
)
|
||||
|
||||
// Converse's inferenceConfig has no topK; Anthropic/Nova read it from
|
||||
@@ -123,13 +154,13 @@ describe("Bedrock Converse route", () => {
|
||||
it.effect("prepares tool config with toolSpec and toolChoice", () =>
|
||||
Effect.gen(function* () {
|
||||
const prepared = yield* LLMClient.prepare(
|
||||
LLM.updateRequest(baseRequest, {
|
||||
LLMRequest.update(baseRequest, {
|
||||
tools: [
|
||||
{
|
||||
ToolDefinition.make({
|
||||
name: "lookup",
|
||||
description: "Lookup data",
|
||||
inputSchema: { type: "object", properties: { query: { type: "string" } }, required: ["query"] },
|
||||
},
|
||||
}),
|
||||
],
|
||||
toolChoice: ToolChoice.make({ type: "required" }),
|
||||
}),
|
||||
@@ -154,6 +185,36 @@ describe("Bedrock Converse route", () => {
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("keeps tools and omits the unsupported choice when tool choice is none", () =>
|
||||
Effect.gen(function* () {
|
||||
const prepared = yield* LLMClient.prepare(
|
||||
LLMRequest.update(baseRequest, {
|
||||
tools: [
|
||||
ToolDefinition.make({
|
||||
name: "lookup",
|
||||
description: "Lookup data",
|
||||
inputSchema: { type: "object", properties: { query: { type: "string" } } },
|
||||
}),
|
||||
],
|
||||
toolChoice: ToolChoice.make({ type: "none" }),
|
||||
}),
|
||||
)
|
||||
|
||||
expect(prepared.body.toolConfig).toMatchObject({
|
||||
tools: [
|
||||
{
|
||||
toolSpec: {
|
||||
name: "lookup",
|
||||
description: "Lookup data",
|
||||
inputSchema: { json: { type: "object", properties: { query: { type: "string" } } } },
|
||||
},
|
||||
},
|
||||
],
|
||||
})
|
||||
expect(prepared.body.toolConfig?.toolChoice).toBeUndefined()
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("lowers assistant tool-call + tool-result message history", () =>
|
||||
Effect.gen(function* () {
|
||||
const prepared = yield* LLMClient.prepare(
|
||||
@@ -260,7 +321,10 @@ describe("Bedrock Converse route", () => {
|
||||
// `metadata` (carries usage). We consolidate them into a single
|
||||
// terminal `finish` event with both.
|
||||
expect(finishes).toHaveLength(1)
|
||||
expect(finishes[0]).toMatchObject({ type: "finish", reason: "stop" })
|
||||
expect(finishes[0]).toMatchObject({
|
||||
type: "finish",
|
||||
reason: { normalized: "stop", raw: "end_turn" },
|
||||
})
|
||||
expect(response.usage).toMatchObject({
|
||||
inputTokens: 5,
|
||||
outputTokens: 2,
|
||||
@@ -269,6 +333,69 @@ describe("Bedrock Converse route", () => {
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("maps truncation and malformed output stop reasons", () =>
|
||||
Effect.gen(function* () {
|
||||
const reasons = [
|
||||
["model_context_window_exceeded", "length"],
|
||||
["malformed_model_output", "error"],
|
||||
["malformed_tool_use", "error"],
|
||||
] as const
|
||||
|
||||
for (const [raw, normalized] of reasons) {
|
||||
const response = yield* LLMClient.generate(baseRequest).pipe(
|
||||
Effect.provide(fixedBytes(eventStreamBody(["messageStop", { stopReason: raw }]))),
|
||||
)
|
||||
expect(response.finishReason).toEqual({ normalized, raw })
|
||||
}
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("adds cache reads and writes to Bedrock input usage", () =>
|
||||
Effect.gen(function* () {
|
||||
const body = eventStreamBody(
|
||||
["messageStart", { role: "assistant" }],
|
||||
["contentBlockDelta", { contentBlockIndex: 0, delta: { text: "Hello" } }],
|
||||
["contentBlockStop", { contentBlockIndex: 0 }],
|
||||
["messageStop", { stopReason: "end_turn" }],
|
||||
[
|
||||
"metadata",
|
||||
{
|
||||
usage: {
|
||||
inputTokens: 5,
|
||||
outputTokens: 2,
|
||||
totalTokens: 12,
|
||||
cacheReadInputTokens: 3,
|
||||
cacheWriteInputTokens: 2,
|
||||
},
|
||||
},
|
||||
],
|
||||
)
|
||||
const response = yield* LLMClient.generate(baseRequest).pipe(Effect.provide(fixedBytes(body)))
|
||||
|
||||
expect(response.usage).toMatchObject({
|
||||
inputTokens: 10,
|
||||
nonCachedInputTokens: 5,
|
||||
cacheReadInputTokens: 3,
|
||||
cacheWriteInputTokens: 2,
|
||||
outputTokens: 2,
|
||||
totalTokens: 12,
|
||||
})
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("preserves usage across later metadata events without usage", () =>
|
||||
Effect.gen(function* () {
|
||||
const body = eventStreamBody(
|
||||
["messageStop", { stopReason: "end_turn" }],
|
||||
["metadata", { usage: { inputTokens: 5, outputTokens: 2, totalTokens: 7 } }],
|
||||
["metadata", { metrics: { latencyMs: 100 } }],
|
||||
)
|
||||
const response = yield* LLMClient.generate(baseRequest).pipe(Effect.provide(fixedBytes(body)))
|
||||
|
||||
expect(response.usage).toMatchObject({ inputTokens: 5, outputTokens: 2, totalTokens: 7 })
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("assembles streamed tool call input", () =>
|
||||
Effect.gen(function* () {
|
||||
const body = eventStreamBody(
|
||||
@@ -286,8 +413,8 @@ describe("Bedrock Converse route", () => {
|
||||
["messageStop", { stopReason: "tool_use" }],
|
||||
)
|
||||
const response = yield* LLMClient.generate(
|
||||
LLM.updateRequest(baseRequest, {
|
||||
tools: [{ name: "lookup", description: "Lookup", inputSchema: { type: "object" } }],
|
||||
LLMRequest.update(baseRequest, {
|
||||
tools: [ToolDefinition.make({ name: "lookup", description: "Lookup", inputSchema: { type: "object" } })],
|
||||
}),
|
||||
).pipe(Effect.provide(fixedBytes(body)))
|
||||
|
||||
@@ -299,7 +426,10 @@ describe("Bedrock Converse route", () => {
|
||||
{ type: "tool-input-delta", id: "tool_1", name: "lookup", text: '{"query"' },
|
||||
{ type: "tool-input-delta", id: "tool_1", name: "lookup", text: ':"weather"}' },
|
||||
])
|
||||
expect(response.events.at(-1)).toMatchObject({ type: "finish", reason: "tool-calls" })
|
||||
expect(response.events.at(-1)).toMatchObject({
|
||||
type: "finish",
|
||||
reason: { normalized: "tool-calls", raw: "tool_use" },
|
||||
})
|
||||
}),
|
||||
)
|
||||
|
||||
@@ -325,7 +455,7 @@ describe("Bedrock Converse route", () => {
|
||||
name: "lookup",
|
||||
raw: '{"query":"partial',
|
||||
})
|
||||
expect(response.finishReason).toBe("tool-calls")
|
||||
expect(response.finishReason).toEqual({ normalized: "tool-calls", raw: "end_turn" })
|
||||
}),
|
||||
)
|
||||
|
||||
@@ -381,12 +511,170 @@ describe("Bedrock Converse route", () => {
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("classifies throttlingException as a rate limit", () =>
|
||||
it.effect("preserves reasoning signatures when contentBlockStop is missing", () =>
|
||||
Effect.gen(function* () {
|
||||
const response = yield* LLMClient.generate(baseRequest).pipe(
|
||||
Effect.provide(
|
||||
fixedBytes(
|
||||
eventStreamBody(
|
||||
["messageStart", { role: "assistant" }],
|
||||
[
|
||||
"contentBlockDelta",
|
||||
{ contentBlockIndex: 0, delta: { reasoningContent: { text: "Let me think." } } },
|
||||
],
|
||||
[
|
||||
"contentBlockDelta",
|
||||
{ contentBlockIndex: 0, delta: { reasoningContent: { signature: "sig_1" } } },
|
||||
],
|
||||
["messageStop", { stopReason: "end_turn" }],
|
||||
),
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
expect(response.events.find((event) => event.type === "reasoning-delta" && event.text === "")).toEqual({
|
||||
type: "reasoning-delta",
|
||||
id: "reasoning-0",
|
||||
text: "",
|
||||
providerMetadata: { bedrock: { signature: "sig_1" } },
|
||||
})
|
||||
expect(response.message.content).toEqual([
|
||||
{
|
||||
type: "reasoning",
|
||||
text: "Let me think.",
|
||||
providerMetadata: { bedrock: { signature: "sig_1" } },
|
||||
},
|
||||
])
|
||||
|
||||
const prepared = yield* LLMClient.prepare<BedrockConverse.BedrockConverseBody>(
|
||||
LLM.request({ model, messages: [response.message], cache: "none" }),
|
||||
)
|
||||
expect(prepared.body.messages).toEqual([
|
||||
{
|
||||
role: "assistant",
|
||||
content: [{ reasoningContent: { reasoningText: { text: "Let me think.", signature: "sig_1" } } }],
|
||||
},
|
||||
])
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("preserves signature-only reasoning blocks", () =>
|
||||
Effect.gen(function* () {
|
||||
const body = eventStreamBody(
|
||||
["messageStart", { role: "assistant" }],
|
||||
["throttlingException", { message: "Slow down" }],
|
||||
[
|
||||
"contentBlockDelta",
|
||||
{ contentBlockIndex: 0, delta: { reasoningContent: { signature: "sig_1" } } },
|
||||
],
|
||||
["contentBlockStop", { contentBlockIndex: 0 }],
|
||||
["messageStop", { stopReason: "end_turn" }],
|
||||
)
|
||||
const response = yield* LLMClient.generate(baseRequest).pipe(Effect.provide(fixedBytes(body)))
|
||||
|
||||
expect(response.message.content).toEqual([
|
||||
{ type: "reasoning", text: "", providerMetadata: { bedrock: { signature: "sig_1" } } },
|
||||
])
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("accepts Vercel-compatible redacted reasoning data deltas", () =>
|
||||
Effect.gen(function* () {
|
||||
const redactedData = "cmVkYWN0ZWQtdGhpbmtpbmc="
|
||||
const body = eventStreamBody(
|
||||
["messageStart", { role: "assistant" }],
|
||||
["contentBlockDelta", { contentBlockIndex: 0, delta: { reasoningContent: { data: redactedData } } }],
|
||||
["contentBlockStop", { contentBlockIndex: 0 }],
|
||||
["messageStop", { stopReason: "end_turn" }],
|
||||
)
|
||||
const response = yield* LLMClient.generate(baseRequest).pipe(Effect.provide(fixedBytes(body)))
|
||||
|
||||
expect(response.events.find((event) => event.type === "reasoning-delta" && event.text === "")).toEqual({
|
||||
type: "reasoning-delta",
|
||||
id: "reasoning-0",
|
||||
text: "",
|
||||
providerMetadata: { bedrock: { redactedData } },
|
||||
})
|
||||
expect(response.message.content).toEqual([
|
||||
{ type: "reasoning", text: "", providerMetadata: { bedrock: { redactedData } } },
|
||||
])
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("round-trips streamed redacted reasoning with tool use into a continuation request", () =>
|
||||
Effect.gen(function* () {
|
||||
// Bedrock represents redactedContent blobs as base64 strings on its JSON
|
||||
// wire. The provider owns the payload and requires byte-exact replay.
|
||||
const redactedData = "cmVkYWN0ZWQtdGhpbmtpbmc="
|
||||
const response = yield* LLMClient.generate(
|
||||
LLMRequest.update(baseRequest, {
|
||||
tools: [ToolDefinition.make({ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } })],
|
||||
}),
|
||||
).pipe(
|
||||
Effect.provide(
|
||||
fixedBytes(
|
||||
eventStreamBody(
|
||||
["messageStart", { role: "assistant" }],
|
||||
[
|
||||
"contentBlockDelta",
|
||||
{ contentBlockIndex: 0, delta: { reasoningContent: { redactedContent: redactedData } } },
|
||||
],
|
||||
["contentBlockStop", { contentBlockIndex: 0 }],
|
||||
[
|
||||
"contentBlockStart",
|
||||
{
|
||||
contentBlockIndex: 1,
|
||||
start: { toolUse: { toolUseId: "tool_1", name: "lookup" } },
|
||||
},
|
||||
],
|
||||
["contentBlockDelta", { contentBlockIndex: 1, delta: { toolUse: { input: '{"query":"weather"}' } } }],
|
||||
["contentBlockStop", { contentBlockIndex: 1 }],
|
||||
["messageStop", { stopReason: "tool_use" }],
|
||||
),
|
||||
),
|
||||
),
|
||||
)
|
||||
expect(response.events.find((event) => event.type === "reasoning-delta" && event.text === "")).toEqual({
|
||||
type: "reasoning-delta",
|
||||
id: "reasoning-0",
|
||||
text: "",
|
||||
providerMetadata: { bedrock: { redactedData } },
|
||||
})
|
||||
const prepared = yield* LLMClient.prepare<BedrockConverse.BedrockConverseBody>(
|
||||
LLM.request({
|
||||
model,
|
||||
messages: [
|
||||
Message.user("Say hello."),
|
||||
response.message,
|
||||
Message.tool({ id: "tool_1", name: "lookup", result: "sunny", resultType: "text" }),
|
||||
],
|
||||
tools: [{ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } }],
|
||||
cache: "none",
|
||||
}),
|
||||
)
|
||||
|
||||
expect(prepared.body.messages).toEqual([
|
||||
{ role: "user", content: [{ text: "Say hello." }] },
|
||||
{
|
||||
role: "assistant",
|
||||
content: [
|
||||
{ reasoningContent: { redactedContent: redactedData } },
|
||||
{ toolUse: { toolUseId: "tool_1", name: "lookup", input: { query: "weather" } } },
|
||||
],
|
||||
},
|
||||
{
|
||||
role: "user",
|
||||
content: [{ toolResult: { toolUseId: "tool_1", content: [{ text: "sunny" }], status: "success" } }],
|
||||
},
|
||||
])
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("classifies throttlingException as a rate limit", () =>
|
||||
Effect.gen(function* () {
|
||||
const body = concat([
|
||||
eventFrame("messageStart", { role: "assistant" }),
|
||||
exceptionFrame("throttlingException", { message: "Slow down" }),
|
||||
])
|
||||
const error = yield* LLMClient.generate(baseRequest).pipe(Effect.provide(fixedBytes(body)), Effect.flip)
|
||||
|
||||
expect(error.reason).toMatchObject({ _tag: "RateLimit", message: "Slow down" })
|
||||
@@ -397,7 +685,7 @@ describe("Bedrock Converse route", () => {
|
||||
Effect.gen(function* () {
|
||||
const error = yield* LLMClient.generate(baseRequest).pipe(
|
||||
Effect.provide(
|
||||
fixedBytes(eventStreamBody(["validationException", { message: "Input is too long for requested model" }])),
|
||||
fixedBytes(exceptionFrame("validationException", { message: "Input is too long for requested model" })),
|
||||
),
|
||||
Effect.flip,
|
||||
)
|
||||
@@ -410,12 +698,44 @@ describe("Bedrock Converse route", () => {
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("uses originalMessage from model stream exception frames", () =>
|
||||
Effect.gen(function* () {
|
||||
const error = yield* LLMClient.generate(baseRequest).pipe(
|
||||
Effect.provide(
|
||||
fixedBytes(
|
||||
exceptionFrame("modelStreamErrorException", {
|
||||
originalMessage: "Upstream model failed",
|
||||
originalStatusCode: 500,
|
||||
}),
|
||||
),
|
||||
),
|
||||
Effect.flip,
|
||||
)
|
||||
|
||||
expect(error.reason).toMatchObject({ _tag: "ProviderInternal", message: "Upstream model failed" })
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("fails unmodeled AWS event-stream errors", () =>
|
||||
Effect.gen(function* () {
|
||||
const error = yield* LLMClient.generate(baseRequest).pipe(
|
||||
Effect.provide(fixedBytes(errorFrame("BadStream", "Stream failed"))),
|
||||
Effect.flip,
|
||||
)
|
||||
|
||||
expect(error.reason).toMatchObject({
|
||||
_tag: "InvalidProviderOutput",
|
||||
message: "BadStream: Stream failed",
|
||||
})
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("rejects requests with no auth path", () =>
|
||||
Effect.gen(function* () {
|
||||
const unsignedModel = AmazonBedrock.configure({
|
||||
baseURL: "https://bedrock-runtime.test",
|
||||
}).model("anthropic.claude-3-5-sonnet-20240620-v1:0")
|
||||
const error = yield* LLMClient.generate(LLM.updateRequest(baseRequest, { model: unsignedModel })).pipe(
|
||||
const error = yield* LLMClient.generate(LLMRequest.update(baseRequest, { model: unsignedModel })).pipe(
|
||||
Effect.provide(fixedBytes(eventStreamBody(["messageStop", { stopReason: "end_turn" }]))),
|
||||
Effect.flip,
|
||||
)
|
||||
@@ -434,7 +754,7 @@ describe("Bedrock Converse route", () => {
|
||||
secretAccessKey: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
|
||||
},
|
||||
}).model("anthropic.claude-3-5-sonnet-20240620-v1:0")
|
||||
const prepared = yield* LLMClient.prepare(LLM.updateRequest(baseRequest, { model: signed }))
|
||||
const prepared = yield* LLMClient.prepare(LLMRequest.update(baseRequest, { model: signed }))
|
||||
|
||||
expect(prepared.route).toBe("bedrock-converse")
|
||||
expect(prepared.model).toBe(signed)
|
||||
@@ -619,6 +939,7 @@ describe("Bedrock Converse route", () => {
|
||||
const prepared = yield* LLMClient.prepare<BedrockConverse.BedrockConverseBody>(
|
||||
LLM.request({
|
||||
model,
|
||||
cache: "none",
|
||||
messages: [
|
||||
Message.assistant([ToolCallPart.make({ id: "call_1", name: "read", input: { path: "report.pdf" } })]),
|
||||
Message.tool({
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect } from "effect"
|
||||
import { LLM, LLMError, Message, ToolCallPart, Usage } from "../../src"
|
||||
import { LLM, LLMError, LLMRequest, Message, ToolCallPart, ToolDefinition, Usage } from "../../src"
|
||||
import { Auth, LLMClient } from "../../src/route"
|
||||
import * as Gemini from "../../src/protocols/gemini"
|
||||
import { ProviderShared } from "../../src/protocols/shared"
|
||||
@@ -36,6 +36,27 @@ describe("Gemini route", () => {
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("normalizes Gemini thinking options", () =>
|
||||
Effect.gen(function* () {
|
||||
const prepared = yield* LLMClient.prepare<Gemini.GeminiBody>(
|
||||
LLMRequest.update(request, {
|
||||
providerOptions: { gemini: { thinkingConfig: { thinkingBudget: 0, includeThoughts: false } } },
|
||||
}),
|
||||
)
|
||||
const filtered = yield* LLMClient.prepare<Gemini.GeminiBody>(
|
||||
LLMRequest.update(request, {
|
||||
providerOptions: { gemini: { thinkingConfig: { thinkingBudget: "invalid", includeThoughts: false } } },
|
||||
}),
|
||||
)
|
||||
|
||||
expect(prepared.body.generationConfig?.thinkingConfig).toEqual({
|
||||
thinkingBudget: 0,
|
||||
includeThoughts: false,
|
||||
})
|
||||
expect(filtered.body.generationConfig?.thinkingConfig).toEqual({ includeThoughts: false })
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("lowers chronological system updates to wrapped user text in order", () =>
|
||||
Effect.gen(function* () {
|
||||
const prepared = yield* LLMClient.prepare<Gemini.GeminiBody>(
|
||||
@@ -233,20 +254,22 @@ describe("Gemini route", () => {
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("omits tools when tool choice is none", () =>
|
||||
it.effect("keeps tools and sends function calling mode NONE", () =>
|
||||
Effect.gen(function* () {
|
||||
const prepared = yield* LLMClient.prepare(
|
||||
LLM.request({
|
||||
id: "req_no_tools",
|
||||
id: "req_tool_choice_none",
|
||||
model,
|
||||
prompt: "Say hello.",
|
||||
tools: [{ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } }],
|
||||
tools: [ToolDefinition.make({ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } })],
|
||||
toolChoice: { type: "none" },
|
||||
}),
|
||||
)
|
||||
|
||||
expect(prepared.body).toEqual({
|
||||
expect(prepared.body).toMatchObject({
|
||||
contents: [{ role: "user", parts: [{ text: "Say hello." }] }],
|
||||
tools: [{ functionDeclarations: [{ name: "lookup", description: "Lookup data" }] }],
|
||||
toolConfig: { functionCallingConfig: { mode: "NONE" } },
|
||||
})
|
||||
}),
|
||||
)
|
||||
@@ -371,10 +394,16 @@ describe("Gemini route", () => {
|
||||
{ type: "text-delta", id: "text-0", text: "Hello" },
|
||||
{ type: "text-delta", id: "text-0", text: "!" },
|
||||
{ type: "text-end", id: "text-0" },
|
||||
{ type: "step-finish", index: 0, reason: "stop", usage, providerMetadata: undefined },
|
||||
{
|
||||
type: "step-finish",
|
||||
index: 0,
|
||||
reason: { normalized: "stop", raw: "STOP" },
|
||||
usage,
|
||||
providerMetadata: undefined,
|
||||
},
|
||||
{
|
||||
type: "finish",
|
||||
reason: "stop",
|
||||
reason: { normalized: "stop", raw: "STOP" },
|
||||
usage,
|
||||
},
|
||||
])
|
||||
@@ -402,8 +431,8 @@ describe("Gemini route", () => {
|
||||
],
|
||||
})
|
||||
const response = yield* LLMClient.generate(
|
||||
LLM.updateRequest(request, {
|
||||
tools: [{ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } }],
|
||||
LLMRequest.update(request, {
|
||||
tools: [ToolDefinition.make({ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } })],
|
||||
}),
|
||||
).pipe(Effect.provide(fixedResponse(body)))
|
||||
const reasoning = response.events.find((event) => event.type === "reasoning-start")
|
||||
@@ -493,8 +522,8 @@ describe("Gemini route", () => {
|
||||
usageMetadata: { promptTokenCount: 5, candidatesTokenCount: 1 },
|
||||
})
|
||||
const response = yield* LLMClient.generate(
|
||||
LLM.updateRequest(request, {
|
||||
tools: [{ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } }],
|
||||
LLMRequest.update(request, {
|
||||
tools: [ToolDefinition.make({ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } })],
|
||||
}),
|
||||
).pipe(Effect.provide(fixedResponse(body)))
|
||||
const usage = new Usage({
|
||||
@@ -527,10 +556,16 @@ describe("Gemini route", () => {
|
||||
providerExecuted: undefined,
|
||||
providerMetadata: undefined,
|
||||
},
|
||||
{ type: "step-finish", index: 0, reason: "tool-calls", usage, providerMetadata: undefined },
|
||||
{
|
||||
type: "step-finish",
|
||||
index: 0,
|
||||
reason: { normalized: "tool-calls", raw: "STOP" },
|
||||
usage,
|
||||
providerMetadata: undefined,
|
||||
},
|
||||
{
|
||||
type: "finish",
|
||||
reason: "tool-calls",
|
||||
reason: { normalized: "tool-calls", raw: "STOP" },
|
||||
usage,
|
||||
},
|
||||
])
|
||||
@@ -554,8 +589,8 @@ describe("Gemini route", () => {
|
||||
],
|
||||
})
|
||||
const response = yield* LLMClient.generate(
|
||||
LLM.updateRequest(request, {
|
||||
tools: [{ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } }],
|
||||
LLMRequest.update(request, {
|
||||
tools: [ToolDefinition.make({ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } })],
|
||||
}),
|
||||
).pipe(Effect.provide(fixedResponse(body)))
|
||||
|
||||
@@ -569,7 +604,10 @@ describe("Gemini route", () => {
|
||||
},
|
||||
{ type: "tool-call", id: "tool_1", name: "lookup", input: { query: "news" } },
|
||||
])
|
||||
expect(response.events.at(-1)).toMatchObject({ type: "finish", reason: "tool-calls" })
|
||||
expect(response.events.at(-1)).toMatchObject({
|
||||
type: "finish",
|
||||
reason: { normalized: "tool-calls", raw: "STOP" },
|
||||
})
|
||||
}),
|
||||
)
|
||||
|
||||
@@ -589,9 +627,41 @@ describe("Gemini route", () => {
|
||||
)
|
||||
|
||||
expect(length.events.map((event) => event.type)).toEqual(["step-start", "step-finish", "finish"])
|
||||
expect(length.events.at(-1)).toMatchObject({ type: "finish", reason: "length" })
|
||||
expect(length.events.at(-1)).toMatchObject({
|
||||
type: "finish",
|
||||
reason: { normalized: "length", raw: "MAX_TOKENS" },
|
||||
})
|
||||
expect(filtered.events.map((event) => event.type)).toEqual(["step-start", "step-finish", "finish"])
|
||||
expect(filtered.events.at(-1)).toMatchObject({ type: "finish", reason: "content-filter" })
|
||||
expect(filtered.events.at(-1)).toMatchObject({
|
||||
type: "finish",
|
||||
reason: { normalized: "content-filter", raw: "SAFETY" },
|
||||
})
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("maps current blocking and invalid-output finish reasons", () =>
|
||||
Effect.gen(function* () {
|
||||
const reasons = [
|
||||
["MODEL_ARMOR", "content-filter"],
|
||||
["IMAGE_PROHIBITED_CONTENT", "content-filter"],
|
||||
["IMAGE_RECITATION", "content-filter"],
|
||||
["LANGUAGE", "content-filter"],
|
||||
["UNEXPECTED_TOOL_CALL", "error"],
|
||||
["NO_IMAGE", "error"],
|
||||
["IMAGE_OTHER", "unknown"],
|
||||
["TOO_MANY_TOOL_CALLS", "error"],
|
||||
["MISSING_THOUGHT_SIGNATURE", "error"],
|
||||
["MALFORMED_RESPONSE", "error"],
|
||||
] as const
|
||||
|
||||
for (const [raw, normalized] of reasons) {
|
||||
const response = yield* LLMClient.generate(request).pipe(
|
||||
Effect.provide(
|
||||
fixedResponse(sseEvents({ candidates: [{ content: { role: "model", parts: [] }, finishReason: raw }] })),
|
||||
),
|
||||
)
|
||||
expect(response.finishReason).toEqual({ normalized, raw })
|
||||
}
|
||||
}),
|
||||
)
|
||||
|
||||
|
||||
@@ -1,7 +1,18 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect, Schema, Stream } from "effect"
|
||||
import { HttpClientRequest } from "effect/unstable/http"
|
||||
import { LLM, LLMError, LLMEvent, Message, Model, ToolCallPart, Usage } from "../../src"
|
||||
import {
|
||||
HttpOptions,
|
||||
LLM,
|
||||
LLMError,
|
||||
LLMEvent,
|
||||
LLMRequest,
|
||||
Message,
|
||||
Model,
|
||||
ToolCallPart,
|
||||
ToolDefinition,
|
||||
Usage,
|
||||
} from "../../src"
|
||||
import * as Azure from "../../src/providers/azure"
|
||||
import * as OpenAI from "../../src/providers/openai"
|
||||
import * as OpenAIChat from "../../src/protocols/openai-chat"
|
||||
@@ -162,7 +173,7 @@ describe("OpenAI Chat route", () => {
|
||||
|
||||
it.effect("adds native query params to the Chat Completions URL", () =>
|
||||
LLMClient.generate(
|
||||
LLM.updateRequest(request, {
|
||||
LLMRequest.update(request, {
|
||||
model: Model.update(model, { route: model.route.with({ endpoint: { query: { "api-version": "v1" } } }) }),
|
||||
}),
|
||||
).pipe(
|
||||
@@ -182,7 +193,7 @@ describe("OpenAI Chat route", () => {
|
||||
|
||||
it.effect("uses Azure api-key header for static OpenAI Chat keys", () =>
|
||||
LLMClient.generate(
|
||||
LLM.updateRequest(request, {
|
||||
LLMRequest.update(request, {
|
||||
model: Azure.configure({
|
||||
baseURL: "https://opencode-test.openai.azure.com/openai/v1/",
|
||||
apiKey: "azure-key",
|
||||
@@ -208,15 +219,15 @@ describe("OpenAI Chat route", () => {
|
||||
|
||||
it.effect("applies serializable HTTP overlays after payload lowering", () =>
|
||||
LLMClient.generate(
|
||||
LLM.updateRequest(request, {
|
||||
LLMRequest.update(request, {
|
||||
model: model.route
|
||||
.with({ auth: Auth.bearer("fresh-key"), headers: { authorization: "Bearer stale" } })
|
||||
.model({ id: model.id }),
|
||||
http: {
|
||||
http: HttpOptions.make({
|
||||
body: { metadata: { source: "test" } },
|
||||
headers: { authorization: "Bearer request", "x-custom": "yes" },
|
||||
query: { debug: "1" },
|
||||
},
|
||||
}),
|
||||
}),
|
||||
).pipe(
|
||||
Effect.provide(
|
||||
@@ -539,7 +550,7 @@ describe("OpenAI Chat route", () => {
|
||||
prompt_tokens: 5,
|
||||
completion_tokens: 2,
|
||||
total_tokens: 7,
|
||||
prompt_tokens_details: { cached_tokens: 1 },
|
||||
prompt_tokens_details: { cached_tokens: 1, cache_write_tokens: 2 },
|
||||
completion_tokens_details: { reasoning_tokens: 0 },
|
||||
}),
|
||||
)
|
||||
@@ -547,8 +558,9 @@ describe("OpenAI Chat route", () => {
|
||||
const usage = new Usage({
|
||||
inputTokens: 5,
|
||||
outputTokens: 2,
|
||||
nonCachedInputTokens: 4,
|
||||
nonCachedInputTokens: 2,
|
||||
cacheReadInputTokens: 1,
|
||||
cacheWriteInputTokens: 2,
|
||||
reasoningTokens: 0,
|
||||
totalTokens: 7,
|
||||
providerMetadata: {
|
||||
@@ -556,7 +568,7 @@ describe("OpenAI Chat route", () => {
|
||||
prompt_tokens: 5,
|
||||
completion_tokens: 2,
|
||||
total_tokens: 7,
|
||||
prompt_tokens_details: { cached_tokens: 1 },
|
||||
prompt_tokens_details: { cached_tokens: 1, cache_write_tokens: 2 },
|
||||
completion_tokens_details: { reasoning_tokens: 0 },
|
||||
},
|
||||
},
|
||||
@@ -569,10 +581,16 @@ describe("OpenAI Chat route", () => {
|
||||
{ type: "text-delta", id: "text-0", text: "Hello" },
|
||||
{ type: "text-delta", id: "text-0", text: "!" },
|
||||
{ type: "text-end", id: "text-0" },
|
||||
{ type: "step-finish", index: 0, reason: "stop", usage, providerMetadata: undefined },
|
||||
{
|
||||
type: "step-finish",
|
||||
index: 0,
|
||||
reason: { normalized: "stop", raw: "stop" },
|
||||
usage,
|
||||
providerMetadata: undefined,
|
||||
},
|
||||
{
|
||||
type: "finish",
|
||||
reason: "stop",
|
||||
reason: { normalized: "stop", raw: "stop" },
|
||||
usage,
|
||||
},
|
||||
])
|
||||
@@ -612,7 +630,7 @@ describe("OpenAI Chat route", () => {
|
||||
it.effect("parses and replays a configured custom reasoning field", () =>
|
||||
Effect.gen(function* () {
|
||||
const custom = Model.update(model, { compatibility: { reasoningField: "vendor_reasoning" } })
|
||||
const response = yield* LLMClient.generate(LLM.updateRequest(request, { model: custom })).pipe(
|
||||
const response = yield* LLMClient.generate(LLMRequest.update(request, { model: custom })).pipe(
|
||||
Effect.provide(
|
||||
fixedResponse(
|
||||
sseEvents(
|
||||
@@ -632,9 +650,7 @@ describe("OpenAI Chat route", () => {
|
||||
const replay = yield* LLMClient.prepare<OpenAIChat.OpenAIChatBody>(
|
||||
LLM.request({ model: custom, messages: [response.message] }),
|
||||
)
|
||||
expect(replay.body.messages).toEqual([
|
||||
{ role: "assistant", content: "Hello", vendor_reasoning: "thinking" },
|
||||
])
|
||||
expect(replay.body.messages).toEqual([{ role: "assistant", content: "Hello", vendor_reasoning: "thinking" }])
|
||||
}),
|
||||
)
|
||||
|
||||
@@ -645,8 +661,8 @@ describe("OpenAI Chat route", () => {
|
||||
{ type: "reasoning.encrypted", data: "opaque", format: "anthropic-claude-v1", index: 1 },
|
||||
]
|
||||
const response = yield* LLMClient.generate(
|
||||
LLM.updateRequest(request, {
|
||||
tools: [{ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } }],
|
||||
LLMRequest.update(request, {
|
||||
tools: [ToolDefinition.make({ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } })],
|
||||
}),
|
||||
).pipe(
|
||||
Effect.provide(
|
||||
@@ -1018,8 +1034,8 @@ describe("OpenAI Chat route", () => {
|
||||
deltaChunk({}, "tool_calls"),
|
||||
)
|
||||
const response = yield* LLMClient.generate(
|
||||
LLM.updateRequest(request, {
|
||||
tools: [{ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } }],
|
||||
LLMRequest.update(request, {
|
||||
tools: [ToolDefinition.make({ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } })],
|
||||
}),
|
||||
).pipe(Effect.provide(fixedResponse(body)))
|
||||
|
||||
@@ -1037,8 +1053,14 @@ describe("OpenAI Chat route", () => {
|
||||
providerExecuted: undefined,
|
||||
providerMetadata: undefined,
|
||||
},
|
||||
{ type: "step-finish", index: 0, reason: "tool-calls", usage: undefined, providerMetadata: undefined },
|
||||
{ type: "finish", reason: "tool-calls", usage: undefined },
|
||||
{
|
||||
type: "step-finish",
|
||||
index: 0,
|
||||
reason: { normalized: "tool-calls", raw: "tool_calls" },
|
||||
usage: undefined,
|
||||
providerMetadata: undefined,
|
||||
},
|
||||
{ type: "finish", reason: { normalized: "tool-calls", raw: "tool_calls" }, usage: undefined },
|
||||
])
|
||||
}),
|
||||
)
|
||||
@@ -1055,8 +1077,8 @@ describe("OpenAI Chat route", () => {
|
||||
deltaChunk({}, "tool_calls"),
|
||||
)
|
||||
const response = yield* LLMClient.generate(
|
||||
LLM.updateRequest(request, {
|
||||
tools: [{ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } }],
|
||||
LLMRequest.update(request, {
|
||||
tools: [ToolDefinition.make({ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } })],
|
||||
}),
|
||||
).pipe(Effect.provide(fixedResponse(body)))
|
||||
|
||||
@@ -1077,8 +1099,8 @@ describe("OpenAI Chat route", () => {
|
||||
deltaChunk({}, "tool_calls"),
|
||||
)
|
||||
const response = yield* LLMClient.generate(
|
||||
LLM.updateRequest(request, {
|
||||
tools: [{ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } }],
|
||||
LLMRequest.update(request, {
|
||||
tools: [ToolDefinition.make({ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } })],
|
||||
}),
|
||||
).pipe(Effect.provide(fixedResponse(body)))
|
||||
|
||||
@@ -1095,8 +1117,8 @@ describe("OpenAI Chat route", () => {
|
||||
deltaChunk({}, "tool_calls"),
|
||||
)
|
||||
const error = yield* LLMClient.generate(
|
||||
LLM.updateRequest(request, {
|
||||
tools: [{ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } }],
|
||||
LLMRequest.update(request, {
|
||||
tools: [ToolDefinition.make({ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } })],
|
||||
}),
|
||||
).pipe(Effect.provide(fixedResponse(body)), Effect.flip)
|
||||
|
||||
@@ -1113,8 +1135,8 @@ describe("OpenAI Chat route", () => {
|
||||
}),
|
||||
deltaChunk({ tool_calls: [{ index: 0, function: { arguments: ':"weather"}' } }] }),
|
||||
)
|
||||
const input = LLM.updateRequest(request, {
|
||||
tools: [{ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } }],
|
||||
const input = LLMRequest.update(request, {
|
||||
tools: [ToolDefinition.make({ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } })],
|
||||
})
|
||||
const events: LLMEvent[] = []
|
||||
const streamError = yield* LLMClient.stream(input).pipe(
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect, Schema } from "effect"
|
||||
import { HttpClientRequest } from "effect/unstable/http"
|
||||
import { LLM, Message, ToolCallPart } from "../../src"
|
||||
import { LLM, LLMRequest, Message, ToolCallPart, ToolChoice, ToolDefinition } from "../../src"
|
||||
import { Auth, LLMClient } from "../../src/route"
|
||||
import * as OpenAICompatible from "../../src/providers/openai-compatible"
|
||||
import * as OpenAICompatibleChat from "../../src/protocols/openai-compatible-chat"
|
||||
@@ -53,9 +53,9 @@ describe("OpenAI-compatible Chat route", () => {
|
||||
it.effect("prepares generic Chat target", () =>
|
||||
Effect.gen(function* () {
|
||||
const prepared = yield* LLMClient.prepare(
|
||||
LLM.updateRequest(request, {
|
||||
tools: [{ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } }],
|
||||
toolChoice: { type: "required" },
|
||||
LLMRequest.update(request, {
|
||||
tools: [ToolDefinition.make({ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } })],
|
||||
toolChoice: ToolChoice.make({ type: "required" }),
|
||||
}),
|
||||
)
|
||||
|
||||
@@ -232,7 +232,10 @@ describe("OpenAI-compatible Chat route", () => {
|
||||
|
||||
expect(response.text).toBe("Hello!")
|
||||
expect(response.usage).toMatchObject({ inputTokens: 5, outputTokens: 2, totalTokens: 7 })
|
||||
expect(response.events.at(-1)).toMatchObject({ type: "finish", reason: "stop" })
|
||||
expect(response.events.at(-1)).toMatchObject({
|
||||
type: "finish",
|
||||
reason: { normalized: "stop", raw: "stop" },
|
||||
})
|
||||
}),
|
||||
)
|
||||
})
|
||||
|
||||
@@ -1,17 +1,22 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect } from "effect"
|
||||
import { LLM } from "../../src"
|
||||
import { LLM, LLMEvent } from "../../src"
|
||||
import { configure } from "../../src/providers/openai-compatible-responses"
|
||||
import { OpenAI } from "../../src/providers"
|
||||
import { OpenResponses } from "../../src/protocols/open-responses"
|
||||
import { OpenAICompatibleResponses } from "../../src/protocols/openai-compatible-responses"
|
||||
import { OpenAIResponses } from "../../src/protocols/openai-responses"
|
||||
import { LLMClient } from "../../src/route"
|
||||
import { it } from "../lib/effect"
|
||||
import { fixedResponse } from "../lib/http"
|
||||
import { sseEvents } from "../lib/sse"
|
||||
|
||||
describe("OpenAI-compatible Responses route", () => {
|
||||
it.effect("reuses the OpenAI Responses protocol for a configured deployment", () =>
|
||||
describe("Open Responses-compatible route", () => {
|
||||
it.effect("uses the Open Responses baseline for a configured deployment", () =>
|
||||
Effect.gen(function* () {
|
||||
expect(OpenAICompatibleResponses.route.body).toBe(OpenAIResponses.protocol.body)
|
||||
expect(OpenAICompatibleResponses.route.transport).toBe(OpenAIResponses.httpTransport)
|
||||
expect(OpenAICompatibleResponses.route.body).toBe(OpenResponses.protocol.body)
|
||||
expect(OpenAICompatibleResponses.route.transport).toBe(OpenResponses.httpTransport)
|
||||
expect(OpenAICompatibleResponses.route.body).not.toBe(OpenAIResponses.protocol.body)
|
||||
|
||||
const model = configure({
|
||||
apiKey: "test-key",
|
||||
@@ -27,7 +32,7 @@ describe("OpenAI-compatible Responses route", () => {
|
||||
)
|
||||
|
||||
expect(prepared.route).toBe("openai-compatible-responses")
|
||||
expect(prepared.protocol).toBe("openai-responses")
|
||||
expect(prepared.protocol).toBe("open-responses")
|
||||
expect(prepared.model).toMatchObject({
|
||||
id: "example-model",
|
||||
provider: "example",
|
||||
@@ -45,9 +50,67 @@ describe("OpenAI-compatible Responses route", () => {
|
||||
{ role: "system", content: "You are concise." },
|
||||
{ role: "user", content: [{ type: "input_text", text: "Say hello." }] },
|
||||
],
|
||||
store: false,
|
||||
stream: true,
|
||||
})
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("rejects OpenAI-native tools", () =>
|
||||
Effect.gen(function* () {
|
||||
const model = configure({
|
||||
apiKey: "test-key",
|
||||
baseURL: "https://responses.example.test/v1",
|
||||
}).model("example-model")
|
||||
const error = yield* LLMClient.prepare(
|
||||
LLM.request({ model, prompt: "Draw.", tools: [OpenAI.imageGeneration()] }),
|
||||
).pipe(Effect.flip)
|
||||
|
||||
expect(error.reason._tag).toBe("InvalidRequest")
|
||||
expect(error.message).toContain("Open Responses does not support provider-native tool image_generation")
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("reads standard options from the Open Responses namespace", () =>
|
||||
Effect.gen(function* () {
|
||||
const model = configure({
|
||||
apiKey: "test-key",
|
||||
baseURL: "https://responses.example.test/v1",
|
||||
providerOptions: { openresponses: { reasoningEffort: "low", store: true } },
|
||||
}).model("example-model")
|
||||
const prepared = yield* LLMClient.prepare(LLM.request({ model, prompt: "Think." }))
|
||||
|
||||
expect(prepared.body).toMatchObject({
|
||||
reasoning: { effort: "low" },
|
||||
store: true,
|
||||
})
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("does not interpret OpenAI hosted-tool items", () =>
|
||||
Effect.gen(function* () {
|
||||
const model = configure({
|
||||
apiKey: "test-key",
|
||||
baseURL: "https://responses.example.test/v1",
|
||||
provider: "example",
|
||||
}).model("example-model")
|
||||
const response = yield* LLMClient.generate(LLM.request({ model, prompt: "Search." })).pipe(
|
||||
Effect.provide(
|
||||
fixedResponse(
|
||||
sseEvents(
|
||||
{
|
||||
type: "response.output_item.done",
|
||||
item: { type: "web_search_call", id: "ws_1", status: "completed", action: { query: "news" } },
|
||||
},
|
||||
{ type: "response.completed", response: { id: "resp_1" } },
|
||||
),
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
expect(response.toolCalls).toEqual([])
|
||||
expect(response.events.find(LLMEvent.is.finish)).toMatchObject({
|
||||
providerMetadata: { openresponses: { responseId: "resp_1" } },
|
||||
})
|
||||
}),
|
||||
)
|
||||
})
|
||||
|
||||
@@ -1,7 +1,18 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { ConfigProvider, Effect, Layer, Stream } from "effect"
|
||||
import { Headers, HttpClientRequest } from "effect/unstable/http"
|
||||
import { LLM, LLMError, LLMEvent, Message, Model, ToolCallPart, ToolResultPart, Usage } from "../../src"
|
||||
import {
|
||||
LLM,
|
||||
LLMError,
|
||||
LLMEvent,
|
||||
LLMRequest,
|
||||
Message,
|
||||
Model,
|
||||
ToolCallPart,
|
||||
ToolDefinition,
|
||||
ToolResultPart,
|
||||
Usage,
|
||||
} from "../../src"
|
||||
import { Auth, LLMClient, RequestExecutor, WebSocketExecutor } from "../../src/route"
|
||||
import * as Azure from "../../src/providers/azure"
|
||||
import * as OpenAI from "../../src/providers/openai"
|
||||
@@ -96,7 +107,7 @@ describe("OpenAI Responses route", () => {
|
||||
|
||||
it.effect("lowers semantic service tier options", () =>
|
||||
Effect.gen(function* () {
|
||||
const input = LLM.updateRequest(request, { providerOptions: { openai: { serviceTier: "priority" } } })
|
||||
const input = LLMRequest.update(request, { providerOptions: { openai: { serviceTier: "priority" } } })
|
||||
expect(input.providerOptions).toEqual({ openai: { serviceTier: "priority" } })
|
||||
const prepared = yield* LLMClient.prepare(input)
|
||||
|
||||
@@ -108,7 +119,7 @@ describe("OpenAI Responses route", () => {
|
||||
it.effect("passes through custom OpenAI reasoning effort strings", () =>
|
||||
Effect.gen(function* () {
|
||||
const prepared = yield* LLMClient.prepare<OpenAIResponses.OpenAIResponsesBody>(
|
||||
LLM.updateRequest(request, { providerOptions: { openai: { reasoningEffort: "experimental" } } }),
|
||||
LLMRequest.update(request, { providerOptions: { openai: { reasoningEffort: "experimental" } } }),
|
||||
)
|
||||
|
||||
expect(prepared.body.reasoning).toEqual({ effort: "experimental" })
|
||||
@@ -118,7 +129,7 @@ describe("OpenAI Responses route", () => {
|
||||
it.effect("omits unsupported semantic service tiers", () =>
|
||||
Effect.gen(function* () {
|
||||
const prepared = yield* LLMClient.prepare(
|
||||
LLM.updateRequest(request, { providerOptions: { openai: { serviceTier: "unsupported" } } }),
|
||||
LLMRequest.update(request, { providerOptions: { openai: { serviceTier: "unsupported" } } }),
|
||||
)
|
||||
|
||||
expect(prepared.body).not.toHaveProperty("service_tier")
|
||||
@@ -128,9 +139,9 @@ describe("OpenAI Responses route", () => {
|
||||
it.effect("flattens top-level object unions in function schemas", () =>
|
||||
Effect.gen(function* () {
|
||||
const prepared = yield* LLMClient.prepare<OpenAIResponses.OpenAIResponsesBody>(
|
||||
LLM.updateRequest(request, {
|
||||
LLMRequest.update(request, {
|
||||
tools: [
|
||||
{
|
||||
ToolDefinition.make({
|
||||
name: "read",
|
||||
description: "Read a path or resource.",
|
||||
inputSchema: {
|
||||
@@ -152,7 +163,7 @@ describe("OpenAI Responses route", () => {
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
}),
|
||||
],
|
||||
}),
|
||||
)
|
||||
@@ -207,7 +218,7 @@ describe("OpenAI Responses route", () => {
|
||||
it.effect("prepares OpenAI Responses WebSocket target", () =>
|
||||
Effect.gen(function* () {
|
||||
const prepared = yield* LLMClient.prepare(
|
||||
LLM.updateRequest(request, {
|
||||
LLMRequest.update(request, {
|
||||
model: OpenAIResponses.webSocketRoute
|
||||
.with({ endpoint: { baseURL: "https://api.openai.test/v1/" }, auth: Auth.bearer("test") })
|
||||
.model({ id: "gpt-4.1-mini" }),
|
||||
@@ -291,7 +302,7 @@ describe("OpenAI Responses route", () => {
|
||||
it.effect("adds native query params to the Responses URL", () =>
|
||||
Effect.gen(function* () {
|
||||
yield* LLMClient.generate(
|
||||
LLM.updateRequest(request, {
|
||||
LLMRequest.update(request, {
|
||||
model: Model.update(model, { route: model.route.with({ endpoint: { query: { "api-version": "v1" } } }) }),
|
||||
}),
|
||||
).pipe(
|
||||
@@ -313,7 +324,7 @@ describe("OpenAI Responses route", () => {
|
||||
it.effect("uses Azure api-key header for static OpenAI Responses keys", () =>
|
||||
Effect.gen(function* () {
|
||||
yield* LLMClient.generate(
|
||||
LLM.updateRequest(request, {
|
||||
LLMRequest.update(request, {
|
||||
model: Azure.configure({
|
||||
baseURL: "https://opencode-test.openai.azure.com/openai/v1/",
|
||||
apiKey: "azure-key",
|
||||
@@ -340,7 +351,7 @@ describe("OpenAI Responses route", () => {
|
||||
|
||||
it.effect("loads OpenAI default auth from Effect Config", () =>
|
||||
LLMClient.generate(
|
||||
LLM.updateRequest(request, {
|
||||
LLMRequest.update(request, {
|
||||
model: OpenAI.configure({ baseURL: "https://api.openai.test/v1/" }).responses("gpt-4.1-mini"),
|
||||
}),
|
||||
).pipe(
|
||||
@@ -361,7 +372,7 @@ describe("OpenAI Responses route", () => {
|
||||
|
||||
it.effect("lets explicit auth override OpenAI default API key auth", () =>
|
||||
LLMClient.generate(
|
||||
LLM.updateRequest(request, {
|
||||
LLMRequest.update(request, {
|
||||
model: OpenAI.configure({
|
||||
baseURL: "https://api.openai.test/v1/",
|
||||
auth: Auth.bearer("oauth-token"),
|
||||
@@ -821,7 +832,7 @@ describe("OpenAI Responses route", () => {
|
||||
input_tokens: 5,
|
||||
output_tokens: 2,
|
||||
total_tokens: 7,
|
||||
input_tokens_details: { cached_tokens: 1 },
|
||||
input_tokens_details: { cached_tokens: 1, cache_write_tokens: 2 },
|
||||
output_tokens_details: { reasoning_tokens: 0 },
|
||||
},
|
||||
},
|
||||
@@ -831,8 +842,9 @@ describe("OpenAI Responses route", () => {
|
||||
const usage = new Usage({
|
||||
inputTokens: 5,
|
||||
outputTokens: 2,
|
||||
nonCachedInputTokens: 4,
|
||||
nonCachedInputTokens: 2,
|
||||
cacheReadInputTokens: 1,
|
||||
cacheWriteInputTokens: 2,
|
||||
reasoningTokens: 0,
|
||||
totalTokens: 7,
|
||||
providerMetadata: {
|
||||
@@ -840,7 +852,7 @@ describe("OpenAI Responses route", () => {
|
||||
input_tokens: 5,
|
||||
output_tokens: 2,
|
||||
total_tokens: 7,
|
||||
input_tokens_details: { cached_tokens: 1 },
|
||||
input_tokens_details: { cached_tokens: 1, cache_write_tokens: 2 },
|
||||
output_tokens_details: { reasoning_tokens: 0 },
|
||||
},
|
||||
},
|
||||
@@ -856,13 +868,13 @@ describe("OpenAI Responses route", () => {
|
||||
{
|
||||
type: "step-finish",
|
||||
index: 0,
|
||||
reason: "stop",
|
||||
reason: { normalized: "stop", raw: undefined },
|
||||
providerMetadata: { openai: { responseId: "resp_1", serviceTier: "default" } },
|
||||
usage,
|
||||
},
|
||||
{
|
||||
type: "finish",
|
||||
reason: "stop",
|
||||
reason: { normalized: "stop", raw: undefined },
|
||||
providerMetadata: { openai: { responseId: "resp_1", serviceTier: "default" } },
|
||||
usage,
|
||||
},
|
||||
@@ -870,6 +882,34 @@ describe("OpenAI Responses route", () => {
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("maps incomplete response reasons", () =>
|
||||
Effect.gen(function* () {
|
||||
const generate = (incompleteDetails: object) =>
|
||||
LLMClient.generate(request).pipe(
|
||||
Effect.provide(
|
||||
fixedResponse(
|
||||
sseEvents({
|
||||
type: "response.incomplete",
|
||||
response: { id: "resp_incomplete", incomplete_details: incompleteDetails },
|
||||
}),
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
const length = yield* generate({ reason: "max_output_tokens" })
|
||||
const contentFilter = yield* generate({ reason: "content_filter" })
|
||||
const unknown = yield* generate({})
|
||||
const custom = yield* generate({ reason: "provider_limit" })
|
||||
|
||||
expect([length.finishReason, contentFilter.finishReason, unknown.finishReason, custom.finishReason]).toEqual([
|
||||
{ normalized: "length", raw: "max_output_tokens" },
|
||||
{ normalized: "content-filter", raw: "content_filter" },
|
||||
{ normalized: "unknown", raw: undefined },
|
||||
{ normalized: "unknown", raw: "provider_limit" },
|
||||
])
|
||||
}),
|
||||
)
|
||||
|
||||
// OpenAI's documented stream orders output text within one message item; no
|
||||
// provider-valid same-kind overlap is evidenced, so done boundaries close it.
|
||||
it.effect("closes sequential output messages before starting the next", () =>
|
||||
@@ -920,8 +960,8 @@ describe("OpenAI Responses route", () => {
|
||||
{ type: "text-delta", id: "msg_1", text: "Hello" },
|
||||
{ type: "reasoning-end", id: "rs_1" },
|
||||
{ type: "text-end", id: "msg_1" },
|
||||
{ type: "step-finish", index: 0, reason: "stop" },
|
||||
{ type: "finish", reason: "stop" },
|
||||
{ type: "step-finish", index: 0, reason: { normalized: "stop", raw: undefined } },
|
||||
{ type: "finish", reason: { normalized: "stop", raw: undefined } },
|
||||
])
|
||||
expect(response.events.filter((event) => event.type === "finish")).toHaveLength(1)
|
||||
expect(response.message.content).toEqual([
|
||||
@@ -966,7 +1006,7 @@ describe("OpenAI Responses route", () => {
|
||||
it.effect("streams each reasoning summary part as a separate block", () =>
|
||||
Effect.gen(function* () {
|
||||
const response = yield* LLMClient.generate(
|
||||
LLM.updateRequest(request, { providerOptions: { openai: { store: false } } }),
|
||||
LLMRequest.update(request, { providerOptions: { openai: { store: false } } }),
|
||||
).pipe(
|
||||
Effect.provide(
|
||||
fixedResponse(
|
||||
@@ -1012,8 +1052,8 @@ describe("OpenAI Responses route", () => {
|
||||
id: "rs_1:1",
|
||||
providerMetadata: { openai: { itemId: "rs_1", reasoningEncryptedContent: "encrypted-state" } },
|
||||
},
|
||||
{ type: "step-finish", index: 0, reason: "stop" },
|
||||
{ type: "finish", reason: "stop" },
|
||||
{ type: "step-finish", index: 0, reason: { normalized: "stop", raw: undefined } },
|
||||
{ type: "finish", reason: { normalized: "stop", raw: undefined } },
|
||||
])
|
||||
}),
|
||||
)
|
||||
@@ -1021,7 +1061,7 @@ describe("OpenAI Responses route", () => {
|
||||
it.effect("closes reasoning summary parts when storage is not disabled", () =>
|
||||
Effect.gen(function* () {
|
||||
const response = yield* LLMClient.generate(
|
||||
LLM.updateRequest(request, { providerOptions: { openai: { store: true } } }),
|
||||
LLMRequest.update(request, { providerOptions: { openai: { store: true } } }),
|
||||
).pipe(
|
||||
Effect.provide(
|
||||
fixedResponse(
|
||||
@@ -1348,8 +1388,8 @@ describe("OpenAI Responses route", () => {
|
||||
{ type: "response.completed", response: { usage: { input_tokens: 5, output_tokens: 1 } } },
|
||||
)
|
||||
const response = yield* LLMClient.generate(
|
||||
LLM.updateRequest(request, {
|
||||
tools: [{ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } }],
|
||||
LLMRequest.update(request, {
|
||||
tools: [ToolDefinition.make({ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } })],
|
||||
}),
|
||||
).pipe(Effect.provide(fixedResponse(body)))
|
||||
const usage = new Usage({
|
||||
@@ -1396,10 +1436,16 @@ describe("OpenAI Responses route", () => {
|
||||
providerExecuted: undefined,
|
||||
providerMetadata: { openai: { itemId: "item_1" } },
|
||||
},
|
||||
{ type: "step-finish", index: 0, reason: "tool-calls", usage, providerMetadata: undefined },
|
||||
{
|
||||
type: "step-finish",
|
||||
index: 0,
|
||||
reason: { normalized: "tool-calls", raw: undefined },
|
||||
usage,
|
||||
providerMetadata: undefined,
|
||||
},
|
||||
{
|
||||
type: "finish",
|
||||
reason: "tool-calls",
|
||||
reason: { normalized: "tool-calls", raw: undefined },
|
||||
providerMetadata: undefined,
|
||||
usage,
|
||||
},
|
||||
@@ -1428,8 +1474,8 @@ describe("OpenAI Responses route", () => {
|
||||
{ type: "response.completed", response: { usage: { input_tokens: 5, output_tokens: 1 } } },
|
||||
)
|
||||
const response = yield* LLMClient.generate(
|
||||
LLM.updateRequest(request, {
|
||||
tools: [{ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } }],
|
||||
LLMRequest.update(request, {
|
||||
tools: [ToolDefinition.make({ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } })],
|
||||
}),
|
||||
).pipe(Effect.provide(fixedResponse(body)))
|
||||
|
||||
@@ -1439,7 +1485,7 @@ describe("OpenAI Responses route", () => {
|
||||
name: "lookup",
|
||||
raw: '{"query":"partial',
|
||||
})
|
||||
expect(response.finishReason).toBe("tool-calls")
|
||||
expect(response.finishReason.normalized).toBe("tool-calls")
|
||||
expect(response.events.some(LLMEvent.is.toolCall)).toBeFalse()
|
||||
}),
|
||||
)
|
||||
@@ -1466,7 +1512,7 @@ describe("OpenAI Responses route", () => {
|
||||
name: "lookup",
|
||||
raw: '{"query":"partial',
|
||||
})
|
||||
expect(response.finishReason).toBe("tool-calls")
|
||||
expect(response.finishReason.normalized).toBe("tool-calls")
|
||||
}),
|
||||
)
|
||||
|
||||
|
||||
@@ -4,6 +4,8 @@ import { LLM, Message } from "../../src"
|
||||
import { LLMClient } from "../../src/route"
|
||||
import * as OpenRouter from "../../src/providers/openrouter"
|
||||
import { it } from "../lib/effect"
|
||||
import { fixedResponse } from "../lib/http"
|
||||
import { sseEvents } from "../lib/sse"
|
||||
|
||||
describe("OpenRouter", () => {
|
||||
it.effect("prepares OpenRouter models through the OpenAI-compatible Chat route", () =>
|
||||
@@ -54,6 +56,42 @@ describe("OpenRouter", () => {
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("preserves the upstream provider finish reason", () =>
|
||||
Effect.gen(function* () {
|
||||
const model = OpenRouter.configure({ apiKey: "test-key" }).model("anthropic/claude-sonnet-4.6")
|
||||
const response = yield* LLMClient.generate(LLM.request({ model, prompt: "Say hello." })).pipe(
|
||||
Effect.provide(
|
||||
fixedResponse(
|
||||
sseEvents({
|
||||
choices: [{ delta: { content: "Hello" }, finish_reason: "stop", native_finish_reason: "end_turn" }],
|
||||
}),
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
expect(response.finishReason).toEqual({ normalized: "stop", raw: "end_turn" })
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("fails on a mid-stream provider error", () =>
|
||||
Effect.gen(function* () {
|
||||
const model = OpenRouter.configure({ apiKey: "test-key" }).model("openai/gpt-4o-mini")
|
||||
const error = yield* LLMClient.generate(LLM.request({ model, prompt: "Say hello." })).pipe(
|
||||
Effect.provide(
|
||||
fixedResponse(
|
||||
sseEvents({
|
||||
error: { code: 502, message: "Provider disconnected" },
|
||||
}),
|
||||
),
|
||||
),
|
||||
Effect.flip,
|
||||
)
|
||||
|
||||
expect(error.reason).toMatchObject({ _tag: "ProviderInternal" })
|
||||
expect(error.message).toContain("Provider disconnected")
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("preserves manually supplied reasoning details", () =>
|
||||
Effect.gen(function* () {
|
||||
const details = [
|
||||
|
||||
@@ -106,7 +106,7 @@ const readPdfRuntime = Tool.make({
|
||||
})
|
||||
|
||||
const expectCode = (response: LLMResponse) => {
|
||||
expect(response.finishReason).toBe("stop")
|
||||
expect(response.finishReason.normalized).toBe("stop")
|
||||
expect(response.text.toUpperCase()).toContain(CODE)
|
||||
}
|
||||
|
||||
@@ -166,7 +166,7 @@ describe("PDF recorded", () => {
|
||||
tools: { read_pdf: readPdfRuntime },
|
||||
}).pipe(Stream.runCollect),
|
||||
)
|
||||
expect(events.at(-1)).toMatchObject({ type: "finish", reason: "stop" })
|
||||
expect(events.at(-1)).toMatchObject({ type: "finish", reason: { normalized: "stop" } })
|
||||
expect(LLMResponse.text({ events }).toUpperCase()).toContain(CODE)
|
||||
return
|
||||
}
|
||||
|
||||
@@ -3,6 +3,7 @@ import { Effect, Schema } from "effect"
|
||||
import {
|
||||
LLM,
|
||||
LLMEvent,
|
||||
LLMRequest,
|
||||
LLMResponse,
|
||||
Message,
|
||||
ToolRuntime,
|
||||
@@ -11,7 +12,6 @@ import {
|
||||
toDefinitions,
|
||||
type ContentPart,
|
||||
type FinishReason,
|
||||
type LLMRequest,
|
||||
type Model,
|
||||
} from "../src"
|
||||
import { LLMClient } from "../src/route"
|
||||
@@ -91,7 +91,7 @@ const restroomImage = () =>
|
||||
export const runWeatherToolLoop = (request: LLMRequest) =>
|
||||
Effect.gen(function* () {
|
||||
const tools = { [weatherToolName]: weatherRuntimeTool }
|
||||
let next = LLM.updateRequest(request, { tools: toDefinitions(tools) })
|
||||
let next = LLMRequest.update(request, { tools: toDefinitions(tools) })
|
||||
const events: LLMEvent[] = []
|
||||
|
||||
for (let step = 0; step < 10; step++) {
|
||||
@@ -108,7 +108,7 @@ export const runWeatherToolLoop = (request: LLMRequest) =>
|
||||
ToolRuntime.dispatch(tools, call).pipe(Effect.map((result) => [call, result] as const)),
|
||||
)
|
||||
events.push(...dispatched.flatMap(([, result]) => result.events))
|
||||
next = LLM.updateRequest(next, {
|
||||
next = LLMRequest.update(next, {
|
||||
messages: [
|
||||
...next.messages,
|
||||
Message.assistant(assistantContent(response.events)),
|
||||
@@ -123,10 +123,8 @@ export const runWeatherToolLoop = (request: LLMRequest) =>
|
||||
const assistantContent = (events: ReadonlyArray<LLMEvent>) =>
|
||||
events.reduce(LLMResponse.reduce, LLMResponse.empty()).message.content
|
||||
|
||||
export const expectFinish = (
|
||||
events: ReadonlyArray<LLMEvent>,
|
||||
reason: Extract<LLMEvent, { readonly type: "finish" }>["reason"],
|
||||
) => expect(events.at(-1)).toMatchObject({ type: "finish", reason })
|
||||
export const expectFinish = (events: ReadonlyArray<LLMEvent>, reason: FinishReason) =>
|
||||
expect(events.at(-1)).toMatchObject({ type: "finish", reason: { normalized: reason } })
|
||||
|
||||
export const expectWeatherToolCall = (response: LLMResponse) =>
|
||||
expect(response.toolCalls).toMatchObject([
|
||||
@@ -136,10 +134,10 @@ export const expectWeatherToolCall = (response: LLMResponse) =>
|
||||
export const expectWeatherToolLoop = (events: ReadonlyArray<LLMEvent>) => {
|
||||
const finishes = events.filter(LLMEvent.is.finish)
|
||||
expect(finishes).toHaveLength(1)
|
||||
expect(finishes[0]?.reason).toBe("stop")
|
||||
expect(finishes[0]?.reason.normalized).toBe("stop")
|
||||
|
||||
const stepFinishes = events.filter(LLMEvent.is.stepFinish)
|
||||
expect(stepFinishes.map((event) => event.reason)).toEqual(["tool-calls", "stop"])
|
||||
expect(stepFinishes.map((event) => event.reason.normalized)).toEqual(["tool-calls", "stop"])
|
||||
|
||||
const toolCalls = events.filter(LLMEvent.is.toolCall)
|
||||
expect(toolCalls).toHaveLength(1)
|
||||
@@ -503,7 +501,7 @@ export const eventSummary = (events: ReadonlyArray<LLMEvent>) => {
|
||||
continue
|
||||
}
|
||||
if (event.type === "finish") {
|
||||
summary.push({ type: "finish", reason: event.reason, usage: usageSummary(event.usage) })
|
||||
summary.push({ type: "finish", reason: event.reason.normalized, usage: usageSummary(event.usage) })
|
||||
}
|
||||
}
|
||||
return summary.map((item) => Object.fromEntries(Object.entries(item).filter((entry) => entry[1] !== undefined)))
|
||||
|
||||
@@ -14,11 +14,11 @@ describe("LLMResponse reducer", () => {
|
||||
LLMEvent.reasoningEnd({ id: "r1", providerMetadata: { anthropic: { signature: "sig" } } }),
|
||||
LLMEvent.textDelta({ id: "t1", text: "Answer" }),
|
||||
LLMEvent.textEnd({ id: "t1" }),
|
||||
LLMEvent.finish({ reason: "stop", usage: { outputTokens: 5 } }),
|
||||
LLMEvent.finish({ reason: { normalized: "stop" }, usage: { outputTokens: 5 } }),
|
||||
]
|
||||
const response = LLMResponse.fromEvents(events)
|
||||
|
||||
expect(response?.finishReason).toBe("stop")
|
||||
expect(response?.finishReason).toEqual({ normalized: "stop" })
|
||||
expect(response?.usage).toMatchObject({ outputTokens: 5 })
|
||||
expect(response?.events).toEqual(events)
|
||||
expect(response?.events.map((event) => event.type)).toEqual([
|
||||
@@ -62,18 +62,26 @@ describe("LLMResponse reducer", () => {
|
||||
|
||||
test("uses terminal usage when present and keeps prior usage when finish omits it", () => {
|
||||
const withFinishUsage = LLMResponse.fromEvents([
|
||||
LLMEvent.stepFinish({ index: 0, reason: "stop", usage: { inputTokens: 3 } }),
|
||||
LLMEvent.finish({ reason: "stop", usage: { outputTokens: 2 } }),
|
||||
LLMEvent.stepFinish({ index: 0, reason: { normalized: "stop" }, usage: { inputTokens: 3 } }),
|
||||
LLMEvent.finish({ reason: { normalized: "stop" }, usage: { outputTokens: 2 } }),
|
||||
])
|
||||
const withoutFinishUsage = LLMResponse.fromEvents([
|
||||
LLMEvent.stepFinish({ index: 0, reason: "stop", usage: { inputTokens: 3 } }),
|
||||
LLMEvent.finish({ reason: "stop" }),
|
||||
LLMEvent.stepFinish({ index: 0, reason: { normalized: "stop" }, usage: { inputTokens: 3 } }),
|
||||
LLMEvent.finish({ reason: { normalized: "stop" } }),
|
||||
])
|
||||
|
||||
expect(withFinishUsage?.usage).toMatchObject({ outputTokens: 2 })
|
||||
expect(withoutFinishUsage?.usage).toMatchObject({ inputTokens: 3 })
|
||||
})
|
||||
|
||||
test("preserves the raw finish reason", () => {
|
||||
const response = LLMResponse.fromEvents([
|
||||
LLMEvent.finish({ reason: { normalized: "unknown", raw: "provider_limit" } }),
|
||||
])
|
||||
|
||||
expect(response?.finishReason).toEqual({ normalized: "unknown", raw: "provider_limit" })
|
||||
})
|
||||
|
||||
test("assembles tool-call content only after the completed tool call event", () => {
|
||||
const pending = reduce([
|
||||
LLMEvent.toolInputStart({ id: "call_1", name: "lookup" }),
|
||||
@@ -88,7 +96,7 @@ describe("LLMResponse reducer", () => {
|
||||
LLMEvent.toolInputDelta({ id: "call_1", name: "lookup", text: ':"weather"}' }),
|
||||
LLMEvent.toolInputEnd({ id: "call_1", name: "lookup" }),
|
||||
LLMEvent.toolCall({ id: "call_1", name: "lookup", input: { query: "weather" } }),
|
||||
LLMEvent.finish({ reason: "tool-calls" }),
|
||||
LLMEvent.finish({ reason: { normalized: "tool-calls" } }),
|
||||
])
|
||||
|
||||
expect(response?.message.content).toEqual([
|
||||
|
||||
@@ -48,8 +48,12 @@ describe("llm schema", () => {
|
||||
})
|
||||
|
||||
test("finish constructors accept usage input", () => {
|
||||
expect(LLMEvent.stepFinish({ index: 0, reason: "stop", usage: { inputTokens: 1 } }).usage).toBeInstanceOf(Usage)
|
||||
expect(LLMEvent.finish({ reason: "stop", usage: { outputTokens: 2 } }).usage).toBeInstanceOf(Usage)
|
||||
expect(
|
||||
LLMEvent.stepFinish({ index: 0, reason: { normalized: "stop" }, usage: { inputTokens: 1 } }).usage,
|
||||
).toBeInstanceOf(Usage)
|
||||
expect(LLMEvent.finish({ reason: { normalized: "stop" }, usage: { outputTokens: 2 } }).usage).toBeInstanceOf(
|
||||
Usage,
|
||||
)
|
||||
})
|
||||
|
||||
test("content part tagged union exposes guards", () => {
|
||||
|
||||
@@ -539,6 +539,7 @@ describe("LLMClient tools", () => {
|
||||
},
|
||||
{ type: "content_block_stop", index: 1 },
|
||||
{ type: "message_delta", delta: { stop_reason: "tool_use" }, usage: { output_tokens: 5 } },
|
||||
{ type: "message_stop" },
|
||||
)
|
||||
: sseEvents(
|
||||
{ type: "message_start", message: { usage: { input_tokens: 5 } } },
|
||||
@@ -546,6 +547,7 @@ describe("LLMClient tools", () => {
|
||||
{ type: "content_block_delta", index: 0, delta: { type: "text_delta", text: "Done." } },
|
||||
{ type: "content_block_stop", index: 0 },
|
||||
{ type: "message_delta", delta: { stop_reason: "end_turn" }, usage: { output_tokens: 1 } },
|
||||
{ type: "message_stop" },
|
||||
),
|
||||
{ headers: { "content-type": "text/event-stream" } },
|
||||
)
|
||||
@@ -553,7 +555,7 @@ describe("LLMClient tools", () => {
|
||||
)
|
||||
|
||||
yield* TestToolRuntime.runTools({
|
||||
request: LLM.updateRequest(baseRequest, {
|
||||
request: LLMRequest.update(baseRequest, {
|
||||
model: AnthropicMessages.route
|
||||
.with({ auth: Auth.header("x-api-key", "test") })
|
||||
.model({ id: "claude-sonnet-4-5" }),
|
||||
@@ -801,6 +803,7 @@ describe("LLMClient tools", () => {
|
||||
{ type: "content_block_delta", index: 2, delta: { type: "text_delta", text: "Done." } },
|
||||
{ type: "content_block_stop", index: 2 },
|
||||
{ type: "message_delta", delta: { stop_reason: "end_turn" }, usage: { output_tokens: 8 } },
|
||||
{ type: "message_stop" },
|
||||
),
|
||||
{ headers: { "content-type": "text/event-stream" } },
|
||||
)
|
||||
@@ -808,7 +811,7 @@ describe("LLMClient tools", () => {
|
||||
)
|
||||
const events = Array.from(
|
||||
yield* TestToolRuntime.runTools({
|
||||
request: LLM.updateRequest(baseRequest, {
|
||||
request: LLMRequest.update(baseRequest, {
|
||||
model: AnthropicMessages.route
|
||||
.with({ auth: Auth.header("x-api-key", "test") })
|
||||
.model({ id: "claude-sonnet-4-5" }),
|
||||
|
||||
@@ -97,6 +97,7 @@ export async function setupTimeline(
|
||||
locale?: string
|
||||
deviceScaleFactor?: number
|
||||
seedHistory?: boolean
|
||||
protocol?: "v1" | "v2"
|
||||
} = {},
|
||||
) {
|
||||
const sessions = input.sessions ?? [session()]
|
||||
@@ -114,6 +115,7 @@ export async function setupTimeline(
|
||||
retry: input.eventRetry ?? 20,
|
||||
})
|
||||
await mockOpenCodeServer(page, {
|
||||
protocol: input.protocol,
|
||||
directory,
|
||||
project: project(),
|
||||
provider: provider(),
|
||||
|
||||
@@ -89,8 +89,8 @@ test("reconnects after a stream error", async ({ page }) => {
|
||||
expect((await timeline.transport.connections())[0]?.endedBy).toBe("error")
|
||||
})
|
||||
|
||||
test("records event IDs and reconnect Last-Event-ID headers", async ({ page }) => {
|
||||
const timeline = await setupTimeline(page, { eventRetry: 10 })
|
||||
test("does not request replay when reconnecting the volatile V2 event stream", async ({ page }) => {
|
||||
const timeline = await setupTimeline(page, { eventRetry: 10, protocol: "v2" })
|
||||
const first = await timeline.transport.send(partUpdated(textPart("prt_transport_id", "event with id")), {
|
||||
id: "timeline-event-7",
|
||||
})
|
||||
@@ -100,7 +100,7 @@ test("records event IDs and reconnect Last-Event-ID headers", async ({ page }) =
|
||||
const connection = await timeline.transport.waitForConnection({ after: first.connectionID })
|
||||
|
||||
expect(first.eventID).toBe("timeline-event-7")
|
||||
expect(connection.headers["last-event-id"]).toBe("timeline-event-7")
|
||||
expect(connection.headers["last-event-id"]).toBeUndefined()
|
||||
})
|
||||
|
||||
test("passes through non-event fetches", async ({ page }) => {
|
||||
|
||||
@@ -4,6 +4,7 @@ const emptyList = new Set(["/skill", "/command", "/lsp", "/formatter", "/vcs/sta
|
||||
const emptyObject = new Set(["/global/config", "/config", "/provider/auth", "/mcp", "/experimental/resource"])
|
||||
|
||||
export interface MockServerConfig {
|
||||
protocol?: "v1" | "v2"
|
||||
provider: unknown
|
||||
directory: string
|
||||
project: unknown
|
||||
@@ -53,8 +54,20 @@ export async function mockOpenCodeServer(page: Page, config: MockServerConfig) {
|
||||
if (url.port !== targetPort && url.port !== appPort) return route.fallback()
|
||||
|
||||
const path = url.pathname
|
||||
if (path === "/global/event" || path === "/event") return sse(route, config.events?.(), config.eventRetry)
|
||||
if (path === "/global/health") return json(route, { healthy: true })
|
||||
if (path === "/global/event" || path === "/event" || path === "/api/event") {
|
||||
const events = config.events?.()
|
||||
return sse(
|
||||
route,
|
||||
path === "/api/event"
|
||||
? [{ id: "evt_mock_connected", type: "server.connected", data: {} }, ...(events?.map(currentEvent) ?? [])]
|
||||
: events,
|
||||
config.eventRetry,
|
||||
)
|
||||
}
|
||||
if (path === "/global/health")
|
||||
return config.protocol === "v2" ? json(route, {}, undefined, 404) : json(route, { healthy: true })
|
||||
if (path === "/api/health" && config.protocol === "v2")
|
||||
return json(route, { healthy: true, version: "2.0.0", pid: 1 })
|
||||
if (path === "/experimental/capabilities") return json(route, { backgroundSubagents: true })
|
||||
if (path === "/permission")
|
||||
return json(route, typeof config.permissions === "function" ? config.permissions() : (config.permissions ?? []))
|
||||
@@ -83,10 +96,129 @@ export async function mockOpenCodeServer(page: Page, config: MockServerConfig) {
|
||||
},
|
||||
data: [],
|
||||
})
|
||||
if (path === "/api/agent")
|
||||
return json(route, {
|
||||
location: location(config),
|
||||
data: [
|
||||
{
|
||||
id: "build",
|
||||
name: "Build",
|
||||
mode: "primary",
|
||||
hidden: false,
|
||||
request: { settings: {}, headers: {}, body: {} },
|
||||
permissions: [],
|
||||
},
|
||||
],
|
||||
})
|
||||
if (path === "/api/command") return json(route, { location: location(config), data: [] })
|
||||
if (path === "/api/mcp") return json(route, { location: location(config), data: [] })
|
||||
if (path === "/api/mcp/resource")
|
||||
return json(route, { location: location(config), data: { resources: [], templates: [] } })
|
||||
const integration = path.match(/^\/api\/integration\/([^/]+)$/)?.[1]
|
||||
if (integration && route.request().method() === "GET")
|
||||
return json(route, {
|
||||
location: location(config),
|
||||
data: { id: integration, name: integration, methods: [{ type: "key", label: "API key" }], connections: [] },
|
||||
})
|
||||
if (/^\/api\/integration\/[^/]+\/connect\/key$/.test(path) && route.request().method() === "POST")
|
||||
return route.fulfill({ status: 204, headers: { "access-control-allow-origin": "*" } })
|
||||
if (path === "/api/project") return json(route, [config.project])
|
||||
if (path === "/api/project/current")
|
||||
return json(route, { id: (config.project as { id?: string }).id, directory: config.directory })
|
||||
if (path.startsWith("/api/project/") && route.request().method() === "PATCH") return json(route, config.project)
|
||||
if (path === "/api/path")
|
||||
return json(route, {
|
||||
state: config.directory,
|
||||
config: config.directory,
|
||||
worktree: config.directory,
|
||||
directory: config.directory,
|
||||
home: "C:/OpenCode",
|
||||
})
|
||||
if (path === "/api/permission/request")
|
||||
return json(route, {
|
||||
location: location(config),
|
||||
data: (typeof config.permissions === "function" ? config.permissions() : (config.permissions ?? [])).map(
|
||||
currentPermission,
|
||||
),
|
||||
})
|
||||
if (path === "/api/question/request")
|
||||
return json(route, {
|
||||
location: location(config),
|
||||
data: typeof config.questions === "function" ? config.questions() : (config.questions ?? []),
|
||||
})
|
||||
if (path === "/api/vcs")
|
||||
return json(route, { location: location(config), data: { branch: "main", defaultBranch: "main" } })
|
||||
if (path === "/api/vcs/status") return json(route, { location: location(config), data: [] })
|
||||
if (path === "/api/vcs/diff") return json(route, { location: location(config), data: config.vcsDiff ?? [] })
|
||||
if (path === "/api/pty/shells") return json(route, { location: location(config), data: [] })
|
||||
if (/^\/api\/pty\/[^/]+\/connect-token$/.test(path))
|
||||
return json(route, { location: location(config), data: { ticket: "e2e-ticket", expires_in: 60 } })
|
||||
if (emptyObject.has(path)) return json(route, {})
|
||||
if (emptyList.has(path)) return json(route, [])
|
||||
if (path === "/api/session") {
|
||||
const directory = url.searchParams.get("directory")
|
||||
const parentID = url.searchParams.get("parentID")
|
||||
const limit = Number(url.searchParams.get("limit") ?? 50)
|
||||
const offset = Number(url.searchParams.get("cursor") ?? 0)
|
||||
const sessions = config.sessions
|
||||
.filter((session) => !directory || session.directory === directory)
|
||||
.filter((session) => parentID !== "null" || session.parentID === undefined)
|
||||
.filter((session) => {
|
||||
const search = url.searchParams.get("search")?.toLowerCase()
|
||||
return (
|
||||
!search ||
|
||||
String(session.title ?? "")
|
||||
.toLowerCase()
|
||||
.includes(search)
|
||||
)
|
||||
})
|
||||
const ordered = url.searchParams.get("order") === "asc" ? sessions.toReversed() : sessions
|
||||
const data = ordered.slice(offset, offset + limit)
|
||||
const next = offset + limit < ordered.length ? String(offset + limit) : undefined
|
||||
return json(route, {
|
||||
data: data.map((session) => currentSession(session, config.directory)),
|
||||
cursor: { next },
|
||||
})
|
||||
}
|
||||
if (path === "/api/session/active") {
|
||||
const statuses = (config.sessionStatus ?? {}) as Record<string, { type?: string }>
|
||||
return json(route, {
|
||||
data: Object.fromEntries(
|
||||
Object.entries(statuses).flatMap(([id, status]) =>
|
||||
status.type === "idle" ? [] : [[id, { type: "running" }]],
|
||||
),
|
||||
),
|
||||
})
|
||||
}
|
||||
if (/^\/api\/session\/[^/]+\/shell$/.test(path) && route.request().method() === "POST") {
|
||||
return route.fulfill({ status: 204, headers: { "access-control-allow-origin": "*" } })
|
||||
}
|
||||
if (/^\/api\/session\/[^/]+\/question\/[^/]+\/(reply|reject)$/.test(path) && route.request().method() === "POST") {
|
||||
return route.fulfill({ status: 204, headers: { "access-control-allow-origin": "*" } })
|
||||
}
|
||||
if (/^\/api\/session\/[^/]+\/permission\/[^/]+\/reply$/.test(path) && route.request().method() === "POST") {
|
||||
return route.fulfill({ status: 204, headers: { "access-control-allow-origin": "*" } })
|
||||
}
|
||||
if (
|
||||
/^\/api\/session\/[^/]+\/(archive|rename|interrupt|revert\/clear|revert\/commit)$/.test(path) &&
|
||||
route.request().method() === "POST"
|
||||
) {
|
||||
return route.fulfill({ status: 204, headers: { "access-control-allow-origin": "*" } })
|
||||
}
|
||||
if (/^\/api\/session\/[^/]+$/.test(path) && route.request().method() === "DELETE") {
|
||||
return route.fulfill({ status: 204, headers: { "access-control-allow-origin": "*" } })
|
||||
}
|
||||
if (path in staticRoutes) return json(route, staticRoutes[path])
|
||||
|
||||
const currentSessionMatch = path.match(/^\/api\/session\/([^/]+)$/)
|
||||
if (currentSessionMatch) {
|
||||
const session = config.sessions.find((item) => item.id === currentSessionMatch[1])
|
||||
if (!session) return json(route, { error: "Session not found" }, undefined, 404)
|
||||
return json(route, {
|
||||
data: currentSession(session, config.directory),
|
||||
})
|
||||
}
|
||||
|
||||
const sessionMatch = path.match(/^\/session\/([^/]+)$/)
|
||||
if (sessionMatch) {
|
||||
const session = config.sessions.find((s) => s.id === sessionMatch[1])
|
||||
@@ -107,6 +239,24 @@ export async function mockOpenCodeServer(page: Page, config: MockServerConfig) {
|
||||
|
||||
if (/^\/session\/[^/]+\/(children|diff)$/.test(path)) return json(route, [])
|
||||
|
||||
const currentMessagesMatch = path.match(/^\/api\/session\/([^/]+)\/message$/)
|
||||
if (currentMessagesMatch) {
|
||||
const token = url.searchParams.get("cursor") ?? undefined
|
||||
const before = token ? cursors.get(token) : undefined
|
||||
if (token && !before) return json(route, { error: "Invalid cursor" }, undefined, 400)
|
||||
config.onMessages?.({ sessionID: currentMessagesMatch[1], before, phase: "start" })
|
||||
await config.beforeMessagesResponse?.({ sessionID: currentMessagesMatch[1]!, before })
|
||||
if (config.messageDelay !== undefined) await new Promise((resolve) => setTimeout(resolve, config.messageDelay))
|
||||
const pageData = config.pageMessages(currentMessagesMatch[1], Number(url.searchParams.get("limit") ?? 50), before)
|
||||
config.onMessages?.({ sessionID: currentMessagesMatch[1], before, phase: "end" })
|
||||
const cursor = pageData.cursor ? `cursor_${++nextCursor}` : undefined
|
||||
if (cursor) cursors.set(cursor, pageData.cursor!)
|
||||
return json(route, {
|
||||
data: pageData.items.map(currentMessage).reverse(),
|
||||
cursor: { next: cursor },
|
||||
})
|
||||
}
|
||||
|
||||
const messagesMatch = path.match(/^\/session\/([^/]+)\/message$/)
|
||||
if (messagesMatch) {
|
||||
const token = url.searchParams.get("before") ?? undefined
|
||||
@@ -129,6 +279,115 @@ export async function mockOpenCodeServer(page: Page, config: MockServerConfig) {
|
||||
})
|
||||
}
|
||||
|
||||
function location(config: MockServerConfig) {
|
||||
return {
|
||||
directory: config.directory,
|
||||
project: { id: (config.project as { id?: string }).id, directory: config.directory },
|
||||
}
|
||||
}
|
||||
|
||||
function currentPermission(value: unknown) {
|
||||
const permission = value as Record<string, unknown>
|
||||
if (permission.action) return permission
|
||||
const tool = permission.tool as { messageID?: string; callID?: string } | undefined
|
||||
return {
|
||||
id: permission.id,
|
||||
sessionID: permission.sessionID,
|
||||
action: permission.permission,
|
||||
resources: permission.patterns ?? [],
|
||||
save: permission.always,
|
||||
metadata: permission.metadata,
|
||||
source:
|
||||
tool?.messageID && tool.callID ? { type: "tool", messageID: tool.messageID, callID: tool.callID } : undefined,
|
||||
}
|
||||
}
|
||||
|
||||
export function currentSession(session: { id: string } & Record<string, unknown>, fallbackDirectory?: string) {
|
||||
const time = session.time && typeof session.time === "object" ? session.time : {}
|
||||
return {
|
||||
id: session.id,
|
||||
parentID: session.parentID,
|
||||
projectID: session.projectID ?? "project",
|
||||
agent: session.agent ?? "build",
|
||||
model: session.model ?? { id: "mock-model", providerID: "mock-provider" },
|
||||
cost: session.cost ?? 0,
|
||||
tokens: session.tokens ?? { input: 0, output: 0, reasoning: 0, cache: { read: 0, write: 0 } },
|
||||
time: {
|
||||
created: "created" in time && typeof time.created === "number" ? time.created : 0,
|
||||
updated: "updated" in time && typeof time.updated === "number" ? time.updated : 0,
|
||||
...(session.time && typeof session.time === "object" && "archived" in session.time
|
||||
? { archived: session.time.archived }
|
||||
: {}),
|
||||
},
|
||||
title: session.title ?? session.id,
|
||||
location: {
|
||||
directory: typeof session.directory === "string" ? session.directory : fallbackDirectory,
|
||||
...(typeof session.workspaceID === "string" ? { workspaceID: session.workspaceID } : {}),
|
||||
},
|
||||
subpath: session.path,
|
||||
revert: session.revert,
|
||||
}
|
||||
}
|
||||
|
||||
function currentMessage(value: unknown) {
|
||||
const item = value as {
|
||||
info: Record<string, unknown> & { id: string; role: "user" | "assistant"; time: { created: number } }
|
||||
parts: Array<Record<string, unknown> & { type: string }>
|
||||
}
|
||||
if (item.info.role === "user") {
|
||||
return {
|
||||
id: item.info.id,
|
||||
type: "user",
|
||||
time: item.info.time,
|
||||
text: item.parts
|
||||
.flatMap((part) => (part.type === "text" && typeof part.text === "string" ? [part.text] : []))
|
||||
.join("\n"),
|
||||
}
|
||||
}
|
||||
return {
|
||||
id: item.info.id,
|
||||
type: "assistant",
|
||||
time: item.info.time,
|
||||
agent: item.info.agent ?? "build",
|
||||
model: { id: item.info.modelID ?? "model", providerID: item.info.providerID ?? "provider" },
|
||||
cost: item.info.cost,
|
||||
tokens: item.info.tokens,
|
||||
error: item.info.error,
|
||||
content: item.parts.flatMap<unknown>((part) => {
|
||||
if (part.type === "text" || part.type === "reasoning") return [{ type: part.type, text: part.text ?? "" }]
|
||||
if (part.type !== "tool") return []
|
||||
const state = part.state as Record<string, unknown>
|
||||
return [
|
||||
{
|
||||
type: "tool",
|
||||
id: part.id,
|
||||
name: part.tool,
|
||||
time: state.time ?? { created: item.info.time.created },
|
||||
state:
|
||||
state.status === "pending"
|
||||
? { status: "streaming", input: state.raw ?? JSON.stringify(state.input ?? {}) }
|
||||
: state.status === "completed"
|
||||
? {
|
||||
status: "completed",
|
||||
input: state.input ?? {},
|
||||
structured: state.metadata ?? {},
|
||||
content: [{ type: "text", text: state.output ?? "" }],
|
||||
}
|
||||
: state.status === "error"
|
||||
? {
|
||||
status: "error",
|
||||
input: state.input ?? {},
|
||||
structured: state.metadata ?? {},
|
||||
content: [],
|
||||
error: { type: "ToolError", message: state.error ?? "Tool failed" },
|
||||
}
|
||||
: { status: "running", input: state.input ?? {}, structured: state.metadata ?? {}, content: [] },
|
||||
},
|
||||
]
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
function json(route: Route, body: unknown, headers?: Record<string, string>, status = 200) {
|
||||
return route.fulfill({
|
||||
status,
|
||||
@@ -149,3 +408,18 @@ function sse(route: Route, events?: unknown[], retry?: number) {
|
||||
body: `${retry === undefined ? "" : `retry: ${retry}\n\n`}${events?.map((event) => `data: ${JSON.stringify(event)}\n\n`).join("") || ": ok\n\n"}`,
|
||||
})
|
||||
}
|
||||
|
||||
function currentEvent(input: unknown) {
|
||||
if (!input || typeof input !== "object" || !("payload" in input)) return input
|
||||
const envelope = input as { directory?: string; payload?: unknown }
|
||||
if (!envelope.payload || typeof envelope.payload !== "object") return input
|
||||
const payload = envelope.payload as { id?: string; type?: string; properties?: unknown }
|
||||
if (!payload.type) return input
|
||||
return {
|
||||
id: payload.id ?? `evt_mock_${Date.now()}`,
|
||||
created: Date.now(),
|
||||
type: payload.type,
|
||||
data: payload.properties ?? {},
|
||||
location: envelope.directory && envelope.directory !== "global" ? { directory: envelope.directory } : undefined,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3,7 +3,7 @@ import type { Page } from "@playwright/test"
|
||||
export type SseConnectionRecord = {
|
||||
id: number
|
||||
url: string
|
||||
path: "/global/event" | "/event"
|
||||
path: "/global/event" | "/event" | "/api/event"
|
||||
headers: Record<string, string>
|
||||
openedAt: number
|
||||
endedAt?: number
|
||||
@@ -93,6 +93,21 @@ export async function installSseTransport<T>(
|
||||
eventOptions.retry === undefined ? "" : `retry: ${eventOptions.retry}\n`,
|
||||
`data: ${JSON.stringify(payload)}\n\n`,
|
||||
].join("")
|
||||
const currentEvent = (input: unknown) => {
|
||||
if (!input || typeof input !== "object" || !("payload" in input)) return input
|
||||
const envelope = input as { directory?: string; payload?: unknown }
|
||||
if (!envelope.payload || typeof envelope.payload !== "object") return input
|
||||
const payload = envelope.payload as { id?: string; type?: string; properties?: unknown }
|
||||
if (!payload.type) return input
|
||||
return {
|
||||
id: payload.id ?? `evt_mock_${Date.now()}`,
|
||||
created: Date.now(),
|
||||
type: payload.type,
|
||||
data: payload.properties ?? {},
|
||||
location:
|
||||
envelope.directory && envelope.directory !== "global" ? { directory: envelope.directory } : undefined,
|
||||
}
|
||||
}
|
||||
const acknowledge = (
|
||||
connection: Connection,
|
||||
bytes: number,
|
||||
@@ -140,15 +155,13 @@ export async function installSseTransport<T>(
|
||||
output.forEach((chunk) => connection.controller.enqueue(chunk))
|
||||
return acknowledge(connection, input.bytes.length, output.length)
|
||||
}
|
||||
const encoded = input.deliveries.map((delivery) => ({
|
||||
delivery,
|
||||
bytes: encoder.encode(frame(delivery.payload, delivery.options)),
|
||||
}))
|
||||
const encoded = input.deliveries.map((delivery) => {
|
||||
const payload = connection.path === "/api/event" ? currentEvent(delivery.payload) : delivery.payload
|
||||
return { delivery, payload, bytes: encoder.encode(frame(payload, delivery.options)) }
|
||||
})
|
||||
encoded.forEach((item) => marker(item.delivery.options?.marker))
|
||||
if (input.burst) {
|
||||
const bytes = encoder.encode(
|
||||
encoded.map((item) => frame(item.delivery.payload, item.delivery.options)).join(""),
|
||||
)
|
||||
const bytes = encoder.encode(encoded.map((item) => frame(item.payload, item.delivery.options)).join(""))
|
||||
connection.controller.enqueue(bytes)
|
||||
return encoded.map((item) => acknowledge(connection, item.bytes.byteLength, 1, item.delivery.options?.id))
|
||||
}
|
||||
@@ -161,7 +174,10 @@ export async function installSseTransport<T>(
|
||||
const fetch = (input: RequestInfo | URL, init?: RequestInit) => {
|
||||
const request = new Request(input, init)
|
||||
const url = new URL(request.url)
|
||||
if (url.origin !== server || (url.pathname !== "/global/event" && url.pathname !== "/event"))
|
||||
if (
|
||||
url.origin !== server ||
|
||||
(url.pathname !== "/global/event" && url.pathname !== "/event" && url.pathname !== "/api/event")
|
||||
)
|
||||
return originalFetch(request)
|
||||
|
||||
const id = ++nextConnectionID
|
||||
@@ -177,6 +193,10 @@ export async function installSseTransport<T>(
|
||||
record.controller = controller
|
||||
connections.push(record)
|
||||
if (retry !== undefined) controller.enqueue(encoder.encode(`retry: ${retry}\n\n`))
|
||||
if (url.pathname === "/api/event")
|
||||
controller.enqueue(
|
||||
encoder.encode(frame({ id: `evt_mock_connected_${id}`, type: "server.connected", data: {} })),
|
||||
)
|
||||
request.signal.addEventListener(
|
||||
"abort",
|
||||
() => {
|
||||
|
||||
@@ -53,6 +53,7 @@
|
||||
"@dnd-kit/helpers": "0.5.0",
|
||||
"@dnd-kit/solid": "0.5.0",
|
||||
"@kobalte/core": "catalog:",
|
||||
"@opencode-ai/client": "file:vendor/opencode-ai-client-1.17.13.tgz",
|
||||
"@opencode-ai/core": "workspace:*",
|
||||
"@opencode-ai/schema": "workspace:*",
|
||||
"@opencode-ai/sdk": "workspace:*",
|
||||
|
||||
@@ -88,7 +88,7 @@ export const DialogSelectModelUnpaidV2: Component<{ model?: ModelState }> = (pro
|
||||
class="w-full"
|
||||
placement="right-start"
|
||||
gutter={6}
|
||||
delay="intent"
|
||||
openDelay={0}
|
||||
contentStyle={{ "font-family": "var(--v2-font-family-sans)" }}
|
||||
value={
|
||||
<ModelTooltip
|
||||
|
||||
@@ -57,7 +57,6 @@ export const DialogSelectModelUnpaid: Component<{ model?: ModelState }> = (props
|
||||
class="w-full"
|
||||
placement="right-start"
|
||||
gutter={12}
|
||||
delay="intent"
|
||||
value={
|
||||
<ModelTooltip
|
||||
model={item}
|
||||
|
||||
@@ -93,7 +93,7 @@ const ModelList: Component<{
|
||||
class="w-full"
|
||||
placement="right-start"
|
||||
gutter={12}
|
||||
delay="intent"
|
||||
openDelay={0}
|
||||
value={<ModelTooltip model={item} latest={item.latest} free={isFree(item.provider.id, item.cost)} />}
|
||||
>
|
||||
{node}
|
||||
@@ -452,7 +452,7 @@ export function ModelSelectorPopoverV2(props: {
|
||||
class="w-full"
|
||||
placement="right-start"
|
||||
gutter={6}
|
||||
delay="intent"
|
||||
openDelay={0}
|
||||
value={
|
||||
<ModelTooltip
|
||||
model={item}
|
||||
|
||||
@@ -47,6 +47,7 @@ export const PromptContextItems: Component<ContextItemsProps> = (props) => {
|
||||
</span>
|
||||
}
|
||||
placement="top"
|
||||
openDelay={800}
|
||||
>
|
||||
<div
|
||||
classList={{
|
||||
|
||||
@@ -52,7 +52,12 @@ export const PromptImageAttachments: Component<PromptImageAttachmentsProps> = (p
|
||||
<For each={props.comments ?? []}>
|
||||
{(item) => (
|
||||
<div class="relative group shrink-0">
|
||||
<TooltipV2 value={item.comment} placement="top" contentClass="max-w-[300px] break-words">
|
||||
<TooltipV2
|
||||
value={item.comment}
|
||||
placement="top"
|
||||
openDelay={800}
|
||||
contentClass="max-w-[300px] break-words"
|
||||
>
|
||||
<CommentCardV2
|
||||
comment={item.comment ?? ""}
|
||||
path={item.path}
|
||||
|
||||
@@ -607,6 +607,7 @@ export function Titlebar(props: { update?: TitlebarUpdate; debugTools?: { visibl
|
||||
placement="bottom"
|
||||
title={language.t("command.session.new")}
|
||||
keybind={command.keybind("session.new")}
|
||||
openDelay={800}
|
||||
>
|
||||
<Button
|
||||
variant="ghost"
|
||||
@@ -636,7 +637,7 @@ export function Titlebar(props: { update?: TitlebarUpdate; debugTools?: { visibl
|
||||
>
|
||||
<Show when={hasProjects() && nav()}>
|
||||
<div class="flex items-center gap-0 transition-transform">
|
||||
<Tooltip placement="bottom" value={language.t("common.goBack")}>
|
||||
<Tooltip placement="bottom" value={language.t("common.goBack")} openDelay={800}>
|
||||
<Button
|
||||
variant="ghost"
|
||||
icon="chevron-left"
|
||||
@@ -646,7 +647,7 @@ export function Titlebar(props: { update?: TitlebarUpdate; debugTools?: { visibl
|
||||
aria-label={language.t("common.goBack")}
|
||||
/>
|
||||
</Tooltip>
|
||||
<Tooltip placement="bottom" value={language.t("common.goForward")}>
|
||||
<Tooltip placement="bottom" value={language.t("common.goForward")} openDelay={800}>
|
||||
<Button
|
||||
variant="ghost"
|
||||
icon="chevron-right"
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { describe, expect, test } from "bun:test"
|
||||
import { coalesceServerEvents, enqueueServerEvent, resumeStreamAfterPageShow } from "./server-sdk"
|
||||
import { adaptServerEvent, coalesceServerEvents, enqueueServerEvent, resumeStreamAfterPageShow } from "./server-sdk"
|
||||
import type { OpenCodeEvent } from "@opencode-ai/client/promise"
|
||||
import type { Event } from "@opencode-ai/sdk/v2/client"
|
||||
|
||||
describe("resumeStreamAfterPageShow", () => {
|
||||
@@ -14,6 +15,23 @@ describe("resumeStreamAfterPageShow", () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe("adaptServerEvent", () => {
|
||||
test("preserves V2 events while adapting permission requests for existing consumers", () => {
|
||||
const current = {
|
||||
id: "evt_1",
|
||||
created: 1,
|
||||
type: "permission.v2.asked",
|
||||
data: { id: "perm_1", sessionID: "ses_1", action: "read", resources: ["src/**"] },
|
||||
} as OpenCodeEvent
|
||||
|
||||
expect(adaptServerEvent(current)).toMatchObject({
|
||||
type: "permission.asked",
|
||||
properties: { id: "perm_1", sessionID: "ses_1", permission: "read", patterns: ["src/**"] },
|
||||
current,
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
describe("coalesceServerEvents", () => {
|
||||
const delta = (value: string, field = "text", partID = "part") => ({
|
||||
directory: "/repo",
|
||||
@@ -34,6 +52,24 @@ describe("coalesceServerEvents", () => {
|
||||
expect(result[0]?.payload).toMatchObject({ id: "second", properties: { delta: "hello world" } })
|
||||
})
|
||||
|
||||
test("merges adjacent current text deltas", () => {
|
||||
const current = (id: string, value: string) =>
|
||||
adaptServerEvent({
|
||||
id,
|
||||
created: 1,
|
||||
type: "session.text.delta",
|
||||
location: { directory: "/repo" },
|
||||
data: { sessionID: "ses", assistantMessageID: "msg", ordinal: 0, delta: value },
|
||||
} as OpenCodeEvent)
|
||||
const result = coalesceServerEvents([
|
||||
{ directory: "/repo", payload: current("evt_1", "hello ") },
|
||||
{ directory: "/repo", payload: current("evt_2", "world") },
|
||||
])
|
||||
|
||||
expect(result).toHaveLength(1)
|
||||
expect(result[0]?.payload.current).toMatchObject({ id: "evt_2", data: { delta: "hello world" } })
|
||||
})
|
||||
|
||||
test("preserves event boundaries and distinct fields", () => {
|
||||
const status = {
|
||||
directory: "/repo",
|
||||
|
||||
@@ -1,21 +1,60 @@
|
||||
import type { OpenCodeEvent } from "@opencode-ai/client/promise"
|
||||
import type { Event } from "@opencode-ai/sdk/v2/client"
|
||||
import { createSimpleContext } from "@opencode-ai/ui/context"
|
||||
import { createGlobalEmitter } from "@solid-primitives/event-bus"
|
||||
import { makeEventListener } from "@solid-primitives/event-listener"
|
||||
import { type Accessor, batch, createMemo, onCleanup, onMount } from "solid-js"
|
||||
import { createSdkForServer } from "@/utils/server"
|
||||
import { createApiForServer, createSdkForServer, type ServerApi } from "@/utils/server"
|
||||
import { useLanguage } from "./language"
|
||||
import { usePlatform } from "./platform"
|
||||
import { ServerConnection, useServer } from "./server"
|
||||
import { createRefCountMap } from "@/utils/refcount"
|
||||
import { useGlobal } from "./global"
|
||||
import { ServerScope } from "@/utils/server-scope"
|
||||
import { detectServerProtocol, type ServerProtocol } from "@/utils/server-protocol"
|
||||
import { createCompatibleApi, type CompatibleApi } from "@/utils/server-compat"
|
||||
|
||||
const isAbortError = (error: unknown) =>
|
||||
error !== null && typeof error === "object" && "name" in error && error.name === "AbortError"
|
||||
|
||||
const isStreamClosed = (error: unknown, signal?: AbortSignal) => isAbortError(error) || signal?.aborted === true
|
||||
type QueuedServerEvent = { directory: string; payload: Event }
|
||||
export type ServerEvent = Event & { current?: OpenCodeEvent }
|
||||
type QueuedServerEvent = { directory: string; payload: ServerEvent }
|
||||
type CurrentDelta = Extract<
|
||||
OpenCodeEvent,
|
||||
{ type: "session.text.delta" | "session.reasoning.delta" | "session.tool.input.delta" | "session.compaction.delta" }
|
||||
>
|
||||
|
||||
export function adaptServerEvent(event: OpenCodeEvent): ServerEvent {
|
||||
if (event.type === "permission.v2.asked") {
|
||||
return {
|
||||
id: event.id,
|
||||
type: "permission.asked",
|
||||
properties: {
|
||||
id: event.data.id,
|
||||
sessionID: event.data.sessionID,
|
||||
permission: event.data.action,
|
||||
patterns: event.data.resources,
|
||||
always: event.data.save ?? [],
|
||||
metadata: event.data.metadata ?? {},
|
||||
tool:
|
||||
event.data.source?.type === "tool"
|
||||
? { messageID: event.data.source.messageID, callID: event.data.source.callID }
|
||||
: undefined,
|
||||
},
|
||||
current: event,
|
||||
} as ServerEvent
|
||||
}
|
||||
if (event.type === "permission.v2.replied")
|
||||
return { id: event.id, type: "permission.replied", properties: event.data, current: event } as ServerEvent
|
||||
if (event.type === "question.v2.asked")
|
||||
return { id: event.id, type: "question.asked", properties: event.data, current: event } as ServerEvent
|
||||
if (event.type === "question.v2.replied")
|
||||
return { id: event.id, type: "question.replied", properties: event.data, current: event } as ServerEvent
|
||||
if (event.type === "question.v2.rejected")
|
||||
return { id: event.id, type: "question.rejected", properties: event.data, current: event } as ServerEvent
|
||||
return { id: event.id, type: event.type, properties: event.data, current: event } as ServerEvent
|
||||
}
|
||||
|
||||
const coalescedKey = (event: QueuedServerEvent) => {
|
||||
if (event.payload.type === "lsp.updated") return `lsp.updated:${event.directory}`
|
||||
@@ -40,6 +79,34 @@ export function enqueueServerEvent(queue: QueuedServerEvent[], event: QueuedServ
|
||||
export function coalesceServerEvents(events: QueuedServerEvent[]) {
|
||||
const output: QueuedServerEvent[] = []
|
||||
events.forEach((event) => {
|
||||
const current = currentDelta(event.payload.current)
|
||||
if (current) {
|
||||
const previous = output[output.length - 1]
|
||||
const prior = currentDelta(previous?.payload.current)
|
||||
if (
|
||||
previous &&
|
||||
prior &&
|
||||
previous.directory === event.directory &&
|
||||
currentDeltaKey(prior) === currentDeltaKey(current)
|
||||
) {
|
||||
const fragment = currentDeltaFragment(prior) + currentDeltaFragment(current)
|
||||
const data =
|
||||
current.type === "session.compaction.delta"
|
||||
? { ...current.data, text: fragment }
|
||||
: { ...current.data, delta: fragment }
|
||||
output[output.length - 1] = {
|
||||
directory: event.directory,
|
||||
payload: {
|
||||
...event.payload,
|
||||
properties: data,
|
||||
current: { ...current, data } as CurrentDelta,
|
||||
} as ServerEvent,
|
||||
}
|
||||
return
|
||||
}
|
||||
output.push(event)
|
||||
return
|
||||
}
|
||||
if (event.payload.type !== "message.part.delta") {
|
||||
output.push(event)
|
||||
return
|
||||
@@ -71,12 +138,52 @@ export function coalesceServerEvents(events: QueuedServerEvent[]) {
|
||||
return output
|
||||
}
|
||||
|
||||
function currentDelta(event: OpenCodeEvent | undefined): CurrentDelta | undefined {
|
||||
if (
|
||||
event?.type === "session.text.delta" ||
|
||||
event?.type === "session.reasoning.delta" ||
|
||||
event?.type === "session.tool.input.delta" ||
|
||||
event?.type === "session.compaction.delta"
|
||||
)
|
||||
return event
|
||||
}
|
||||
|
||||
function currentDeltaKey(event: CurrentDelta) {
|
||||
if (event.type === "session.tool.input.delta")
|
||||
return `${event.type}:${event.data.sessionID}:${event.data.assistantMessageID}:${event.data.callID}`
|
||||
if (event.type === "session.compaction.delta") return `${event.type}:${event.data.sessionID}`
|
||||
return `${event.type}:${event.data.sessionID}:${event.data.assistantMessageID}:${event.data.ordinal}`
|
||||
}
|
||||
|
||||
function currentDeltaFragment(event: CurrentDelta) {
|
||||
return event.type === "session.compaction.delta" ? event.data.text : event.data.delta
|
||||
}
|
||||
|
||||
export function resumeStreamAfterPageShow(event: PageTransitionEvent, start: () => unknown) {
|
||||
if (!event.persisted) return
|
||||
start()
|
||||
}
|
||||
|
||||
function createServerSdkContextBase(server: ServerConnection.Any, scope: ServerScope) {
|
||||
type ServerEventEmitter = ReturnType<typeof createGlobalEmitter<{ [key: string]: ServerEvent }>>
|
||||
type ServerSDKBase = {
|
||||
server: ServerConnection.Any
|
||||
scope: ServerScope
|
||||
protocol: Promise<ServerProtocol>
|
||||
url: string
|
||||
client: ReturnType<typeof createSdkForServer>
|
||||
api: CompatibleApi
|
||||
currentApi: ServerApi
|
||||
event: {
|
||||
on: ServerEventEmitter["on"]
|
||||
listen: ServerEventEmitter["listen"]
|
||||
start: () => Promise<void> | undefined
|
||||
}
|
||||
createClient: (
|
||||
opts: Omit<Parameters<typeof createSdkForServer>[0], "server" | "fetch">,
|
||||
) => ReturnType<typeof createSdkForServer>
|
||||
}
|
||||
|
||||
function createServerSdkContextBase(server: ServerConnection.Any, scope: ServerScope): ServerSDKBase {
|
||||
const platform = usePlatform()
|
||||
const abort = new AbortController()
|
||||
|
||||
@@ -91,13 +198,15 @@ function createServerSdkContextBase(server: ServerConnection.Any, scope: ServerS
|
||||
}
|
||||
})()
|
||||
|
||||
const eventApi = createApiForServer({ server: server.http, fetch: eventFetch })
|
||||
const eventSdk = createSdkForServer({
|
||||
signal: abort.signal,
|
||||
fetch: eventFetch,
|
||||
server: server.http,
|
||||
})
|
||||
const protocol = detectServerProtocol(server.http, platform.fetch ?? globalThis.fetch)
|
||||
const emitter = createGlobalEmitter<{
|
||||
[key: string]: Event
|
||||
[key: string]: ServerEvent
|
||||
}>()
|
||||
|
||||
type Queued = QueuedServerEvent
|
||||
@@ -142,21 +251,6 @@ function createServerSdkContextBase(server: ServerConnection.Any, scope: ServerS
|
||||
let run: Promise<void> | undefined
|
||||
let started = false
|
||||
let generation = 0
|
||||
const HEARTBEAT_TIMEOUT_MS = 15_000
|
||||
let lastEventAt = Date.now()
|
||||
let heartbeat: ReturnType<typeof setTimeout> | undefined
|
||||
const resetHeartbeat = () => {
|
||||
lastEventAt = Date.now()
|
||||
if (heartbeat) clearTimeout(heartbeat)
|
||||
heartbeat = setTimeout(() => {
|
||||
attempt?.abort()
|
||||
}, HEARTBEAT_TIMEOUT_MS)
|
||||
}
|
||||
const clearHeartbeat = () => {
|
||||
if (!heartbeat) return
|
||||
clearTimeout(heartbeat)
|
||||
heartbeat = undefined
|
||||
}
|
||||
|
||||
const start = () => {
|
||||
if (started) return run
|
||||
@@ -168,35 +262,24 @@ function createServerSdkContextBase(server: ServerConnection.Any, scope: ServerS
|
||||
// oxlint-disable-next-line no-unmodified-loop-condition -- `started` is set to false by stop() which also aborts; both flags are checked to allow graceful exit
|
||||
while (!abort.signal.aborted && started && generation === active) {
|
||||
attempt = new AbortController()
|
||||
lastEventAt = Date.now()
|
||||
const onAbort = () => {
|
||||
attempt?.abort()
|
||||
}
|
||||
abort.signal.addEventListener("abort", onAbort)
|
||||
try {
|
||||
const events = await eventSdk.global.event({
|
||||
signal: attempt.signal,
|
||||
onSseError: (error) => {
|
||||
if (isStreamClosed(error, attempt?.signal)) return
|
||||
if (streamErrorLogged) return
|
||||
streamErrorLogged = true
|
||||
console.error("[global-sdk] event stream error", {
|
||||
url: server.http.url,
|
||||
fetch: eventFetch ? "platform" : "webview",
|
||||
error,
|
||||
})
|
||||
},
|
||||
})
|
||||
const kind = await protocol
|
||||
const events =
|
||||
kind === "v1"
|
||||
? (await eventSdk.global.event({ signal: attempt.signal })).stream
|
||||
: eventApi.event.subscribe({ signal: attempt.signal })
|
||||
let yielded = Date.now()
|
||||
resetHeartbeat()
|
||||
for await (const event of events.stream) {
|
||||
resetHeartbeat()
|
||||
for await (const event of events) {
|
||||
streamErrorLogged = false
|
||||
if (event.payload.type !== "sync") {
|
||||
const directory = event.directory ?? "global"
|
||||
const payload = event.payload as Event
|
||||
if (enqueueServerEvent(queue, { directory, payload })) schedule()
|
||||
}
|
||||
const legacy = "payload" in event
|
||||
if (legacy && event.payload.type === "sync") continue
|
||||
const directory = legacy ? (event.directory ?? "global") : (event.location?.directory ?? "global")
|
||||
const payload = legacy ? (event.payload as Event) : adaptServerEvent(event)
|
||||
if (enqueueServerEvent(queue, { directory, payload })) schedule()
|
||||
|
||||
if (Date.now() - yielded < STREAM_YIELD_MS) continue
|
||||
yielded = Date.now()
|
||||
@@ -214,7 +297,6 @@ function createServerSdkContextBase(server: ServerConnection.Any, scope: ServerS
|
||||
} finally {
|
||||
abort.signal.removeEventListener("abort", onAbort)
|
||||
attempt = undefined
|
||||
clearHeartbeat()
|
||||
}
|
||||
|
||||
if (abort.signal.aborted || !started || generation !== active) return
|
||||
@@ -233,18 +315,11 @@ function createServerSdkContextBase(server: ServerConnection.Any, scope: ServerS
|
||||
started = false
|
||||
generation++
|
||||
attempt?.abort()
|
||||
clearHeartbeat()
|
||||
}
|
||||
|
||||
onMount(() => {
|
||||
makeEventListener(window, "pagehide", stop)
|
||||
makeEventListener(window, "pageshow", (event) => resumeStreamAfterPageShow(event, start))
|
||||
makeEventListener(document, "visibilitychange", () => {
|
||||
if (document.visibilityState !== "visible") return
|
||||
if (!started) return
|
||||
if (Date.now() - lastEventAt < HEARTBEAT_TIMEOUT_MS) return
|
||||
attempt?.abort()
|
||||
})
|
||||
})
|
||||
|
||||
onCleanup(() => {
|
||||
@@ -258,12 +333,24 @@ function createServerSdkContextBase(server: ServerConnection.Any, scope: ServerS
|
||||
fetch: platform.fetch,
|
||||
throwOnError: true,
|
||||
})
|
||||
const currentApi: ServerApi = createApiForServer({ server: server.http, fetch: platform.fetch })
|
||||
const legacy = (directory?: string) =>
|
||||
createSdkForServer({
|
||||
server: server.http,
|
||||
fetch: platform.fetch,
|
||||
throwOnError: true,
|
||||
directory,
|
||||
})
|
||||
const api = createCompatibleApi({ protocol, current: currentApi, legacy })
|
||||
|
||||
return {
|
||||
server,
|
||||
scope,
|
||||
protocol,
|
||||
url: server.http.url,
|
||||
client: sdk,
|
||||
api,
|
||||
currentApi,
|
||||
event: {
|
||||
on: emitter.on.bind(emitter),
|
||||
listen: emitter.listen.bind(emitter),
|
||||
@@ -279,7 +366,6 @@ function createServerSdkContextBase(server: ServerConnection.Any, scope: ServerS
|
||||
}
|
||||
}
|
||||
|
||||
type ServerSDKBase = ReturnType<typeof createServerSdkContextBase>
|
||||
export type ServerSDK = ServerSDKBase & {
|
||||
ensureDirSdkContext: (directory: string) => ReturnType<typeof createDirSdkContext>
|
||||
}
|
||||
@@ -309,7 +395,7 @@ export const { use: useServerSDK, provider: ServerSDKProvider } = createSimpleCo
|
||||
})
|
||||
|
||||
type SDKEventMap = {
|
||||
[key in Event["type"]]: Extract<Event, { type: key }>
|
||||
[key in Event["type"]]: Extract<ServerEvent, { type: key }>
|
||||
}
|
||||
|
||||
function createDirSdkContext(directory: string, serverSDK: ServerSDKBase) {
|
||||
@@ -329,6 +415,12 @@ function createDirSdkContext(directory: string, serverSDK: ServerSDKBase) {
|
||||
scope: serverSDK.scope,
|
||||
directory,
|
||||
client,
|
||||
api: createCompatibleApi({
|
||||
protocol: serverSDK.protocol,
|
||||
current: serverSDK.currentApi,
|
||||
legacy: (next) => serverSDK.createClient({ directory: next ?? directory, throwOnError: true }),
|
||||
directory,
|
||||
}),
|
||||
event: emitter,
|
||||
get url() {
|
||||
return serverSDK.url
|
||||
|
||||
@@ -263,6 +263,7 @@ function ProviderTip(props: { ready: () => boolean; connected: () => boolean; op
|
||||
<TooltipV2
|
||||
class="hover-reveal absolute left-full top-0 flex h-6 w-7 items-center justify-end delay-0 duration-0 group-hover/provider-tip:delay-[250ms] group-hover/provider-tip:duration-150 group-hover/provider-tip:opacity-100 focus-within:delay-0 focus-within:duration-0 focus-within:opacity-100"
|
||||
placement="top"
|
||||
openDelay={1000}
|
||||
value={language.t("common.dismiss")}
|
||||
>
|
||||
<button
|
||||
|
||||
@@ -0,0 +1,113 @@
|
||||
import { describe, expect, test } from "bun:test"
|
||||
import { createApiForServer, createSdkForServer } from "./server"
|
||||
import { createCompatibleApi } from "./server-compat"
|
||||
|
||||
function setup(protocol: "v1" | "v2" | Promise<"v1" | "v2">) {
|
||||
const requests: Request[] = []
|
||||
const fetcher = Object.assign(
|
||||
async (input: string | URL | Request, init?: RequestInit) => {
|
||||
const request = new Request(input, init)
|
||||
requests.push(request)
|
||||
if (request.method === "PATCH") {
|
||||
return Response.json({
|
||||
id: "ses_1",
|
||||
slug: "ses_1",
|
||||
projectID: "project",
|
||||
directory: "/repo",
|
||||
title: "Session",
|
||||
version: "1",
|
||||
time: { created: 1, updated: 1 },
|
||||
})
|
||||
}
|
||||
if (request.method === "POST" && request.url.endsWith("/prompt_async"))
|
||||
return new Response(undefined, { status: 204 })
|
||||
if (request.method === "POST" && request.url.endsWith("/prompt")) {
|
||||
return Response.json({
|
||||
admittedSeq: 1,
|
||||
id: "msg_1",
|
||||
sessionID: "ses_1",
|
||||
timeCreated: 1,
|
||||
type: "user",
|
||||
data: { text: "hello" },
|
||||
delivery: "steer",
|
||||
})
|
||||
}
|
||||
if (request.method === "GET") return Response.json([])
|
||||
return new Response(undefined, { status: 204 })
|
||||
},
|
||||
{ preconnect: globalThis.fetch.preconnect },
|
||||
)
|
||||
const server = { url: "http://localhost:4096" }
|
||||
const api = createCompatibleApi({
|
||||
protocol: typeof protocol === "string" ? Promise.resolve(protocol) : protocol,
|
||||
current: createApiForServer({ server, fetch: fetcher }),
|
||||
legacy: (directory) => createSdkForServer({ server, fetch: fetcher, directory, throwOnError: true }),
|
||||
directory: "/repo",
|
||||
})
|
||||
return { api, requests }
|
||||
}
|
||||
|
||||
describe("createCompatibleApi", () => {
|
||||
test("routes V1 archive through the legacy session update", async () => {
|
||||
const { api, requests } = setup("v1")
|
||||
await api.session.archive({ sessionID: "ses_1", directory: "/repo" })
|
||||
|
||||
const url = new URL(requests[0]!.url)
|
||||
expect(url.pathname).toBe("/session/ses_1")
|
||||
expect(requests[0]!.headers.get("x-opencode-directory")).toBe("%2Frepo")
|
||||
expect(requests[0]!.method).toBe("PATCH")
|
||||
expect(await requests[0]!.json()).toMatchObject({ time: { archived: expect.any(Number) } })
|
||||
})
|
||||
|
||||
test("converts current prompts to the V1 prompt contract", async () => {
|
||||
const { api, requests } = setup("v1")
|
||||
await api.session.prompt({
|
||||
sessionID: "ses_1",
|
||||
id: "msg_1",
|
||||
text: "hello",
|
||||
agent: "build",
|
||||
model: { providerID: "provider", modelID: "model" },
|
||||
})
|
||||
|
||||
expect(new URL(requests[0]!.url).pathname).toBe("/session/ses_1/prompt_async")
|
||||
expect(await requests[0]!.json()).toMatchObject({
|
||||
messageID: "msg_1",
|
||||
agent: "build",
|
||||
model: { providerID: "provider", modelID: "model" },
|
||||
parts: [{ type: "text", text: "hello" }],
|
||||
})
|
||||
})
|
||||
|
||||
test("keeps V2 session actions on the current API", async () => {
|
||||
const { api, requests } = setup("v2")
|
||||
await api.session.archive({ sessionID: "ses_1" })
|
||||
|
||||
expect(new URL(requests[0]!.url).pathname).toBe("/api/session/ses_1/archive")
|
||||
expect(requests[0]!.method).toBe("POST")
|
||||
})
|
||||
|
||||
test("resolves protocol detection once across implementation methods", async () => {
|
||||
let detections = 0
|
||||
const resolved = Promise.resolve<"v1" | "v2">("v2")
|
||||
const protocol = new Proxy(resolved, {
|
||||
get(target, property) {
|
||||
if (property !== "then") return Reflect.get(target, property, target)
|
||||
detections++
|
||||
return target.then.bind(target)
|
||||
},
|
||||
})
|
||||
const { api } = setup(protocol)
|
||||
|
||||
await api.session.archive({ sessionID: "ses_1" })
|
||||
await api.session.list()
|
||||
|
||||
expect(detections).toBe(1)
|
||||
})
|
||||
|
||||
test("uses the global V1 session search endpoint", async () => {
|
||||
const { api, requests } = setup("v1")
|
||||
await api.session.list({ parentID: null, search: "session", limit: 50 })
|
||||
|
||||
expect(new URL(requests[0]!.url).pathname).toBe("/experimental/session")
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,495 @@
|
||||
import type { ServerApi } from "./server"
|
||||
import type { ServerProtocol } from "./server-protocol"
|
||||
import type { OpencodeClient, Session } from "@opencode-ai/sdk/v2/client"
|
||||
import type {
|
||||
Project,
|
||||
ProjectCurrent,
|
||||
SessionApi,
|
||||
SessionCommandInput,
|
||||
SessionCommandOutput,
|
||||
SessionCompactInput,
|
||||
SessionCompactOutput,
|
||||
SessionInfo,
|
||||
SessionPromptInput,
|
||||
SessionPromptOutput,
|
||||
SessionShellInput,
|
||||
SessionShellOutput,
|
||||
} from "@opencode-ai/client/promise"
|
||||
|
||||
type LegacyClient = OpencodeClient
|
||||
type LegacyFor = (directory?: string) => LegacyClient
|
||||
type CompatibleSessionApi = Omit<
|
||||
SessionApi,
|
||||
"prompt" | "command" | "shell" | "compact" | "rename" | "archive" | "remove"
|
||||
> & {
|
||||
prompt: (input: SessionPromptInput & LegacyPrompt) => Promise<SessionPromptOutput>
|
||||
command: (input: SessionCommandInput) => Promise<SessionCommandOutput>
|
||||
shell: (input: SessionShellInput & LegacyPrompt) => Promise<SessionShellOutput>
|
||||
compact: (input: SessionCompactInput & { model?: LegacyPrompt["model"] }) => Promise<SessionCompactOutput>
|
||||
rename: (input: Parameters<SessionApi["rename"]>[0] & LegacyLocation) => ReturnType<SessionApi["rename"]>
|
||||
archive: (input: Parameters<SessionApi["archive"]>[0] & LegacyLocation) => ReturnType<SessionApi["archive"]>
|
||||
remove: (input: Parameters<SessionApi["remove"]>[0] & LegacyLocation) => ReturnType<SessionApi["remove"]>
|
||||
}
|
||||
export type CompatibleApi = Omit<ServerApi, "session"> & { readonly session: CompatibleSessionApi }
|
||||
type LegacyPrompt = {
|
||||
agent?: string
|
||||
model?: { providerID: string; modelID: string }
|
||||
variant?: string
|
||||
}
|
||||
type LegacyLocation = { directory?: string }
|
||||
type CompatibleInput = {
|
||||
protocol: Promise<ServerProtocol>
|
||||
current: ServerApi
|
||||
legacy: LegacyFor
|
||||
directory?: string
|
||||
}
|
||||
|
||||
function mime(uri: string) {
|
||||
const match = /^data:([^;,]+)/.exec(uri)
|
||||
return match?.[1] ?? "application/octet-stream"
|
||||
}
|
||||
|
||||
function sessionInfo(session: Session): SessionInfo {
|
||||
return {
|
||||
id: session.id,
|
||||
parentID: session.parentID,
|
||||
projectID: session.projectID,
|
||||
agent: session.agent,
|
||||
model: session.model && {
|
||||
id: session.model.id,
|
||||
providerID: session.model.providerID,
|
||||
variant: session.model.variant,
|
||||
},
|
||||
cost: session.cost ?? 0,
|
||||
tokens: session.tokens ?? { input: 0, output: 0, reasoning: 0, cache: { read: 0, write: 0 } },
|
||||
time: session.time,
|
||||
title: session.title,
|
||||
location: { directory: session.directory, workspaceID: session.workspaceID },
|
||||
subpath: session.path,
|
||||
revert: session.revert && {
|
||||
messageID: session.revert.messageID,
|
||||
partID: session.revert.partID,
|
||||
snapshot: session.revert.snapshot,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
export function createCompatibleApi(input: CompatibleInput): CompatibleApi {
|
||||
const v1 = createV1Api(input)
|
||||
return lazyApi(
|
||||
input.protocol.then((protocol) => (protocol === "v1" ? v1 : input.current)),
|
||||
input.current,
|
||||
)
|
||||
}
|
||||
|
||||
function lazyApi<T extends object>(implementation: Promise<T>, shape: T): T {
|
||||
const cache = new Map<PropertyKey, unknown>()
|
||||
return new Proxy(shape, {
|
||||
get(target, property, receiver) {
|
||||
const sample = Reflect.get(target, property, receiver)
|
||||
if (typeof sample === "function") {
|
||||
return (...args: unknown[]) =>
|
||||
implementation.then((value) => {
|
||||
const method = Reflect.get(value, property)
|
||||
if (typeof method !== "function") throw new Error(`API method unavailable: ${String(property)}`)
|
||||
return Reflect.apply(method, value, args)
|
||||
})
|
||||
}
|
||||
if (sample === null || typeof sample !== "object") return sample
|
||||
if (cache.has(property)) return cache.get(property)
|
||||
const nested = lazyApi(
|
||||
implementation.then((value) => {
|
||||
const result = Reflect.get(value, property)
|
||||
if (result === null || typeof result !== "object") {
|
||||
throw new Error(`API namespace unavailable: ${String(property)}`)
|
||||
}
|
||||
return result
|
||||
}),
|
||||
sample,
|
||||
)
|
||||
cache.set(property, nested)
|
||||
return nested
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
function createV1Api(input: CompatibleInput): CompatibleApi {
|
||||
const directory = (location?: { directory?: string }) => location?.directory ?? input.directory
|
||||
const legacy = (location?: { directory?: string }) => input.legacy(directory(location))
|
||||
const located = <T>(data: T, value?: { directory?: string }) => ({
|
||||
location: {
|
||||
directory: directory(value) ?? "",
|
||||
project: { id: "", directory: directory(value) ?? "" },
|
||||
},
|
||||
data,
|
||||
})
|
||||
|
||||
return {
|
||||
...input.current,
|
||||
session: {
|
||||
...input.current.session,
|
||||
async list(
|
||||
value?: Parameters<ServerApi["session"]["list"]>[0],
|
||||
options?: Parameters<ServerApi["session"]["list"]>[1],
|
||||
) {
|
||||
if (!value?.directory && value?.search !== undefined) {
|
||||
const result = await legacy().experimental.session.list(
|
||||
{
|
||||
roots: value.parentID === null ? true : undefined,
|
||||
search: value.search,
|
||||
limit: value.limit,
|
||||
},
|
||||
options,
|
||||
)
|
||||
return { data: (result.data ?? []).map(sessionInfo), cursor: {} }
|
||||
}
|
||||
const result = await legacy({ directory: value?.directory }).session.list({
|
||||
directory: value?.directory,
|
||||
roots: value?.parentID === null ? true : undefined,
|
||||
search: value?.search,
|
||||
limit: value?.limit,
|
||||
})
|
||||
return { data: (result.data ?? []).map(sessionInfo), cursor: {} }
|
||||
},
|
||||
async create(value?: Parameters<ServerApi["session"]["create"]>[0]) {
|
||||
const result = await legacy(value?.location ?? undefined).session.create({
|
||||
directory: directory(value?.location ?? undefined),
|
||||
})
|
||||
if (!result.data) throw new Error("Failed to create session")
|
||||
return sessionInfo(result.data)
|
||||
},
|
||||
async get(value: Parameters<ServerApi["session"]["get"]>[0]) {
|
||||
const result = await legacy().session.get(value)
|
||||
if (!result.data) throw new Error(`Session not found: ${value.sessionID}`)
|
||||
return sessionInfo(result.data)
|
||||
},
|
||||
async active() {
|
||||
const result = await legacy().session.status()
|
||||
return Object.fromEntries(
|
||||
Object.entries(result.data ?? {}).flatMap(([sessionID, status]) =>
|
||||
status.type === "idle" ? [] : [[sessionID, { type: "running" as const }]],
|
||||
),
|
||||
)
|
||||
},
|
||||
async rename(value: Parameters<ServerApi["session"]["rename"]>[0] & LegacyLocation) {
|
||||
await legacy(value).session.update({ sessionID: value.sessionID, title: value.title })
|
||||
},
|
||||
async archive(value: Parameters<ServerApi["session"]["archive"]>[0] & LegacyLocation) {
|
||||
await legacy(value).session.update({ sessionID: value.sessionID, time: { archived: Date.now() } })
|
||||
},
|
||||
async remove(value: Parameters<ServerApi["session"]["remove"]>[0] & LegacyLocation) {
|
||||
await legacy(value).session.delete(value)
|
||||
},
|
||||
async fork(value: Parameters<ServerApi["session"]["fork"]>[0]) {
|
||||
const result = await legacy().session.fork(value)
|
||||
if (!result.data) throw new Error("Failed to fork session")
|
||||
return sessionInfo(result.data)
|
||||
},
|
||||
async interrupt(value: Parameters<ServerApi["session"]["interrupt"]>[0]) {
|
||||
await legacy().session.abort(value)
|
||||
},
|
||||
async prompt(value: SessionPromptInput & LegacyPrompt) {
|
||||
await legacy().session.promptAsync({
|
||||
sessionID: value.sessionID,
|
||||
messageID: value.id ?? undefined,
|
||||
agent: value.agent,
|
||||
model: value.model,
|
||||
variant: value.variant,
|
||||
parts: [
|
||||
{ type: "text", text: value.text },
|
||||
...(value.files ?? []).map((file) => ({
|
||||
type: "file" as const,
|
||||
mime: mime(file.uri),
|
||||
url: file.uri,
|
||||
filename: file.name,
|
||||
})),
|
||||
...(value.agents ?? []).map((agent) => ({
|
||||
type: "agent" as const,
|
||||
name: agent.name,
|
||||
source: agent.mention
|
||||
? { value: agent.mention.text, start: agent.mention.start, end: agent.mention.end }
|
||||
: undefined,
|
||||
})),
|
||||
],
|
||||
})
|
||||
return {
|
||||
admittedSeq: 0,
|
||||
id: value.id ?? "",
|
||||
sessionID: value.sessionID,
|
||||
timeCreated: Date.now(),
|
||||
type: "user",
|
||||
data: { text: value.text },
|
||||
delivery: value.delivery ?? "steer",
|
||||
}
|
||||
},
|
||||
async command(value: SessionCommandInput) {
|
||||
await legacy().session.command({
|
||||
sessionID: value.sessionID,
|
||||
messageID: value.id ?? undefined,
|
||||
command: value.command,
|
||||
arguments: value.arguments ?? "",
|
||||
agent: value.agent ?? undefined,
|
||||
model: value.model ? `${value.model.providerID}/${value.model.id}` : undefined,
|
||||
variant: value.model?.variant,
|
||||
parts: value.files?.map((file) => ({
|
||||
type: "file" as const,
|
||||
mime: mime(file.uri),
|
||||
url: file.uri,
|
||||
filename: file.name,
|
||||
})),
|
||||
})
|
||||
return {
|
||||
admittedSeq: 0,
|
||||
id: value.id ?? "",
|
||||
sessionID: value.sessionID,
|
||||
timeCreated: Date.now(),
|
||||
type: "user",
|
||||
data: { text: `/${value.command} ${value.arguments ?? ""}`.trim() },
|
||||
delivery: value.delivery ?? "steer",
|
||||
}
|
||||
},
|
||||
async shell(value: SessionShellInput & LegacyPrompt) {
|
||||
await legacy().session.shell({
|
||||
sessionID: value.sessionID,
|
||||
command: value.command,
|
||||
agent: value.agent,
|
||||
model: value.model,
|
||||
})
|
||||
},
|
||||
compact: async (value: SessionCompactInput & { model?: LegacyPrompt["model"] }) => {
|
||||
if (!value.model) throw new Error("A model is required to compact a V1 session")
|
||||
await legacy().session.summarize({
|
||||
sessionID: value.sessionID,
|
||||
providerID: value.model.providerID,
|
||||
modelID: value.model.modelID,
|
||||
})
|
||||
return {
|
||||
admittedSeq: 0,
|
||||
id: value.id ?? "",
|
||||
sessionID: value.sessionID,
|
||||
timeCreated: Date.now(),
|
||||
type: "compaction",
|
||||
}
|
||||
},
|
||||
revert: {
|
||||
stage: async (value: Parameters<ServerApi["session"]["revert"]["stage"]>[0]) => {
|
||||
await legacy().session.revert(value)
|
||||
return { messageID: value.messageID }
|
||||
},
|
||||
clear: async (value: Parameters<ServerApi["session"]["revert"]["clear"]>[0]) => {
|
||||
await legacy().session.unrevert(value)
|
||||
},
|
||||
commit: input.current.session.revert.commit,
|
||||
},
|
||||
},
|
||||
project: {
|
||||
...input.current.project,
|
||||
async list() {
|
||||
return ((await legacy().project.list()).data ?? []) as Project[]
|
||||
},
|
||||
async current(value?: Parameters<ServerApi["project"]["current"]>[0]) {
|
||||
const result = await legacy(value?.location).project.current()
|
||||
if (!result.data) throw new Error("Project not found")
|
||||
return { id: result.data.id, directory: result.data.worktree } satisfies ProjectCurrent
|
||||
},
|
||||
async update(value: Parameters<ServerApi["project"]["update"]>[0]) {
|
||||
const project = (await legacy().project.list()).data?.find((item) => item.id === value.projectID)
|
||||
const result = await legacy({ directory: project?.worktree }).project.update({
|
||||
...value,
|
||||
directory: project?.worktree,
|
||||
})
|
||||
if (!result.data) throw new Error(`Project not found: ${value.projectID}`)
|
||||
return result.data as Project
|
||||
},
|
||||
async directories(value: Parameters<ServerApi["project"]["directories"]>[0]) {
|
||||
const result = await legacy(value.location).worktree.list()
|
||||
return (result.data ?? []).map((item) => ({ directory: item }))
|
||||
},
|
||||
},
|
||||
path: {
|
||||
...input.current.path,
|
||||
async get(value?: Parameters<ServerApi["path"]["get"]>[0]) {
|
||||
const result = await legacy(value?.location).path.get()
|
||||
if (!result.data) throw new Error("Path unavailable")
|
||||
return result.data
|
||||
},
|
||||
},
|
||||
vcs: {
|
||||
...input.current.vcs,
|
||||
async get(value?: Parameters<ServerApi["vcs"]["get"]>[0]) {
|
||||
const result = await legacy(value?.location).vcs.get()
|
||||
return located({ branch: result.data?.branch, defaultBranch: undefined }, value?.location)
|
||||
},
|
||||
async status(value?: Parameters<ServerApi["vcs"]["status"]>[0]) {
|
||||
const result = await legacy(value?.location).vcs.status()
|
||||
return located(result.data ?? [], value?.location)
|
||||
},
|
||||
async diff(value: Parameters<ServerApi["vcs"]["diff"]>[0]) {
|
||||
const result = await legacy(value.location).vcs.diff({
|
||||
mode: value.mode === "working" ? "git" : value.mode,
|
||||
context: value.context,
|
||||
})
|
||||
return located(
|
||||
(result.data ?? []).map((file) => ({
|
||||
file: file.file,
|
||||
patch: file.patch ?? "",
|
||||
additions: file.additions,
|
||||
deletions: file.deletions,
|
||||
status: file.status ?? "modified",
|
||||
})),
|
||||
value.location,
|
||||
)
|
||||
},
|
||||
},
|
||||
file: {
|
||||
...input.current.file,
|
||||
async list(value?: Parameters<ServerApi["file"]["list"]>[0]) {
|
||||
const result = await legacy(value?.location).file.list({ path: value?.path ?? "" })
|
||||
return located(result.data ?? [], value?.location)
|
||||
},
|
||||
async find(value: Parameters<ServerApi["file"]["find"]>[0]) {
|
||||
const result = await legacy(value.location).find.files({
|
||||
query: value.query,
|
||||
type: value.type,
|
||||
limit: value.limit,
|
||||
})
|
||||
return located(
|
||||
(result.data ?? []).map((path) => ({ path, type: value.type ?? "file" })),
|
||||
value.location,
|
||||
)
|
||||
},
|
||||
},
|
||||
integration: {
|
||||
...input.current.integration,
|
||||
async get(value: Parameters<ServerApi["integration"]["get"]>[0]) {
|
||||
const methods = ((await legacy(value.location).provider.auth()).data?.[value.integrationID] ?? []).map(
|
||||
(method, index) =>
|
||||
method.type === "api"
|
||||
? { type: "key" as const, label: method.label }
|
||||
: { type: "oauth" as const, id: String(index), label: method.label, prompts: method.prompts },
|
||||
)
|
||||
return located(
|
||||
{
|
||||
id: value.integrationID,
|
||||
name: value.integrationID,
|
||||
methods,
|
||||
connections: [],
|
||||
},
|
||||
value.location,
|
||||
)
|
||||
},
|
||||
connect: {
|
||||
...input.current.integration.connect,
|
||||
key: async (value: Parameters<ServerApi["integration"]["connect"]["key"]>[0]) => {
|
||||
await legacy(value.location).auth.set({
|
||||
providerID: value.integrationID,
|
||||
auth: { type: "api", key: value.key },
|
||||
})
|
||||
},
|
||||
},
|
||||
oauth: {
|
||||
...input.current.integration.oauth,
|
||||
connect: async (value: Parameters<ServerApi["integration"]["oauth"]["connect"]>[0]) => {
|
||||
const method = Number(value.methodID)
|
||||
const result = await legacy(value.location).provider.oauth.authorize(
|
||||
{ providerID: value.integrationID, method, inputs: value.inputs },
|
||||
{ throwOnError: true },
|
||||
)
|
||||
if (!result.data) throw new Error("Failed to start OAuth authorization")
|
||||
return located(
|
||||
{
|
||||
attemptID: `${value.integrationID}:${method}`,
|
||||
url: result.data.url,
|
||||
instructions: result.data.instructions,
|
||||
mode: result.data.method,
|
||||
time: { created: Date.now(), expires: Date.now() + 10 * 60 * 1000 },
|
||||
},
|
||||
value.location,
|
||||
)
|
||||
},
|
||||
complete: async (value: Parameters<ServerApi["integration"]["oauth"]["complete"]>[0]) => {
|
||||
const method = Number(value.attemptID.split(":").at(-1))
|
||||
await legacy(value.location).provider.oauth.callback(
|
||||
{ providerID: value.integrationID, method, code: value.code },
|
||||
{ throwOnError: true },
|
||||
)
|
||||
},
|
||||
status: async (value: Parameters<ServerApi["integration"]["oauth"]["status"]>[0]) => {
|
||||
const method = Number(value.attemptID.split(":").at(-1))
|
||||
await legacy(value.location).provider.oauth.callback(
|
||||
{ providerID: value.integrationID, method },
|
||||
{ throwOnError: true },
|
||||
)
|
||||
return located(
|
||||
{ status: "complete" as const, time: { created: Date.now(), expires: Date.now() } },
|
||||
value.location,
|
||||
)
|
||||
},
|
||||
},
|
||||
},
|
||||
pty: {
|
||||
...input.current.pty,
|
||||
async shells(value?: Parameters<ServerApi["pty"]["shells"]>[0]) {
|
||||
return located((await legacy(value?.location).pty.shells()).data ?? [], value?.location)
|
||||
},
|
||||
async list(value?: Parameters<ServerApi["pty"]["list"]>[0]) {
|
||||
return located((await legacy(value?.location).pty.list()).data ?? [], value?.location)
|
||||
},
|
||||
async create(value?: Parameters<ServerApi["pty"]["create"]>[0]) {
|
||||
const result = await legacy(value?.location).pty.create({
|
||||
command: value?.command,
|
||||
args: value?.args ? [...value.args] : undefined,
|
||||
cwd: value?.cwd,
|
||||
title: value?.title,
|
||||
env: value?.env,
|
||||
})
|
||||
if (!result.data) throw new Error("Failed to create terminal")
|
||||
return located(result.data, value?.location)
|
||||
},
|
||||
async get(value: Parameters<ServerApi["pty"]["get"]>[0]) {
|
||||
const result = await legacy(value.location).pty.get({ ptyID: value.ptyID })
|
||||
if (!result.data) throw new Error(`Terminal not found: ${value.ptyID}`)
|
||||
return located(result.data, value.location)
|
||||
},
|
||||
async update(value: Parameters<ServerApi["pty"]["update"]>[0]) {
|
||||
const result = await legacy(value.location).pty.update({
|
||||
ptyID: value.ptyID,
|
||||
title: value.title,
|
||||
size: value.size,
|
||||
})
|
||||
if (!result.data) throw new Error(`Terminal not found: ${value.ptyID}`)
|
||||
return located(result.data, value.location)
|
||||
},
|
||||
async remove(value: Parameters<ServerApi["pty"]["remove"]>[0]) {
|
||||
await legacy(value.location).pty.remove({ ptyID: value.ptyID })
|
||||
},
|
||||
async connectToken(value: Parameters<ServerApi["pty"]["connectToken"]>[0]) {
|
||||
const result = await legacy(value.location).pty.connectToken({ ptyID: value.ptyID })
|
||||
if (!result.data) throw new Error(`Failed to connect terminal: ${value.ptyID}`)
|
||||
return located(result.data, value.location)
|
||||
},
|
||||
},
|
||||
permission: {
|
||||
...input.current.permission,
|
||||
async reply(value: Parameters<ServerApi["permission"]["reply"]>[0]) {
|
||||
await legacy().permission.respond({
|
||||
sessionID: value.sessionID,
|
||||
permissionID: value.requestID,
|
||||
response: value.reply,
|
||||
})
|
||||
},
|
||||
},
|
||||
question: {
|
||||
...input.current.question,
|
||||
async reply(value: Parameters<ServerApi["question"]["reply"]>[0]) {
|
||||
await legacy().question.reply({
|
||||
requestID: value.requestID,
|
||||
answers: value.answers.map((answer) => [...answer]),
|
||||
})
|
||||
},
|
||||
async reject(value: Parameters<ServerApi["question"]["reject"]>[0]) {
|
||||
await legacy().question.reject({ requestID: value.requestID })
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -14,15 +14,45 @@ function abortFromInput(input: RequestInfo | URL, init?: RequestInit) {
|
||||
|
||||
describe("checkServerHealth", () => {
|
||||
test("returns healthy response with version", async () => {
|
||||
const fetch = (async () =>
|
||||
new Response(JSON.stringify({ healthy: true, version: "1.2.3" }), {
|
||||
let request: URL | undefined
|
||||
const fetch = (async (input: RequestInfo | URL) => {
|
||||
request = input instanceof URL ? input : new URL(input instanceof Request ? input.url : input)
|
||||
return new Response(JSON.stringify({ healthy: true, version: "1.2.3" }), {
|
||||
status: 200,
|
||||
headers: { "content-type": "application/json" },
|
||||
})) as unknown as typeof globalThis.fetch
|
||||
})
|
||||
}) as unknown as typeof globalThis.fetch
|
||||
|
||||
const result = await checkServerHealth(server, fetch)
|
||||
|
||||
expect(result).toEqual({ healthy: true, version: "1.2.3" })
|
||||
expect(request?.pathname).toBe("/api/health")
|
||||
})
|
||||
|
||||
test("falls back to the V1 health endpoint", async () => {
|
||||
const paths: string[] = []
|
||||
const fetch = (async (input: RequestInfo | URL) => {
|
||||
const url = input instanceof URL ? input : new URL(input instanceof Request ? input.url : input)
|
||||
paths.push(url.pathname)
|
||||
if (url.pathname === "/api/health") return new Response(undefined, { status: 404 })
|
||||
return Response.json({ healthy: true, version: "1.18.4" })
|
||||
}) as unknown as typeof globalThis.fetch
|
||||
|
||||
expect(await checkServerHealth(server, fetch)).toEqual({ healthy: true, version: "1.18.4" })
|
||||
expect(paths).toEqual(["/api/health", "/global/health"])
|
||||
})
|
||||
|
||||
test("falls back when the current health response is malformed", async () => {
|
||||
const paths: string[] = []
|
||||
const fetch = (async (input: RequestInfo | URL) => {
|
||||
const url = input instanceof URL ? input : new URL(input instanceof Request ? input.url : input)
|
||||
paths.push(url.pathname)
|
||||
if (url.pathname === "/api/health") return Response.json({})
|
||||
return Response.json({ healthy: true, version: "1.18.4" })
|
||||
}) as unknown as typeof globalThis.fetch
|
||||
|
||||
expect(await checkServerHealth(server, fetch)).toEqual({ healthy: true, version: "1.18.4" })
|
||||
expect(paths).toEqual(["/api/health", "/global/health"])
|
||||
})
|
||||
|
||||
test("allows slow servers thirty seconds by default", async () => {
|
||||
@@ -142,7 +172,7 @@ describe("checkServerHealth", () => {
|
||||
retryDelayMs: 1,
|
||||
})
|
||||
|
||||
expect(count).toBe(3)
|
||||
expect(count).toBe(6)
|
||||
expect(result).toEqual({ healthy: false })
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { usePlatform } from "@/context/platform"
|
||||
import { ServerConnection } from "@/context/server"
|
||||
import { createSdkForServer } from "./server"
|
||||
import { authTokenFromCredentials, createSdkForServer } from "./server"
|
||||
import { ClientError, OpenCode } from "@opencode-ai/client"
|
||||
import { Accessor, createEffect, onCleanup } from "solid-js"
|
||||
import { createStore, reconcile } from "solid-js/store"
|
||||
|
||||
@@ -61,6 +62,7 @@ function wait(ms: number, signal?: AbortSignal) {
|
||||
|
||||
function retryable(error: unknown, signal?: AbortSignal) {
|
||||
if (signal?.aborted) return false
|
||||
if (error instanceof ClientError) return error.reason === "Transport"
|
||||
if (!(error instanceof Error)) return false
|
||||
if (error.name === "AbortError" || error.name === "TimeoutError") return false
|
||||
if (error instanceof TypeError) return true
|
||||
@@ -82,15 +84,31 @@ export async function checkServerHealth(
|
||||
.then(() => attempt(count + 1))
|
||||
.catch(() => ({ healthy: false }))
|
||||
}
|
||||
const attempt = (count: number): Promise<ServerHealth> =>
|
||||
createSdkForServer({
|
||||
server,
|
||||
const attempt = async (count: number): Promise<ServerHealth> => {
|
||||
const current = await OpenCode.make({
|
||||
baseUrl: server.url,
|
||||
fetch,
|
||||
signal,
|
||||
headers: server.password
|
||||
? {
|
||||
Authorization: `Basic ${authTokenFromCredentials({ username: server.username, password: server.password })}`,
|
||||
}
|
||||
: undefined,
|
||||
})
|
||||
.health.get({ signal })
|
||||
.then((x) =>
|
||||
typeof x.healthy === "boolean"
|
||||
? { data: { healthy: x.healthy, version: x.version } }
|
||||
: { error: new Error("Invalid health response") },
|
||||
)
|
||||
.catch((error) => ({ error }))
|
||||
if ("data" in current && current.data) return current.data
|
||||
if (signal?.aborted) return { healthy: false }
|
||||
|
||||
return createSdkForServer({ server, fetch, signal })
|
||||
.global.health()
|
||||
.then((x) => (x.error ? next(count, x.error) : { healthy: x.data?.healthy === true, version: x.data?.version }))
|
||||
.catch((error) => next(count, error))
|
||||
}
|
||||
return attempt(0).finally(() => timeout?.clear?.())
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
import { describe, expect, test } from "bun:test"
|
||||
import { detectServerProtocol } from "./server-protocol"
|
||||
|
||||
const server = { url: "http://localhost:4096" }
|
||||
const json = (value: unknown, status = 200) =>
|
||||
new Response(JSON.stringify(value), { status, headers: { "content-type": "application/json" } })
|
||||
const mockFetch = (run: (input: string | URL | Request) => Promise<Response>) =>
|
||||
Object.assign(run, { preconnect: globalThis.fetch.preconnect })
|
||||
|
||||
describe("detectServerProtocol", () => {
|
||||
test("prefers the legacy health endpoint when both API generations exist", async () => {
|
||||
const fetcher = mockFetch((input) => {
|
||||
const path = new URL(input instanceof Request ? input.url : input).pathname
|
||||
if (path === "/global/health") return Promise.resolve(json({ healthy: true, version: "1.18.4" }))
|
||||
return Promise.resolve(json({ healthy: true, version: "2.0.0", pid: 123 }))
|
||||
})
|
||||
|
||||
expect(await detectServerProtocol(server, fetcher)).toBe("v1")
|
||||
})
|
||||
|
||||
test("recognizes V2 health by its process identifier", async () => {
|
||||
const fetcher = mockFetch((input) => {
|
||||
const path = new URL(input instanceof Request ? input.url : input).pathname
|
||||
if (path === "/global/health") return Promise.resolve(json({}, 404))
|
||||
return Promise.resolve(json({ healthy: true, version: "2.0.0", pid: 123 }))
|
||||
})
|
||||
|
||||
expect(await detectServerProtocol(server, fetcher)).toBe("v2")
|
||||
})
|
||||
|
||||
test("recognizes the transitional V1 API health response", async () => {
|
||||
const fetcher = mockFetch((input) => {
|
||||
const path = new URL(input instanceof Request ? input.url : input).pathname
|
||||
if (path === "/global/health") return Promise.resolve(json({}, 404))
|
||||
return Promise.resolve(json({ healthy: true }))
|
||||
})
|
||||
|
||||
expect(await detectServerProtocol(server, fetcher)).toBe("v1")
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,35 @@
|
||||
import type { ServerConnection } from "@/context/server"
|
||||
import { authTokenFromCredentials } from "./server"
|
||||
|
||||
export type ServerProtocol = "v1" | "v2"
|
||||
|
||||
function headers(server: ServerConnection.HttpBase) {
|
||||
if (!server.password) return
|
||||
return {
|
||||
Authorization: `Basic ${authTokenFromCredentials({ username: server.username, password: server.password })}`,
|
||||
}
|
||||
}
|
||||
|
||||
async function probe(server: ServerConnection.HttpBase, fetch: typeof globalThis.fetch, path: string) {
|
||||
const response = await fetch(new URL(path, server.url), {
|
||||
headers: headers(server),
|
||||
signal: AbortSignal.timeout(5_000),
|
||||
})
|
||||
if (!response.ok || !response.headers.get("content-type")?.includes("application/json")) return
|
||||
const value: unknown = await response.json()
|
||||
if (!value || typeof value !== "object") return
|
||||
return value
|
||||
}
|
||||
|
||||
export async function detectServerProtocol(
|
||||
server: ServerConnection.HttpBase,
|
||||
fetch: typeof globalThis.fetch,
|
||||
): Promise<ServerProtocol> {
|
||||
const legacy = await probe(server, fetch, "/global/health").catch(() => undefined)
|
||||
if (legacy && "healthy" in legacy && legacy.healthy === true) return "v1"
|
||||
|
||||
const current = await probe(server, fetch, "/api/health").catch(() => undefined)
|
||||
if (current && "pid" in current && typeof current.pid === "number") return "v2"
|
||||
if (current && "healthy" in current && current.healthy === true) return "v1"
|
||||
return "v2"
|
||||
}
|
||||
@@ -1,4 +1,5 @@
|
||||
import { createOpencodeClient } from "@opencode-ai/sdk/v2/client"
|
||||
import { OpenCode, type OpenCodeClient } from "@opencode-ai/client/promise"
|
||||
import type { ServerConnection } from "@/context/server"
|
||||
import { decode64 } from "@/utils/base64"
|
||||
|
||||
@@ -39,3 +40,23 @@ export function createSdkForServer({
|
||||
baseUrl: server.url,
|
||||
})
|
||||
}
|
||||
|
||||
export function createApiForServer(input: {
|
||||
server: ServerConnection.HttpBase
|
||||
fetch?: typeof globalThis.fetch
|
||||
}): OpenCodeClient {
|
||||
return OpenCode.make({
|
||||
baseUrl: input.server.url,
|
||||
fetch: input.fetch,
|
||||
headers: input.server.password
|
||||
? {
|
||||
Authorization: `Basic ${authTokenFromCredentials({
|
||||
username: input.server.username,
|
||||
password: input.server.password,
|
||||
})}`,
|
||||
}
|
||||
: undefined,
|
||||
})
|
||||
}
|
||||
|
||||
export type ServerApi = OpenCodeClient
|
||||
|
||||
Binary file not shown.
@@ -22,7 +22,7 @@
|
||||
"typecheck": "tsgo --noEmit"
|
||||
},
|
||||
"dependencies": {
|
||||
"@agentclientprotocol/sdk": "0.21.0",
|
||||
"@agentclientprotocol/sdk": "1.2.1",
|
||||
"@effect/platform-node": "catalog:",
|
||||
"@opencode-ai/client": "workspace:*",
|
||||
"@opencode-ai/plugin": "workspace:*",
|
||||
|
||||
@@ -5,6 +5,7 @@ import {
|
||||
type AuthenticateRequest,
|
||||
type CancelNotification,
|
||||
type CloseSessionRequest,
|
||||
type DeleteSessionRequest,
|
||||
type ForkSessionRequest,
|
||||
type InitializeRequest,
|
||||
type ListSessionsRequest,
|
||||
@@ -13,7 +14,6 @@ import {
|
||||
type PromptRequest,
|
||||
type ResumeSessionRequest,
|
||||
type SetSessionConfigOptionRequest,
|
||||
type SetSessionModelRequest,
|
||||
type SetSessionModeRequest,
|
||||
} from "@agentclientprotocol/sdk"
|
||||
import type { OpenCodeClient } from "@opencode-ai/client/promise"
|
||||
@@ -28,12 +28,12 @@ export function create(client: OpenCodeClient, connection: AgentSideConnection)
|
||||
newSession: (params: NewSessionRequest) => run(service.newSession(params)),
|
||||
loadSession: (params: LoadSessionRequest) => run(service.loadSession(params)),
|
||||
listSessions: (params: ListSessionsRequest) => run(service.listSessions(params)),
|
||||
deleteSession: (params: DeleteSessionRequest) => run(service.deleteSession(params)),
|
||||
resumeSession: (params: ResumeSessionRequest) => run(service.resumeSession(params)),
|
||||
closeSession: (params: CloseSessionRequest) => run(service.closeSession(params)),
|
||||
unstable_forkSession: (params: ForkSessionRequest) => run(service.forkSession(params)),
|
||||
setSessionConfigOption: (params: SetSessionConfigOptionRequest) => run(service.setSessionConfigOption(params)),
|
||||
setSessionMode: (params: SetSessionModeRequest) => run(service.setSessionMode(params)),
|
||||
unstable_setSessionModel: (params: SetSessionModelRequest) => run(service.setSessionModel(params)),
|
||||
prompt: (params: PromptRequest) => run(service.prompt(params)),
|
||||
cancel: (params: CancelNotification) => run(service.cancel(params)),
|
||||
} satisfies Agent
|
||||
|
||||
@@ -28,7 +28,7 @@ export type TurnControl = {
|
||||
type ToolState = {
|
||||
readonly name: string
|
||||
input: ToolInput
|
||||
structured: Record<string, unknown>
|
||||
metadata: Record<string, unknown>
|
||||
content: ToolContent
|
||||
}
|
||||
|
||||
@@ -38,7 +38,7 @@ export type TurnStart =
|
||||
| { readonly type: "compaction"; readonly id: string }
|
||||
|
||||
function emptyToolState(): ToolState {
|
||||
return { name: "tool", input: {}, structured: {}, content: [] }
|
||||
return { name: "tool", input: {}, metadata: {}, content: [] }
|
||||
}
|
||||
|
||||
export async function streamTurn(input: {
|
||||
@@ -47,7 +47,7 @@ export async function streamTurn(input: {
|
||||
readonly sessionID: string
|
||||
readonly cwd: string
|
||||
readonly start: TurnStart
|
||||
readonly userMessageID?: string | null
|
||||
readonly writeTextFile: boolean
|
||||
readonly submit: (signal: AbortSignal) => Promise<unknown>
|
||||
readonly control: TurnControl
|
||||
}): Promise<PromptResponse> {
|
||||
@@ -120,7 +120,7 @@ export async function streamTurn(input: {
|
||||
}
|
||||
if (event.type === "session.tool.input.started") {
|
||||
assistantMessageID = event.data.assistantMessageID
|
||||
tools.set(event.data.callID, { name: event.data.name, input: {}, structured: {}, content: [] })
|
||||
tools.set(event.data.callID, { name: event.data.name, input: {}, metadata: {}, content: [] })
|
||||
await update({
|
||||
sessionUpdate: "tool_call",
|
||||
...pendingToolCall({
|
||||
@@ -151,15 +151,13 @@ export async function streamTurn(input: {
|
||||
if (event.type === "session.tool.progress") {
|
||||
const current = tools.get(event.data.callID)
|
||||
if (!current) continue
|
||||
current.structured = event.data.structured
|
||||
current.content = event.data.content
|
||||
current.metadata = event.data.metadata
|
||||
await update({
|
||||
sessionUpdate: "tool_call_update",
|
||||
...runningToolUpdate({
|
||||
toolCallId: event.data.callID,
|
||||
toolName: current.name,
|
||||
state: { input: current.input },
|
||||
content: current.content,
|
||||
cwd: input.cwd,
|
||||
}),
|
||||
})
|
||||
@@ -170,11 +168,12 @@ export async function streamTurn(input: {
|
||||
tools.delete(event.data.callID)
|
||||
await syncEditedFiles({
|
||||
connection: input.connection,
|
||||
writeTextFile: input.writeTextFile,
|
||||
sessionID: input.sessionID,
|
||||
cwd: input.cwd,
|
||||
toolName: current.name,
|
||||
toolInput: current.input,
|
||||
structured: event.data.structured,
|
||||
metadata: event.data.metadata ?? {},
|
||||
}).catch(() => {})
|
||||
await update({
|
||||
sessionUpdate: "tool_call_update",
|
||||
@@ -182,9 +181,8 @@ export async function streamTurn(input: {
|
||||
toolCallId: event.data.callID,
|
||||
toolName: current.name,
|
||||
input: current.input,
|
||||
structured: event.data.structured,
|
||||
metadata: event.data.metadata,
|
||||
content: event.data.content,
|
||||
result: event.data.result,
|
||||
}),
|
||||
})
|
||||
continue
|
||||
@@ -198,7 +196,7 @@ export async function streamTurn(input: {
|
||||
toolCallId: event.data.callID,
|
||||
toolName: current.name,
|
||||
input: current.input,
|
||||
structured: event.data.metadata ?? current.structured,
|
||||
metadata: event.data.metadata ?? current.metadata,
|
||||
content: event.data.content ?? current.content,
|
||||
error: event.data.error.message,
|
||||
cwd: input.cwd,
|
||||
@@ -231,7 +229,7 @@ export async function streamTurn(input: {
|
||||
if (!started) {
|
||||
streamController.abort()
|
||||
await completed.catch(() => {})
|
||||
return response(undefined, undefined, "interrupted", true, undefined, input.userMessageID)
|
||||
return response(undefined, undefined, "interrupted", true, undefined)
|
||||
}
|
||||
}
|
||||
const terminal = await completed
|
||||
@@ -246,7 +244,6 @@ export async function streamTurn(input: {
|
||||
terminal,
|
||||
control.cancelled,
|
||||
finish,
|
||||
input.userMessageID,
|
||||
)
|
||||
} catch (error) {
|
||||
streamController.abort()
|
||||
@@ -342,9 +339,8 @@ async function replayMessage(
|
||||
toolCallId: part.id,
|
||||
toolName: part.name,
|
||||
input: part.state.input,
|
||||
structured: part.state.structured,
|
||||
metadata: part.state.metadata,
|
||||
content: part.state.content,
|
||||
result: part.state.result,
|
||||
}),
|
||||
},
|
||||
})
|
||||
@@ -358,7 +354,6 @@ async function replayMessage(
|
||||
toolCallId: part.id,
|
||||
toolName: part.name,
|
||||
state: { input: part.state.input },
|
||||
content: part.state.content,
|
||||
cwd,
|
||||
}),
|
||||
},
|
||||
@@ -373,7 +368,7 @@ async function replayMessage(
|
||||
toolCallId: part.id,
|
||||
toolName: part.name,
|
||||
input: part.state.input,
|
||||
structured: part.state.structured,
|
||||
metadata: part.state.metadata,
|
||||
content: part.state.content,
|
||||
error: part.state.error.message,
|
||||
cwd,
|
||||
@@ -400,7 +395,6 @@ function response(
|
||||
terminal: "succeeded" | "failed" | "interrupted",
|
||||
cancelled: boolean,
|
||||
finish: SessionMessageAssistant["finish"],
|
||||
messageID: string | null | undefined,
|
||||
): PromptResponse {
|
||||
const error = assistant?.error ?? executionError
|
||||
if (error?.type === "provider.auth") throw new ACPError.AuthRequiredError()
|
||||
@@ -423,7 +417,7 @@ function response(
|
||||
}
|
||||
: undefined
|
||||
const stopReason = resolveStopReason({ terminal, cancelled, finish, error: error?.type })
|
||||
return { stopReason, ...(usage ? { usage } : {}), ...(messageID ? { userMessageId: messageID } : {}), _meta: {} }
|
||||
return { stopReason, ...(usage ? { usage } : {}), _meta: {} }
|
||||
}
|
||||
|
||||
function resolveStopReason(input: {
|
||||
|
||||
@@ -53,15 +53,16 @@ export async function replyPermission(input: {
|
||||
|
||||
export async function syncEditedFiles(input: {
|
||||
readonly connection: Partial<Pick<AgentSideConnection, "writeTextFile">>
|
||||
readonly writeTextFile: boolean
|
||||
readonly sessionID: string
|
||||
readonly cwd: string
|
||||
readonly toolName: string
|
||||
readonly toolInput: ToolInput
|
||||
readonly structured: Readonly<Record<string, unknown>>
|
||||
readonly metadata: Readonly<Record<string, unknown>>
|
||||
}) {
|
||||
if (!input.connection.writeTextFile || toToolKind(input.toolName) !== "edit") return
|
||||
const files = Array.isArray(input.structured.files)
|
||||
? input.structured.files.flatMap((file): string[] => {
|
||||
if (!input.writeTextFile || !input.connection.writeTextFile || toToolKind(input.toolName) !== "edit") return
|
||||
const files = Array.isArray(input.metadata.files)
|
||||
? input.metadata.files.flatMap((file): string[] => {
|
||||
if (!file || typeof file !== "object") return []
|
||||
const path = Reflect.get(file, "file")
|
||||
return typeof path === "string" ? [path] : []
|
||||
|
||||
@@ -16,6 +16,8 @@ import type {
|
||||
CancelNotification,
|
||||
CloseSessionRequest,
|
||||
CloseSessionResponse,
|
||||
DeleteSessionRequest,
|
||||
DeleteSessionResponse,
|
||||
ForkSessionRequest,
|
||||
ForkSessionResponse,
|
||||
InitializeRequest,
|
||||
@@ -33,8 +35,6 @@ import type {
|
||||
ResumeSessionResponse,
|
||||
SetSessionConfigOptionRequest,
|
||||
SetSessionConfigOptionResponse,
|
||||
SetSessionModelRequest,
|
||||
SetSessionModelResponse,
|
||||
SetSessionModeRequest,
|
||||
SetSessionModeResponse,
|
||||
} from "@agentclientprotocol/sdk"
|
||||
@@ -47,7 +47,8 @@ import { ACPError } from "./error"
|
||||
|
||||
export const AuthMethodID = "opencode-login"
|
||||
|
||||
type Connection = Pick<AgentSideConnection, "sessionUpdate" | "requestPermission">
|
||||
type Connection = Pick<AgentSideConnection, "sessionUpdate" | "requestPermission"> &
|
||||
Partial<Pick<AgentSideConnection, "writeTextFile">>
|
||||
|
||||
type Catalog = {
|
||||
readonly providers: ConfigOptionProvider[]
|
||||
@@ -83,12 +84,12 @@ export interface Interface {
|
||||
newSession(input: NewSessionRequest): Promise<NewSessionResponse>
|
||||
loadSession(input: LoadSessionRequest): Promise<LoadSessionResponse>
|
||||
listSessions(input: ListSessionsRequest): Promise<ListSessionsResponse>
|
||||
deleteSession(input: DeleteSessionRequest): Promise<DeleteSessionResponse>
|
||||
resumeSession(input: ResumeSessionRequest): Promise<ResumeSessionResponse>
|
||||
closeSession(input: CloseSessionRequest): Promise<CloseSessionResponse>
|
||||
forkSession(input: ForkSessionRequest): Promise<ForkSessionResponse>
|
||||
setSessionConfigOption(input: SetSessionConfigOptionRequest): Promise<SetSessionConfigOptionResponse>
|
||||
setSessionMode(input: SetSessionModeRequest): Promise<SetSessionModeResponse>
|
||||
setSessionModel(input: SetSessionModelRequest): Promise<SetSessionModelResponse>
|
||||
prompt(input: PromptRequest): Promise<PromptResponse>
|
||||
cancel(input: CancelNotification): Promise<void>
|
||||
}
|
||||
@@ -98,6 +99,7 @@ export function make(input: { readonly client: OpenCodeClient; readonly connecti
|
||||
const catalogs = new Map<string, Promise<Catalog>>()
|
||||
const registeredMcp = new Map<string, Set<string>>()
|
||||
const active = new Map<string, TurnControl>()
|
||||
const capabilities = { writeTextFile: false }
|
||||
|
||||
const catalog = (cwd: string) => {
|
||||
const cached = catalogs.get(cwd)
|
||||
@@ -157,6 +159,7 @@ export function make(input: { readonly client: OpenCodeClient; readonly connecti
|
||||
|
||||
return {
|
||||
initialize: async (params) => {
|
||||
capabilities.writeTextFile = params.clientCapabilities?.fs?.writeTextFile === true
|
||||
const authMethod: AuthMethod = {
|
||||
description: "Run `opencode auth login` in the terminal",
|
||||
name: "Login with opencode",
|
||||
@@ -173,7 +176,7 @@ export function make(input: { readonly client: OpenCodeClient; readonly connecti
|
||||
loadSession: true,
|
||||
mcpCapabilities: { http: true, sse: false },
|
||||
promptCapabilities: { embeddedContext: true, image: true },
|
||||
sessionCapabilities: { close: {}, fork: {}, list: {}, resume: {} },
|
||||
sessionCapabilities: { close: {}, delete: {}, fork: {}, list: {}, resume: {} },
|
||||
},
|
||||
authMethods: [authMethod],
|
||||
agentInfo: { name: "OpenCode", version: OPENCODE_VERSION },
|
||||
@@ -216,6 +219,14 @@ export function make(input: { readonly client: OpenCodeClient; readonly connecti
|
||||
...(page.cursor.next ? { nextCursor: page.cursor.next } : {}),
|
||||
}
|
||||
},
|
||||
deleteSession: async (params) => {
|
||||
await input.client.session.remove({ sessionID: params.sessionId }).catch((error) => {
|
||||
if (!isSessionNotFoundError(error)) throw error
|
||||
})
|
||||
sessions.delete(params.sessionId)
|
||||
registeredMcp.delete(params.sessionId)
|
||||
return {}
|
||||
},
|
||||
resumeSession: async (params) => {
|
||||
const session = await getSession(input.client, params.sessionId)
|
||||
const state = await attach(session, session.location.directory, params.mcpServers ?? [])
|
||||
@@ -270,13 +281,6 @@ export function make(input: { readonly client: OpenCodeClient; readonly connecti
|
||||
await selectMode(input.client, await requireSession(params.sessionId), params.modeId)
|
||||
return {}
|
||||
},
|
||||
setSessionModel: async (params) => {
|
||||
const state = await requireSession(params.sessionId)
|
||||
const selected = requireModel(state.catalog, params.modelId)
|
||||
state.model = selected
|
||||
await input.client.session.switchModel({ sessionID: state.id, model: selected })
|
||||
return {}
|
||||
},
|
||||
prompt: async (params) => {
|
||||
const state = await requireSession(params.sessionId)
|
||||
if (active.has(state.id)) {
|
||||
@@ -295,7 +299,7 @@ export function make(input: { readonly client: OpenCodeClient; readonly connecti
|
||||
sessionID: state.id,
|
||||
cwd: state.cwd,
|
||||
start: prepared.start,
|
||||
userMessageID: params.messageId,
|
||||
writeTextFile: capabilities.writeTextFile,
|
||||
control,
|
||||
submit: (signal) => submitPrompt(input.client, state, prepared, signal),
|
||||
}).finally(() => {
|
||||
@@ -479,6 +483,7 @@ async function registerMcpServers(
|
||||
|
||||
function mcpConfig(server: McpServer) {
|
||||
if ("type" in server) {
|
||||
if (server.type === "acp") throw new Error("MCP-over-ACP is not supported")
|
||||
return {
|
||||
type: "remote" as const,
|
||||
url: server.url,
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { isAbsolute, resolve } from "node:path"
|
||||
import type { ToolCall, ToolCallContent, ToolCallLocation, ToolCallUpdate, ToolKind } from "@agentclientprotocol/sdk"
|
||||
import { readDisplayText } from "@opencode-ai/tui/mini/tool"
|
||||
|
||||
export type ToolInput = Record<string, unknown>
|
||||
export type ToolContent = ReadonlyArray<
|
||||
@@ -100,11 +101,12 @@ export function completedToolUpdate(input: {
|
||||
readonly toolName: string
|
||||
readonly input: ToolInput
|
||||
readonly content: ToolContent
|
||||
readonly structured: Readonly<Record<string, unknown>>
|
||||
readonly result?: unknown
|
||||
readonly metadata?: Readonly<Record<string, unknown>>
|
||||
}): ToolCallUpdate {
|
||||
const normalized = toolContent(input.content)
|
||||
const read = input.toolName.toLocaleLowerCase() === "read" ? readDisplayText(input.structured) : undefined
|
||||
// Read's model content is a JSON page envelope; show the clean text instead.
|
||||
const firstText = input.content.find((part) => part.type === "text")
|
||||
const read = input.toolName.toLocaleLowerCase() === "read" && firstText ? readDisplayText(firstText.text) : undefined
|
||||
const images = normalized.filter((part) => part.type === "content" && part.content.type === "image")
|
||||
const primary =
|
||||
read === undefined
|
||||
@@ -128,8 +130,7 @@ export function completedToolUpdate(input: {
|
||||
status: "completed",
|
||||
content: [...primary, ...diff, ...images],
|
||||
rawOutput: {
|
||||
structured: input.structured,
|
||||
...(input.result === undefined ? {} : { result: input.result }),
|
||||
...(input.metadata === undefined ? {} : { metadata: input.metadata }),
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -138,8 +139,8 @@ export function errorToolUpdate(input: {
|
||||
readonly toolCallId: string
|
||||
readonly toolName: string
|
||||
readonly input: ToolInput
|
||||
readonly content: ToolContent
|
||||
readonly structured: Readonly<Record<string, unknown>>
|
||||
readonly content?: ToolContent
|
||||
readonly metadata?: Readonly<Record<string, unknown>>
|
||||
readonly error: string
|
||||
readonly cwd?: string
|
||||
}): ToolCallUpdate {
|
||||
@@ -150,8 +151,11 @@ export function errorToolUpdate(input: {
|
||||
title: toolTitle(input.toolName, input.input, undefined),
|
||||
locations: toLocations(input.toolName, input.input, input.cwd),
|
||||
rawInput: rawInput(input.toolName, input.input, input.cwd),
|
||||
content: [...toolContent(input.content), { type: "content", content: { type: "text", text: input.error } }],
|
||||
rawOutput: { structured: input.structured, error: input.error },
|
||||
content: [...toolContent(input.content ?? []), { type: "content", content: { type: "text", text: input.error } }],
|
||||
rawOutput: {
|
||||
...(input.metadata === undefined ? {} : { metadata: input.metadata }),
|
||||
error: input.error,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -164,21 +168,6 @@ function toolContent(content: ToolContent): ToolCallContent[] {
|
||||
})
|
||||
}
|
||||
|
||||
function readDisplayText(structured: Readonly<Record<string, unknown>>) {
|
||||
if (typeof structured.content === "string") {
|
||||
if (structured.type === "text-page" || structured.encoding === "utf8") return structured.content
|
||||
}
|
||||
if (!Array.isArray(structured.entries)) return undefined
|
||||
return structured.entries
|
||||
.flatMap((entry): string[] => {
|
||||
if (typeof entry === "string") return [entry]
|
||||
if (!entry || typeof entry !== "object") return []
|
||||
const path = Reflect.get(entry, "path")
|
||||
return typeof path === "string" ? [path] : []
|
||||
})
|
||||
.join("\n")
|
||||
}
|
||||
|
||||
function toolTitle(toolName: string, input: ToolInput, fallback: string | undefined) {
|
||||
if (isShell(toolName)) return stringValue(input.command) ?? stringValue(input.cmd) ?? fallback ?? toolName
|
||||
return fallback || toolName
|
||||
|
||||
@@ -140,11 +140,11 @@ export const Commands = Spec.make(typeof OPENCODE_CLI_NAME === "string" ? OPENCO
|
||||
Flag.withDefault(false),
|
||||
),
|
||||
replay: Flag.boolean("replay").pipe(
|
||||
Flag.withDescription("Replay session history on resume and after resize"),
|
||||
Flag.withDefault(true),
|
||||
Flag.withDescription("Restore session history on resume and resize (disable with --no-replay)"),
|
||||
Flag.optional,
|
||||
),
|
||||
replayLimit: Flag.integer("replay-limit").pipe(
|
||||
Flag.withDescription("Cap visible replay to the newest N messages"),
|
||||
Flag.withDescription("Limit replay to the newest N messages (default: 200)"),
|
||||
Flag.optional,
|
||||
),
|
||||
model: Flag.string("model").pipe(
|
||||
|
||||
@@ -28,8 +28,8 @@ export default Runtime.handler(Commands.commands.mini, (input) =>
|
||||
model: Option.getOrUndefined(input.model),
|
||||
agent: Option.getOrUndefined(input.agent),
|
||||
prompt: Option.getOrUndefined(input.prompt),
|
||||
replay: input.replay,
|
||||
replayLimit: Option.getOrUndefined(input.replayLimit),
|
||||
replay: Option.getOrUndefined(input.replay) ?? resolved.mini?.replay ?? true,
|
||||
replayLimit: Option.getOrUndefined(input.replayLimit) ?? resolved.mini?.replay_limit,
|
||||
demo: input.demo,
|
||||
tuiConfig: resolved,
|
||||
config: {
|
||||
|
||||
@@ -10,6 +10,7 @@ import {
|
||||
Reference,
|
||||
Skill,
|
||||
} from "@opencode-ai/plugin/v2"
|
||||
import { Tool } from "@opencode-ai/plugin/v2/tool"
|
||||
|
||||
const key = Symbol.for("opencode.plugin.v2.promise")
|
||||
;(globalThis as typeof globalThis & { [key]?: unknown })[key] = {
|
||||
@@ -23,4 +24,5 @@ const key = Symbol.for("opencode.plugin.v2.promise")
|
||||
Provider,
|
||||
Reference,
|
||||
Skill,
|
||||
Tool,
|
||||
}
|
||||
|
||||
@@ -10,7 +10,7 @@ import type {
|
||||
import { SessionMessage } from "@opencode-ai/schema/session-message"
|
||||
import { EOL } from "node:os"
|
||||
import { readFile } from "node:fs/promises"
|
||||
import { toolOutputText, type MiniToolPart } from "@opencode-ai/tui/mini/tool"
|
||||
import { nonEmptyToolContent, toolOutputText, type MiniToolPart } from "@opencode-ai/tui/mini/tool"
|
||||
import { UI } from "./ui"
|
||||
|
||||
type Model = {
|
||||
@@ -55,7 +55,7 @@ type ToolState = StartedPart & {
|
||||
raw?: string
|
||||
provider?: unknown
|
||||
providerState?: SessionMessageAssistantTool["providerState"]
|
||||
structured: Record<string, JsonValue>
|
||||
metadata: Record<string, JsonValue>
|
||||
content: LLMToolContent[]
|
||||
}
|
||||
|
||||
@@ -306,7 +306,7 @@ export async function runNonInteractivePrompt(input: Input) {
|
||||
assistantMessageID: event.data.assistantMessageID,
|
||||
tool: event.data.name,
|
||||
input: {},
|
||||
structured: {},
|
||||
metadata: {},
|
||||
content: [],
|
||||
})
|
||||
continue
|
||||
@@ -334,7 +334,7 @@ export async function runNonInteractivePrompt(input: Input) {
|
||||
raw: current?.raw,
|
||||
provider: { executed: event.data.executed, state: event.data.state },
|
||||
providerState: event.data.state,
|
||||
structured: {},
|
||||
metadata: {},
|
||||
content: [],
|
||||
})
|
||||
continue
|
||||
@@ -342,8 +342,7 @@ export async function runNonInteractivePrompt(input: Input) {
|
||||
if (event.type === "session.tool.progress") {
|
||||
const current = tools.get(toolKey(event.data.assistantMessageID, event.data.callID))
|
||||
if (current) {
|
||||
current.structured = event.data.structured
|
||||
current.content = event.data.content
|
||||
current.metadata = event.data.metadata
|
||||
}
|
||||
continue
|
||||
}
|
||||
@@ -360,9 +359,8 @@ export async function runNonInteractivePrompt(input: Input) {
|
||||
state: {
|
||||
status: "completed",
|
||||
input: current.input,
|
||||
structured: event.data.structured,
|
||||
metadata: event.data.metadata,
|
||||
content: event.data.content,
|
||||
result: event.data.result,
|
||||
},
|
||||
time: { created: current.timestamp, ran: current.timestamp, completed: time },
|
||||
}
|
||||
@@ -379,9 +377,8 @@ export async function runNonInteractivePrompt(input: Input) {
|
||||
output: toolOutputText(current.tool, event.data.content),
|
||||
title: current.tool,
|
||||
metadata: {
|
||||
structured: event.data.structured,
|
||||
metadata: event.data.metadata,
|
||||
content: event.data.content,
|
||||
result: event.data.result,
|
||||
providerCall: current.provider,
|
||||
providerResult: { executed: event.data.executed, state: event.data.resultState },
|
||||
rawInput: current.raw,
|
||||
@@ -398,8 +395,8 @@ export async function runNonInteractivePrompt(input: Input) {
|
||||
const key = toolKey(event.data.assistantMessageID, event.data.callID)
|
||||
const current = tools.get(key) ?? fallbackTool(event)
|
||||
const error = event.data.error.message
|
||||
const structured = event.data.metadata ?? current.structured
|
||||
const content = event.data.content ?? current.content
|
||||
const metadata = event.data.metadata ?? current.metadata
|
||||
const content = event.data.content ?? nonEmptyToolContent(current.content)
|
||||
const tool: SessionMessageAssistantTool = {
|
||||
type: "tool",
|
||||
id: event.data.callID,
|
||||
@@ -410,10 +407,9 @@ export async function runNonInteractivePrompt(input: Input) {
|
||||
state: {
|
||||
status: "error",
|
||||
input: current.input,
|
||||
structured,
|
||||
metadata,
|
||||
content,
|
||||
error: event.data.error,
|
||||
result: event.data.result,
|
||||
},
|
||||
time: { created: current.timestamp, ran: current.timestamp, completed: time },
|
||||
}
|
||||
@@ -429,7 +425,6 @@ export async function runNonInteractivePrompt(input: Input) {
|
||||
input: current.input,
|
||||
error,
|
||||
metadata: {
|
||||
result: event.data.result,
|
||||
providerCall: current.provider,
|
||||
providerResult: { executed: event.data.executed, state: event.data.resultState },
|
||||
rawInput: current.raw,
|
||||
@@ -441,15 +436,14 @@ export async function runNonInteractivePrompt(input: Input) {
|
||||
renderedTools.add(key)
|
||||
if (input.compatibility === "v1" && (permissionRejected || formCancelled)) continue
|
||||
if (!emit("tool_use", time, { part })) {
|
||||
if (toolOutputText(current.tool, content).trim())
|
||||
if (content && toolOutputText(current.tool, content).trim())
|
||||
await input.renderTool({
|
||||
...tool,
|
||||
state: {
|
||||
status: "completed",
|
||||
input: current.input,
|
||||
structured,
|
||||
metadata,
|
||||
content,
|
||||
result: event.data.result,
|
||||
},
|
||||
})
|
||||
await input.renderToolError(tool)
|
||||
@@ -597,14 +591,14 @@ export async function runNonInteractivePrompt(input: Input) {
|
||||
input: item.state.input,
|
||||
output: toolOutputText(item.name, item.state.content),
|
||||
title: item.name,
|
||||
metadata: { structured: item.state.structured, content: item.state.content, result: item.state.result },
|
||||
metadata: { metadata: item.state.metadata, content: item.state.content },
|
||||
time: { start: item.time.ran ?? item.time.created, end: item.time.completed ?? timestamp },
|
||||
}
|
||||
: {
|
||||
status: "error",
|
||||
input: item.state.input,
|
||||
error: item.state.error.message,
|
||||
metadata: { structured: item.state.structured, content: item.state.content, result: item.state.result },
|
||||
metadata: { metadata: item.state.metadata, content: item.state.content },
|
||||
time: { start: item.time.ran ?? item.time.created, end: item.time.completed ?? timestamp },
|
||||
},
|
||||
}
|
||||
@@ -614,8 +608,16 @@ export async function runNonInteractivePrompt(input: Input) {
|
||||
await input.renderTool(item)
|
||||
continue
|
||||
}
|
||||
if (toolOutputText(item.name, item.state.content).trim()) {
|
||||
await input.renderTool({ ...item, state: { ...item.state, status: "completed" } })
|
||||
if (item.state.content && toolOutputText(item.name, item.state.content).trim()) {
|
||||
await input.renderTool({
|
||||
...item,
|
||||
state: {
|
||||
status: "completed",
|
||||
input: item.state.input,
|
||||
metadata: item.state.metadata,
|
||||
content: item.state.content,
|
||||
},
|
||||
})
|
||||
}
|
||||
await input.renderToolError(item)
|
||||
UI.error(item.state.error.message)
|
||||
@@ -792,7 +794,7 @@ function fallbackTool(event: {
|
||||
assistantMessageID: event.data.assistantMessageID,
|
||||
tool: "tool",
|
||||
input: {},
|
||||
structured: {},
|
||||
metadata: {},
|
||||
content: [],
|
||||
}
|
||||
}
|
||||
|
||||
@@ -218,8 +218,7 @@ describe("acp event behavior", () => {
|
||||
sessionID: "ses_tools",
|
||||
assistantMessageID: "msg_tools",
|
||||
callID: "call_ok",
|
||||
structured: { phase: 1 },
|
||||
content: [{ type: "text", text: "working" }],
|
||||
metadata: { phase: 1 },
|
||||
}),
|
||||
)
|
||||
send(
|
||||
@@ -227,9 +226,8 @@ describe("acp event behavior", () => {
|
||||
sessionID: "ses_tools",
|
||||
assistantMessageID: "msg_tools",
|
||||
callID: "call_ok",
|
||||
structured: { exit: 0 },
|
||||
metadata: { exit: 0 },
|
||||
content: [{ type: "text", text: "done" }],
|
||||
result: { code: 0 },
|
||||
executed: true,
|
||||
}),
|
||||
)
|
||||
@@ -255,8 +253,7 @@ describe("acp event behavior", () => {
|
||||
sessionID: "ses_tools",
|
||||
assistantMessageID: "msg_tools",
|
||||
callID: "call_fail",
|
||||
structured: { bytes: 0 },
|
||||
content: [{ type: "text", text: "opening" }],
|
||||
metadata: { bytes: 0 },
|
||||
}),
|
||||
)
|
||||
send(
|
||||
@@ -313,12 +310,10 @@ describe("acp event behavior", () => {
|
||||
locations: [{ path: resolve("/workspace", "sub") }],
|
||||
rawInput: { command: "printf done", workdir: "sub" },
|
||||
})
|
||||
expect(updates[2]?.update).toMatchObject({
|
||||
content: [{ type: "content", content: { type: "text", text: "working" } }],
|
||||
})
|
||||
expect(updates[2]?.update).not.toHaveProperty("content")
|
||||
expect(updates[3]?.update).toMatchObject({
|
||||
content: [{ type: "content", content: { type: "text", text: "done" } }],
|
||||
rawOutput: { structured: { exit: 0 }, result: { code: 0 } },
|
||||
rawOutput: { metadata: { exit: 0 } },
|
||||
})
|
||||
expect(updates[7]?.update).toMatchObject({
|
||||
kind: "read",
|
||||
@@ -327,7 +322,7 @@ describe("acp event behavior", () => {
|
||||
{ type: "content", content: { type: "text", text: "opening" } },
|
||||
{ type: "content", content: { type: "text", text: "not found" } },
|
||||
],
|
||||
rawOutput: { structured: { bytes: 0 }, error: "not found" },
|
||||
rawOutput: { metadata: { bytes: 0 }, error: "not found" },
|
||||
})
|
||||
expect(response.stopReason).toBe("end_turn")
|
||||
} finally {
|
||||
@@ -379,7 +374,7 @@ describe("acp event behavior", () => {
|
||||
{ type: "content", content: { type: "text", text: "done" } },
|
||||
{ type: "content", content: { type: "image", mimeType: "image/png", data: "AAAA" } },
|
||||
],
|
||||
rawOutput: { structured: { exit: 0 }, result: { code: 0 } },
|
||||
rawOutput: { metadata: { exit: 0 } },
|
||||
})
|
||||
expect(updates[8]?.update).toMatchObject({
|
||||
toolCallId: "call_running",
|
||||
@@ -439,6 +434,7 @@ describe("acp event behavior", () => {
|
||||
sessionID: "ses_cancel",
|
||||
cwd: "/workspace",
|
||||
start: { type: "input", id: "input_cancel" },
|
||||
writeTextFile: false,
|
||||
control,
|
||||
submit: async (signal) => {
|
||||
await fixture.client.session.prompt(
|
||||
@@ -481,6 +477,7 @@ describe("acp event behavior", () => {
|
||||
sessionID: "ses_cancel_admission",
|
||||
cwd: "/workspace",
|
||||
start: { type: "input", id: "input_cancel_admission" },
|
||||
writeTextFile: false,
|
||||
control,
|
||||
submit: (signal) =>
|
||||
fixture.client.session.prompt(
|
||||
@@ -566,7 +563,7 @@ function turn(input: {
|
||||
sessionID: input.sessionID,
|
||||
cwd: "/workspace",
|
||||
start: { type: "input", id: input.inputID },
|
||||
userMessageID: `client_${input.inputID}`,
|
||||
writeTextFile: false,
|
||||
control: { cancelled: false, admission: new AbortController() },
|
||||
submit: (signal) =>
|
||||
input.fixture.client.session.prompt({ sessionID: input.sessionID, id: input.inputID, text: "hello" }, { signal }),
|
||||
@@ -616,12 +613,11 @@ function replayFixtureMessages(): SessionMessageInfo[] {
|
||||
state: {
|
||||
status: "completed",
|
||||
input: { command: "printf done" },
|
||||
structured: { exit: 0 },
|
||||
metadata: { exit: 0 },
|
||||
content: [
|
||||
{ type: "text", text: "done" },
|
||||
{ type: "file", uri: "data:image/png;base64,AAAA", mime: "image/png", name: "image.png" },
|
||||
],
|
||||
result: { code: 0 },
|
||||
},
|
||||
},
|
||||
{
|
||||
@@ -632,8 +628,7 @@ function replayFixtureMessages(): SessionMessageInfo[] {
|
||||
state: {
|
||||
status: "running",
|
||||
input: { command: "pwd" },
|
||||
structured: {},
|
||||
content: [{ type: "text", text: "/workspace" }],
|
||||
metadata: {},
|
||||
},
|
||||
},
|
||||
{
|
||||
@@ -644,7 +639,7 @@ function replayFixtureMessages(): SessionMessageInfo[] {
|
||||
state: {
|
||||
status: "error",
|
||||
input: { filePath: "/workspace/missing.ts" },
|
||||
structured: { bytes: 0 },
|
||||
metadata: { bytes: 0 },
|
||||
content: [{ type: "text", text: "partial" }],
|
||||
error: { type: "tool.error", message: "failed hard" },
|
||||
},
|
||||
@@ -677,7 +672,7 @@ function replayToolMessage(id: string) {
|
||||
state: {
|
||||
status: "completed",
|
||||
input: { command: "printf done" },
|
||||
structured: { exit: 0 },
|
||||
metadata: { exit: 0 },
|
||||
content: [{ type: "text", text: "done" }],
|
||||
},
|
||||
},
|
||||
|
||||
@@ -85,7 +85,6 @@ test("acp prompt resolves after ordered turn updates", async () => {
|
||||
|
||||
try {
|
||||
const id = "msg_prompt"
|
||||
const userMessageID = "client-message"
|
||||
const response = await streamTurn({
|
||||
client,
|
||||
connection: {
|
||||
@@ -97,7 +96,7 @@ test("acp prompt resolves after ordered turn updates", async () => {
|
||||
sessionID: "ses_test",
|
||||
cwd: "/workspace",
|
||||
start: { type: "input", id },
|
||||
userMessageID,
|
||||
writeTextFile: false,
|
||||
control: { cancelled: false, admission: new AbortController() },
|
||||
submit: () => client.session.prompt({ sessionID: "ses_test", id, text: "hi" }),
|
||||
})
|
||||
@@ -112,7 +111,7 @@ test("acp prompt resolves after ordered turn updates", async () => {
|
||||
},
|
||||
},
|
||||
])
|
||||
expect(response).toMatchObject({ stopReason: "end_turn", userMessageId: userMessageID, usage: { totalTokens: 2 } })
|
||||
expect(response).toMatchObject({ stopReason: "end_turn", usage: { totalTokens: 2 } })
|
||||
} finally {
|
||||
events?.close()
|
||||
await server.stop(true)
|
||||
|
||||
@@ -14,6 +14,7 @@ describe("acp initialize/auth subprocess", () => {
|
||||
expect(initialized.agentCapabilities?.mcpCapabilities?.sse).toBe(false)
|
||||
expect(initialized.agentCapabilities?.loadSession).toBe(true)
|
||||
expect(initialized.agentCapabilities?.sessionCapabilities?.close).toEqual({})
|
||||
expect(initialized.agentCapabilities?.sessionCapabilities?.delete).toEqual({})
|
||||
expect(initialized.agentCapabilities?.sessionCapabilities?.fork).toEqual({})
|
||||
expect(initialized.agentCapabilities?.sessionCapabilities?.list).toEqual({})
|
||||
expect(initialized.agentCapabilities?.sessionCapabilities?.resume).toEqual({})
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import type {
|
||||
CloseSessionResponse,
|
||||
DeleteSessionResponse,
|
||||
ListSessionsResponse,
|
||||
LoadSessionResponse,
|
||||
ResumeSessionResponse,
|
||||
@@ -60,6 +61,20 @@ describe("acp lifecycle subprocess", () => {
|
||||
expect(listed.sessions.some((item) => item.sessionId === session.sessionId)).toBe(true)
|
||||
}, 60_000)
|
||||
|
||||
test("delete capability and delete request", async () => {
|
||||
await using fixture = await createAcpFixture()
|
||||
const acp = fixture.spawn()
|
||||
const initialized = await initialize(acp)
|
||||
expect(initialized.agentCapabilities?.sessionCapabilities?.delete).toEqual({})
|
||||
const session = await newSession(acp, fixture.home)
|
||||
|
||||
expect(
|
||||
expectOk(await acp.request<DeleteSessionResponse>("session/delete", { sessionId: session.sessionId })),
|
||||
).toEqual({})
|
||||
const listed = expectOk(await acp.request<ListSessionsResponse>("session/list", { cwd: fixture.home }))
|
||||
expect(listed.sessions.some((item) => item.sessionId === session.sessionId)).toBe(false)
|
||||
}, 60_000)
|
||||
|
||||
test("resume capability advertisement", async () => {
|
||||
await using fixture = await createAcpFixture()
|
||||
const initialized = await initialize(fixture.spawn())
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user