Compare commits

...
Author SHA1 Message Date
Aiden Cline a9da993e28 fix(core): trigger compaction at 85% of the input window 2026-09-24 17:36:58 -05:00
Aiden Cline 61ecf404b9 fix(core): apply GPT verbosity defaults at request time (#51166) 2026-09-24 17:13:39 -05:00
Shoubhit Dash 92d2b1700f refactor(ai): one media client shape and route types erased over options (#51226) 2026-09-25 03:15:45 +05:30
Aiden Cline 56262121ee feat(codemode): bind this and arguments in functions, add Function.prototype.call, apply, and bind (#50831) 2026-09-24 16:34:46 -05:00
Aiden Cline e3b588e7d2 refactor(ai): apply tool schema rules once per request (#51162) 2026-09-24 16:27:30 -05:00
opencode-agent[bot]andnexxeln 1de648cb13 feat(ai): restore direct LLM input overloads (#51211)
Co-authored-by: nexxeln <95541290+nexxeln@users.noreply.github.com>
2026-09-25 02:49:07 +05:30
Aiden Cline 03be7f385b fix(core): accept 128-character tool names (#51207) 2026-09-24 14:59:57 -05:00
Aiden Cline 8118690839 fix(core): show line counts when tool output is truncated (#51200) 2026-09-24 14:50:55 -05:00
Aiden Cline a16eedfed7 fix(codemode): make derived error constructors inherit from Error (#51045) 2026-09-24 14:03:10 -05:00
Aiden Cline e796f2f9a5 fix(core): read provider errors from plain AI SDK stream errors (#51194) 2026-09-24 13:52:32 -05:00
Aiden Cline 20610e6645 fix(ai): fit Claude thinking budgets on Bedrock Converse (#51190) 2026-09-24 13:48:39 -05:00
Aiden Cline 7f245b0968 fix(ai): fit OpenRouter reasoning budgets to the output limit (#51189) 2026-09-24 13:42:10 -05:00
Shoubhit Dash 7013e925f5 refactor(ai): keep LLM calls request-only in the promise client (#51180) 2026-09-24 23:47:42 +05:30
Aiden Cline 499c2feaa3 fix(core): cap Alibaba thinking budget variants at 64k (#51154) 2026-09-24 13:14:49 -05:00
Aiden Cline 03af821aa5 fix(core): restore the shell no-output placeholder (#51187) 2026-09-24 13:12:42 -05:00
Aiden Cline c903774556 fix(ai): fit thinking budgets to the output limit (#51157) 2026-09-24 12:58:54 -05:00
James Long 14aaf91e65 fix(tui): mark failed groups with a plain ✗ (#51175) 2026-09-24 13:52:14 -04:00
James Long c832432d89 refactor(tui): drop unused yellow alias from opencode theme (#51177) 2026-09-24 13:17:21 -04:00
opencode-agent[bot]andjlongster 1d431a80df fix(cli): reuse core declarations during typecheck (#51165)
Co-authored-by: jlongster <jlongster@users.noreply.github.com>
2026-09-24 13:16:46 -04:00
105 changed files with 2411 additions and 1005 deletions
+3 -2
View File
@@ -10,7 +10,7 @@
## Conventions
Per-type constructors live on the type, not as top-level re-exports. Use `Message.system(...)`, `Message.user(...)`, `Message.assistant(...)`, `Message.tool(...)`, `Message.media(...)`, `LanguageModel.make(...)`, `ToolDefinition.make(...)`, `ToolCallPart.make(...)`, `ToolResultPart.make(...)`, `ToolChoice.make(...)`, `ToolChoice.named(...)`, `SystemPart.make(...)`, and `GenerationOptions.make(...)` directly. The top-level `LLM` namespace is reserved for request-shaped call APIs: `LLM.request`, `LLM.generate`, `LLM.stream`, and `LLM.generateObject`. Use `LLMRequest.update(...)` when deriving canonical request data; do not add a duplicate `LLM.updateRequest(...)` path. Two ways to construct the same thing is one too many.
Per-type constructors live on the type, not as top-level re-exports. Use `Message.system(...)`, `Message.user(...)`, `Message.assistant(...)`, `Message.tool(...)`, `Message.media(...)`, `LanguageModel.make(...)`, `ToolDefinition.make(...)`, `ToolCallPart.make(...)`, `ToolResultPart.make(...)`, `ToolChoice.make(...)`, `ToolChoice.named(...)`, `SystemPart.make(...)`, and `GenerationOptions.make(...)` directly. The top-level `LLM` namespace is reserved for request-shaped call APIs: `LLM.request`, `LLM.generate`, `LLM.stream`, and `LLM.generateObject`. `LLM.generate`/`LLM.stream` and Promise `ai.llm.generate`/`ai.llm.stream` accept ergonomic input or an `LLMRequest`; both paths use the same canonical request. Core still builds, logs, replays, and updates that durable `LLMRequest` boundary. Use `LLMRequest.update(...)` when deriving canonical request data; do not add a duplicate `LLM.updateRequest(...)` path.
Modality namespaces mirror `LLM` exactly: `Image.request`, `Image.generate`, `Image.stream` (later `Video`, `Speech`, `Transcription`). Common request fields (`images`, `mask`, `n`, `size`, `aspectRatio`, `seed`, `format`) lower natively or fail with a typed `AIError`; provider-native controls always live under `providerOptions`, never under a modality-specific `options` key.
@@ -102,7 +102,7 @@ Media does not fit the SSE-frames-to-event-state-machine LLM route. `MediaRoute.
`MediaProtocol.stream` is the incremental kind every speech route uses, with the same discipline as LLM protocols. `MediaRoute.stream` submits the caller's request as `MediaProtocol.Addressed<Request>` (`{ ...request, mode }`, `mode: "generate" | "stream"`), so one provider stays one protocol: `body.from`, the endpoint path, and `frames` read `request.mode` to pick the body, path, and framing. `frames(bytes, context)` returns frames — `Framing.sse`, `Framing.lines`, `Framing.document` (a single-document response shaped like a streamed record), or the raw `bytes` for chunked audio. `initial()` is fresh per-response parser state; `step` folds each frame into it and emits modality events; `finish(state, context)` runs once after the last frame with the request, body, and observed `http` (header-only usage lives there) and emits exactly one terminal event or fails with `route.incomplete()`. Keep parser state to real accumulators and derive anything the request or body determines in `finish`. `generate` runs the same stream and folds it with the modality's `collect`. Request-derived URL parameters go on the body's `query` (array values repeat the parameter), applied before route and caller `http.query`. Decode frames with `route.decodeFrame` and raise stream-time failures with `route.frameError` (the frame stays on `reason.body`); protocols never thread HTTP context, because the route fills `reason.http` on stream errors that lack it. Speech protocols share `protocols/utils/speech-stream.ts` for deltas, timestamps, voice ids, PCM and container descriptions, and the terminal asset.
Transcription uses all three kinds (OpenAI and Gemini stream, Deepgram is inline, AssemblyAI is queued): every route carries its `kind` and `TranscriptionClient` dispatches on it. `ImageRoute` is the same union; both clients dispatch through `MediaRoute.dispatch` and models compose through `composeAnyRoute`, and fal queue protocols come from `protocols/utils/fal-queue.ts`. Bodies are `json`, `multipart`, or `binary` (a raw upload), and a queued protocol that must upload media before submitting implements `start.prepare` (`MediaProtocol.Prepare`; AssemblyAI `/v2/upload`).
Every modality route is the inline | stream | queued union (transcription uses all three: OpenAI and Gemini stream, Deepgram is inline, AssemblyAI is queued), every client is `MediaClient.make(Service, { modality, responseEvents })` (`src/media-client.ts`), which dispatches on the route's `kind`, and every model composes through `composeRoute`. fal queue protocols come from `protocols/utils/fal-queue.ts`, bodies are `json`, `multipart`, or `binary` (a raw upload), and a queued protocol that must upload media before submitting implements `start.prepare` (`MediaProtocol.Prepare`; AssemblyAI `/v2/upload`).
### URL Construction
@@ -275,6 +275,7 @@ Use this order for every protocol module:
### Rules
- Keep protocol files focused on the protocol. Move provider-specific projection, signing, media normalization, or other bulky transformations into `src/protocols/utils/*`.
- Send `tool.inputSchema` as given. `prepareRequest` applies the tool schema rules (`ToolSchemaProjection.tools`) once per request, including tools in namespaces. A protocol whose API needs a model family's rules for every model declares `sanitizer` instead of transforming schemas itself.
- Use `Effect.fn("Provider.fromRequest")` for request body construction entrypoints. Use `Effect.fn(...)` for event handlers that yield effects; keep purely synchronous handlers as plain functions returning a `StepResult` that the dispatcher lifts via `Effect.succeed(...)`.
- Parser state owns terminal information. The state machine records finish reason, usage, and pending tool calls; emit one terminal `finish` event (or `provider-error`) for each completed response. If a provider splits reason and usage across events, merge them in parser state before flushing.
- Emit exactly one terminal `finish` event for a completed response, normally after a matching `step-finish`. Use `stream.terminal` to stop reading when the provider has a completion sentinel; use `stream.onHalt` when the final event must be flushed after the framed stream ends.
+12 -12
View File
@@ -9,15 +9,13 @@ import { OpenAI } from "@opencode/ai/providers"
const openai = OpenAI.configure({ apiKey: process.env.OPENAI_API_KEY })
const request = LLM.request({
model: openai.responses("gpt-4o-mini"), // `.chat(...)` selects the Chat Completions API instead
system: "You are concise.",
prompt: "Say hello in one short sentence.",
generation: { maxTokens: 40 },
})
const program = Effect.gen(function* () {
const response = yield* LLM.generate(request)
const response = yield* LLM.generate({
model: openai.responses("gpt-4o-mini"), // `.chat(...)` selects the Chat Completions API instead
system: "You are concise.",
prompt: "Say hello in one short sentence.",
generation: { maxTokens: 40 },
})
console.log(response.text)
})
@@ -25,7 +23,8 @@ const program = Effect.gen(function* () {
await Effect.runPromise(program.pipe(Effect.provide(AIClient.layer)))
```
Run `LLM.stream(request)` instead of `generate` when you want incremental `LLMEvent`s. The event stream is provider-neutral — same shape across OpenAI Chat, OpenAI Responses,
Run `LLM.stream(...)` instead of `generate` when you want incremental `LLMEvent`s. Both accept input or a prebuilt
`LLM.request(...)`. The event stream is provider-neutral — same shape across OpenAI Chat, OpenAI Responses,
Anthropic Messages, Gemini, Bedrock Converse, and any OpenAI-compatible deployment.
The same configured facade names image, video, speech, and transcription models. `Image.generate` resolves the
@@ -72,10 +71,11 @@ helpers; `ai.file` and `ai.write` load `node:fs/promises` on first use, so no Ef
import { AI } from "@opencode/ai/promise"
const ai = AI.make()
const text = await ai.llm.generate({ model: openai.responses("gpt-4o-mini"), prompt: "Say hello." })
const input = { model: openai.responses("gpt-4o-mini"), prompt: "Say hello." }
const text = await ai.llm.generate(input)
const generated = await ai.image.generate({ model: openai.image("gpt-image-2"), prompt: "A lighthouse" })
await ai.write(generated.image, "./lighthouse.png") // also ai.file(path), ai.bytes(asset), ai.base64(asset), ai.materialize(asset)
for await (const event of ai.llm.stream({ model: openai.responses("gpt-4o-mini"), prompt: "Stream hello." })) {
for await (const event of ai.llm.stream(ai.llm.request(input))) {
// LLMEvent
}
await ai.dispose()
@@ -936,7 +936,7 @@ const transcript = await generation.await({ poll: { interval: 3_000 } })
## Public API
- **`LLM.request({...})`** — build a provider-neutral `LLMRequest`. Accepts ergonomic inputs (`system: string`, `prompt: string`) that normalize into the canonical Schema classes.
- **`LLM.generate` / `LLM.stream`** — re-exported from `LLMClient` for one-import use.
- **`LLM.generate` / `LLM.stream`** — run direct input or an `LLMRequest` through `LLMClient` for one-import use.
- **`Message.user(...)` / `Message.assistant(...)` / `Message.tool(...)`** — message constructors from the canonical schema model.
- **`LanguageModel.make(...)` / `ToolCallPart.make(...)` / `ToolResultPart.make(...)` / `ToolDefinition.make(...)`** — model and tool-related constructors from the canonical schema model.
- **`LLMEvent.is.*`** — typed guards (`is.textDelta`, `is.toolCall`, `is.finish`, …) for filtering streams.
+5 -4
View File
@@ -140,7 +140,7 @@ portability matrix.
Editing is not a separate function; `images`/`mask` on the request select the edit path in the route (OpenAI `/images/edits`, Gemini multimodal parts, xAI `/images/edits`). Routes that cannot honor `mask` fail with `Unsupported`.
`ImageRoute` is the inline | stream | queued union, dispatched on `route.kind`. `Image.stream` on a streaming route emits `image-partial` previews before each `image`; on a queued route it emits `generation-queued` / `generation-progress` observations, then the result's `image` and `finish` events.
`ImageRoute` is the inline | stream | queued union, dispatched on `route.kind`, like every modality route. `Image.stream` on a streaming route emits `image-partial` previews before each `image`; on a queued route it emits `generation-queued` / `generation-progress` observations, then the result's `image` and `finish` events.
#### Video
@@ -215,7 +215,7 @@ const request = Speech.request({
})
const response = yield* Speech.generate(request) // SpeechResponse: audio: Media.Asset, timestamps?, usage?, providerMetadata?
yield* Speech.stream(request) // Stream<SpeechEvent>: audio-delta { chunk } | timestamps { items } | finish { audio, usage? }
yield* Speech.stream(request) // Stream<SpeechEvent>: generation-queued | generation-progress | audio-delta { chunk } | timestamps { items } | finish { audio, usage? }
```
Execution is `MediaProtocol.stream` for every provider: one request whose body is framed and folded by a `step`
@@ -389,7 +389,8 @@ for await (const event of generation.events({ poll: { interval: 10_000 } })) {
const video = await generation.await({ poll: { interval: 10_000 }, signal })
const resumed = await ai.video.resume(model, JSON.parse(saved)) // persist provider + model ID with the token
const text = await ai.llm.generate({ model, prompt })
const request = ai.llm.request({ model, prompt })
const text = await ai.llm.generate(request)
for await (const event of ai.llm.stream(request)) { … }
await ai.dispose()
@@ -419,7 +420,7 @@ Existing facades gain per-modality selectors; the modality routes each facade pr
New facades follow the existing one-file-per-provider rule. The facade selector is the public path for media models; modality-specific package entrypoints (for example `@opencode/ai/providers/openai/images`) are deferred until Core has a modality-aware model resolver.
`ImageModel<Options>` gives typed `providerOptions` per model; `VideoModel`, `SpeechModel`, and `TranscriptionModel` follow the same generic. They share an internal `MediaModel` base class (ids, route, `http` overlays) that is not part of the public exports; `Generation` and the promise client work with the concrete modality models.
`ImageModel<Options>` gives typed `providerOptions` per model; `VideoModel`, `SpeechModel`, and `TranscriptionModel` follow the same generic. As with `LanguageModel`, the route type does not carry `Options`, so `ImageModel<OpenAIImageOptions>` is an `ImageModel` and client methods take plain `ImageRequestFor`. They share an internal `MediaModel` base class (ids, route, `http` overlays) that is not part of the public exports; `Generation` and the promise client work with the concrete modality models.
### Routes and protocols
+16 -85
View File
@@ -1,99 +1,30 @@
import { Context, Effect, Layer, Stream } from "effect"
import type { AwaitOptions, Generation } from "./generation.js"
import { RequestExecutor } from "./route/executor.js"
import { MediaRoute } from "./route/media.js"
import type { AIError } from "./schema/index.js"
import { Context } from "effect"
import { MediaClient } from "./media-client.js"
import {
responseEvents,
ImageOutputEvent,
ImageFinishEvent,
type ImageEvent,
type ImageModel,
type ImageOptions,
type ImageRequestFor,
type ImageResponse,
} from "./image.js"
export interface Interface {
readonly generate: <Options extends ImageOptions>(
request: ImageRequestFor<Options>,
options?: AwaitOptions,
) => Effect.Effect<ImageResponse, AIError>
readonly stream: <Options extends ImageOptions>(
request: ImageRequestFor<Options>,
options?: AwaitOptions,
) => Stream.Stream<ImageEvent, AIError>
readonly start: <Options extends ImageOptions>(
request: ImageRequestFor<Options>,
) => Effect.Effect<Generation<ImageResponse>, AIError>
readonly resume: <Options extends ImageOptions>(
model: ImageModel<Options>,
token: unknown,
) => Effect.Effect<Generation<ImageResponse>, AIError>
}
export type Interface = MediaClient.Interface<ImageRequestFor, ImageEvent, ImageResponse>
export class ImageClientService extends Context.Service<ImageClientService, Interface>()("@opencode/ImageClient") {}
export const Service = ImageClientService
export type Service = ImageClientService
export const generate = <Options extends ImageOptions>(
request: ImageRequestFor<Options>,
options?: AwaitOptions,
): Effect.Effect<ImageResponse, AIError, Service> =>
Effect.gen(function* () {
const client = yield* Service
return yield* client.generate(request, options)
})
export const stream = <Options extends ImageOptions>(
request: ImageRequestFor<Options>,
options?: AwaitOptions,
): Stream.Stream<ImageEvent, AIError, Service> =>
Stream.unwrap(
Effect.gen(function* () {
const client = yield* Service
return client.stream(request, options)
}),
)
export const start = <Options extends ImageOptions>(
request: ImageRequestFor<Options>,
): Effect.Effect<Generation<ImageResponse>, AIError, Service> =>
Effect.gen(function* () {
const client = yield* Service
return yield* client.start(request)
})
export const resume = <Options extends ImageOptions>(
model: ImageModel<Options>,
token: unknown,
): Effect.Effect<Generation<ImageResponse>, AIError, Service> =>
Effect.gen(function* () {
const client = yield* Service
return yield* client.resume(model, token)
})
export const layer: Layer.Layer<Service, never, RequestExecutor.Service> = Layer.effect(
Service,
Effect.gen(function* () {
const executor = yield* RequestExecutor.Service
const dispatch = MediaRoute.dispatch<ImageEvent, ImageResponse>({
modality: "image",
execute: executor.execute,
responseEvents,
})
return Service.of({
start: (request) => dispatch.start(request.model.route, request),
resume: (model, token) => dispatch.resume(model.route, model, token),
generate: (request, options) => dispatch.generate(request.model.route, request, options),
stream: (request, options) => dispatch.stream(request.model.route, request, options),
})
}),
)
export const ImageClient = {
Service,
layer,
generate,
stream,
start,
resume,
...MediaClient.make(Service, {
modality: "image",
responseEvents: (response: ImageResponse) => [
...response.images.map((image, index) => ImageOutputEvent.make({ index, image })),
ImageFinishEvent.make({
usage: response.usage,
notices: response.notices,
providerMetadata: response.providerMetadata,
}),
],
}),
} as const
+14 -65
View File
@@ -1,9 +1,8 @@
import { Effect, Schema, Stream } from "effect"
import { Generation, ProgressEvent, QueuedEvent, type AwaitOptions } from "./generation.js"
import { Media } from "./media.js"
import { MediaModel, composeAnyRoute, tryRequest } from "./media-model.js"
import { MediaModel, composeRoute, tryRequest } from "./media-model.js"
import { MediaRoute } from "./route/media.js"
import type { MediaProtocol } from "./route/media-protocol.js"
import { AIError, HttpOptions, MediaUsage, ProviderMetadata, type OpenString } from "./schema/index.js"
import { ImageClient, Service } from "./image-client.js"
@@ -11,75 +10,39 @@ import { ImageClient, Service } from "./image-client.js"
// Model
// ---------------------------------------------------------------------------
export type ImageOptions = Record<string, unknown>
export type ImageOptions = MediaModel.Options
export type ImageRoute<Options extends ImageOptions = ImageOptions> = MediaRoute.AnyRoute<
ImageRequestFor<Options>,
ImageEvent,
ImageResponse
>
export type ImageRoute = MediaRoute.AnyRoute<ImageRequestFor, ImageEvent, ImageResponse>
export class ImageModel<Options extends ImageOptions = ImageOptions> extends MediaModel<ImageRoute<Options>, Options> {
export class ImageModel<Options extends ImageOptions = ImageOptions> extends MediaModel<ImageRoute, Options> {
declare protected readonly _ImageModel: void
static make<Options extends ImageOptions = ImageOptions>(input: MediaModel.Input<ImageRoute<Options>>) {
return new ImageModel<Options>(input)
}
/** The number of type arguments selects the kind: `<Options>`, `<Options, Frame, State>`, or `<Options, Token>`. */
static fromRoute<Options extends ImageOptions>(
route: ImageModel.InlineRouteInput<Options>,
route: MediaModel.InlineRouteInput<ImageRequestFor<Options>, ImageResponse>,
input: MediaRoute.ModelInput,
): ImageModel<Options>
static fromRoute<Options extends ImageOptions, Frame, State>(
route: ImageModel.StreamRouteInput<Options, Frame, State>,
route: MediaModel.StreamRouteInput<ImageRequestFor<Options>, ImageEvent, Frame, State>,
input: MediaRoute.ModelInput,
): ImageModel<Options>
static fromRoute<Options extends ImageOptions, Token>(
route: ImageModel.QueuedRouteInput<Options, Token>,
route: MediaModel.QueuedRouteInput<ImageRequestFor<Options>, ImageResponse, Token>,
input: MediaRoute.ModelInput,
): ImageModel<Options>
static fromRoute<Options extends ImageOptions, Frame, State, Token>(
route: ImageModel.RouteInput<Options, Frame, State, Token>,
route: MediaModel.AnyRouteInput<ImageRequestFor<Options>, ImageEvent, ImageResponse, Frame, State, Token>,
input: MediaRoute.ModelInput,
) {
return new ImageModel<Options>({
id: input.id,
provider: route.protocol.provider,
http: input.http,
route: composeAnyRoute(route, input, collectResponse),
route: composeRoute(route, input, collectResponse) as ImageRoute,
})
}
}
export namespace ImageModel {
export type InlineRouteInput<Options extends ImageOptions = ImageOptions> = MediaModel.RouteInput<
ImageRequestFor<Options>,
MediaProtocol.Inline<ImageRequestFor<Options>, ImageResponse>
>
export type StreamRouteInput<
Options extends ImageOptions = ImageOptions,
Frame = unknown,
State = unknown,
> = MediaModel.RouteInput<
MediaProtocol.Addressed<ImageRequestFor<Options>>,
MediaProtocol.Streamed<ImageRequestFor<Options>, ImageEvent, Frame, State>
>
export type QueuedRouteInput<Options extends ImageOptions = ImageOptions, Token = unknown> = MediaModel.RouteInput<
ImageRequestFor<Options>,
MediaProtocol.Queued<ImageRequestFor<Options>, ImageResponse, Token>
>
export type RouteInput<
Options extends ImageOptions = ImageOptions,
Frame = unknown,
State = unknown,
Token = unknown,
> = MediaModel.AnyRouteInput<ImageRequestFor<Options>, ImageEvent, ImageResponse, Frame, State, Token>
}
export const ImageModelSchema = Schema.declare((value): value is ImageModel => value instanceof ImageModel, {
expected: "Image.Model",
})
@@ -190,15 +153,6 @@ export const ImageEvent = Object.assign(imageEventTagged, {
})
export type ImageEvent = Schema.Schema.Type<typeof imageEventTagged>
export const responseEvents = (response: ImageResponse): ReadonlyArray<ImageEvent> => [
...response.images.map((image, index) => ImageOutputEvent.make({ index, image })),
ImageFinishEvent.make({
usage: response.usage,
notices: response.notices,
providerMetadata: response.providerMetadata,
}),
]
const collectResponse = (events: ReadonlyArray<ImageEvent>): Effect.Effect<ImageResponse> => {
const finish = events.find(ImageEvent.is.finish)
// Every image protocol's `finish` emits the terminal event or fails, so a completed stream always has one.
@@ -232,36 +186,31 @@ export function request(input: ImageRequest | ImageRequestInput) {
const requestEffect = (input: ImageRequest | ImageRequestInput) => tryRequest(() => request(input))
export function generate<const Model extends ImageModel>(
input: ImageRequestInput<Model>,
input: ImageRequest | ImageRequestInput<Model>,
options?: AwaitOptions,
): Effect.Effect<ImageResponse, AIError, Service>
export function generate(input: ImageRequest, options?: AwaitOptions): Effect.Effect<ImageResponse, AIError, Service>
export function generate(input: ImageRequest | ImageRequestInput, options?: AwaitOptions) {
return requestEffect(input).pipe(Effect.flatMap((request) => ImageClient.generate(request, options)))
}
export function stream<const Model extends ImageModel>(
input: ImageRequestInput<Model>,
input: ImageRequest | ImageRequestInput<Model>,
options?: AwaitOptions,
): Stream.Stream<ImageEvent, AIError, Service>
export function stream(input: ImageRequest, options?: AwaitOptions): Stream.Stream<ImageEvent, AIError, Service>
export function stream(input: ImageRequest | ImageRequestInput, options?: AwaitOptions) {
return Stream.unwrap(requestEffect(input).pipe(Effect.map((request) => ImageClient.stream(request, options))))
}
/** Inline and streaming routes fail with `UnsupportedOperation`. */
export function start<const Model extends ImageModel>(
input: ImageRequestInput<Model>,
input: ImageRequest | ImageRequestInput<Model>,
): Effect.Effect<Generation<ImageResponse>, AIError, Service>
export function start(input: ImageRequest): Effect.Effect<Generation<ImageResponse>, AIError, Service>
export function start(input: ImageRequest | ImageRequestInput) {
return requestEffect(input).pipe(Effect.flatMap((request) => ImageClient.start(request)))
}
export const resume = <Options extends ImageOptions>(
model: ImageModel<Options>,
token: unknown,
): Effect.Effect<Generation<ImageResponse>, AIError, Service> => ImageClient.resume(model, token)
export const resume = (model: ImageModel, token: unknown): Effect.Effect<Generation<ImageResponse>, AIError, Service> =>
ImageClient.resume(model, token)
export const Image = {
request,
+22 -4
View File
@@ -1,5 +1,6 @@
import { Effect, JsonSchema, Schema } from "effect"
import { LLMClient, Service } from "./route/client.js"
import { Effect, JsonSchema, Schema, Stream } from "effect"
import { tryRequest } from "./media-model.js"
import { LLMClient, Service, type StreamOptions } from "./route/client.js"
import {
GenerationOptions,
HttpOptions,
@@ -35,9 +36,26 @@ export type RequestInput<SelectedLanguageModel extends LanguageModel = LanguageM
readonly http?: HttpOptions.Input
}
export const generate = LLMClient.generate
export function generate<const Model extends LanguageModel>(
input: RequestInput<Model>,
options?: StreamOptions,
): Effect.Effect<LLMResponse, AIError, Service>
export function generate(input: LLMRequest, options?: StreamOptions): Effect.Effect<LLMResponse, AIError, Service>
export function generate(input: RequestInput | LLMRequest, options?: StreamOptions) {
return requestEffect(input).pipe(Effect.flatMap((request) => LLMClient.generate(request, options)))
}
export const stream = LLMClient.stream
export function stream<const Model extends LanguageModel>(
input: RequestInput<Model>,
options?: StreamOptions,
): Stream.Stream<LLMEvent, AIError, Service>
export function stream(input: LLMRequest, options?: StreamOptions): Stream.Stream<LLMEvent, AIError, Service>
export function stream(input: RequestInput | LLMRequest, options?: StreamOptions) {
return Stream.unwrap(requestEffect(input).pipe(Effect.map((request) => LLMClient.stream(request, options))))
}
const requestEffect = (input: RequestInput | LLMRequest) =>
input instanceof LLMRequest ? Effect.succeed(input) : tryRequest(() => request(input))
export const request = <const SelectedLanguageModel extends LanguageModel>(
input: RequestInput<SelectedLanguageModel>,
+77
View File
@@ -0,0 +1,77 @@
import { type Context, Effect, Layer, Stream } from "effect"
import { resultEvents, type AwaitOptions, type Generation, type Observation } from "./generation.js"
import { RequestExecutor } from "./route/executor.js"
import type { MediaRoute } from "./route/media.js"
import { AIError, UnsupportedOperationError } from "./schema/index.js"
/** A media request whose model carries the route that executes it. */
export interface RoutedRequest<Self extends MediaRoute.MediaRequest, Event, Response> extends MediaRoute.MediaRequest {
readonly model: MediaRoute.MediaRequest["model"] & { readonly route: MediaRoute.AnyRoute<Self, Event, Response> }
}
/** `start` and `resume` fail with `UnsupportedOperation` on inline and stream routes. */
export interface Interface<Req extends RoutedRequest<Req, Event, Response>, Event, Response> {
readonly generate: (request: Req, options?: AwaitOptions) => Effect.Effect<Response, AIError>
readonly stream: (request: Req, options?: AwaitOptions) => Stream.Stream<Event | Observation, AIError>
readonly start: (request: Req) => Effect.Effect<Generation<Response>, AIError>
readonly resume: (model: Req["model"], token: unknown) => Effect.Effect<Generation<Response>, AIError>
}
/** One modality's layer and service accessors, dispatching each request on its route's `kind`. */
export const make = <Self, Req extends RoutedRequest<Req, Event, Response>, Event, Response>(
service: Context.Service<Self, Interface<Req, Event, Response>>,
input: {
readonly modality: string
/** A completed response expanded into the streaming event shape. */
readonly responseEvents: (response: Response) => ReadonlyArray<Event>
},
) => ({
layer: Layer.effect(
service,
Effect.gen(function* () {
const executor = yield* RequestExecutor.Service
const notQueued = (route: MediaRoute.AnyRoute<Req, Event, Response>, operation: string) =>
new AIError({
reason: new UnsupportedOperationError({
operation: `${input.modality}.${operation}`,
provider: route.provider,
route: route.id,
message: `${route.provider}/${route.id} is not a queued route; use generate or stream`,
}),
})
const start = (request: Req) => {
const route = request.model.route
if (route.kind !== "queued") return Effect.fail(notQueued(route, "start"))
return route.start(request, executor.execute)
}
return service.of({
start,
resume: (model, token) => {
if (model.route.kind !== "queued") return Effect.fail(notQueued(model.route, "resume"))
return model.route.resume(model, token, executor.execute)
},
generate: (request, options) => {
const route = request.model.route
if (route.kind !== "queued") return route.generate(request, executor.execute)
return start(request).pipe(Effect.flatMap((generation) => generation.await(options)))
},
stream: (request, options) => {
const route = request.model.route
if (route.kind === "stream") return route.stream(request, executor.execute)
if (route.kind === "queued")
return Stream.unwrap(
start(request).pipe(Effect.map((generation) => resultEvents(generation, input.responseEvents, options))),
)
return Stream.fromIterableEffect(Effect.map(route.generate(request, executor.execute), input.responseEvents))
},
})
}),
),
generate: (request: Req, options?: AwaitOptions) => service.use((client) => client.generate(request, options)),
stream: (request: Req, options?: AwaitOptions) =>
Stream.unwrap(service.useSync((client) => client.stream(request, options))),
start: (request: Req) => service.use((client) => client.start(request)),
resume: (model: Req["model"], token: unknown) => service.use((client) => client.resume(model, token)),
})
export * as MediaClient from "./media-client.js"
+42 -30
View File
@@ -6,11 +6,13 @@ import { AIError, HttpOptions, InvalidRequestError, ModelID, ProviderID } from "
/**
* What every media model carries: ids, the configured route, and deployment `http` overlays. Modality classes
* (`ImageModel`, `VideoModel`, `SpeechModel`) extend it with their route type and a nominal marker so one cannot stand
* in for the other in requests.
* (`ImageModel`, `VideoModel`, `SpeechModel`, `TranscriptionModel`) extend it with their route type and a nominal
* marker so one cannot stand in for the other in requests.
*/
export class MediaModel<Route, Options> {
declare protected readonly _Options: (options: Options) => Options
// As with `LanguageModel`, the route type is erased over `Options`; `fromRoute` and the constructor trust that the
// route accepts every request this model's `Options` admit.
declare protected readonly _Options: Options
readonly id: ModelID
readonly provider: ProviderID
readonly route: Route
@@ -25,6 +27,8 @@ export class MediaModel<Route, Options> {
}
export namespace MediaModel {
export type Options = Record<string, unknown>
export interface Input<Route> {
readonly id: string | ModelID
readonly provider: string | ProviderID
@@ -41,48 +45,56 @@ export namespace MediaModel {
readonly headers?: Record<string, string>
}
export type InlineRouteInput<Request extends MediaRoute.MediaRequest, Response> = RouteInput<
Request,
MediaProtocol.Inline<Request, Response>
>
export type StreamRouteInput<Request extends MediaRoute.MediaRequest, Event, Frame, State> = RouteInput<
MediaProtocol.Addressed<Request>,
MediaProtocol.Streamed<Request, Event, Frame, State>
>
export type QueuedRouteInput<Request extends MediaRoute.MediaRequest, Response, Token> = RouteInput<
Request,
MediaProtocol.Queued<Request, Response, Token>
>
export type AnyRouteInput<Request extends MediaRoute.MediaRequest, Event, Response, Frame, State, Token> =
| RouteInput<Request, MediaProtocol.Inline<Request, Response>>
| RouteInput<MediaProtocol.Addressed<Request>, MediaProtocol.Streamed<Request, Event, Frame, State>>
| RouteInput<Request, MediaProtocol.Queued<Request, Response, Token>>
| InlineRouteInput<Request, Response>
| StreamRouteInput<Request, Event, Frame, State>
| QueuedRouteInput<Request, Response, Token>
}
/** Compose a protocol route input with one deployment through `MediaRoute.inline`, `queued`, or `stream`. */
export const composeRoute = <Request extends MediaRoute.MediaRequest, Protocol, Route>(
compose: (input: MediaRoute.Composition<Request> & { readonly protocol: Protocol }) => Route,
route: MediaModel.RouteInput<Request, Protocol>,
input: MediaRoute.ModelInput,
): Route =>
compose({
protocol: route.protocol,
endpoint: Endpoint.path(route.path, { baseURL: input.baseURL ?? route.baseURL }),
auth: input.auth,
headers:
route.headers === undefined && input.headers === undefined ? undefined : { ...route.headers, ...input.headers },
})
export const composeAnyRoute = <Request extends MediaRoute.MediaRequest, Event, Response, Frame, State, Token>(
export const composeRoute = <Request extends MediaRoute.MediaRequest, Event, Response, Frame, State, Token>(
route: MediaModel.AnyRouteInput<Request, Event, Response, Frame, State, Token>,
input: MediaRoute.ModelInput,
collect: (events: ReadonlyArray<Event>) => Effect.Effect<Response, AIError>,
): MediaRoute.AnyRoute<Request, Event, Response> => {
if (isStreamInput(route))
return composeRoute((composition) => MediaRoute.stream({ ...composition, collect }), route, input)
if (isQueuedInput(route)) return composeRoute(MediaRoute.queued, route, input)
return composeRoute(MediaRoute.inline, route, input)
if (isStreamInput(route)) return MediaRoute.stream({ ...composition(route, input), collect })
if (isQueuedInput(route)) return MediaRoute.queued(composition(route, input))
return MediaRoute.inline(composition(route, input))
}
const composition = <Request extends MediaRoute.MediaRequest, Protocol>(
route: MediaModel.RouteInput<Request, Protocol>,
input: MediaRoute.ModelInput,
): MediaRoute.Composition<Request> & { readonly protocol: Protocol } => ({
protocol: route.protocol,
endpoint: Endpoint.path(route.path, { baseURL: input.baseURL ?? route.baseURL }),
auth: input.auth,
headers:
route.headers === undefined && input.headers === undefined ? undefined : { ...route.headers, ...input.headers },
})
const isStreamInput = <Request extends MediaRoute.MediaRequest, Event, Response, Frame, State, Token>(
route: MediaModel.AnyRouteInput<Request, Event, Response, Frame, State, Token>,
): route is MediaModel.RouteInput<
MediaProtocol.Addressed<Request>,
MediaProtocol.Streamed<Request, Event, Frame, State>
> => route.protocol.kind === "stream"
): route is MediaModel.StreamRouteInput<Request, Event, Frame, State> => route.protocol.kind === "stream"
const isQueuedInput = <Request extends MediaRoute.MediaRequest, Event, Response, Frame, State, Token>(
route: MediaModel.AnyRouteInput<Request, Event, Response, Frame, State, Token>,
): route is MediaModel.RouteInput<Request, MediaProtocol.Queued<Request, Response, Token>> =>
route.protocol.kind === "queued"
): route is MediaModel.QueuedRouteInput<Request, Response, Token> => route.protocol.kind === "queued"
/** Lift a synchronous Schema-class constructor into a typed `InvalidRequest` failure. */
export const tryRequest = <A>(make: () => A): Effect.Effect<A, AIError> =>
+20 -33
View File
@@ -1,23 +1,22 @@
import { Effect, Layer, ManagedRuntime, Stream } from "effect"
import { AIClient } from "./ai-client.js"
import type { AwaitOptions, Event, Generation, Snapshot } from "./generation.js"
import { Image, ImageModel, ImageRequest, type ImageOptions, type ImageRequestInput } from "./image.js"
import { Image, type ImageModel, type ImageRequest, type ImageRequestInput } from "./image.js"
import { LLM } from "./index.js"
import { Media } from "./media.js"
import { tryRequest } from "./media-model.js"
import { RequestExecutor } from "./route/executor.js"
import { AIError, InvalidRequestError, LanguageModel, LLMRequest } from "./schema/index.js"
import type { RequestInput } from "./llm.js"
import { Speech, SpeechModel, SpeechRequest, type SpeechRequestInput } from "./speech.js"
import { Speech, type SpeechModel, type SpeechRequest, type SpeechRequestInput } from "./speech.js"
import {
Transcription,
TranscriptionModel,
TranscriptionRequest,
type TranscriptionOptions,
type TranscriptionModel,
type TranscriptionRequest,
type TranscriptionRequestInput,
} from "./transcription.js"
import { fileMediaType } from "./utils/media-type.js"
import { Video, VideoModel, VideoRequest, type VideoOptions, type VideoRequestInput } from "./video.js"
import { Video, type VideoModel, type VideoRequest, type VideoRequestInput } from "./video.js"
/**
* Promise-first entrypoint for scripts and non-Effect callers. One `ManagedRuntime` hosts the LLM, image, video, speech,
@@ -93,17 +92,8 @@ export const make = (options: Options = {}) => {
cancel: (options) => run(generation.cancel(), options),
})
// The typed `generate`/`stream` overloads take a concrete input or a request, not the union; normalize once here.
const llmRequest = (input: RequestInput | LLMRequest) =>
input instanceof LLMRequest ? Effect.succeed(input) : tryRequest(() => LLM.request(input))
const imageRequest = (input: ImageRequestInput | ImageRequest) =>
input instanceof ImageRequest ? input : Image.request(input)
const videoRequest = (input: VideoRequestInput | VideoRequest) =>
input instanceof VideoRequest ? input : Video.request(input)
const speechRequest = (input: SpeechRequestInput | SpeechRequest) =>
input instanceof SpeechRequest ? input : Speech.request(input)
const transcriptionRequest = (input: TranscriptionRequestInput | TranscriptionRequest) =>
input instanceof TranscriptionRequest ? input : Transcription.request(input)
return {
run,
@@ -155,61 +145,58 @@ export const make = (options: Options = {}) => {
generate: <const Model extends ImageModel>(
input: ImageRequestInput<Model> | ImageRequest,
options?: AwaitOptions & RunOptions,
) => run(Image.generate(imageRequest(input), { poll: options?.poll }), options),
) => run(Image.generate(input, { poll: options?.poll }), options),
stream: <const Model extends ImageModel>(
input: ImageRequestInput<Model> | ImageRequest,
options?: AwaitOptions & RunOptions,
) => iterate(Image.stream(imageRequest(input), { poll: options?.poll }), options),
) => iterate(Image.stream(input, { poll: options?.poll }), options),
start: <const Model extends ImageModel>(input: ImageRequestInput<Model> | ImageRequest, options?: RunOptions) =>
run(Image.start(imageRequest(input)), options).then(handle),
resume: <Options extends ImageOptions>(model: ImageModel<Options>, token: unknown, options?: RunOptions) =>
run(Image.start(input), options).then(handle),
resume: (model: ImageModel, token: unknown, options?: RunOptions) =>
run(Image.resume(model, token), options).then(handle),
},
video: {
request: Video.request,
start: <const Model extends VideoModel>(input: VideoRequestInput<Model> | VideoRequest, options?: RunOptions) =>
run(Video.start(videoRequest(input)), options).then(handle),
run(Video.start(input), options).then(handle),
generate: <const Model extends VideoModel>(
input: VideoRequestInput<Model> | VideoRequest,
options?: AwaitOptions & RunOptions,
) => run(Video.generate(videoRequest(input), { poll: options?.poll }), options),
resume: <Options extends VideoOptions>(model: VideoModel<Options>, token: unknown, options?: RunOptions) =>
) => run(Video.generate(input, { poll: options?.poll }), options),
resume: (model: VideoModel, token: unknown, options?: RunOptions) =>
run(Video.resume(model, token), options).then(handle),
stream: <const Model extends VideoModel>(
input: VideoRequestInput<Model> | VideoRequest,
options?: AwaitOptions & RunOptions,
) => iterate(Video.stream(videoRequest(input), { poll: options?.poll }), options),
) => iterate(Video.stream(input, { poll: options?.poll }), options),
},
speech: {
request: Speech.request,
generate: <const Model extends SpeechModel>(
input: SpeechRequestInput<Model> | SpeechRequest,
options?: RunOptions,
) => run(Speech.generate(speechRequest(input)), options),
) => run(Speech.generate(input), options),
stream: <const Model extends SpeechModel>(
input: SpeechRequestInput<Model> | SpeechRequest,
options?: RunOptions,
) => iterate(Speech.stream(speechRequest(input)), options),
) => iterate(Speech.stream(input), options),
},
transcription: {
request: Transcription.request,
generate: <const Model extends TranscriptionModel>(
input: TranscriptionRequestInput<Model> | TranscriptionRequest,
options?: AwaitOptions & RunOptions,
) => run(Transcription.generate(transcriptionRequest(input), { poll: options?.poll }), options),
) => run(Transcription.generate(input, { poll: options?.poll }), options),
stream: <const Model extends TranscriptionModel>(
input: TranscriptionRequestInput<Model> | TranscriptionRequest,
options?: AwaitOptions & RunOptions,
) => iterate(Transcription.stream(transcriptionRequest(input), { poll: options?.poll }), options),
) => iterate(Transcription.stream(input, { poll: options?.poll }), options),
start: <const Model extends TranscriptionModel>(
input: TranscriptionRequestInput<Model> | TranscriptionRequest,
options?: RunOptions,
) => run(Transcription.start(transcriptionRequest(input)), options).then(handle),
resume: <Options extends TranscriptionOptions>(
model: TranscriptionModel<Options>,
token: unknown,
options?: RunOptions,
) => run(Transcription.resume(model, token), options).then(handle),
) => run(Transcription.start(input), options).then(handle),
resume: (model: TranscriptionModel, token: unknown, options?: RunOptions) =>
run(Transcription.resume(model, token), options).then(handle),
},
dispose: () => runtime.dispose(),
}
+5 -1
View File
@@ -70,7 +70,11 @@ export const protocol = Protocol.make({
return {
...(yield* OpenAIChat.protocol.body.from(req)),
enable_thinking: opts.enableThinking,
thinking_budget: opts.thinkingBudget,
// Alibaba also rejects an explicit budget that is not below `max_completion_tokens`.
thinking_budget:
opts.thinkingBudget === undefined
? undefined
: ProviderShared.fitThinkingBudget(opts.thinkingBudget, req.generation?.maxTokens),
preserve_thinking: opts.preserveThinking,
clear_thinking: opts.clearThinking,
thinking: opts.thinking,
@@ -26,18 +26,21 @@ export const protocol = Protocol.make({
from: Effect.fn("AlibabaMessages.fromRequest")(function* (req) {
const opts = yield* ProviderShared.validateWith(Schema.decodeUnknownEffect(Options))(req.providerOptions ?? {})
// Model Studio accepts enabled thinking without Anthropic's mandatory token budget.
const body = yield* AnthropicMessages.protocol.body.from(
LLMRequest.update(req, {
providerOptions: { ...req.providerOptions, thinking: undefined },
}),
)
const budget = opts.thinking?.budgetTokens ?? opts.thinking?.budget_tokens
return {
...(yield* AnthropicMessages.protocol.body.from(
LLMRequest.update(req, {
providerOptions: { ...req.providerOptions, thinking: undefined },
}),
)),
...body,
thinking:
opts.thinking === undefined
? undefined
: {
type: opts.thinking.type,
budget_tokens: opts.thinking.budgetTokens ?? opts.thinking.budget_tokens,
budget_tokens:
budget === undefined ? undefined : ProviderShared.fitThinkingBudget(budget, body.max_tokens),
},
}
}),
+16 -12
View File
@@ -18,7 +18,6 @@ import {
type CacheHint,
type FinishReasonDetails,
type FinishReason,
type JsonSchema,
type MediaPart,
type ProviderMetadata,
type ProviderOptions,
@@ -31,13 +30,13 @@ import { classifyProviderFailure } from "../provider-error.js"
import { effortUpdate, resolveEffortUpdates } from "../effort-updates.js"
import * as Cache from "./utils/cache.js"
import { Lifecycle } from "./utils/lifecycle.js"
import { ToolSchemaProjection } from "./utils/tool-schema.js"
import { ToolStream } from "./utils/tool-stream.js"
const ADAPTER = "anthropic-messages"
export const DEFAULT_BASE_URL = "https://api.anthropic.com/v1"
export const PATH = "/messages"
export const DEFAULT_MAX_TOKENS = 32_000
const MIN_THINKING_BUDGET = 1_024
const DEFAULT_EFFORT = "high"
const SSE_EVENTS = new Set([
@@ -524,10 +523,10 @@ const redactedDataFromMetadata = (metadata: ProviderMetadata | undefined, key: s
return typeof provider.redactedData === "string" ? provider.redactedData : undefined
}
const lowerTool = (breakpoints: Cache.Breakpoints, tool: ToolDefinition, inputSchema: JsonSchema): AnthropicTool => ({
const lowerTool = (breakpoints: Cache.Breakpoints, tool: ToolDefinition): AnthropicTool => ({
name: tool.name,
description: tool.description,
input_schema: inputSchema,
input_schema: tool.inputSchema,
cache_control: cacheControl(breakpoints, tool.cache),
})
@@ -1027,6 +1026,15 @@ const applyThinkingBindingDefault = (model: LLMRequest["model"], thinking: Anthr
}
}
// Anthropic also requires an explicit thinking budget below `max_tokens` and at or above its minimum.
const fitThinking = (thinking: AnthropicThinking | undefined, maxTokens: number) =>
thinking?.type === "enabled"
? {
...thinking,
budget_tokens: ProviderShared.fitThinkingBudget(thinking.budget_tokens, maxTokens, MIN_THINKING_BUDGET),
}
: thinking
const fromRequest = Effect.fn("AnthropicMessages.fromRequest")(function* (request: LLMRequest) {
const options = yield* decodeOptions(request.providerOptions ?? {})
const management = options.contextManagement
@@ -1039,12 +1047,7 @@ const fromRequest = Effect.fn("AnthropicMessages.fromRequest")(function* (reques
// over-mark we keep their tool hints and shed the message-tail ones first.
const breakpoints = Cache.newBreakpoints(ANTHROPIC_BREAKPOINT_CAP)
const flattened = ProviderShared.flattenToolRequest(updates.request)
const tools =
flattened.tools.length === 0
? undefined
: flattened.tools.map((tool) =>
lowerTool(breakpoints, tool, ToolSchemaProjection.modelCompatibility(tool.inputSchema, request.model)),
)
const tools = flattened.tools.length === 0 ? undefined : flattened.tools.map((tool) => lowerTool(breakpoints, tool))
// Anthropic rejects tool_choice when tools are absent; "none" is only meaningful with tools present.
const toolChoice = tools === undefined || !request.toolChoice ? undefined : yield* lowerToolChoice(request.toolChoice)
const systemParts = request.system.filter((part) => part.text.length > 0)
@@ -1064,6 +1067,7 @@ const fromRequest = Effect.fn("AnthropicMessages.fromRequest")(function* (reques
}
const output_config =
updates.effort === undefined && format === undefined ? undefined : { effort: updates.effort, format }
const maxTokens = generation?.maxTokens ?? DEFAULT_MAX_TOKENS
const body = {
model: request.model.id,
system,
@@ -1071,12 +1075,12 @@ const fromRequest = Effect.fn("AnthropicMessages.fromRequest")(function* (reques
tools,
tool_choice: toolChoice,
stream: true as const,
max_tokens: generation?.maxTokens ?? DEFAULT_MAX_TOKENS,
max_tokens: maxTokens,
temperature: generation?.temperature,
top_p: generation?.topP,
top_k: generation?.topK,
stop_sequences: generation?.stop,
thinking: applyThinkingBindingDefault(request.model, options.thinking),
thinking: applyThinkingBindingDefault(request.model, fitThinking(options.thinking, maxTokens)),
output_config,
// top-level passthrough per SDK MessageCreateParamsBase:4638,4643,4649,4654,4670
cache_control: options.cache_control ?? options.cacheControl,
+34 -14
View File
@@ -9,7 +9,6 @@ import {
type CacheHint,
type FinishReason,
type FinishReasonDetails,
type JsonSchema,
type LLMRequest,
type LanguageModel,
type ProviderMetadata,
@@ -26,7 +25,6 @@ import { BedrockCache } from "./utils/bedrock-cache.js"
import { BedrockMedia } from "./utils/bedrock-media.js"
import { Lifecycle } from "./utils/lifecycle.js"
import { MistralToolID } from "./utils/mistral-tool-id.js"
import { ToolSchemaProjection } from "./utils/tool-schema.js"
import { ToolStream } from "./utils/tool-stream.js"
import { concatBytes } from "../utils/bytes.js"
@@ -221,22 +219,18 @@ type BedrockEvent = Schema.Schema.Type<typeof BedrockEvent>
// =============================================================================
// Request Lowering
// =============================================================================
const lowerToolSpec = (tool: ToolDefinition, inputSchema: JsonSchema): BedrockToolSpec => ({
const lowerToolSpec = (tool: ToolDefinition): BedrockToolSpec => ({
toolSpec: {
name: tool.name,
...(tool.description.trim().length > 0 ? { description: tool.description } : {}),
inputSchema: { json: inputSchema },
inputSchema: { json: tool.inputSchema },
},
})
const lowerTools = (
model: LanguageModel,
breakpoints: BedrockCache.Breakpoints,
tools: ReadonlyArray<ToolDefinition>,
): BedrockTool[] => {
const lowerTools = (breakpoints: BedrockCache.Breakpoints, tools: ReadonlyArray<ToolDefinition>): BedrockTool[] => {
const result: BedrockTool[] = []
for (const tool of tools) {
result.push(lowerToolSpec(tool, ToolSchemaProjection.modelCompatibility(tool.inputSchema, model)))
result.push(lowerToolSpec(tool))
const cachePoint = BedrockCache.block(breakpoints, tool.cache)
if (cachePoint) result.push(cachePoint)
}
@@ -441,19 +435,39 @@ const isHighReasoningEffort = Schema.is(
}),
)
const Options = Schema.Struct({
thinking: Schema.optional(Schema.Struct({ type: Schema.Literal("enabled"), budgetTokens: Schema.Number })),
})
export type OptionsInput = typeof Options.Type
const decodeOptions = ProviderShared.validateWith(Schema.decodeUnknownEffect(Options))
// Claude on Bedrock requires the thinking budget below `maxTokens`, with a minimum of 1,024.
const MIN_THINKING_BUDGET = 1_024
const fromRequest = Effect.fn("BedrockConverse.fromRequest")(function* (request: LLMRequest) {
const toolChoice = request.toolChoice ? yield* lowerToolChoice(request.toolChoice) : undefined
const flattened = ProviderShared.flattenToolRequest(request)
const generation = request.generation
const options = yield* decodeOptions(request.providerOptions ?? {})
const maxTokens =
isNova2(request.model) && isHighReasoningEffort(request.http?.body) ? undefined : generation?.maxTokens
const thinking =
options.thinking === undefined
? undefined
: {
type: "enabled",
budget_tokens: ProviderShared.fitThinkingBudget(
options.thinking.budgetTokens,
maxTokens,
MIN_THINKING_BUDGET,
),
}
// Bedrock-Claude shares Anthropic's 4-breakpoint cap. Spend the budget in
// tools → system → messages order to favour the highest-impact prefixes.
const breakpoints = BedrockCache.breakpoints(request.model.id)
const toolConfig = (() => {
if (flattened.tools.length === 0) return undefined
return {
tools: lowerTools(request.model, breakpoints, flattened.tools),
tools: lowerTools(breakpoints, flattened.tools),
// Converse has no native "none". Keep definitions stable for prompt
// caching and omit only the unsupported choice.
toolChoice,
@@ -487,9 +501,15 @@ const fromRequest = Effect.fn("BedrockConverse.fromRequest")(function* (request:
system,
inferenceConfig,
toolConfig,
// Converse's base inferenceConfig has no topK; Anthropic/Nova accept it
// as a model-specific field, so it goes through additionalModelRequestFields.
additionalModelRequestFields: generation?.topK === undefined ? undefined : { top_k: generation.topK },
// Converse's base inferenceConfig has no topK or thinking; Anthropic/Nova accept them
// as model-specific fields, so they go through additionalModelRequestFields.
additionalModelRequestFields:
generation?.topK === undefined && thinking === undefined
? undefined
: {
...(generation?.topK === undefined ? {} : { top_k: generation.topK }),
...(thinking === undefined ? {} : { thinking }),
},
}
})
+21 -8
View File
@@ -11,7 +11,6 @@ import {
Usage,
type FinishReason,
type LLMRequest,
type LanguageModel,
type MediaPart,
type ProviderMetadata,
type ProviderOptions,
@@ -24,11 +23,12 @@ import { Media } from "../media.js"
import { JsonObject, knownString, lenient, optionalArray, optionalNull, ProviderShared } from "./shared.js"
import { GeminiGenerateContent } from "./utils/gemini-generate-content.js"
import { Lifecycle } from "./utils/lifecycle.js"
import { ToolSchemaProjection } from "./utils/tool-schema.js"
const ADAPTER = "gemini"
// Google documents this sentinel for replaying Gemini 3 function calls after their original signature was lost.
const SKIP_THOUGHT_SIGNATURE_VALIDATOR = "skip_thought_signature_validator"
// Gemini 2.5 rejects a budget under the model's minimum: 512 on Flash-Lite, the highest, and 128 on Pro.
const MIN_THINKING_BUDGET = 512
export const DEFAULT_BASE_URL = "https://generativelanguage.googleapis.com/v1beta"
// Gemini 3 rejects replayed function calls without a thought signature. Google's SDKs avoid that in normal chats by
@@ -268,12 +268,11 @@ interface ParserState {
// =============================================================================
// Request Lowering
// =============================================================================
// Tool schemas go in `parametersJsonSchema`, which accepts standard JSON Schema. Gemini's schema
// rules are this API's default, including for tuned endpoints whose IDs do not name Gemini.
const lowerTool = (tool: ToolDefinition, model: LanguageModel) => ({
// Tool schemas go in `parametersJsonSchema`, which accepts standard JSON Schema.
const lowerTool = (tool: ToolDefinition) => ({
name: tool.name,
description: tool.description,
parametersJsonSchema: ToolSchemaProjection.modelCompatibility(tool.inputSchema, model, "gemini"),
parametersJsonSchema: tool.inputSchema,
})
const lowerToolConfig = (toolChoice: NonNullable<LLMRequest["toolChoice"]>) =>
@@ -452,10 +451,22 @@ const fromRequest = Effect.fn("Gemini.fromRequest")(function* (request: LLMReque
presencePenalty: generation?.presencePenalty,
seed: generation?.seed,
stopSequences: generation?.stop,
// Gemini accepts a budget above `maxOutputTokens`, but thinking then leaves the answer empty.
thinkingConfig:
options.thinkingConfig === undefined
? undefined
: { ...options.thinkingConfig, includeThoughts: options.thinkingConfig.includeThoughts ?? true },
: {
...options.thinkingConfig,
includeThoughts: options.thinkingConfig.includeThoughts ?? true,
thinkingBudget:
options.thinkingConfig.thinkingBudget === undefined
? undefined
: ProviderShared.fitThinkingBudget(
options.thinkingConfig.thinkingBudget,
generation?.maxTokens,
MIN_THINKING_BUDGET,
),
},
}
return {
@@ -468,7 +479,7 @@ const fromRequest = Effect.fn("Gemini.fromRequest")(function* (request: LLMReque
tools: hasTools
? [
{
functionDeclarations: flattened.tools.map((tool) => lowerTool(tool, request.model)),
functionDeclarations: flattened.tools.map(lowerTool),
},
]
: undefined,
@@ -804,6 +815,8 @@ export const protocol = Protocol.make({
schema: GeminiBody,
from: fromRequest,
},
// Gemini's schema rules are this API's default, including for tuned endpoints whose IDs do not name Gemini.
sanitizer: "gemini",
stream: {
event: Protocol.jsonEvent(GeminiEvent),
initial: (request) => ({
+1 -7
View File
@@ -5,7 +5,6 @@ import { LLMEvent, LLMRequest, Message, ToolResultPart } from "../schema/index.j
import { OpenResponses } from "./open-responses.js"
import { JsonObject, optionalArray, optionalNull, ProviderShared } from "./shared.js"
import { ResponsesHostedTools } from "./utils/responses-hosted-tools.js"
import { ToolSchemaProjection } from "./utils/tool-schema.js"
import { detectMediaType } from "../utils/media-type.js"
const ADAPTER = "meta-responses"
@@ -103,12 +102,7 @@ const fromRequest = Effect.fn("MetaResponses.fromRequest")(function* (request: L
? undefined
: yield* Effect.forEach(projected.tools, (tool) =>
Effect.gen(function* () {
if (tool.native === undefined)
return yield* OpenResponses.lowerTool(
NAME,
tool,
ToolSchemaProjection.modelCompatibility(tool.inputSchema, request.model),
)
if (tool.native === undefined) return yield* OpenResponses.lowerTool(NAME, tool)
return yield* ProviderShared.validateWith(Schema.decodeUnknownEffect(NativeTool))(tool.native.meta)
}),
),
+3 -10
View File
@@ -13,7 +13,6 @@ import {
UnknownProviderError,
Usage,
type FinishReasonDetails,
type JsonSchema,
type LLMRequest,
type MediaPart,
type ToolCallPart,
@@ -23,7 +22,6 @@ import { classifyProviderFailure } from "../provider-error.js"
import { JsonObject, optionalArray, optionalNull, ProviderShared } from "./shared.js"
import { Lifecycle } from "./utils/lifecycle.js"
import { MistralToolID } from "./utils/mistral-tool-id.js"
import { ToolSchemaProjection } from "./utils/tool-schema.js"
import { ToolStream } from "./utils/tool-stream.js"
const ADAPTER = "mistral-chat"
@@ -368,9 +366,9 @@ const lowerMessages = Effect.fn("MistralChat.lowerMessages")(function* (request:
return messages
})
const lowerTool = (tool: ToolDefinition, inputSchema: JsonSchema): MistralTool => ({
const lowerTool = (tool: ToolDefinition): MistralTool => ({
type: "function",
function: { name: tool.name, description: tool.description, parameters: inputSchema, strict: false },
function: { name: tool.name, description: tool.description, parameters: tool.inputSchema, strict: false },
})
export const fromRequest = Effect.fn("MistralChat.fromRequest")(function* (request: LLMRequest) {
@@ -396,12 +394,7 @@ export const fromRequest = Effect.fn("MistralChat.fromRequest")(function* (reque
return {
model: request.model.id,
messages: yield* lowerMessages(flattened.request),
tools:
flattened.tools.length > 0
? flattened.tools.map((tool) =>
lowerTool(tool, ToolSchemaProjection.modelCompatibility(tool.inputSchema, request.model)),
)
: undefined,
tools: flattened.tools.length > 0 ? flattened.tools.map(lowerTool) : undefined,
tool_choice: toolChoice,
stream: true as const,
max_tokens: request.generation?.maxTokens,
+8 -16
View File
@@ -8,7 +8,6 @@ import {
ProviderInternalError,
Usage,
type FinishReason,
type JsonSchema,
type LLMRequest,
type MediaPart,
type ProviderMetadata,
@@ -24,7 +23,6 @@ import { classifyProviderFailure } from "../provider-error.js"
import { effortUpdate } from "../effort-updates.js"
import { OpenResponsesOptions } from "./utils/open-responses-options.js"
import { Lifecycle } from "./utils/lifecycle.js"
import { ToolSchemaProjection } from "./utils/tool-schema.js"
import { ToolStream } from "./utils/tool-stream.js"
const ADAPTER = "open-responses"
@@ -443,23 +441,24 @@ interface ReasoningStreamItem {
// =============================================================================
// Request Lowering
// =============================================================================
export const lowerTool = Effect.fn("OpenResponses.lowerTool")(function* (
protocolName: string,
tool: ToolDefinition,
inputSchema: JsonSchema,
) {
export const lowerTool = Effect.fn("OpenResponses.lowerTool")(function* (protocolName: string, tool: ToolDefinition) {
if (tool.native !== undefined)
return yield* ProviderShared.invalidRequest(`${protocolName} does not support provider-native tool ${tool.name}`)
return {
type: "function" as const,
name: tool.name,
description: tool.description,
parameters: inputSchema,
parameters: tool.inputSchema,
// The common tool definition does not currently express Responses strict-schema policy.
strict: false,
}
})
export const lowerTools = (tools: ReadonlyArray<ToolDefinition>, adapter: ProviderAdapter) =>
Effect.forEach(tools, (tool) =>
tool.native !== undefined && adapter.nativeTool ? adapter.nativeTool(tool.native) : lowerTool(adapter.name, tool),
)
export const lowerToolChoice = (protocolName: string, toolChoice: NonNullable<LLMRequest["toolChoice"]>) =>
ProviderShared.matchToolChoice(protocolName, toolChoice, {
auto: () => "auto" as const,
@@ -821,14 +820,7 @@ export const fromRequestWithAdapter = Effect.fn("OpenResponses.fromRequestWithAd
return {
...(yield* lowerConversation(projected.request, adapter)),
...lowerGeneration(request),
tools:
projected.tools.length === 0
? undefined
: yield* Effect.forEach(projected.tools, (tool) =>
tool.native !== undefined && adapter.nativeTool
? adapter.nativeTool(tool.native)
: lowerTool(adapter.name, tool, ToolSchemaProjection.modelCompatibility(tool.inputSchema, request.model)),
),
tools: projected.tools.length === 0 ? undefined : yield* lowerTools(projected.tools, adapter),
tool_choice:
allowedToolChoice(request) ??
(request.toolChoice ? yield* lowerToolChoice(adapter.name, request.toolChoice) : undefined),
+3 -17
View File
@@ -17,7 +17,6 @@ import {
type FinishReason,
type FinishReasonDetails,
type CacheHint,
type JsonSchema,
type LLMRequest,
type MediaPart,
type ReasoningPart,
@@ -29,7 +28,6 @@ import { classifyProviderFailure } from "../provider-error.js"
import { isRecord, JsonObject, optionalArray, optionalNull, ProviderShared } from "./shared.js"
import { OpenAIOptions } from "./utils/openai-options.js"
import { Lifecycle } from "./utils/lifecycle.js"
import { ToolSchemaProjection } from "./utils/tool-schema.js"
import { ToolStream } from "./utils/tool-stream.js"
const ADAPTER = "openai-chat"
@@ -330,17 +328,12 @@ interface LoweringOptions {
readonly toolCallID?: (id: string) => string
}
const lowerTool = (
tool: ToolDefinition,
inputSchema: JsonSchema,
options: LoweringOptions,
supportsStrictMode: boolean,
): OpenAIChatTool => ({
const lowerTool = (tool: ToolDefinition, options: LoweringOptions, supportsStrictMode: boolean): OpenAIChatTool => ({
type: "function",
function: {
name: tool.name,
description: tool.description,
parameters: inputSchema,
parameters: tool.inputSchema,
...(supportsStrictMode ? { strict: false } : {}),
},
cache_control: options.cacheControl?.(tool.cache),
@@ -825,14 +818,7 @@ export const fromRequest = Effect.fn("OpenAIChat.fromRequest")(function* (
? hasHistory
? []
: undefined
: flattened.tools.map((tool) =>
lowerTool(
tool,
ToolSchemaProjection.modelCompatibility(tool.inputSchema, request.model),
options,
supportsStrictMode,
),
),
: flattened.tools.map((tool) => lowerTool(tool, options, supportsStrictMode)),
tool_choice: hasActiveTools && request.toolChoice ? yield* lowerToolChoice(request.toolChoice) : undefined,
stream: true as const,
...(supportsUsageInStreaming ? { stream_options: { include_usage: true } } : {}),
+20 -38
View File
@@ -5,20 +5,12 @@ import { Auth } from "../route/auth.js"
import { Endpoint } from "../route/endpoint.js"
import { Protocol } from "../route/protocol.js"
import { HttpTransport } from "../route/transport/index.js"
import {
LLMRequest,
mergeJsonRecords,
type JsonSchema,
type LanguageModel,
type ToolDefinition,
type ToolEntry,
} from "../schema/index.js"
import { LLMRequest, type ToolDefinition, type ToolEntry } from "../schema/index.js"
import { resolveEffortUpdates } from "../effort-updates.js"
import { OpenResponses } from "./open-responses.js"
import { OpenResponsesOptions } from "./utils/open-responses-options.js"
import { JsonObject, optionalArray, optionalNull, ProviderShared } from "./shared.js"
import { ResponsesHostedTools } from "./utils/responses-hosted-tools.js"
import { ToolSchemaProjection } from "./utils/tool-schema.js"
import { OpenResponsesChannel } from "./open-responses-channel.js"
import { ResponsesCompaction } from "./utils/responses-compaction.js"
import { ResponsesCheckpoint } from "./utils/responses-checkpoint.js"
@@ -143,11 +135,6 @@ export const CompactionTrigger = Schema.Struct({ type: Schema.Literal("compactio
const CheckpointBody = Schema.Struct({
...OpenAIResponsesBody.fields,
input: Schema.Array(Schema.Union([OpenAIResponsesInputItem, CompactionTrigger])),
store: Schema.Literal(false),
prompt_cache_retention: optionalNull(Schema.String),
prompt_cache_options: optionalNull(
Schema.Struct({ mode: Schema.optional(Schema.String), ttl: Schema.optional(Schema.String) }),
),
})
const adapter = {
@@ -174,20 +161,19 @@ const nativeImageTool = (tool: ToolDefinition) => {
return Schema.is(OpenAIResponsesImageGenerationTool)(native) ? native : undefined
}
const lowerTool = Effect.fn("OpenAIResponses.lowerTool")(function* (tool: ToolDefinition, inputSchema: JsonSchema) {
const lowerTool = Effect.fn("OpenAIResponses.lowerTool")(function* (tool: ToolDefinition) {
const native = nativeImageToolInput(tool)
if (native !== undefined) {
if (Schema.is(OpenAIResponsesImageGenerationTool)(native)) return native
return yield* ProviderShared.invalidRequest("OpenAI Responses image generation tool options are invalid")
}
return yield* OpenResponses.lowerTool(NAME, tool, inputSchema)
return yield* OpenResponses.lowerTool(NAME, tool)
})
// Native namespaces hold only function tools, so deeper levels flatten into
// the leaf names the same way non-native protocols flatten the whole tree.
const lowerToolEntry = Effect.fn("OpenAIResponses.lowerToolEntry")(function* (tool: ToolEntry, model: LanguageModel) {
if (tool.type === "tool")
return yield* lowerTool(tool, ToolSchemaProjection.modelCompatibility(tool.inputSchema, model))
const lowerToolEntry = Effect.fn("OpenAIResponses.lowerToolEntry")(function* (tool: ToolEntry) {
if (tool.type === "tool") return yield* lowerTool(tool)
// OpenAI requires a namespace description; fall back to a generic one so a
// missing description never blocks the request.
return {
@@ -195,11 +181,13 @@ const lowerToolEntry = Effect.fn("OpenAIResponses.lowerToolEntry")(function* (to
name: tool.name,
description: tool.description ?? `Tools in the ${tool.name} namespace.`,
tools: yield* Effect.forEach(ProviderShared.flattenTools(tool.tools), (leaf) =>
OpenResponses.lowerTool(NAME, leaf, ToolSchemaProjection.modelCompatibility(leaf.inputSchema, model)),
OpenResponses.lowerTool(NAME, leaf),
),
}
})
const lowerTools = (request: LLMRequest) => Effect.forEach(request.tools, lowerToolEntry)
const lowerToolChoice = (toolChoice: NonNullable<LLMRequest["toolChoice"]>, tools: ReadonlyArray<ToolEntry>) =>
ProviderShared.matchToolChoice(NAME, toolChoice, {
auto: () => "auto" as const,
@@ -223,10 +211,7 @@ const fromRequest = Effect.fn("OpenAIResponses.fromRequest")(function* (request:
...(yield* OpenResponses.lowerConversation(updates.request, adapter)),
...OpenResponses.lowerGeneration(request, { ...options, reasoningEffort: updates.effort }),
context_management: management?.map((edit) => ({ type: edit.type, compact_threshold: edit.compactThreshold })),
tools:
request.tools.length === 0
? undefined
: yield* Effect.forEach(request.tools, (tool) => lowerToolEntry(tool, request.model)),
tools: request.tools.length === 0 ? undefined : yield* lowerTools(request),
tool_choice:
request.tools.length === 0
? undefined
@@ -238,7 +223,6 @@ const fromRequest = Effect.fn("OpenAIResponses.fromRequest")(function* (request:
const checkpointBody = {
schema: CheckpointBody,
from: Effect.fn("OpenAIResponses.checkpointBody")(function* (request: LLMRequest) {
const native = yield* fromRequest(LLMRequest.update(request, { toolChoice: undefined }))
const overlay = request.http?.body
// Complete history is required for stateless replay and SSE recovery. Raw input overrides bypass that contract.
if (
@@ -249,18 +233,13 @@ const checkpointBody = {
return yield* ProviderShared.invalidRequest(
"Trigger compaction requires complete canonical history, not an input or continuation override",
)
return yield* ProviderShared.validateWith(Schema.decodeUnknownEffect(CheckpointBody))({
...mergeJsonRecords(native, overlay),
input: [...native.input, { type: "compaction_trigger" }],
stream: true,
store: false,
parallel_tool_calls: true,
tool_choice: undefined,
context_management: undefined,
text: undefined,
max_output_tokens: undefined,
max_tool_calls: undefined,
})
if (overlay?.stream !== undefined && overlay.stream !== true)
return yield* ProviderShared.invalidRequest("Trigger compaction requires a streamed response")
const native = yield* fromRequest(request)
return {
...native,
input: [...native.input, { type: "compaction_trigger" as const }],
}
}),
}
@@ -342,7 +321,10 @@ export const transport = channelTransport({
})
export const route = Route.make({
compact: { endpoint: ResponsesCompaction.make(adapter), trigger: ResponsesCheckpoint.make(checkpointBody) },
compact: {
endpoint: ResponsesCompaction.make(adapter, lowerTools),
trigger: ResponsesCheckpoint.make(checkpointBody),
},
id: ADAPTER,
provider: "openai",
providerMetadataKey: "openai",
+8
View File
@@ -110,6 +110,14 @@ export const sumTokens = (...values: ReadonlyArray<number | undefined>): number
return values.reduce((acc: number, value) => acc + (value ?? 0), 0)
}
/**
* Caps an explicit thinking budget at half the output limit. Thinking counts against the output limit, so a budget
* near it leaves the answer, a tool call, or a summary without room. Smaller budgets, special values such as `-1` and
* `0`, and requests without an output limit pass through unchanged.
*/
export const fitThinkingBudget = (budget: number, maxTokens: number | undefined, minimum = 1) =>
maxTokens === undefined || budget <= maxTokens / 2 ? budget : Math.max(minimum, Math.floor(maxTokens / 2))
export const eventError = (route: string, message: string, body?: string, cause?: unknown) =>
new AIError({
reason: new InvalidProviderOutputError({ route, message, body, cause }),
@@ -1,7 +1,7 @@
import { Effect, Schema, Stream } from "effect"
import { Route, type RouteBody, type TriggerCompactOperation } from "../../route/client.js"
import { Protocol } from "../../route/protocol.js"
import { CompactionCheckpointResponse, HttpOptions, LLMEvent, LLMRequest } from "../../schema/index.js"
import { CompactionCheckpointResponse, LLMEvent, LLMRequest } from "../../schema/index.js"
import { OpenResponses } from "../open-responses.js"
import { ProviderShared } from "../shared.js"
@@ -109,12 +109,8 @@ export const make = <Body>(body: RouteBody<Body>): TriggerCompactOperation =>
transport: source.transport,
})
const native = yield* body.from(request)
// The body builder already applied and validated overlays. Do not let transport reapply them.
const preparedRequest = LLMRequest.update(request, {
http: request.http === undefined ? undefined : new HttpOptions({ ...request.http, body: undefined }),
})
const prepared = yield* route.prepareTransport(native, preparedRequest, options)
yield* route.streamPrepared(prepared, preparedRequest, { http: executor }, options).pipe(Stream.runDrain)
const prepared = yield* route.prepareTransport(native, request, options)
yield* route.streamPrepared(prepared, request, { http: executor }, options).pipe(Stream.runDrain)
if (!result) return yield* ProviderShared.eventError(source.id, "Compaction response ended without a checkpoint")
return result
})
@@ -19,12 +19,18 @@ import { OpenResponses } from "../open-responses.js"
import { JsonObject, optionalNull, ProviderShared } from "../shared.js"
import { Media } from "../../media.js"
// /compact has a smaller wire contract than /responses; keep the request controls it accepts.
const Body = Schema.Struct({
model: Schema.String,
input: Schema.Array(Schema.Unknown),
instructions: optionalNull(Schema.String),
previous_response_id: optionalNull(Schema.String),
service_tier: optionalNull(Schema.String),
reasoning: Schema.optional(JsonObject),
text: Schema.optional(JsonObject),
include: OpenResponses.coreFields.include,
parallel_tool_calls: OpenResponses.coreFields.parallel_tool_calls,
tools: Schema.optional(Schema.Array(JsonObject)),
prompt_cache_key: optionalNull(Schema.String),
prompt_cache_retention: optionalNull(Schema.String),
prompt_cache_options: optionalNull(
@@ -74,17 +80,27 @@ const Response = Schema.Struct({
usage: Schema.optional(Schema.StructWithRest(OpenResponses.OpenResponsesUsage, [JsonObject])),
})
export const make = (adapter: OpenResponses.ProviderAdapter): CompactOperation =>
export const make = (
adapter: OpenResponses.ProviderAdapter,
lowerTools: (request: LLMRequest) => Effect.Effect<ReadonlyArray<Record<string, unknown>>, AIError>,
): CompactOperation =>
Effect.fn("ResponsesCompaction.execute")(function* (request, executor, options) {
const route = request.model.route
// The standalone compaction endpoint rejects histories containing configuration updates.
const native = yield* OpenResponses.lowerConversation(stripEffortUpdates(request), adapter)
const generation = OpenResponses.lowerGeneration(request)
const tools = request.tools.length === 0 ? undefined : yield* lowerTools(request)
const body = yield* ProviderShared.validateWith(Schema.decodeUnknownEffect(Body))(
mergeJsonRecords(
{
...native,
service_tier: request.providerOptions?.serviceTier,
prompt_cache_key: ProviderShared.promptCacheKey(request),
service_tier: generation.service_tier,
reasoning: generation.reasoning,
text: generation.text,
include: generation.include,
parallel_tool_calls: generation.parallel_tool_calls,
tools,
prompt_cache_key: generation.prompt_cache_key,
},
request.http?.body,
),
+16 -8
View File
@@ -1,4 +1,4 @@
import type { JsonSchema, LanguageModel, LanguageModelSanitizerCompatibility } from "../../schema/index.js"
import { ToolDefinition, type JsonSchema, type LanguageModel, type LLMRequest } from "../../schema/index.js"
import { isRecord } from "../../utils/record.js"
import { GeminiJsonSchema } from "./gemini-json-schema.js"
@@ -70,13 +70,13 @@ const objectRoot = (schema: JsonSchema): JsonSchema => {
// Otherwise the protocol's own default applies (the Gemini API always uses Gemini's rules), then the
// model name selects the family's rules so models reached through gateways and OpenAI-compatible
// endpoints get the same handling.
const modelCompatibility = (
schema: JsonSchema,
model: LanguageModel,
protocolDefault?: LanguageModelSanitizerCompatibility,
): JsonSchema => {
const modelCompatibility = (schema: JsonSchema, model: LanguageModel): JsonSchema => {
const root = objectRoot(schema)
switch (model.compatibility?.sanitizer ?? protocolDefault ?? MODEL_NAMES.find(([name]) => name.test(model.id))?.[1]) {
switch (
model.compatibility?.sanitizer ??
model.route.sanitizer ??
MODEL_NAMES.find(([name]) => name.test(model.id))?.[1]
) {
case "gemini":
return gemini(root)
case "moonshot":
@@ -87,10 +87,18 @@ const modelCompatibility = (
}
}
// Applied once to every request before any protocol builds its body, including tools in namespaces.
const tools = (entries: LLMRequest["tools"], model: LanguageModel): LLMRequest["tools"] =>
entries.map((tool) =>
tool.type === "tool"
? new ToolDefinition({ ...tool, inputSchema: modelCompatibility(tool.inputSchema, model) })
: { ...tool, tools: tools(tool.tools, model) },
)
export const ToolSchemaProjection = {
gemini,
modelCompatibility,
moonshot,
openAI,
responses,
tools,
} as const
+5 -2
View File
@@ -50,7 +50,8 @@ const fromRequest = Effect.fn("XAIResponses.fromRequest")(function* (request: LL
operation: "in-band-compaction",
provider: request.model.provider,
route: request.model.route.id,
message: "xAI requires explicit compaction through LLMClient.compact; automatic context management is not supported",
message:
"xAI requires explicit compaction through LLMClient.compact; automatic context management is not supported",
})
return yield* decodeBody(yield* OpenResponses.fromRequestWithAdapter(request, adapter))
})
@@ -93,6 +94,8 @@ export const protocol = Protocol.make({
},
})
export const compact = ResponsesCompaction.make(adapter)
export const compact = ResponsesCompaction.make(adapter, (request) =>
OpenResponses.lowerTools(ProviderShared.flattenTools(request.tools), adapter),
)
export * as XAIResponses from "./xai-responses.js"
+7 -1
View File
@@ -80,7 +80,13 @@ const SERVER_CODES = new Set([
"slow_down",
"serviceunavailableexception",
])
const INVALID_REQUEST_CODES = new Set(["invalid_prompt", "invalid_request_error", "validationexception"])
// `invalid_request` is the Vercel AI Gateway's code for an upstream request rejection.
const INVALID_REQUEST_CODES = new Set([
"invalid_prompt",
"invalid_request",
"invalid_request_error",
"validationexception",
])
// Azure OpenAI reports `content_filter` with `innererror.code` ResponsibleAIPolicyViolation.
// OpenRouter tags provider failures with a typed `error_type`; its Responses skin also
// emits `image_content_policy_violation` as the native code.
@@ -31,6 +31,7 @@ export interface Settings extends ProviderPackage.Settings {
readonly profile?: string
readonly region?: string
readonly topP?: number
readonly thinking?: BedrockConverse.OptionsInput["thinking"]
}
export const routes = [BedrockConverse.route]
@@ -71,6 +72,7 @@ export const model: ProviderPackage.Definition<Settings>["model"] = (modelID, se
generation: settings.topP === undefined ? undefined : { topP: settings.topP },
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
providerOptions: settings.thinking === undefined ? undefined : { thinking: settings.thinking },
profile: settings.profile,
region: settings.region,
}).model(modelID)
+4 -15
View File
@@ -32,10 +32,7 @@ const openAIProviderOptions = (options: OpenAIOptionsInput | undefined): Provide
return result
}
export const gpt5DefaultOptions = (
modelID: string,
options: { readonly textVerbosity?: boolean } = {},
): ProviderOptions | undefined => {
export const gpt5DefaultOptions = (modelID: string): ProviderOptions | undefined => {
const id = modelID.toLowerCase()
if (!id.includes("gpt-5") || id.includes("gpt-5-chat") || id.includes("gpt-5-pro")) return undefined
return openAIProviderOptions({
@@ -47,27 +44,19 @@ export const gpt5DefaultOptions = (
// this, callers using the default model facade get reasoning summaries
// they cannot replay statelessly.
include: ["reasoning.encrypted_content"],
textVerbosity:
options.textVerbosity === true && id.includes("gpt-5.") && !id.includes("codex") && !id.includes("-chat")
? "low"
: undefined,
})
}
export const openAIDefaultOptions = (
modelID: string,
options: { readonly textVerbosity?: boolean } = {},
): ProviderOptions | undefined =>
mergeProviderOptions(openAIProviderOptions({ store: false }), gpt5DefaultOptions(modelID, options))
export const openAIDefaultOptions = (modelID: string): ProviderOptions | undefined =>
mergeProviderOptions(openAIProviderOptions({ store: false }), gpt5DefaultOptions(modelID))
export const withOpenAIOptions = <Options extends { readonly providerOptions?: OpenAIProviderOptionsInput }>(
modelID: string,
options: Options,
defaults: { readonly textVerbosity?: boolean } = {},
): Omit<Options, "providerOptions"> & { readonly providerOptions?: ProviderOptions } => {
return {
...options,
providerOptions: mergeProviderOptions(openAIDefaultOptions(modelID, defaults), options.providerOptions),
providerOptions: mergeProviderOptions(openAIDefaultOptions(modelID), options.providerOptions),
}
}
+1 -1
View File
@@ -100,7 +100,7 @@ export const configure = (input: Config = {}) => {
const modelDefaults = defaults(input)
const responses = (id: string | ModelID) =>
responsesRoute
.with(withOpenAIOptions(id, modelDefaults, { textVerbosity: true }))
.with(withOpenAIOptions(id, modelDefaults))
.model<OpenAIProviderOptionsInput>({ id })
const chat = (id: string | ModelID) =>
chatRoute.with(withOpenAIOptions(id, modelDefaults)).model<OpenAIProviderOptionsInput>({
+11 -4
View File
@@ -8,7 +8,7 @@ import type { ProviderPackage } from "../provider-package.js"
import { SystemOne } from "../experimental/system-one.js"
import { OpenAIChat } from "../protocols/openai-chat.js"
import { newBreakpoints, ttlBucket } from "../protocols/utils/cache.js"
import { isRecord } from "../protocols/shared.js"
import { isRecord, ProviderShared } from "../protocols/shared.js"
export const id = ProviderID.make("openrouter")
const baseURL = "https://openrouter.ai/api/v1"
@@ -123,7 +123,7 @@ export const protocol = Protocol.make({
return {
...body,
messages,
...bodyOptions(request.providerOptions),
...bodyOptions(request.providerOptions, request.generation?.maxTokens),
} as OpenRouterBody
}),
),
@@ -143,7 +143,14 @@ const cacheControl = () => {
}
}
const bodyOptions = (input: unknown) => {
// OpenRouter forwards `reasoning.max_tokens` as the upstream thinking budget. Upstreams such as Anthropic and Alibaba
// reject one that is not below the output limit; 1,024 is Anthropic's minimum budget.
const fitReasoning = (reasoning: Record<string, unknown>, maxTokens: number | undefined) =>
typeof reasoning.max_tokens === "number"
? { ...reasoning, max_tokens: ProviderShared.fitThinkingBudget(reasoning.max_tokens, maxTokens, 1_024) }
: reasoning
const bodyOptions = (input: unknown, maxTokens: number | undefined) => {
const openrouter = isRecord(input) ? input : {}
const { usage, models, provider, plugins, web_search_options, debug, user, reasoning, promptCacheKey, ...options } =
openrouter
@@ -162,7 +169,7 @@ const bodyOptions = (input: unknown) => {
...(isRecord(web_search_options) ? { web_search_options } : {}),
...(isRecord(debug) ? { debug } : {}),
...(typeof user === "string" ? { user } : {}),
...(isRecord(reasoning) ? { reasoning } : {}),
...(isRecord(reasoning) ? { reasoning: fitReasoning(reasoning, maxTokens) } : {}),
}
}
+7 -2
View File
@@ -11,7 +11,8 @@ import { applyEffortUpdates } from "../effort-updates.js"
import { normalizeToolHistory } from "../tool-history.js"
import { sanitizeSurrogates } from "../utils/sanitize.js"
import * as ProviderShared from "../protocols/shared.js"
import type { ProtocolID, ProviderOptions } from "../schema/index.js"
import { ToolSchemaProjection } from "../protocols/utils/tool-schema.js"
import type { LanguageModelSanitizerCompatibility, ProtocolID, ProviderOptions } from "../schema/index.js"
import {
AIError,
CompactionResponse,
@@ -57,6 +58,7 @@ export interface Route<
readonly defaults: RouteDefaults
readonly body: RouteBody<Body>
readonly supportsEffortUpdates?: (request: LLMRequest) => boolean
readonly sanitizer?: LanguageModelSanitizerCompatibility
readonly with: {
<Next extends CompactionOperations | undefined>(
patch: RoutePatch<Body, Prepared> & { readonly compact: Next },
@@ -388,6 +390,7 @@ function makeFromTransport<Body, Prepared, Frame, Event, State>(
defaults: routeInput.defaults ?? {},
body: protocol.body,
supportsEffortUpdates: protocol.supportsEffortUpdates,
sanitizer: protocol.sanitizer,
with: (patch: RoutePatch<Body, Prepared>) => {
const { compact, id, provider, providerMetadataKey, auth, transport, endpoint, ...defaults } = patch
return build({
@@ -559,7 +562,9 @@ const prepareRequest = (request: LLMRequest) => {
tool.type === "tool" ? tool : { ...tool, tools: dedupe(tool.tools) },
)
const resolved = applyCachePolicy(
applyEffortUpdates(LLMRequest.update(sanitized, { tools: dedupe(sanitized.tools) })),
applyEffortUpdates(
LLMRequest.update(sanitized, { tools: ToolSchemaProjection.tools(dedupe(sanitized.tools), sanitized.model) }),
),
)
const headers = resolved.model.route.headers?.({ request: resolved })
return headers === undefined
+4 -57
View File
@@ -5,7 +5,7 @@ import { Endpoint } from "./endpoint.js"
import { RequestExecutorService, type Interface } from "./executor-service.js"
import { RequestExecutor } from "./executor.js"
import { MediaProtocol } from "./media-protocol.js"
import { Generation, resultEvents, type AwaitOptions, type Observation } from "../generation.js"
import { Generation } from "../generation.js"
import type { Media } from "../media.js"
import {
AIError,
@@ -52,7 +52,7 @@ export const deployment = (
// ---------------------------------------------------------------------------
/** One request, one response. */
export interface Route<Request extends MediaRequest, Response> {
export interface InlineRoute<Request extends MediaRequest, Response> {
readonly kind: "inline"
readonly id: string
readonly provider: ProviderID
@@ -86,7 +86,7 @@ export interface StreamRoute<Request extends MediaRequest, Event, Response> {
}
export type AnyRoute<Request extends MediaRequest, Event, Response> =
| Route<Request, Response>
| InlineRoute<Request, Response>
| StreamRoute<Request, Event, Response>
| QueuedRoute<Request, Response>
@@ -119,7 +119,7 @@ export interface StreamInput<Request extends MediaRequest, Event, Response, Fram
*/
export const inline = <Request extends MediaRequest, Response>(
input: InlineInput<Request, Response>,
): Route<Request, Response> => {
): InlineRoute<Request, Response> => {
const transport = makeTransport(input)
return {
kind: "inline",
@@ -267,59 +267,6 @@ export const stream = <Request extends MediaRequest, Event, Response, Frame, Sta
}
}
export const dispatch = <Event, Response>(input: {
readonly modality: string
readonly execute: Execute
readonly responseEvents: (response: Response) => ReadonlyArray<Event>
}) => {
const notQueued = (route: { readonly provider: ProviderID; readonly id: string }, operation: string) =>
new AIError({
reason: new UnsupportedOperationError({
operation: `${input.modality}.${operation}`,
provider: route.provider,
route: route.id,
message: `${route.provider}/${route.id} is not a queued route; use generate or stream`,
}),
})
const start = <Request extends MediaRequest>(route: AnyRoute<Request, Event, Response>, request: Request) => {
if (route.kind !== "queued") return Effect.fail(notQueued(route, "start"))
return route.start(request, input.execute)
}
return {
start,
resume: <Request extends MediaRequest>(
route: AnyRoute<Request, Event, Response>,
model: MediaRequest["model"],
token: unknown,
) => {
if (route.kind !== "queued") return Effect.fail(notQueued(route, "resume"))
return route.resume(model, token, input.execute)
},
generate: <Request extends MediaRequest>(
route: AnyRoute<Request, Event, Response>,
request: Request,
options?: AwaitOptions,
) => {
if (route.kind !== "queued") return route.generate(request, input.execute)
return start(route, request).pipe(Effect.flatMap((generation) => generation.await(options)))
},
stream: <Request extends MediaRequest>(
route: AnyRoute<Request, Event, Response>,
request: Request,
options?: AwaitOptions,
): Stream.Stream<Event | Observation, AIError> => {
if (route.kind === "stream") return route.stream(request, input.execute)
if (route.kind === "queued")
return Stream.unwrap(
start(route, request).pipe(
Effect.map((generation) => resultEvents(generation, input.responseEvents, options)),
),
)
return Stream.fromIterableEffect(Effect.map(route.generate(request, input.execute), input.responseEvents))
},
}
}
// ---------------------------------------------------------------------------
// Transport plumbing shared by every kind
// ---------------------------------------------------------------------------
+3 -1
View File
@@ -1,5 +1,5 @@
import { Schema, type Effect } from "effect"
import type { AIError, LLMEvent, LLMRequest, ProtocolID } from "../schema/index.js"
import type { AIError, LanguageModelSanitizerCompatibility, LLMEvent, LLMRequest, ProtocolID } from "../schema/index.js"
/**
* The semantic API contract of one model server family.
@@ -43,6 +43,8 @@ export interface Protocol<Body, Frame, Event, State> {
readonly stream: ProtocolStream<Frame, Event, State>
/** Whether `body.from` lowers `Message.effort(...)` markers; wrappers around another `body.from` must forward it. */
readonly supportsEffortUpdates?: (request: LLMRequest) => boolean
/** Tool schema sanitizer for every model on this protocol unless the model's compatibility sets one; wrappers around another `body.from` must forward it. */
readonly sanitizer?: LanguageModelSanitizerCompatibility
}
export interface ProtocolBody<Body> {
+22 -44
View File
@@ -1,53 +1,31 @@
import { Context, Effect, Layer, Stream } from "effect"
import { RequestExecutor } from "./route/executor.js"
import type { AIError } from "./schema/index.js"
import type { SpeechEvent, SpeechOptions, SpeechRequestFor, SpeechResponse } from "./speech.js"
import { Context } from "effect"
import { MediaClient } from "./media-client.js"
import {
SpeechTimestampsEvent,
SpeechFinishEvent,
type SpeechEvent,
type SpeechRequestFor,
type SpeechResponse,
} from "./speech.js"
export interface Interface {
readonly generate: <Options extends SpeechOptions>(
request: SpeechRequestFor<Options>,
) => Effect.Effect<SpeechResponse, AIError>
readonly stream: <Options extends SpeechOptions>(
request: SpeechRequestFor<Options>,
) => Stream.Stream<SpeechEvent, AIError>
}
export type Interface = MediaClient.Interface<SpeechRequestFor, SpeechEvent, SpeechResponse>
export class SpeechClientService extends Context.Service<SpeechClientService, Interface>()("@opencode/SpeechClient") {}
export const Service = SpeechClientService
export type Service = SpeechClientService
export const generate = <Options extends SpeechOptions>(
request: SpeechRequestFor<Options>,
): Effect.Effect<SpeechResponse, AIError, Service> =>
Effect.gen(function* () {
const client = yield* Service
return yield* client.generate(request)
})
export const stream = <Options extends SpeechOptions>(
request: SpeechRequestFor<Options>,
): Stream.Stream<SpeechEvent, AIError, Service> =>
Stream.unwrap(
Effect.gen(function* () {
const client = yield* Service
return client.stream(request)
}),
)
export const layer: Layer.Layer<Service, never, RequestExecutor.Service> = Layer.effect(
Service,
Effect.gen(function* () {
const executor = yield* RequestExecutor.Service
return Service.of({
generate: (request) => request.model.route.generate(request, executor.execute),
stream: (request) => request.model.route.stream(request, executor.execute),
})
}),
)
export const SpeechClient = {
Service,
layer,
generate,
stream,
...MediaClient.make(Service, {
modality: "speech",
responseEvents: (response: SpeechResponse) => [
...(response.timestamps === undefined ? [] : [SpeechTimestampsEvent.make({ items: response.timestamps })]),
SpeechFinishEvent.make({
audio: response.audio,
usage: response.usage,
notices: response.notices,
providerMetadata: response.providerMetadata,
}),
],
}),
} as const
+31 -41
View File
@@ -1,8 +1,8 @@
import { Effect, Schema, Stream } from "effect"
import { ProgressEvent, QueuedEvent } from "./generation.js"
import { Media } from "./media.js"
import { MediaModel, composeRoute, tryRequest } from "./media-model.js"
import { MediaRoute } from "./route/media.js"
import type { MediaProtocol } from "./route/media-protocol.js"
import { AIError, HttpOptions, MediaUsage, ProviderMetadata, type OpenString } from "./schema/index.js"
import { SpeechClient, Service } from "./speech-client.js"
@@ -10,53 +10,39 @@ import { SpeechClient, Service } from "./speech-client.js"
// Model
// ---------------------------------------------------------------------------
export type SpeechOptions = Record<string, unknown>
export type SpeechOptions = MediaModel.Options
export type SpeechRoute<Options extends SpeechOptions = SpeechOptions> = MediaRoute.StreamRoute<
SpeechRequestFor<Options>,
SpeechEvent,
SpeechResponse
>
export type SpeechRoute = MediaRoute.AnyRoute<SpeechRequestFor, SpeechEvent, SpeechResponse>
export class SpeechModel<Options extends SpeechOptions = SpeechOptions> extends MediaModel<
SpeechRoute<Options>,
Options
> {
export class SpeechModel<Options extends SpeechOptions = SpeechOptions> extends MediaModel<SpeechRoute, Options> {
declare protected readonly _SpeechModel: void
static make<Options extends SpeechOptions = SpeechOptions>(input: MediaModel.Input<SpeechRoute<Options>>) {
return new SpeechModel<Options>(input)
}
/** Compose a streaming speech protocol with its canonical path into a model for one deployment. */
static fromRoute<Options extends SpeechOptions = SpeechOptions, Frame = unknown, State = unknown>(
route: SpeechModel.RouteInput<Options, Frame, State>,
/** The number of type arguments selects the kind: `<Options>`, `<Options, Frame, State>`, or `<Options, Token>`. */
static fromRoute<Options extends SpeechOptions>(
route: MediaModel.InlineRouteInput<SpeechRequestFor<Options>, SpeechResponse>,
input: MediaRoute.ModelInput,
): SpeechModel<Options>
static fromRoute<Options extends SpeechOptions, Frame, State>(
route: MediaModel.StreamRouteInput<SpeechRequestFor<Options>, SpeechEvent, Frame, State>,
input: MediaRoute.ModelInput,
): SpeechModel<Options>
static fromRoute<Options extends SpeechOptions, Token>(
route: MediaModel.QueuedRouteInput<SpeechRequestFor<Options>, SpeechResponse, Token>,
input: MediaRoute.ModelInput,
): SpeechModel<Options>
static fromRoute<Options extends SpeechOptions, Frame, State, Token>(
route: MediaModel.AnyRouteInput<SpeechRequestFor<Options>, SpeechEvent, SpeechResponse, Frame, State, Token>,
input: MediaRoute.ModelInput,
) {
return new SpeechModel<Options>({
id: input.id,
provider: route.protocol.provider,
http: input.http,
route: composeRoute(
(composition) => MediaRoute.stream({ ...composition, collect: collectResponse }),
route,
input,
),
route: composeRoute(route, input, collectResponse) as SpeechRoute,
})
}
}
export namespace SpeechModel {
export type RouteInput<
Options extends SpeechOptions = SpeechOptions,
Frame = unknown,
State = unknown,
> = MediaModel.RouteInput<
MediaProtocol.Addressed<SpeechRequestFor<Options>>,
MediaProtocol.Streamed<SpeechRequestFor<Options>, SpeechEvent, Frame, State>
>
}
export const SpeechModelSchema = Schema.declare((value): value is SpeechModel => value instanceof SpeechModel, {
expected: "Speech.Model",
})
@@ -148,11 +134,17 @@ export const SpeechFinishEvent = Schema.Struct({
providerMetadata: Schema.optional(ProviderMetadata),
}).annotate({ identifier: "Speech.Event.Finish" })
const speechEventTagged = Schema.Union([SpeechAudioDeltaEvent, SpeechTimestampsEvent, SpeechFinishEvent]).pipe(
Schema.toTaggedUnion("type"),
)
const speechEventTagged = Schema.Union([
QueuedEvent,
ProgressEvent,
SpeechAudioDeltaEvent,
SpeechTimestampsEvent,
SpeechFinishEvent,
]).pipe(Schema.toTaggedUnion("type"))
export const SpeechEvent = Object.assign(speechEventTagged, {
is: {
generationQueued: speechEventTagged.guards["generation-queued"],
generationProgress: speechEventTagged.guards["generation-progress"],
audioDelta: speechEventTagged.guards["audio-delta"],
timestamps: speechEventTagged.guards.timestamps,
finish: speechEventTagged.guards.finish,
@@ -195,17 +187,15 @@ export function request(input: SpeechRequest | SpeechRequestInput) {
const requestEffect = (input: SpeechRequest | SpeechRequestInput) => tryRequest(() => request(input))
export function generate<const Model extends SpeechModel>(
input: SpeechRequestInput<Model>,
input: SpeechRequest | SpeechRequestInput<Model>,
): Effect.Effect<SpeechResponse, AIError, Service>
export function generate(input: SpeechRequest): Effect.Effect<SpeechResponse, AIError, Service>
export function generate(input: SpeechRequest | SpeechRequestInput) {
return requestEffect(input).pipe(Effect.flatMap((request) => SpeechClient.generate(request)))
}
export function stream<const Model extends SpeechModel>(
input: SpeechRequestInput<Model>,
input: SpeechRequest | SpeechRequestInput<Model>,
): Stream.Stream<SpeechEvent, AIError, Service>
export function stream(input: SpeechRequest): Stream.Stream<SpeechEvent, AIError, Service>
export function stream(input: SpeechRequest | SpeechRequestInput) {
return Stream.unwrap(requestEffect(input).pipe(Effect.map((request) => SpeechClient.stream(request))))
}
+8 -85
View File
@@ -1,34 +1,13 @@
import { Context, Effect, Layer, Stream } from "effect"
import type { AwaitOptions, Generation } from "./generation.js"
import { RequestExecutor } from "./route/executor.js"
import { MediaRoute } from "./route/media.js"
import type { AIError } from "./schema/index.js"
import { Context } from "effect"
import { MediaClient } from "./media-client.js"
import {
responseEvents,
TranscriptionFinishEvent,
type TranscriptionEvent,
type TranscriptionModel,
type TranscriptionOptions,
type TranscriptionRequestFor,
type TranscriptionResponse,
} from "./transcription.js"
export interface Interface {
readonly generate: <Options extends TranscriptionOptions>(
request: TranscriptionRequestFor<Options>,
options?: AwaitOptions,
) => Effect.Effect<TranscriptionResponse, AIError>
readonly stream: <Options extends TranscriptionOptions>(
request: TranscriptionRequestFor<Options>,
options?: AwaitOptions,
) => Stream.Stream<TranscriptionEvent, AIError>
readonly start: <Options extends TranscriptionOptions>(
request: TranscriptionRequestFor<Options>,
) => Effect.Effect<Generation<TranscriptionResponse>, AIError>
readonly resume: <Options extends TranscriptionOptions>(
model: TranscriptionModel<Options>,
token: unknown,
) => Effect.Effect<Generation<TranscriptionResponse>, AIError>
}
export type Interface = MediaClient.Interface<TranscriptionRequestFor, TranscriptionEvent, TranscriptionResponse>
export class TranscriptionClientService extends Context.Service<TranscriptionClientService, Interface>()(
"@opencode/TranscriptionClient",
@@ -36,66 +15,10 @@ export class TranscriptionClientService extends Context.Service<TranscriptionCli
export const Service = TranscriptionClientService
export type Service = TranscriptionClientService
export const generate = <Options extends TranscriptionOptions>(
request: TranscriptionRequestFor<Options>,
options?: AwaitOptions,
): Effect.Effect<TranscriptionResponse, AIError, Service> =>
Effect.gen(function* () {
const client = yield* Service
return yield* client.generate(request, options)
})
export const stream = <Options extends TranscriptionOptions>(
request: TranscriptionRequestFor<Options>,
options?: AwaitOptions,
): Stream.Stream<TranscriptionEvent, AIError, Service> =>
Stream.unwrap(
Effect.gen(function* () {
const client = yield* Service
return client.stream(request, options)
}),
)
export const start = <Options extends TranscriptionOptions>(
request: TranscriptionRequestFor<Options>,
): Effect.Effect<Generation<TranscriptionResponse>, AIError, Service> =>
Effect.gen(function* () {
const client = yield* Service
return yield* client.start(request)
})
export const resume = <Options extends TranscriptionOptions>(
model: TranscriptionModel<Options>,
token: unknown,
): Effect.Effect<Generation<TranscriptionResponse>, AIError, Service> =>
Effect.gen(function* () {
const client = yield* Service
return yield* client.resume(model, token)
})
export const layer: Layer.Layer<Service, never, RequestExecutor.Service> = Layer.effect(
Service,
Effect.gen(function* () {
const executor = yield* RequestExecutor.Service
const dispatch = MediaRoute.dispatch<TranscriptionEvent, TranscriptionResponse>({
modality: "transcription",
execute: executor.execute,
responseEvents,
})
return Service.of({
start: (request) => dispatch.start(request.model.route, request),
resume: (model, token) => dispatch.resume(model.route, model, token),
generate: (request, options) => dispatch.generate(request.model.route, request, options),
stream: (request, options) => dispatch.stream(request.model.route, request, options),
})
}),
)
export const TranscriptionClient = {
Service,
layer,
generate,
stream,
start,
resume,
...MediaClient.make(Service, {
modality: "transcription",
responseEvents: (response: TranscriptionResponse) => [TranscriptionFinishEvent.make({ ...response })],
}),
} as const
+21 -76
View File
@@ -1,9 +1,8 @@
import { Effect, Schema, Stream } from "effect"
import { Generation, ProgressEvent, QueuedEvent, type AwaitOptions } from "./generation.js"
import { Media } from "./media.js"
import { MediaModel, composeAnyRoute, tryRequest } from "./media-model.js"
import { MediaModel, composeRoute, tryRequest } from "./media-model.js"
import { MediaRoute } from "./route/media.js"
import type { MediaProtocol } from "./route/media-protocol.js"
import { AIError, HttpOptions, MediaUsage, ProviderMetadata } from "./schema/index.js"
import { TranscriptionClient, Service } from "./transcription-client.js"
@@ -11,90 +10,49 @@ import { TranscriptionClient, Service } from "./transcription-client.js"
// Model
// ---------------------------------------------------------------------------
export type TranscriptionOptions = Record<string, unknown>
export type TranscriptionOptions = MediaModel.Options
export type TranscriptionRoute<Options extends TranscriptionOptions = TranscriptionOptions> = MediaRoute.AnyRoute<
TranscriptionRequestFor<Options>,
TranscriptionEvent,
TranscriptionResponse
>
export type TranscriptionRoute = MediaRoute.AnyRoute<TranscriptionRequestFor, TranscriptionEvent, TranscriptionResponse>
export class TranscriptionModel<Options extends TranscriptionOptions = TranscriptionOptions> extends MediaModel<
TranscriptionRoute<Options>,
TranscriptionRoute,
Options
> {
declare protected readonly _TranscriptionModel: void
static make<Options extends TranscriptionOptions = TranscriptionOptions>(
input: MediaModel.Input<TranscriptionRoute<Options>>,
) {
return new TranscriptionModel<Options>(input)
}
/** The number of type arguments selects the kind: `<Options>`, `<Options, Frame, State>`, or `<Options, Token>`. */
static fromRoute<Options extends TranscriptionOptions>(
route: TranscriptionModel.InlineRouteInput<Options>,
route: MediaModel.InlineRouteInput<TranscriptionRequestFor<Options>, TranscriptionResponse>,
input: MediaRoute.ModelInput,
): TranscriptionModel<Options>
static fromRoute<Options extends TranscriptionOptions, Frame, State>(
route: TranscriptionModel.StreamRouteInput<Options, Frame, State>,
route: MediaModel.StreamRouteInput<TranscriptionRequestFor<Options>, TranscriptionEvent, Frame, State>,
input: MediaRoute.ModelInput,
): TranscriptionModel<Options>
static fromRoute<Options extends TranscriptionOptions, Token>(
route: TranscriptionModel.QueuedRouteInput<Options, Token>,
route: MediaModel.QueuedRouteInput<TranscriptionRequestFor<Options>, TranscriptionResponse, Token>,
input: MediaRoute.ModelInput,
): TranscriptionModel<Options>
static fromRoute<Options extends TranscriptionOptions, Frame, State, Token>(
route: TranscriptionModel.RouteInput<Options, Frame, State, Token>,
route: MediaModel.AnyRouteInput<
TranscriptionRequestFor<Options>,
TranscriptionEvent,
TranscriptionResponse,
Frame,
State,
Token
>,
input: MediaRoute.ModelInput,
) {
return new TranscriptionModel<Options>({
id: input.id,
provider: route.protocol.provider,
http: input.http,
route: composeAnyRoute(route, input, collectResponse),
route: composeRoute(route, input, collectResponse) as TranscriptionRoute,
})
}
}
export namespace TranscriptionModel {
export type InlineRouteInput<Options extends TranscriptionOptions = TranscriptionOptions> = MediaModel.RouteInput<
TranscriptionRequestFor<Options>,
MediaProtocol.Inline<TranscriptionRequestFor<Options>, TranscriptionResponse>
>
export type StreamRouteInput<
Options extends TranscriptionOptions = TranscriptionOptions,
Frame = unknown,
State = unknown,
> = MediaModel.RouteInput<
MediaProtocol.Addressed<TranscriptionRequestFor<Options>>,
MediaProtocol.Streamed<TranscriptionRequestFor<Options>, TranscriptionEvent, Frame, State>
>
export type QueuedRouteInput<
Options extends TranscriptionOptions = TranscriptionOptions,
Token = unknown,
> = MediaModel.RouteInput<
TranscriptionRequestFor<Options>,
MediaProtocol.Queued<TranscriptionRequestFor<Options>, TranscriptionResponse, Token>
>
export type RouteInput<
Options extends TranscriptionOptions = TranscriptionOptions,
Frame = unknown,
State = unknown,
Token = unknown,
> = MediaModel.AnyRouteInput<
TranscriptionRequestFor<Options>,
TranscriptionEvent,
TranscriptionResponse,
Frame,
State,
Token
>
}
export const TranscriptionModelSchema = Schema.declare(
(value): value is TranscriptionModel => value instanceof TranscriptionModel,
{ expected: "Transcription.Model" },
@@ -212,10 +170,6 @@ export const TranscriptionEvent = Object.assign(transcriptionEventTagged, {
})
export type TranscriptionEvent = Schema.Schema.Type<typeof transcriptionEventTagged>
export const responseEvents = (response: TranscriptionResponse): ReadonlyArray<TranscriptionEvent> => [
TranscriptionFinishEvent.make({ ...response }),
]
const collectResponse = (events: ReadonlyArray<TranscriptionEvent>): Effect.Effect<TranscriptionResponse> => {
const finish = events.find(TranscriptionEvent.is.finish)
// Every transcription protocol's `finish` emits the terminal event or fails, so a completed stream always has one.
@@ -243,11 +197,7 @@ export function request(input: TranscriptionRequest | TranscriptionRequestInput)
const requestEffect = (input: TranscriptionRequest | TranscriptionRequestInput) => tryRequest(() => request(input))
export function generate<const Model extends TranscriptionModel>(
input: TranscriptionRequestInput<Model>,
options?: AwaitOptions,
): Effect.Effect<TranscriptionResponse, AIError, Service>
export function generate(
input: TranscriptionRequest,
input: TranscriptionRequest | TranscriptionRequestInput<Model>,
options?: AwaitOptions,
): Effect.Effect<TranscriptionResponse, AIError, Service>
export function generate(input: TranscriptionRequest | TranscriptionRequestInput, options?: AwaitOptions) {
@@ -255,11 +205,7 @@ export function generate(input: TranscriptionRequest | TranscriptionRequestInput
}
export function stream<const Model extends TranscriptionModel>(
input: TranscriptionRequestInput<Model>,
options?: AwaitOptions,
): Stream.Stream<TranscriptionEvent, AIError, Service>
export function stream(
input: TranscriptionRequest,
input: TranscriptionRequest | TranscriptionRequestInput<Model>,
options?: AwaitOptions,
): Stream.Stream<TranscriptionEvent, AIError, Service>
export function stream(input: TranscriptionRequest | TranscriptionRequestInput, options?: AwaitOptions) {
@@ -268,15 +214,14 @@ export function stream(input: TranscriptionRequest | TranscriptionRequestInput,
/** Inline and streaming routes fail with `UnsupportedOperation`. */
export function start<const Model extends TranscriptionModel>(
input: TranscriptionRequestInput<Model>,
input: TranscriptionRequest | TranscriptionRequestInput<Model>,
): Effect.Effect<Generation<TranscriptionResponse>, AIError, Service>
export function start(input: TranscriptionRequest): Effect.Effect<Generation<TranscriptionResponse>, AIError, Service>
export function start(input: TranscriptionRequest | TranscriptionRequestInput) {
return requestEffect(input).pipe(Effect.flatMap((request) => TranscriptionClient.start(request)))
}
export const resume = <Options extends TranscriptionOptions>(
model: TranscriptionModel<Options>,
export const resume = (
model: TranscriptionModel,
token: unknown,
): Effect.Effect<Generation<TranscriptionResponse>, AIError, Service> => TranscriptionClient.resume(model, token)
+16 -84
View File
@@ -1,98 +1,30 @@
import { Context, Effect, Layer, Stream } from "effect"
import { resultEvents, type AwaitOptions, type Generation } from "./generation.js"
import { RequestExecutor } from "./route/executor.js"
import type { AIError } from "./schema/index.js"
import { Context } from "effect"
import { MediaClient } from "./media-client.js"
import {
responseEvents,
VideoOutputEvent,
VideoFinishEvent,
type VideoEvent,
type VideoModel,
type VideoOptions,
type VideoRequestFor,
type VideoResponse,
} from "./video.js"
export interface Interface {
readonly start: <Options extends VideoOptions>(
request: VideoRequestFor<Options>,
) => Effect.Effect<Generation<VideoResponse>, AIError>
readonly resume: <Options extends VideoOptions>(
model: VideoModel<Options>,
token: unknown,
) => Effect.Effect<Generation<VideoResponse>, AIError>
readonly generate: <Options extends VideoOptions>(
request: VideoRequestFor<Options>,
options?: AwaitOptions,
) => Effect.Effect<VideoResponse, AIError>
readonly stream: <Options extends VideoOptions>(
request: VideoRequestFor<Options>,
options?: AwaitOptions,
) => Stream.Stream<VideoEvent, AIError>
}
export type Interface = MediaClient.Interface<VideoRequestFor, VideoEvent, VideoResponse>
export class VideoClientService extends Context.Service<VideoClientService, Interface>()("@opencode/VideoClient") {}
export const Service = VideoClientService
export type Service = VideoClientService
export const start = <Options extends VideoOptions>(
request: VideoRequestFor<Options>,
): Effect.Effect<Generation<VideoResponse>, AIError, Service> =>
Effect.gen(function* () {
const client = yield* Service
return yield* client.start(request)
})
export const resume = <Options extends VideoOptions>(
model: VideoModel<Options>,
token: unknown,
): Effect.Effect<Generation<VideoResponse>, AIError, Service> =>
Effect.gen(function* () {
const client = yield* Service
return yield* client.resume(model, token)
})
export const generate = <Options extends VideoOptions>(
request: VideoRequestFor<Options>,
options?: AwaitOptions,
): Effect.Effect<VideoResponse, AIError, Service> =>
Effect.gen(function* () {
const client = yield* Service
return yield* client.generate(request, options)
})
export const stream = <Options extends VideoOptions>(
request: VideoRequestFor<Options>,
options?: AwaitOptions,
): Stream.Stream<VideoEvent, AIError, Service> =>
Stream.unwrap(
Effect.gen(function* () {
const client = yield* Service
return client.stream(request, options)
}),
)
export const layer: Layer.Layer<Service, never, RequestExecutor.Service> = Layer.effect(
Service,
Effect.gen(function* () {
const executor = yield* RequestExecutor.Service
const start = <Options extends VideoOptions>(request: VideoRequestFor<Options>) =>
request.model.route.start(request, executor.execute)
return Service.of({
start,
resume: (model, token) => model.route.resume(model, token, executor.execute),
generate: (request, options) => start(request).pipe(Effect.flatMap((generation) => generation.await(options))),
stream: (request, options) =>
Stream.unwrap(
start(request).pipe(Effect.map((generation) => resultEvents(generation, responseEvents, options))),
),
})
}),
)
export const VideoClient = {
Service,
layer,
start,
resume,
generate,
stream,
...MediaClient.make(Service, {
modality: "video",
responseEvents: (response: VideoResponse) => [
...response.videos.map((video, index) => VideoOutputEvent.make({ index, video })),
VideoFinishEvent.make({
usage: response.usage,
notices: response.notices,
providerMetadata: response.providerMetadata,
}),
],
}),
} as const
+37 -41
View File
@@ -3,7 +3,6 @@ import { Generation, ProgressEvent, QueuedEvent, type AwaitOptions } from "./gen
import { Media } from "./media.js"
import { MediaModel, composeRoute, tryRequest } from "./media-model.js"
import { MediaRoute } from "./route/media.js"
import type { MediaProtocol } from "./route/media-protocol.js"
import { AIError, HttpOptions, MediaUsage, ProviderMetadata, type OpenString } from "./schema/index.js"
import { VideoClient, Service } from "./video-client.js"
@@ -11,41 +10,39 @@ import { VideoClient, Service } from "./video-client.js"
// Model
// ---------------------------------------------------------------------------
export type VideoOptions = Record<string, unknown>
export type VideoOptions = MediaModel.Options
export type VideoRoute<Options extends VideoOptions = VideoOptions> = MediaRoute.QueuedRoute<
VideoRequestFor<Options>,
VideoResponse
>
export type VideoRoute = MediaRoute.AnyRoute<VideoRequestFor, VideoEvent, VideoResponse>
export class VideoModel<Options extends VideoOptions = VideoOptions> extends MediaModel<VideoRoute<Options>, Options> {
export class VideoModel<Options extends VideoOptions = VideoOptions> extends MediaModel<VideoRoute, Options> {
declare protected readonly _VideoModel: void
static make<Options extends VideoOptions = VideoOptions>(input: MediaModel.Input<VideoRoute<Options>>) {
return new VideoModel<Options>(input)
}
/** Compose a queued video protocol with its canonical start path into a model for one deployment. */
static fromRoute<Options extends VideoOptions = VideoOptions, Token = unknown>(
route: VideoModel.RouteInput<Options, Token>,
/** The number of type arguments selects the kind: `<Options>`, `<Options, Frame, State>`, or `<Options, Token>`. */
static fromRoute<Options extends VideoOptions>(
route: MediaModel.InlineRouteInput<VideoRequestFor<Options>, VideoResponse>,
input: MediaRoute.ModelInput,
): VideoModel<Options>
static fromRoute<Options extends VideoOptions, Frame, State>(
route: MediaModel.StreamRouteInput<VideoRequestFor<Options>, VideoEvent, Frame, State>,
input: MediaRoute.ModelInput,
): VideoModel<Options>
static fromRoute<Options extends VideoOptions, Token>(
route: MediaModel.QueuedRouteInput<VideoRequestFor<Options>, VideoResponse, Token>,
input: MediaRoute.ModelInput,
): VideoModel<Options>
static fromRoute<Options extends VideoOptions, Frame, State, Token>(
route: MediaModel.AnyRouteInput<VideoRequestFor<Options>, VideoEvent, VideoResponse, Frame, State, Token>,
input: MediaRoute.ModelInput,
) {
return new VideoModel<Options>({
id: input.id,
provider: route.protocol.provider,
http: input.http,
route: composeRoute(MediaRoute.queued, route, input),
route: composeRoute(route, input, collectResponse) as VideoRoute,
})
}
}
export namespace VideoModel {
export type RouteInput<Options extends VideoOptions = VideoOptions, Token = unknown> = MediaModel.RouteInput<
VideoRequestFor<Options>,
MediaProtocol.Queued<VideoRequestFor<Options>, VideoResponse, Token>
>
}
export const VideoModelSchema = Schema.declare((value): value is VideoModel => value instanceof VideoModel, {
expected: "Video.Model",
})
@@ -149,15 +146,19 @@ export const VideoEvent = Object.assign(videoEventTagged, {
})
export type VideoEvent = Schema.Schema.Type<typeof videoEventTagged>
/** A completed response expanded into the streaming event shape. */
export const responseEvents = (response: VideoResponse): ReadonlyArray<VideoEvent> => [
...response.videos.map((video, index) => VideoOutputEvent.make({ index, video })),
VideoFinishEvent.make({
usage: response.usage,
notices: response.notices,
providerMetadata: response.providerMetadata,
}),
]
const collectResponse = (events: ReadonlyArray<VideoEvent>): Effect.Effect<VideoResponse> => {
const finish = events.find(VideoEvent.is.finish)
// A streaming video protocol's `finish` emits the terminal event or fails, so a completed stream always has one.
if (finish === undefined) return Effect.die(new Error("The video stream completed without a finish event"))
return Effect.succeed(
new VideoResponse({
videos: events.filter(VideoEvent.is.video).map((event) => event.video),
usage: finish.usage,
notices: finish.notices,
providerMetadata: finish.providerMetadata,
}),
)
}
// ---------------------------------------------------------------------------
// Request-shaped call API
@@ -178,33 +179,28 @@ export function request(input: VideoRequest | VideoRequestInput) {
const requestEffect = (input: VideoRequest | VideoRequestInput) => tryRequest(() => request(input))
export function start<const Model extends VideoModel>(
input: VideoRequestInput<Model>,
input: VideoRequest | VideoRequestInput<Model>,
): Effect.Effect<Generation<VideoResponse>, AIError, Service>
export function start(input: VideoRequest): Effect.Effect<Generation<VideoResponse>, AIError, Service>
export function start(input: VideoRequest | VideoRequestInput) {
return requestEffect(input).pipe(Effect.flatMap((request) => VideoClient.start(request)))
}
export function generate<const Model extends VideoModel>(
input: VideoRequestInput<Model>,
input: VideoRequest | VideoRequestInput<Model>,
options?: AwaitOptions,
): Effect.Effect<VideoResponse, AIError, Service>
export function generate(input: VideoRequest, options?: AwaitOptions): Effect.Effect<VideoResponse, AIError, Service>
export function generate(input: VideoRequest | VideoRequestInput, options?: AwaitOptions) {
return requestEffect(input).pipe(Effect.flatMap((request) => VideoClient.generate(request, options)))
}
/** Rebuild a generation handle from a persisted `Generation.token`, refreshing its status once. */
export const resume = <Options extends VideoOptions>(
model: VideoModel<Options>,
token: unknown,
): Effect.Effect<Generation<VideoResponse>, AIError, Service> => VideoClient.resume(model, token)
export const resume = (model: VideoModel, token: unknown): Effect.Effect<Generation<VideoResponse>, AIError, Service> =>
VideoClient.resume(model, token)
export function stream<const Model extends VideoModel>(
input: VideoRequestInput<Model>,
input: VideoRequest | VideoRequestInput<Model>,
options?: AwaitOptions,
): Stream.Stream<VideoEvent, AIError, Service>
export function stream(input: VideoRequest, options?: AwaitOptions): Stream.Stream<VideoEvent, AIError, Service>
export function stream(input: VideoRequest | VideoRequestInput, options?: AwaitOptions) {
return Stream.unwrap(requestEffect(input).pipe(Effect.map((request) => VideoClient.stream(request, options))))
}
+13
View File
@@ -54,6 +54,19 @@ import { TestLLM } from "@opencode/ai/testing"
import { Evaluation, EvaluationClient } from "@opencode/ai/experimental"
describe("public exports", () => {
test("modality, provider, and protocol entrypoints load first in a fresh process", async () => {
const results = await Promise.all(
["image", "video", "speech", "transcription", "providers", "protocols"].map(async (entry) => {
const child = Bun.spawn(
[process.execPath, "-e", `await import(${JSON.stringify(`${import.meta.dir}/../src/${entry}.ts`)})`],
{ stderr: "pipe" },
)
return { entry, exitCode: await child.exited, stderr: await new Response(child.stderr).text() }
}),
)
expect(results.filter((result) => result.exitCode !== 0)).toEqual([])
})
test("root exposes app-facing runtime APIs", () => {
expect(LLM.request).toBeFunction()
expect(LLMClient.Service).toBeFunction()
@@ -30,7 +30,7 @@
{
"direction": "client",
"kind": "text",
"body": "{\"type\":\"response.create\",\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"What is the weather in Paris?\"}]}],\"instructions\":\"Call get_weather once, then reply exactly: Paris is sunny.\",\"tools\":[{\"type\":\"function\",\"name\":\"get_weather\",\"description\":\"Get current weather for a city.\",\"parameters\":{\"type\":\"object\",\"properties\":{\"city\":{\"type\":\"string\"}},\"required\":[\"city\"],\"additionalProperties\":false},\"strict\":false}],\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"medium\",\"summary\":\"auto\"},\"text\":{\"verbosity\":\"low\"},\"max_output_tokens\":50}"
"body": "{\"type\":\"response.create\",\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"What is the weather in Paris?\"}]}],\"instructions\":\"Call get_weather once, then reply exactly: Paris is sunny.\",\"tools\":[{\"type\":\"function\",\"name\":\"get_weather\",\"description\":\"Get current weather for a city.\",\"parameters\":{\"type\":\"object\",\"properties\":{\"city\":{\"type\":\"string\"}},\"required\":[\"city\"],\"additionalProperties\":false},\"strict\":false}],\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"medium\",\"summary\":\"auto\"},\"max_output_tokens\":50}"
},
{
"direction": "server",
@@ -525,7 +525,7 @@
{
"direction": "client",
"kind": "text",
"body": "{\"type\":\"response.create\",\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"function_call_output\",\"call_id\":\"call_hejtTYDa1IfLyNIzb3fq9gJs\",\"output\":\"{\\\"temperature\\\":22,\\\"condition\\\":\\\"sunny\\\"}\"}],\"instructions\":\"Call get_weather once, then reply exactly: Paris is sunny.\",\"tools\":[{\"type\":\"function\",\"name\":\"get_weather\",\"description\":\"Get current weather for a city.\",\"parameters\":{\"type\":\"object\",\"properties\":{\"city\":{\"type\":\"string\"}},\"required\":[\"city\"],\"additionalProperties\":false},\"strict\":false}],\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"medium\",\"summary\":\"auto\"},\"text\":{\"verbosity\":\"low\"},\"max_output_tokens\":50,\"previous_response_id\":\"resp_0d9a44b6df400533016aa8c8e36de887d1be260913d131b2ca\"}"
"body": "{\"type\":\"response.create\",\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"function_call_output\",\"call_id\":\"call_hejtTYDa1IfLyNIzb3fq9gJs\",\"output\":\"{\\\"temperature\\\":22,\\\"condition\\\":\\\"sunny\\\"}\"}],\"instructions\":\"Call get_weather once, then reply exactly: Paris is sunny.\",\"tools\":[{\"type\":\"function\",\"name\":\"get_weather\",\"description\":\"Get current weather for a city.\",\"parameters\":{\"type\":\"object\",\"properties\":{\"city\":{\"type\":\"string\"}},\"required\":[\"city\"],\"additionalProperties\":false},\"strict\":false}],\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"medium\",\"summary\":\"auto\"},\"max_output_tokens\":50,\"previous_response_id\":\"resp_0d9a44b6df400533016aa8c8e36de887d1be260913d131b2ca\"}"
},
{
"direction": "server",
@@ -30,7 +30,7 @@
{
"direction": "client",
"kind": "text",
"body": "{\"type\":\"response.create\",\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"Reply exactly: Alpha.\"}]}],\"instructions\":\"Follow the user's exact reply instruction.\",\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"medium\",\"summary\":\"auto\"},\"text\":{\"verbosity\":\"low\"},\"max_output_tokens\":30}"
"body": "{\"type\":\"response.create\",\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"Reply exactly: Alpha.\"}]}],\"instructions\":\"Follow the user's exact reply instruction.\",\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"medium\",\"summary\":\"auto\"},\"max_output_tokens\":30}"
},
{
"direction": "server",
@@ -109,7 +109,7 @@
{
"direction": "client",
"kind": "text",
"body": "{\"type\":\"response.create\",\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"Reply exactly: Alpha.\"}]},{\"type\":\"reasoning\",\"id\":\"rs_0a6277dd90b94da1016aa8c946e33487d1b725d8e9dc874d82\",\"summary\":[],\"encrypted_content\":\"gAAAAABqqMlHtG6yltzgW_UjUcYxvl2hoMkk7cSEH5SJMe9CR5gKIaCwCh4peUo8XZrd-EU-tPCXthv7JzHxYXGVG1fIgTJ7BjBoOh-jgd_oGlyCHj2jVPj7nVoB763tZHdyw_ovL3V7GpJ6VeLGIsRGqNTnBz47ZuKikTKrbTDupn6fT2wOM_69zwdg4-UVnoF2J9E_jIS0E5XRtRwOidqANl61HCwo7LV-Ut1aqCbXb-59vkrVGWPxD_8n4smf7Qjywc0FCH9zwuEDX4cPdqj3MmjvYPrn0jand9LKkAy9rblaKFJSQfVuritbHrdQrnF7hLxu7QCpehOzNXOkpNEqTmXnAZjfMc53hq-ahH6_KoyYlZpompwyGPngFeI2fKRqP8rlpDilD1BHBsh-bl7kXzI8HQ_jameXwPZ1La6gjtFkThXp53BD6BOx11SB9Nypqolu2at5rR32UYcGrYeGtTu-HmYGp0oHVFkHumTNuHKmGXV14dI9swgryfygLhX8EAJrWrjm3e8rvAkHKpAZ0IkmlCcp15UCFDKNeS560fQVRaKXWnQ7m0Ih3C1xG0ifJ89j27c8GHo1kAhEJk-lSB1-FZr9Ls_w7N772kmZ2a7LLswu3kNW78kPas_CtcOnBOHwE1DhcVh5YpxwNftOHZnK1v8NPNF8EWqio4ArZy1thMCzH5zXBNcFzgd8tePMFukBblSP8QGQoVYRqTne2sFoOZXjslXnDvEe-Ycj8X38zWgRiwAk7guOloFC7Se36KCDP357773Vah86gWCt55mSEyhVW_GF1oTuHvJ18GZsXcyN21scF1PSr8YaCM_jR7ZkU1GYXbQK2Y4oTAV9XDptQA5YzEREnn7muC_6v5ZTAglZF1lhn9Q0NwmylZEAXJdSGHaqXt1Hv-vQlprA_9m22vrreBOTLPnVK946J8absKrwfe-jK_1n_9YQR43uwH8XwFBFND0c4lICCQGbxwM8pX4ACWR0c19aORCYm-M5FrJsxmG29_aDVNhcvkoQ3mlP7ITQeqzkrjfytSwLb2BYpXYZKjEHNfV9j3JoxJobUkK5hrxXBhTvZzbVBnE0LSXQNwR-JAcOliP_jXBEeQ-28B-aGW8TI1vEP2i260QKgzOzPC-pOoFp0-vCvxojNO0kE16ECVTLfDNLNGSMZs9vekdLJP2akBp6PsUXsQUTbWO_wr1E2oNU7ctfMRxoh-yP0ZW2_xz_NjE72O5LF_6zuDVV7Q==\"},{\"type\":\"message\",\"id\":\"msg_0a6277dd90b94da1016aa8c947253887d184c150fcbcbcd8da\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Alpha.\"}],\"phase\":\"final_answer\",\"status\":\"completed\"},{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"Reply exactly: Beta.\"}]}],\"instructions\":\"Follow the user's exact reply instruction.\",\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"medium\",\"summary\":\"auto\"},\"text\":{\"verbosity\":\"low\"},\"max_output_tokens\":30}"
"body": "{\"type\":\"response.create\",\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"Reply exactly: Alpha.\"}]},{\"type\":\"reasoning\",\"id\":\"rs_0a6277dd90b94da1016aa8c946e33487d1b725d8e9dc874d82\",\"summary\":[],\"encrypted_content\":\"gAAAAABqqMlHtG6yltzgW_UjUcYxvl2hoMkk7cSEH5SJMe9CR5gKIaCwCh4peUo8XZrd-EU-tPCXthv7JzHxYXGVG1fIgTJ7BjBoOh-jgd_oGlyCHj2jVPj7nVoB763tZHdyw_ovL3V7GpJ6VeLGIsRGqNTnBz47ZuKikTKrbTDupn6fT2wOM_69zwdg4-UVnoF2J9E_jIS0E5XRtRwOidqANl61HCwo7LV-Ut1aqCbXb-59vkrVGWPxD_8n4smf7Qjywc0FCH9zwuEDX4cPdqj3MmjvYPrn0jand9LKkAy9rblaKFJSQfVuritbHrdQrnF7hLxu7QCpehOzNXOkpNEqTmXnAZjfMc53hq-ahH6_KoyYlZpompwyGPngFeI2fKRqP8rlpDilD1BHBsh-bl7kXzI8HQ_jameXwPZ1La6gjtFkThXp53BD6BOx11SB9Nypqolu2at5rR32UYcGrYeGtTu-HmYGp0oHVFkHumTNuHKmGXV14dI9swgryfygLhX8EAJrWrjm3e8rvAkHKpAZ0IkmlCcp15UCFDKNeS560fQVRaKXWnQ7m0Ih3C1xG0ifJ89j27c8GHo1kAhEJk-lSB1-FZr9Ls_w7N772kmZ2a7LLswu3kNW78kPas_CtcOnBOHwE1DhcVh5YpxwNftOHZnK1v8NPNF8EWqio4ArZy1thMCzH5zXBNcFzgd8tePMFukBblSP8QGQoVYRqTne2sFoOZXjslXnDvEe-Ycj8X38zWgRiwAk7guOloFC7Se36KCDP357773Vah86gWCt55mSEyhVW_GF1oTuHvJ18GZsXcyN21scF1PSr8YaCM_jR7ZkU1GYXbQK2Y4oTAV9XDptQA5YzEREnn7muC_6v5ZTAglZF1lhn9Q0NwmylZEAXJdSGHaqXt1Hv-vQlprA_9m22vrreBOTLPnVK946J8absKrwfe-jK_1n_9YQR43uwH8XwFBFND0c4lICCQGbxwM8pX4ACWR0c19aORCYm-M5FrJsxmG29_aDVNhcvkoQ3mlP7ITQeqzkrjfytSwLb2BYpXYZKjEHNfV9j3JoxJobUkK5hrxXBhTvZzbVBnE0LSXQNwR-JAcOliP_jXBEeQ-28B-aGW8TI1vEP2i260QKgzOzPC-pOoFp0-vCvxojNO0kE16ECVTLfDNLNGSMZs9vekdLJP2akBp6PsUXsQUTbWO_wr1E2oNU7ctfMRxoh-yP0ZW2_xz_NjE72O5LF_6zuDVV7Q==\"},{\"type\":\"message\",\"id\":\"msg_0a6277dd90b94da1016aa8c947253887d184c150fcbcbcd8da\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Alpha.\"}],\"phase\":\"final_answer\",\"status\":\"completed\"},{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"Reply exactly: Beta.\"}]}],\"instructions\":\"Follow the user's exact reply instruction.\",\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"medium\",\"summary\":\"auto\"},\"max_output_tokens\":30}"
},
{
"direction": "server",
@@ -30,7 +30,7 @@
{
"direction": "client",
"kind": "text",
"body": "{\"type\":\"response.create\",\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"Reply exactly: Ready.\"}]}],\"instructions\":\"Follow the user's exact reply instruction.\",\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"medium\",\"summary\":\"auto\"},\"text\":{\"verbosity\":\"low\"},\"max_output_tokens\":30}"
"body": "{\"type\":\"response.create\",\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"Reply exactly: Ready.\"}]}],\"instructions\":\"Follow the user's exact reply instruction.\",\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"medium\",\"summary\":\"auto\"},\"max_output_tokens\":30}"
},
{
"direction": "server",
@@ -109,7 +109,7 @@
{
"direction": "client",
"kind": "text",
"body": "{\"type\":\"response.create\",\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"Reply exactly: Recovered.\"}]}],\"instructions\":\"Follow the user's exact reply instruction.\",\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"medium\",\"summary\":\"auto\"},\"text\":{\"verbosity\":\"low\"},\"max_output_tokens\":30,\"previous_response_id\":\"resp_01cc0cda24c36acf016aa8ca3c1d3c87d1853283f43675e411\"}"
"body": "{\"type\":\"response.create\",\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"Reply exactly: Recovered.\"}]}],\"instructions\":\"Follow the user's exact reply instruction.\",\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"medium\",\"summary\":\"auto\"},\"max_output_tokens\":30,\"previous_response_id\":\"resp_01cc0cda24c36acf016aa8ca3c1d3c87d1853283f43675e411\"}"
},
{
"direction": "server",
@@ -119,7 +119,7 @@
{
"direction": "client",
"kind": "text",
"body": "{\"type\":\"response.create\",\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"Reply exactly: Ready.\"}]},{\"type\":\"reasoning\",\"id\":\"rs_01cc0cda24c36acf016aa8ca3ced8c87d1a14c5c6a2ced8544\",\"summary\":[],\"encrypted_content\":\"gAAAAABqqMo9M9B15LsV1CLsXpNJpI08GiwCAK97hTheQ2s1lAkKgARMs1HIIXVeAr-tbUq88yN51fiQE8HvGwdF9ZcNI6bZle8D2PS7m8O7UE7C1Z_HX_fz2f0pRHyRfWpVAT2gYQWIDeZu6UdgxbKPBW0lXWzlk_relHG5x6nXkYzZoEeVasqavWoyMMSX7cexe-IYJh6e_3DgRpOchueS8Z-70P0w5R83Ea7UXZQNhMA0yDEYu_td2PmE2Pd2PUTOB3mxF2pb1z7-2t6S0UryhHx0az7Gh2eT60GGUqz9CIZzNE_FX--tszeuO0eI92Cen5tirOUHBTyyDqE0eG26DRl_p_U-xDZwaQONbtYbvkvrj-G7FA2oZXxjJPHuQZsNgBslXS-H0KT1lx3Y8XJ9QMVjFLFaucFG64wCmXPfCH8dYtX_YqfYQR4lwNfiSbyJEX2oDvTVVD_aCJ9NRo7c0aCTtmKBvr6fvvAy3MAFxAp_Sm2nMx4P5GYO4qAmJDByywKw-VK1vHlv3NRmVsAgbArIFgm-axoCs2PLpvZjDqeQGPavaq8zKWTyZYqBsEzKZUtGOZfYjD4mud0Z08I4i2H4K-L00ccVauode3548ZipOIuslbhJxonQXsF6TFdW2Hj8E5JjoEr5IbmwHyI0PBcDWW5AmkjHLwr9v08mFppRoD-2wzPAd5igROAuUbvJhiQd2A-uOaohwMjdpFxrjyUqgGTlI5g7tmI1ceeQWms0bKm8Pd0wIqVM3Nq6YvV7XfEyeogfRMVUQezr_lES42ZMVAoKBSFzMysDwCFkhVNVclcTUpcUUbVp21FChG7Ag-xuq8Cl6OGLA8nWX1C0aCf2HNa-n3dkYr1DtUziurh1MD-UIs5jdGiq3ptrc0VaVZwNdD4jVfAoHB_Ws7GiISXuclfpqsG3DTJEfzlbukI1vxXrt3FArsHiQvQjW5UM7gGel32M6p8AlXRxnez9PgIuU1WrtBUJetk7m39AZwp_aqbqC-AJ-MF70xP1VJZwFN-GeNL3VZsRHePFG4h7Pj---CCZRGlmzuE1-b-sIE7Bn_gbue_qHFMZJhAY55MO25vfZbSoLWHZCGmLMjJVHOYCPoy6l7zyxNmoIcC58QILNWal31KLCDmCsASmZC-xQRjyFwt-kvLbvk38Dc02IKcP3ujhf6WRRr1A0hh1K-gXmv_XI_MUFDcOVIjjhB-rXgjWSCoKfaKI9GCIsb9mnNaBq65BWg==\"},{\"type\":\"message\",\"id\":\"msg_01cc0cda24c36acf016aa8ca3d4a0087d1950121aed4802434\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Ready.\"}],\"phase\":\"final_answer\",\"status\":\"completed\"},{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"Reply exactly: Recovered.\"}]}],\"instructions\":\"Follow the user's exact reply instruction.\",\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"medium\",\"summary\":\"auto\"},\"text\":{\"verbosity\":\"low\"},\"max_output_tokens\":30}"
"body": "{\"type\":\"response.create\",\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"Reply exactly: Ready.\"}]},{\"type\":\"reasoning\",\"id\":\"rs_01cc0cda24c36acf016aa8ca3ced8c87d1a14c5c6a2ced8544\",\"summary\":[],\"encrypted_content\":\"gAAAAABqqMo9M9B15LsV1CLsXpNJpI08GiwCAK97hTheQ2s1lAkKgARMs1HIIXVeAr-tbUq88yN51fiQE8HvGwdF9ZcNI6bZle8D2PS7m8O7UE7C1Z_HX_fz2f0pRHyRfWpVAT2gYQWIDeZu6UdgxbKPBW0lXWzlk_relHG5x6nXkYzZoEeVasqavWoyMMSX7cexe-IYJh6e_3DgRpOchueS8Z-70P0w5R83Ea7UXZQNhMA0yDEYu_td2PmE2Pd2PUTOB3mxF2pb1z7-2t6S0UryhHx0az7Gh2eT60GGUqz9CIZzNE_FX--tszeuO0eI92Cen5tirOUHBTyyDqE0eG26DRl_p_U-xDZwaQONbtYbvkvrj-G7FA2oZXxjJPHuQZsNgBslXS-H0KT1lx3Y8XJ9QMVjFLFaucFG64wCmXPfCH8dYtX_YqfYQR4lwNfiSbyJEX2oDvTVVD_aCJ9NRo7c0aCTtmKBvr6fvvAy3MAFxAp_Sm2nMx4P5GYO4qAmJDByywKw-VK1vHlv3NRmVsAgbArIFgm-axoCs2PLpvZjDqeQGPavaq8zKWTyZYqBsEzKZUtGOZfYjD4mud0Z08I4i2H4K-L00ccVauode3548ZipOIuslbhJxonQXsF6TFdW2Hj8E5JjoEr5IbmwHyI0PBcDWW5AmkjHLwr9v08mFppRoD-2wzPAd5igROAuUbvJhiQd2A-uOaohwMjdpFxrjyUqgGTlI5g7tmI1ceeQWms0bKm8Pd0wIqVM3Nq6YvV7XfEyeogfRMVUQezr_lES42ZMVAoKBSFzMysDwCFkhVNVclcTUpcUUbVp21FChG7Ag-xuq8Cl6OGLA8nWX1C0aCf2HNa-n3dkYr1DtUziurh1MD-UIs5jdGiq3ptrc0VaVZwNdD4jVfAoHB_Ws7GiISXuclfpqsG3DTJEfzlbukI1vxXrt3FArsHiQvQjW5UM7gGel32M6p8AlXRxnez9PgIuU1WrtBUJetk7m39AZwp_aqbqC-AJ-MF70xP1VJZwFN-GeNL3VZsRHePFG4h7Pj---CCZRGlmzuE1-b-sIE7Bn_gbue_qHFMZJhAY55MO25vfZbSoLWHZCGmLMjJVHOYCPoy6l7zyxNmoIcC58QILNWal31KLCDmCsASmZC-xQRjyFwt-kvLbvk38Dc02IKcP3ujhf6WRRr1A0hh1K-gXmv_XI_MUFDcOVIjjhB-rXgjWSCoKfaKI9GCIsb9mnNaBq65BWg==\"},{\"type\":\"message\",\"id\":\"msg_01cc0cda24c36acf016aa8ca3d4a0087d1950121aed4802434\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Ready.\"}],\"phase\":\"final_answer\",\"status\":\"completed\"},{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"Reply exactly: Recovered.\"}]}],\"instructions\":\"Follow the user's exact reply instruction.\",\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"medium\",\"summary\":\"auto\"},\"max_output_tokens\":30}"
},
{
"direction": "server",
File diff suppressed because one or more lines are too long
@@ -26,7 +26,7 @@
"headers": {
"content-type": "application/json"
},
"body": "{\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"Think briefly, then reply exactly with: Hello!\"}]}],\"instructions\":\"Show concise reasoning when the provider supports visible reasoning summaries.\",\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"low\",\"summary\":\"auto\"},\"text\":{\"verbosity\":\"low\"},\"max_output_tokens\":120,\"stream\":true}"
"body": "{\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"Think briefly, then reply exactly with: Hello!\"}]}],\"instructions\":\"Show concise reasoning when the provider supports visible reasoning summaries.\",\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"low\",\"summary\":\"auto\"},\"max_output_tokens\":120,\"stream\":true}"
},
"response": {
"status": 200,
@@ -44,7 +44,7 @@
"headers": {
"content-type": "application/json"
},
"body": "{\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"Think briefly, then reply exactly with: Hello!\"}]},{\"type\":\"reasoning\",\"id\":\"rs_052e7ec551f55289016aa8c8d63eac87d19d6b921611f123e7\",\"summary\":[],\"encrypted_content\":\"gAAAAABqqMjWZ2Eei8_Gf-5FeEFAYp-gSzFL4D4lQBKL_fyyTYXv5iJ-2jql1mOq0wZpqHL8O9MWxebQGW56Ahd-p21qrDD52CyUBqKKlF87eC1d-cTgjXQlFMsPxvwyQeuU6A2l8tanTtJ48sKtzZtHrDuBXZ35u-lONnovjFGMX3Q83xoqG_um_w5rT420TA_SyU4fGt7oiQvOPS1q4PNo97O824oRnI7n_BC1jPYCaJhl2I1rPJg4afuOpjG-u7JcXRD4JPwZdqMa5o2d0uDKHuUYwP25qiPKKDqTTqFka5cDJjZNPF3ZkVHR-cagjZGvMnizXXxgUpPJ9j83gqY4QJLKCkzcaBj9H7mAL-v9yl4I5kn_9_DhpMILs2SZkC8AvIYNgmel3sDV_BG4XZ2JXciZz86ukQ6DwXgQqS4HOB91g-sGHOWMU1ohsZlEvvJBjGkJ_rAdXVMqAbvi2zvE3_NI4sTAGUrIugGJePrQYTe8gqL8f9NsYac6pzHNQL1e_jQNUvp49bu7EsPzCP3KPYVvZCFohDdwe7sMd6wrztrCJwM4CLdAQK61A7sYzU0HyglLtPidmS5QkSmV6U_xgih7JbKnY0oAeCyYw4ADYqdNTi0axmBErh-lbh-XKNG_TnoMa-2IS3X041N8OsDfSdQp3QsAm53fF8seQuLFa27Iaq2etMj3yGeqWVjA-Mae3K34mt2YPGjQ-HbIOMVmYXBLzNr-s2fT35Sp6SDEsFyvzXb0Vij58s1wW5zkuKgaJiroGgkY86NImuaa3_-wpMK3_9O_wwAbRwV4uBCVzT_rY6rDQHR9-VkM0MbGK8drbdtjXwy3KtAzkux4N4g2nadYU0IIIEUNj_JChUFSHC7VRg7L7LpZMgAFGwHmaUyzQt31LyiVix9WFtcKfBgzehoRV6vstln-oBRd-vFjUW-7WLm7R_lFNHQZ0CKUvKCSpQxdIevczpYT0_lQiDU7Rfp1UBBnicndpq4YQwgRppdZX-QHG5IZxxHNWYdIBtn3eO3ooDXmI-rYryyVcFG6VY5xqssf8GgXqgNy84X_SgYfhGmNiPPQTl5wIBa49f1sxAI-7ri1xLr6FJhQMasOoxm_VuZYNkq6NrAXprtKa70cBRpm1wWBNcPdMul4GNMDQatQZRDENcNLe45iMX-HB1YYSvyc-y5rfSJkP4EgKXC-OqcrzxAlRfB3Bm4jOQL_PSsNBbh26CWAdAYSftM88fbnuqZX2w==\"},{\"type\":\"message\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello!\"}],\"status\":\"completed\"},{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"Now reply exactly with: Done.\"}]}],\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"low\",\"summary\":\"auto\"},\"text\":{\"verbosity\":\"low\"},\"max_output_tokens\":40,\"stream\":true}"
"body": "{\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"Think briefly, then reply exactly with: Hello!\"}]},{\"type\":\"reasoning\",\"id\":\"rs_052e7ec551f55289016aa8c8d63eac87d19d6b921611f123e7\",\"summary\":[],\"encrypted_content\":\"gAAAAABqqMjWZ2Eei8_Gf-5FeEFAYp-gSzFL4D4lQBKL_fyyTYXv5iJ-2jql1mOq0wZpqHL8O9MWxebQGW56Ahd-p21qrDD52CyUBqKKlF87eC1d-cTgjXQlFMsPxvwyQeuU6A2l8tanTtJ48sKtzZtHrDuBXZ35u-lONnovjFGMX3Q83xoqG_um_w5rT420TA_SyU4fGt7oiQvOPS1q4PNo97O824oRnI7n_BC1jPYCaJhl2I1rPJg4afuOpjG-u7JcXRD4JPwZdqMa5o2d0uDKHuUYwP25qiPKKDqTTqFka5cDJjZNPF3ZkVHR-cagjZGvMnizXXxgUpPJ9j83gqY4QJLKCkzcaBj9H7mAL-v9yl4I5kn_9_DhpMILs2SZkC8AvIYNgmel3sDV_BG4XZ2JXciZz86ukQ6DwXgQqS4HOB91g-sGHOWMU1ohsZlEvvJBjGkJ_rAdXVMqAbvi2zvE3_NI4sTAGUrIugGJePrQYTe8gqL8f9NsYac6pzHNQL1e_jQNUvp49bu7EsPzCP3KPYVvZCFohDdwe7sMd6wrztrCJwM4CLdAQK61A7sYzU0HyglLtPidmS5QkSmV6U_xgih7JbKnY0oAeCyYw4ADYqdNTi0axmBErh-lbh-XKNG_TnoMa-2IS3X041N8OsDfSdQp3QsAm53fF8seQuLFa27Iaq2etMj3yGeqWVjA-Mae3K34mt2YPGjQ-HbIOMVmYXBLzNr-s2fT35Sp6SDEsFyvzXb0Vij58s1wW5zkuKgaJiroGgkY86NImuaa3_-wpMK3_9O_wwAbRwV4uBCVzT_rY6rDQHR9-VkM0MbGK8drbdtjXwy3KtAzkux4N4g2nadYU0IIIEUNj_JChUFSHC7VRg7L7LpZMgAFGwHmaUyzQt31LyiVix9WFtcKfBgzehoRV6vstln-oBRd-vFjUW-7WLm7R_lFNHQZ0CKUvKCSpQxdIevczpYT0_lQiDU7Rfp1UBBnicndpq4YQwgRppdZX-QHG5IZxxHNWYdIBtn3eO3ooDXmI-rYryyVcFG6VY5xqssf8GgXqgNy84X_SgYfhGmNiPPQTl5wIBa49f1sxAI-7ri1xLr6FJhQMasOoxm_VuZYNkq6NrAXprtKa70cBRpm1wWBNcPdMul4GNMDQatQZRDENcNLe45iMX-HB1YYSvyc-y5rfSJkP4EgKXC-OqcrzxAlRfB3Bm4jOQL_PSsNBbh26CWAdAYSftM88fbnuqZX2w==\"},{\"type\":\"message\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello!\"}],\"status\":\"completed\"},{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"Now reply exactly with: Done.\"}]}],\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"low\",\"summary\":\"auto\"},\"max_output_tokens\":40,\"stream\":true}"
},
"response": {
"status": 200,
@@ -24,7 +24,7 @@
"headers": {
"content-type": "application/json"
},
"body": "{\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"Think briefly, then reply exactly with: Hello!\"}]}],\"instructions\":\"Show concise reasoning when the provider supports visible reasoning summaries.\",\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"low\",\"summary\":\"auto\"},\"text\":{\"verbosity\":\"low\"},\"max_output_tokens\":120,\"stream\":true}"
"body": "{\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"Think briefly, then reply exactly with: Hello!\"}]}],\"instructions\":\"Show concise reasoning when the provider supports visible reasoning summaries.\",\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"low\",\"summary\":\"auto\"},\"max_output_tokens\":120,\"stream\":true}"
},
"response": {
"status": 200,
@@ -25,7 +25,7 @@
"headers": {
"content-type": "application/json"
},
"body": "{\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"What is the weather in Paris?\"}]}],\"instructions\":\"Use the get_weather tool exactly once. After the tool result, reply exactly: Paris is sunny.\",\"tools\":[{\"type\":\"function\",\"name\":\"get_weather\",\"description\":\"Get current weather for a city.\",\"parameters\":{\"type\":\"object\",\"properties\":{\"city\":{\"type\":\"string\"}},\"required\":[\"city\"],\"additionalProperties\":false},\"strict\":false}],\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"medium\",\"summary\":\"auto\"},\"text\":{\"verbosity\":\"low\"},\"max_output_tokens\":80,\"stream\":true}"
"body": "{\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"What is the weather in Paris?\"}]}],\"instructions\":\"Use the get_weather tool exactly once. After the tool result, reply exactly: Paris is sunny.\",\"tools\":[{\"type\":\"function\",\"name\":\"get_weather\",\"description\":\"Get current weather for a city.\",\"parameters\":{\"type\":\"object\",\"properties\":{\"city\":{\"type\":\"string\"}},\"required\":[\"city\"],\"additionalProperties\":false},\"strict\":false}],\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"medium\",\"summary\":\"auto\"},\"max_output_tokens\":80,\"stream\":true}"
},
"response": {
"status": 200,
@@ -43,7 +43,7 @@
"headers": {
"content-type": "application/json"
},
"body": "{\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"What is the weather in Paris?\"}]},{\"type\":\"function_call\",\"id\":\"fc_09525c04931d1487016aa8c8d8e12887d193bd327a0f313bd7\",\"call_id\":\"call_p57PJbKe0bX44nj908fpHdRn\",\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\":\\\"Paris\\\"}\"},{\"type\":\"function_call_output\",\"call_id\":\"call_p57PJbKe0bX44nj908fpHdRn\",\"output\":\"{\\\"temperature\\\":22,\\\"condition\\\":\\\"sunny\\\"}\"}],\"instructions\":\"Use the get_weather tool exactly once. After the tool result, reply exactly: Paris is sunny.\",\"tools\":[{\"type\":\"function\",\"name\":\"get_weather\",\"description\":\"Get current weather for a city.\",\"parameters\":{\"type\":\"object\",\"properties\":{\"city\":{\"type\":\"string\"}},\"required\":[\"city\"],\"additionalProperties\":false},\"strict\":false}],\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"medium\",\"summary\":\"auto\"},\"text\":{\"verbosity\":\"low\"},\"max_output_tokens\":80,\"stream\":true}"
"body": "{\"model\":\"gpt-5.5\",\"input\":[{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"What is the weather in Paris?\"}]},{\"type\":\"function_call\",\"id\":\"fc_09525c04931d1487016aa8c8d8e12887d193bd327a0f313bd7\",\"call_id\":\"call_p57PJbKe0bX44nj908fpHdRn\",\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\":\\\"Paris\\\"}\"},{\"type\":\"function_call_output\",\"call_id\":\"call_p57PJbKe0bX44nj908fpHdRn\",\"output\":\"{\\\"temperature\\\":22,\\\"condition\\\":\\\"sunny\\\"}\"}],\"instructions\":\"Use the get_weather tool exactly once. After the tool result, reply exactly: Paris is sunny.\",\"tools\":[{\"type\":\"function\",\"name\":\"get_weather\",\"description\":\"Get current weather for a city.\",\"parameters\":{\"type\":\"object\",\"properties\":{\"city\":{\"type\":\"string\"}},\"required\":[\"city\"],\"additionalProperties\":false},\"strict\":false}],\"store\":false,\"include\":[\"reasoning.encrypted_content\"],\"reasoning\":{\"effort\":\"medium\",\"summary\":\"auto\"},\"max_output_tokens\":80,\"stream\":true}"
},
"response": {
"status": 200,
+3 -3
View File
@@ -7,7 +7,6 @@ import {
type ImageModelOptions,
type ImageOptions,
type ImageRequestFor,
type ImageRoute,
} from "../src/index.js"
import type { Service } from "../src/image-client.js"
import { Anthropic, BlackForestLabs, Google, OpenAI, Stability, XAI, ZAI } from "../src/providers.js"
@@ -21,8 +20,7 @@ type GoogleLikeOptions = {
readonly thinkingLevel?: "LOW" | "HIGH"
} & Record<string, unknown>
declare const route: ImageRoute<GoogleLikeOptions>
const google = ImageModel.make<GoogleLikeOptions>({ id: "gemini-image", provider: "google", route })
declare const google: ImageModel<GoogleLikeOptions>
// @ts-expect-error Extracted model options retain known provider fields.
const invalidGoogleOptions: ImageModelOptions<typeof google> = { imageSize: "8K" }
void invalidGoogleOptions
@@ -152,6 +150,8 @@ Image.generate({ model: zai, prompt: "A lighthouse", providerOptions: { quality:
Image.generate({ model: zai, prompt: "A lighthouse", providerOptions: { userID: 1 } })
declare const generic: ImageModel<ImageOptions>
const widenImage = <Options extends ImageOptions>(model: ImageModel<Options>): ImageModel => model
void widenImage
Image.generate({ model: generic, prompt: "A lighthouse", providerOptions: { arbitrary: true } })
const explicitAsset: Media.Asset = Media.url("https://example.com/image.png")
void explicitAsset
@@ -6,6 +6,7 @@ import {
type LanguageModelProviderOptions,
type ProviderOptions,
} from "../src/index.js"
import { ai } from "../src/promise.js"
import { OpenAIChat } from "../src/protocols.js"
interface ExampleOptions {
@@ -31,6 +32,10 @@ const generated = LLM.generate(LLM.request({ model, prompt: "Hello" }))
type GenerateRequirements = Assert<Equal<Requirements<typeof generated>, LLMClientService>>
const streamed = LLM.stream(LLM.request({ model, prompt: "Hello" }))
type StreamClientRequirements = Assert<Equal<StreamRequirements<typeof streamed>, LLMClientService>>
const generatedFromInput = LLM.generate({ model, prompt: "Hello", providerOptions: { mode: "fast" } })
type InputGenerateRequirements = Assert<Equal<Requirements<typeof generatedFromInput>, LLMClientService>>
const streamedFromInput = LLM.stream({ model, prompt: "Hello", providerOptions: { mode: "thorough" } })
type InputStreamRequirements = Assert<Equal<StreamRequirements<typeof streamedFromInput>, LLMClientService>>
LLM.request({
model,
@@ -39,6 +44,11 @@ LLM.request({
providerOptions: { mode: "slow" },
})
// @ts-expect-error Direct input keeps the selected model's provider option types.
LLM.generate({ model, prompt: "Hello", providerOptions: { mode: "slow" } })
// @ts-expect-error Stream input keeps the selected model's provider option types.
LLM.stream({ model, prompt: "Hello", providerOptions: { mode: "slow" } })
const generatedObject = LLM.generateObject({
model,
prompt: "Hello",
@@ -69,5 +79,16 @@ const options: LanguageModelProviderOptions<typeof model> = { mode: "fast" }
void (options satisfies LanguageModelProviderOptions<typeof model>)
void (true satisfies GenerateRequirements)
void (true satisfies StreamClientRequirements)
void (true satisfies InputGenerateRequirements)
void (true satisfies InputStreamRequirements)
void (true satisfies GenerateObjectRequirements)
void (true satisfies GenerateDynamicObjectRequirements)
void ai.llm.generate({ model, prompt: "Hello", providerOptions: { mode: "fast" } })
void ai.llm.stream({ model, prompt: "Hello", providerOptions: { mode: "thorough" } })
void ai.llm.generate(ai.llm.request({ model, prompt: "Hello" }))
void ai.llm.stream(ai.llm.request({ model, prompt: "Hello" }))
// @ts-expect-error Promise direct input keeps the selected model's provider option types.
void ai.llm.generate({ model, prompt: "Hello", providerOptions: { mode: "slow" } })
// @ts-expect-error Promise stream input keeps the selected model's provider option types.
void ai.llm.stream({ model, prompt: "Hello", providerOptions: { mode: "slow" } })
+25 -2
View File
@@ -1,6 +1,7 @@
import { describe, expect, test } from "bun:test"
import { Schema } from "effect"
import { CacheHint, LLM, LLMResponse, ToolEntry, ToolNamespace } from "../src/index.js"
import { Effect, Schema, Stream } from "effect"
import { CacheHint, LLM, LLMEvent, LLMResponse, ToolEntry, ToolNamespace } from "../src/index.js"
import { OpenAI } from "../src/providers.js"
import * as OpenAIChat from "../src/protocols/openai-chat.js"
import * as OpenAIResponses from "../src/protocols/openai-responses.js"
import {
@@ -13,6 +14,8 @@ import {
ToolDefinition,
ToolResultPart,
} from "../src/schema/index.js"
import { fixedResponse } from "./lib/http.js"
import { sseEvents } from "./lib/sse.js"
const chatRoute = OpenAIChat.route
const responsesRoute = OpenAIResponses.route
@@ -240,6 +243,26 @@ describe("llm constructors", () => {
expect(request.messages.map((message) => message.role)).toEqual(["user", "system"])
})
test("generates and streams from input or a prebuilt request", async () => {
const model = OpenAI.configure({ apiKey: "test", baseURL: "https://openai.test/v1" }).chat("gpt-4o-mini")
const layer = fixedResponse(
sseEvents({ choices: [{ delta: { content: "Hello" } }] }, { choices: [{ delta: {}, finish_reason: "stop" }] }),
)
const input = { model, prompt: "Say hello." }
const request = LLM.request(input)
const generated = await Effect.runPromise(LLM.generate(input).pipe(Effect.provide(layer)))
const generatedFromRequest = await Effect.runPromise(LLM.generate(request).pipe(Effect.provide(layer)))
expect(generated.text).toBe("Hello")
expect(generatedFromRequest.text).toBe(generated.text)
const streamed = await Effect.runPromise(LLM.stream(input).pipe(Stream.runCollect, Effect.provide(layer)))
const streamedFromRequest = await Effect.runPromise(
LLM.stream(request).pipe(Stream.runCollect, Effect.provide(layer)),
)
expect(Array.from(streamed).some(LLMEvent.is.textDelta)).toBe(true)
expect(streamedFromRequest).toEqual(streamed)
})
test("extracts output text from response events", () => {
expect(
LLMResponse.text({
+27 -6
View File
@@ -116,19 +116,32 @@ describe("AI promise client", () => {
const seen: Array<string> = []
const ai = AI.make({ layer: executor(seen) })
const text = await ai.llm.generate({ model: openai.chat("gpt-4o-mini"), prompt: "Say hello." })
const request = ai.llm.request({ model: openai.chat("gpt-4o-mini"), prompt: "Say hello." })
const text = await ai.llm.generate(request)
expect(text.text).toBe("Hello world")
expect((await ai.llm.generate({ model: openai.chat("gpt-4o-mini"), prompt: "Say hello." })).text).toBe(
"Hello world",
)
const image = await ai.image.generate({ model: openai.image("gpt-image-2"), prompt: "A lighthouse" })
expect(image.image).toBeInstanceOf(Media.Asset)
expect(image.image.mediaType).toBe("image/png")
expect(await ai.run(image.image.bytes())).toEqual(Uint8Array.from([1, 2, 3]))
const requested = await ai.image.generate(
ai.image.request({ model: openai.image("gpt-image-2"), prompt: "A lighthouse" }),
)
expect(requested.image.mediaType).toBe("image/png")
const deltas: Array<string> = []
for await (const event of ai.llm.stream({ model: openai.chat("gpt-4o-mini"), prompt: "Say hello." })) {
for await (const event of ai.llm.stream(request)) {
if (LLMEvent.is.textDelta(event)) deltas.push(event.text)
}
expect(deltas).toEqual(["Hello", " world"])
const directDeltas: Array<string> = []
for await (const event of ai.llm.stream({ model: openai.chat("gpt-4o-mini"), prompt: "Say hello." })) {
if (LLMEvent.is.textDelta(event)) directDeltas.push(event.text)
}
expect(directDeltas).toEqual(deltas)
const imageEvents: Array<string> = []
for await (const event of ai.image.stream({ model: openai.image("gpt-image-2"), prompt: "A lighthouse" })) {
@@ -137,8 +150,11 @@ describe("AI promise client", () => {
expect(imageEvents).toEqual(["image-partial", "image", "finish"])
expect(seen).toEqual([
"https://openai.test/v1/chat/completions",
"https://openai.test/v1/chat/completions",
"https://openai.test/v1/images/generations",
"https://openai.test/v1/images/generations",
"https://openai.test/v1/chat/completions",
"https://openai.test/v1/chat/completions",
"https://openai.test/v1/images/generations",
])
@@ -260,22 +276,27 @@ describe("AI promise client", () => {
const ai = AI.make({ layer: executor([]) })
const failure = await ai.llm
.generate({ model: openai.responses("gpt-5"), prompt: "Hello" })
.generate(ai.llm.request({ model: openai.responses("gpt-5"), prompt: "Hello" }))
.then(() => undefined)
.catch((error: unknown) => error)
expect(failure).toBeInstanceOf(AIError)
expect(failure instanceof AIError && failure.reason.http?.status).toBe(404)
const invalid = await ai.llm
// @ts-expect-error Invalid input must reject with AIError, not throw synchronously.
const invalidLLM = await ai.llm
// @ts-expect-error Invalid input must reject with AIError instead of throwing synchronously.
.generate({ model: openai.responses("gpt-5"), messages: [{ role: "bogus" }] })
.catch((error: unknown) => error)
expect(invalidLLM instanceof AIError && invalidLLM.reason._tag).toBe("InvalidRequest")
const invalid = await ai.image
.generate({ model: openai.image("gpt-image-2"), prompt: "A lighthouse", n: 1.5 })
.catch((error: unknown) => error)
expect(invalid instanceof AIError && invalid.reason._tag).toBe("InvalidRequest")
const controller = new AbortController()
controller.abort()
const aborted = await ai.llm
.generate({ model: openai.chat("gpt-4o-mini"), prompt: "Hello" }, { signal: controller.signal })
.generate(ai.llm.request({ model: openai.chat("gpt-4o-mini"), prompt: "Hello" }), { signal: controller.signal })
.then(() => "completed")
.catch(() => "aborted")
expect(aborted).toBe("aborted")
+27
View File
@@ -216,6 +216,33 @@ it.effect("Alibaba keeps native reasoning controls and future efforts on their s
}),
)
it.effect("Alibaba fits explicit thinking budgets to half the output limit", () =>
Effect.gen(function* () {
const provider = Alibaba.configure({ region: "ap-southeast-1", apiKey: "fixture" })
const chat = (maxTokens?: number) =>
compileRequest(
LLM.request({
model: provider.chat("qwen3.7-plus"),
prompt: "hi",
...(maxTokens === undefined ? {} : { generation: { maxTokens } }),
providerOptions: { enableThinking: true, thinkingBudget: 131_071 },
}),
).pipe(Effect.map((prepared) => prepared.body.thinking_budget))
const messages = yield* compileRequest(
LLM.request({
model: provider.messages("qwen3.7-plus"),
prompt: "hi",
generation: { maxTokens: 32_000 },
providerOptions: { thinking: { type: "enabled", budgetTokens: 131_071 } },
}),
)
expect(yield* chat(32_000)).toBe(16_000)
expect(yield* chat()).toBe(131_071)
expect(messages.body.thinking).toEqual({ type: "enabled", budget_tokens: 16_000 })
}),
)
it.effect("Alibaba validates malformed options before execution", () =>
Effect.gen(function* () {
const provider = Alibaba.configure({ region: "ap-southeast-1", apiKey: "fixture" })
@@ -148,11 +148,13 @@ describe("Anthropic Messages route", () => {
Effect.gen(function* () {
const enabled = yield* compileRequest(
LLMRequest.update(request, {
generation: { maxTokens: 4_096 },
providerOptions: { thinking: { type: "enabled", budgetTokens: 1_024 } },
}),
)
const legacy = yield* compileRequest(
LLMRequest.update(request, {
generation: { maxTokens: 4_096 },
providerOptions: { thinking: { type: "enabled", budget_tokens: 2_048 } },
}),
)
@@ -168,6 +170,22 @@ describe("Anthropic Messages route", () => {
}),
)
it.effect("fits the thinking budget to half the output limit", () =>
Effect.gen(function* () {
const thinking = (maxTokens: number) =>
compileRequest(
LLMRequest.update(request, {
generation: { maxTokens },
providerOptions: { thinking: { type: "enabled", budgetTokens: 31_999 } },
}),
).pipe(Effect.map((prepared) => prepared.body.thinking))
expect(yield* thinking(64_000)).toEqual({ type: "enabled", budget_tokens: 31_999 })
expect(yield* thinking(20_000)).toEqual({ type: "enabled", budget_tokens: 10_000 })
expect(yield* thinking(1_500)).toEqual({ type: "enabled", budget_tokens: 1_024 })
}),
)
it.effect("rejects enabled thinking without a budget", () =>
Effect.gen(function* () {
const error = yield* compileRequest(
@@ -244,6 +244,29 @@ describe("Bedrock Converse route", () => {
}),
)
it.effect("fits a Claude thinking budget below maxTokens", () =>
Effect.gen(function* () {
const fields = (maxTokens: number, budgetTokens: number, topK?: number) =>
compileRequest(
LLMRequest.update(baseRequest, {
model: AmazonBedrock.model("us.anthropic.claude-haiku-4-5-20251001-v1:0", {
baseURL: "https://bedrock-runtime.test",
apiKey: "test-bearer",
thinking: { type: "enabled", budgetTokens },
}),
generation: GenerationOptions.make({ maxTokens, topK }),
}),
).pipe(Effect.map((prepared) => prepared.body.additionalModelRequestFields))
expect(yield* fields(64_000, 31_999)).toEqual({ thinking: { type: "enabled", budget_tokens: 31_999 } })
expect(yield* fields(20_000, 31_999, 40)).toEqual({
top_k: 40,
thinking: { type: "enabled", budget_tokens: 10_000 },
})
expect(yield* fields(1_500, 31_999)).toEqual({ thinking: { type: "enabled", budget_tokens: 1_024 } })
}),
)
it.effect("omits additionalModelRequestFields when topK is unset", () =>
Effect.gen(function* () {
const prepared = yield* compileRequest(baseRequest)
+81 -10
View File
@@ -22,21 +22,22 @@ testEffect(
expect(body).toMatchObject({
model: "fixture",
stream: true,
store: false,
store: true,
instructions: "Keep the context",
parallel_tool_calls: true,
parallel_tool_calls: false,
prompt_cache_key: "session-key",
service_tier: "priority",
reasoning: { effort: "high", summary: "auto" },
context_management: [{ type: "compaction" }],
max_tool_calls: 1,
tool_choice: "required",
text: { verbosity: "high", format: { type: "json_object" } },
prompt_cache_retention: "24h",
prompt_cache_options: { mode: "session", ttl: "1h" },
input: [{ role: "user", content: [{ type: "input_text", text: "hello" }] }, { type: "compaction_trigger" }],
})
expect(body.tools).toHaveLength(1)
expect(body.tools[0].name).toBe("lookup")
expect(body.tool_choice).toBeUndefined()
expect(body.context_management).toBeUndefined()
expect(body.text).toBeUndefined()
expect(body.max_output_tokens).toBeUndefined()
expect(body.previous_response_id).toBeUndefined()
return respond(
@@ -57,7 +58,7 @@ testEffect(
)
}),
),
).effect("trigger uses normal request preparation, configured deployment, and supplied subscription headers", () =>
).effect("trigger keeps request controls, configured deployment, and supplied subscription headers", () =>
Effect.gen(function* () {
const calls: string[] = []
const input = LLM.request({
@@ -67,12 +68,14 @@ testEffect(
promptCacheKey: "session-key",
tools: [{ name: "lookup", description: "Lookup", inputSchema: { type: "object", properties: {} } }],
toolChoice: { type: "tool", name: "lookup" },
generation: { maxTokens: 1 },
providerOptions: {
store: true,
reasoningEffort: "high",
reasoningSummary: "auto",
contextManagement: [{ type: "compaction" }],
parallelToolCalls: false,
maxToolCalls: 1,
textVerbosity: "low",
},
http: {
headers: { "chatgpt-account-id": "fixture-account", "x-codex-beta-features": "remote_compaction_v2" },
@@ -82,8 +85,7 @@ testEffect(
prompt_cache_retention: "24h",
prompt_cache_options: { mode: "session", ttl: "1h" },
store: true,
stream: false,
text: { format: { type: "json_object" } },
text: { verbosity: "high", format: { type: "json_object" } },
tool_choice: "required",
},
},
@@ -114,6 +116,75 @@ testEffect(
}),
)
testEffect(
dynamicResponse(({ text, respond }) =>
Effect.sync(() => {
expect(JSON.parse(text).text).toEqual({ verbosity: "low", format: { type: "json_object" } })
return respond(sseEvents({ type: "response.completed", response: { id: "resp_1", output: [checkpoint] } }), {
headers: { "content-type": "text/event-stream" },
})
}),
),
).effect("keeps explicit verbosity on a trigger checkpoint for prompt cache reuse", () =>
LLMClient.compact(
LLM.request({
model: OpenAI.configure({ apiKey: "fixture" }).responses("gpt-5.5"),
prompt: "Hello.",
providerOptions: { textVerbosity: "low" },
http: { body: { text: { format: { type: "json_object" } } } },
}),
trigger,
),
)
testEffect(
dynamicResponse(({ text, respond }) =>
Effect.sync(() => {
const body = JSON.parse(text)
expect(body.text).toEqual({ verbosity: "high", format: { type: "json_object" } })
expect(body.max_output_tokens).toBe(20_000)
return respond(sseEvents({ type: "response.completed", response: { id: "resp_1", output: [checkpoint] } }), {
headers: { "content-type": "text/event-stream" },
})
}),
),
).effect("keeps the effective body-overlay verbosity and text formatting", () =>
LLMClient.compact(
LLM.request({
model: OpenAI.configure({ apiKey: "fixture" }).responses("gpt-5.5"),
prompt: "Hello.",
generation: { maxTokens: 20_000 },
providerOptions: { textVerbosity: "low" },
http: { body: { text: { verbosity: "high", format: { type: "json_object" } } } },
}),
trigger,
),
)
testEffect(
dynamicResponse(({ text, respond }) =>
Effect.sync(() => {
expect(JSON.parse(text).max_output_tokens).toBe(128)
return respond(JSON.stringify({ error: { message: "max_output_tokens must be at least 20000" } }), {
status: 400,
headers: { "content-type": "application/json" },
})
}),
),
).effect("passes configured output limits through and leaves rejection to the provider", () =>
Effect.gen(function* () {
const error = yield* LLMClient.compact(
LLM.request({
model: OpenAI.configure({ apiKey: "fixture" }).responses("gpt-5.5"),
prompt: "Hello.",
generation: { maxTokens: 128 },
}),
trigger,
).pipe(Effect.flip)
expect(error.message).toContain("at least 20000")
}),
)
const idless = { type: "compaction", encrypted_content: "opaque" }
testEffect(
fixedResponse(
@@ -184,7 +255,7 @@ testEffect(fixedResponse(sseEvents({ type: "response.output_item.done", item: ch
expect(error.reason._tag).toBe("InvalidProviderOutput")
}),
)
for (const body of [{ input: [] }, { previous_response_id: "stale" }]) {
for (const body of [{ input: [] }, { previous_response_id: "stale" }, { stream: false }]) {
testEffect(dynamicResponse(() => Effect.die("Must reject before sending"))).effect(
`rejects caller-supplied ${Object.keys(body)[0]} before sending trigger`,
() =>
@@ -110,11 +110,16 @@ for (const model of [
dynamicResponse(({ request, text, respond }) =>
Effect.sync(() => {
expect(new URL(request.url).pathname).toEndWith("/responses/compact")
expect(JSON.parse(text)).toEqual({ model: "fixture", input: [item], instructions: "Keep the context" })
expect(JSON.parse(text)).toEqual({
model: "fixture",
input: [item],
instructions: "Keep the context",
include: ["reasoning.encrypted_content"],
})
return respond(JSON.stringify({ object: "response.compaction", output: [checkpoint] }))
}),
),
).effect(`${model.provider} compacts provider-specific history without lowering generation settings`, () =>
).effect(`${model.provider} validates tools but ignores unrelated unsupported generation settings`, () =>
Effect.gen(function* () {
const request = LLM.request({
model,
@@ -151,6 +156,11 @@ for (const model of [
] as const) {
const error = yield* LLMClient.generate(candidate).pipe(Effect.flip)
expect(error.reason._tag).toBe(tag)
if (candidate.tools.length > 0) {
const compactError = yield* LLMClient.compact(candidate).pipe(Effect.flip)
expect(compactError.reason._tag).toBe("InvalidRequest")
continue
}
const response = yield* LLMClient.compact(candidate)
expect(response.replacement[0]?.content[0]?.type).toBe("compaction")
}
@@ -255,6 +265,13 @@ for (const overlay of [undefined, { service_tier: "priority", prompt_cache_key:
model: "fixture",
input: [{ type: "message", role: "user", content: [{ type: "input_text", text: "hello" }] }],
service_tier: overlay?.service_tier ?? "flex",
reasoning: { effort: "low" },
text: { verbosity: "low", format: { type: "json_object" } },
include: ["reasoning.encrypted_content"],
parallel_tool_calls: false,
tools: [
{ type: "function", name: "lookup", description: "Lookup", parameters: { type: "object" }, strict: false },
],
prompt_cache_key: overlay?.prompt_cache_key ?? "affinity",
prompt_cache_retention: "24h",
prompt_cache_options: { mode: "explicit", ttl: "30m" },
@@ -268,12 +285,20 @@ for (const overlay of [undefined, { service_tier: "priority", prompt_cache_key:
model: OpenAI.configure({ apiKey: "test" }).responses("fixture"),
prompt: "hello",
promptCacheKey: "affinity",
providerOptions: { serviceTier: "flex" },
providerOptions: {
serviceTier: "flex",
reasoningEffort: "low",
textVerbosity: "low",
include: ["reasoning.encrypted_content"],
parallelToolCalls: false,
},
generation: { maxTokens: 100 },
tools: [{ name: "lookup", description: "Lookup", inputSchema: {} }],
http: {
body: {
stream: true,
store: false,
text: { format: { type: "json_object" } },
prompt_cache_retention: "24h",
prompt_cache_options: { mode: "explicit", ttl: "30m" },
...overlay,
@@ -396,6 +421,8 @@ for (const model of [
model: model.id,
input: [{ type: "message", role: "user", content: [{ type: "input_text", text: "original" }] }],
instructions: "system",
include: ["reasoning.encrypted_content"],
...(model.id === "gpt-5.3-codex" ? { reasoning: { effort: "medium", summary: "auto" } } : {}),
})
return respond(
JSON.stringify({
@@ -407,7 +434,10 @@ for (const model of [
)
}
expect(new URL(request.url).pathname.endsWith("/responses")).toBe(true)
expect(body.input).toEqual([...output, { type: "message", role: "user", content: [{ type: "input_text", text: "continue" }] }])
expect(body.input).toEqual([
...output,
{ type: "message", role: "user", content: [{ type: "input_text", text: "continue" }] },
])
return respond(sseEvents({ type: "response.completed", response: { id: "resp_1", output: [] } }), {
headers: { "content-type": "text/event-stream" },
})
+17
View File
@@ -90,6 +90,23 @@ describe("Gemini route", () => {
}),
)
it.effect("fits the thinking budget to half the output limit", () =>
Effect.gen(function* () {
const thinkingBudget = (budget: number, maxTokens = 32_000) =>
compileRequest(
LLMRequest.update(request, {
generation: { maxTokens },
providerOptions: { thinkingConfig: { thinkingBudget: budget } },
}),
).pipe(Effect.map((prepared) => prepared.body.generationConfig?.thinkingConfig?.thinkingBudget))
expect(yield* thinkingBudget(32_768)).toBe(16_000)
expect(yield* thinkingBudget(8_000)).toBe(8_000)
expect(yield* thinkingBudget(-1)).toBe(-1)
expect(yield* thinkingBudget(8_192, 1_000)).toBe(512)
}),
)
it.effect("forwards standard Gemini generation options", () =>
Effect.gen(function* () {
const prepared = yield* compileRequest(
@@ -1945,7 +1945,7 @@ describe("OpenAI Responses route", () => {
expect(prepared.body.prompt_cache_key).toBe("session_123")
expect(prepared.body.include).toEqual(["reasoning.encrypted_content"])
expect(prepared.body.reasoning).toEqual({ effort: "high", summary: "auto" })
expect(prepared.body.text).toEqual({ verbosity: "low" })
expect(prepared.body.text).toBeUndefined()
expect(prepared.body.metadata).toEqual({ environment: "test", tenant: "acme" })
expect(prepared.body.safety_identifier).toBe("user_123")
expect(prepared.body.stream_options).toEqual({ include_obfuscation: false })
@@ -152,6 +152,27 @@ describe("OpenRouter", () => {
}),
)
it.effect("fits the reasoning budget to half the output limit", () =>
Effect.gen(function* () {
const reasoning = (maxTokens: number | undefined, value: Record<string, unknown>) =>
compileRequest(
LLM.request({
model: OpenRouter.configure({ apiKey: "test-key" }).model("qwen/qwen3.8-flash"),
cache: "none",
prompt: "Hello",
...(maxTokens === undefined ? {} : { generation: { maxTokens } }),
providerOptions: { reasoning: value },
}),
).pipe(Effect.map((prepared) => prepared.body.reasoning))
expect(yield* reasoning(32_000, { max_tokens: 131_071 })).toEqual({ max_tokens: 16_000 })
expect(yield* reasoning(131_072, { max_tokens: 65_536 })).toEqual({ max_tokens: 65_536 })
expect(yield* reasoning(1_500, { max_tokens: 65_536 })).toEqual({ max_tokens: 1_024 })
expect(yield* reasoning(undefined, { max_tokens: 131_071 })).toEqual({ max_tokens: 131_071 })
expect(yield* reasoning(32_000, { effort: "high" })).toEqual({ effort: "high" })
}),
)
it.effect("applies OpenRouter payload options from the model helper", () =>
Effect.gen(function* () {
const prepared = yield* compileRequest(
+3 -1
View File
@@ -1,5 +1,5 @@
import type { Stream } from "effect"
import { Speech, type SpeechEvent } from "../src/index.js"
import { Speech, SpeechModel, type SpeechEvent, type SpeechOptions } from "../src/index.js"
import { ElevenLabs, OpenAI, Runway } from "../src/providers.js"
type StreamItem<T> = T extends Stream.Stream<infer A, infer _E, infer _R> ? A : never
@@ -7,6 +7,8 @@ type Equal<A, B> = [A, B] extends [B, A] ? true : false
type Assert<T extends true> = T
const elevenlabs = ElevenLabs.configure({ apiKey: "test" }).speech("eleven_flash_v2_5")
const widenSpeech = <Options extends SpeechOptions>(model: SpeechModel<Options>): SpeechModel => model
void widenSpeech
Speech.generate({
model: elevenlabs,
+11 -1
View File
@@ -1,5 +1,11 @@
import type { Stream } from "effect"
import { Media, Transcription, type TranscriptionEvent } from "../src/index.js"
import {
Media,
Transcription,
TranscriptionModel,
type TranscriptionEvent,
type TranscriptionOptions,
} from "../src/index.js"
import { AssemblyAI, Deepgram, OpenAI } from "../src/providers.js"
type StreamItem<T> = T extends Stream.Stream<infer A, infer _E, infer _R> ? A : never
@@ -8,6 +14,10 @@ type Assert<T extends true> = T
const audio = Media.url("https://example.com/call.mp3")
const deepgram = Deepgram.configure({ apiKey: "test" }).transcription("nova-3")
const widenTranscription = <Options extends TranscriptionOptions>(
model: TranscriptionModel<Options>,
): TranscriptionModel => model
void widenTranscription
Transcription.generate({
model: deepgram,
+3 -3
View File
@@ -9,7 +9,6 @@ import {
type VideoModelOptions,
type VideoOptions,
type VideoRequestFor,
type VideoRoute,
} from "../src/index.js"
import type { Service } from "../src/video-client.js"
import { Anthropic, Fal, Google, OpenAI, Runway, XAI } from "../src/providers.js"
@@ -23,8 +22,7 @@ type VeoLikeOptions = {
readonly personGeneration?: "allow_all" | "allow_adult"
} & Record<string, unknown>
declare const route: VideoRoute<VeoLikeOptions>
const veo = VideoModel.make<VeoLikeOptions>({ id: "veo", provider: "google", route })
declare const veo: VideoModel<VeoLikeOptions>
// @ts-expect-error Extracted model options retain known provider fields.
const invalidVeoOptions: VideoModelOptions<typeof veo> = { personGeneration: "everyone" }
void invalidVeoOptions
@@ -99,6 +97,8 @@ Video.generate({ model: google, prompt: "A kitten", durationSeconds: "8s" })
Video.generate({ model: google, prompt: "A kitten", options: { personGeneration: "allow_all" } })
declare const generic: VideoModel<VideoOptions>
const widenVideo = <Options extends VideoOptions>(model: VideoModel<Options>): VideoModel => model
void widenVideo
Video.generate({ model: generic, prompt: "A kitten", providerOptions: { arbitrary: true } })
const request = Video.request({ model: veo, prompt: "A kitten", providerOptions: { personGeneration: "allow_all" } })
+1 -1
View File
@@ -21,7 +21,7 @@
"build:node": "bun run script/build-node.ts",
"dev": "bun run src/index.ts",
"test": "bun test --timeout 30000 --only-failures",
"typecheck": "tsgo --noEmit"
"typecheck": "tsgo -b"
},
"dependencies": {
"@agentclientprotocol/sdk": "1.2.1",
+2 -1
View File
@@ -7,5 +7,6 @@
"lib": ["ESNext", "DOM", "DOM.Iterable", "DOM.AsyncIterable"],
"noUncheckedIndexedAccess": false
},
"exclude": ["dist", "dist-node"]
"exclude": ["dist", "dist-node"],
"references": [{ "path": "../core" }]
}
+23 -10
View File
@@ -139,11 +139,24 @@ ultimate source of truth. Upstream test262 files run verbatim from `test/test262
arrow function.
- [x] Promise-returning string replacers are coerced synchronously to `"[object Promise]"`, like JavaScript; they are
not automatically awaited.
- [x] The optional `thisArg` of iteration methods is accepted and ignored: CodeMode functions have no `this`, so
ignoring it matches JS arrow-function semantics exactly.
- [ ] `this` in non-arrow CodeMode functions and callbacks.
- [x] `this` in non-arrow functions is the call's receiver: `obj.m()` and `obj["m"]()` see `obj`, a bare or detached
call (`f()`, `const m = obj.m; m()`, `(0, obj.m)()`) sees `undefined`, as in strict JS. Arrows read the enclosing
function's `this`. Program code has no receiver, so top-level `this` is `undefined`, as in a module.
- [x] `arguments` in non-arrow functions: an unmapped ordinary object with the call's arguments as indexed
properties and a hidden `length`; iterable, so spread, `for...of`, and `Array.from` work. It is not an Array
(`JSON.stringify` gives `{"0":1}`, `String` gives `[object Arguments]`). A parameter named `arguments` shadows
it; arrows read the enclosing function's; it is only created for functions whose body mentions it. `callee`
and `caller` are absent rather than poisoned.
- [ ] Array methods on `arguments` and other array-likes (`Array.prototype.slice.call(arguments, 1)`); use
`[...arguments]` or a rest parameter meanwhile.
- [x] `Function.prototype.call`, `apply`, and `bind` on program functions and built-ins:
`Array.prototype.push.call(arr, 1)`, `Math.max.apply(null, values)`, `fn.bind(obj, first)`. `apply` accepts an
array, an array-like object (its `length` clamped and capped like `Array.from`), or `null`/`undefined`. A
bound function is named `bound f`, has its remaining `length`, and is not constructible.
- [x] `JSON.parse` revivers and `JSON.stringify` function replacers see the holder object as `this`.
- [ ] The optional `thisArg` of iteration methods (`map`, `forEach`, `Map.prototype.forEach`, `Array.from`, …) is
accepted but not yet passed as `this`; callbacks run with `this` undefined.
- [ ] User-defined constructor calls.
- [ ] `Function.prototype.call`, `apply`, and `bind` for CodeMode functions.
- [ ] Classes and private fields.
- [x] Functions are objects: they hold own properties (`fn.count = 1`), enumerate them, and expose read-only `name`
and `length`. Names follow JavaScript's NamedEvaluation: declarations, named expressions, bindings,
@@ -178,7 +191,7 @@ ultimate source of truth. Upstream test262 files run verbatim from `test/test262
yielded promises; mixed async request queues; sync and async `yield*` forwarding; malformed methods/results;
and declaration, expression, and object-method forms with closure and parameter behavior. The adapted suite
deliberately skips Test262 variants whose observation mechanism requires unsupported getter definitions,
proxies, prototype inspection or mutation, non-arrow `this`, classes, or arbitrary symbols. It also skips tests
proxies, prototype inspection or mutation, classes, or arbitrary symbols. It also skips tests
asserting exact promise reaction-turn counts beyond the observable ordering guarantee documented below. These
are interpreter-surface boundaries, not claims that the corresponding full Test262 families pass unchanged.
@@ -269,7 +282,7 @@ reject }` object.
- [x] Recursive assimilation of objects with an own callable `then` field across `Promise.resolve`, combinators,
constructors, reactions, `finally`, `await`, and async returns. Thenable methods run deferred, receive
first-call-wins resolve/reject functions, and ignore throws after settlement. Inherited/accessor `then` fields
and a JavaScript `this` receiver remain outside the supported object/function model.
remain outside the supported object model.
- [x] Dotted tool names are canonicalized into namespace paths; a path can be both callable and a namespace, and the
last tool supplied for a canonical path wins.
- [x] Tool path segments may be named `constructor`, `prototype`, or `__proto__` because paths use inert Map keys.
@@ -419,9 +432,9 @@ reject }` object.
- [x] `JSON.parse` and `JSON.stringify` for supported data objects.
- [x] Numeric/string indentation for `JSON.stringify`.
- [x] `JSON.parse` reviver callbacks, including postorder traversal, deletion through `undefined`, and root replacement.
Revivers receive `(key, value)` but no `this` holder because CodeMode functions intentionally have no `this`.
Revivers receive `(key, value)` with the holder as `this`.
- [x] `JSON.stringify` function and array replacers. Function replacers receive `(key, value)` in preorder, including
the root, but no `this` holder. Array replacers preserve requested property order, deduplicate names, coerce
the root, with the holder as `this`. Array replacers preserve requested property order, deduplicate names, coerce
number primitives, and ignore non-string/non-number entries. Primitive wrapper entries remain unsupported.
- [x] Captured `console.log`, `console.info`, `console.debug`, `console.warn`, and `console.error`. An Error prints as
`Error.prototype.toString` would show it (`Error: boom`), wherever it appears in the logged value.
@@ -596,8 +609,8 @@ Nothing is exposed unless a host provides it; extension calls are not tool calls
non-enumerable: an extension Error carries it, and this JSON form does not. Errors have no `stack`; the diagnostic
carries a 1-based line and column in the submitted source instead.
- [x] `instanceof` against any constructor with a `prototype`, including every built-in and `Function`.
- [ ] The derived error constructors inheriting from `Error`: `Object.getPrototypeOf(TypeError)` is
`Function.prototype` here, while `TypeError.prototype` does inherit from `Error.prototype`.
- [x] Derived error constructors extend `Error` itself: `Object.getPrototypeOf(TypeError) === Error`, so
`TypeError.isError` is inherited, and `TypeError.prototype` inherits from `Error.prototype`.
- [x] Catchable user throws, runtime failures raised during interpreted evaluation, awaited tool failures, and awaited
tool-call-limit failures; parse/compile failures, cooperative timeout, and output bounding remain outside program
`catch`.
+2 -2
View File
@@ -24,7 +24,7 @@ import { typeofValue } from "./interpreter/references.js"
export type Json = Schema.Json
type Replacer<R> = (args: Array<Value>) => Effect.Effect<Value, unknown, R>
type Replacer<R> = (args: Array<Value>, holder: Obj) => Effect.Effect<Value, unknown, R>
/**
* What `JSON.stringify` would serialize for a program value, as host JSON: `toJSON` is honored, functions and
@@ -60,7 +60,7 @@ const walk = <R>(
const settled = raw instanceof PromiseObj ? yield* ctx.await(raw) : raw
const toJSON = settled instanceof Obj ? get(settled, "toJSON") : undefined
const own = toJSON instanceof Callable ? yield* ctx.call(toJSON, settled, [key]) : settled
const value = replacer === undefined ? own : yield* replacer([key, own])
const value = replacer === undefined ? own : yield* replacer([key, own], holder)
if (value === undefined || typeofValue(value) === "function") return undefined
if (typeof value === "number") return Number.isFinite(value) ? value : null
if (value === null || typeof value === "string" || typeof value === "boolean") return value
@@ -78,7 +78,7 @@ export const applyCollectionCallback = <R>(
ctx: Interpreter<R>,
callback: Value,
name: string,
): ((args: Array<Value>) => Effect.Effect<Value, unknown, R>) => {
): ((args: Array<Value>, thisValue?: Value) => Effect.Effect<Value, unknown, R>) => {
if (!isSupportedCallback(callback)) {
if (typeofValue(callback) === "function") {
throw typeError(
@@ -87,5 +87,5 @@ export const applyCollectionCallback = <R>(
}
throw typeError(`${name} expects a function callback.`)
}
return (callbackArgs) => ctx.call(callback, undefined, callbackArgs)
return (callbackArgs, thisValue) => ctx.call(callback, thisValue, callbackArgs)
}
@@ -180,6 +180,9 @@ export const errorGlobal = <R>(type: ErrorType, ctx: Interpreter<R>) => {
["toString", 0, (thisValue) => errorToString(receiver(Obj, thisValue, "Error.prototype.toString"))],
])
methods(builtins, ctor, [["isError", 1, (_, args) => args[0] instanceof ErrorObj]])
return ctor
}
// Derived constructors extend Error itself, so its statics are inherited. The globals table creates Error first.
ctor.proto = get(builtins.Error, "constructor") as Obj
return ctor
}
+31 -2
View File
@@ -1,5 +1,5 @@
import { Effect } from "effect"
import type { Value } from "./objects.js"
import { Arr, Callable, coerceToInteger, coerceToString, get, Obj, type Value } from "./objects.js"
import { arrayGlobal } from "../stdlib/array.js"
import { textDecoderGlobal, textEncoderGlobal, uint8ArrayGlobal } from "../stdlib/bytes.js"
import { mapGlobal, setGlobal } from "../stdlib/collections.js"
@@ -19,8 +19,9 @@ import { base64Global, cryptoGlobal, structuredCloneGlobal } from "../stdlib/web
import { ToolReference } from "../tool-runtime.js"
import { errorGlobal } from "./errors.js"
import { errorTypes } from "./intrinsics.js"
import { constants, constructor, native } from "./native.js"
import { constants, constructor, methods, native, receiver } from "./native.js"
import { AsyncIteratorSymbol, IteratorSymbol, typeError } from "./model.js"
import { checkArrayLength } from "./limits.js"
import { generatorGlobals } from "./generators.js"
import { promiseGlobal } from "./promises.js"
import type { Interpreter } from "./interpreter.js"
@@ -31,6 +32,24 @@ const functionGlobal = <R>(ctx: Interpreter<R>) => {
Effect.sync(() => {
throw typeError("The Function constructor is not supported; write the function inline.")
})
const target = (thisValue: Value, method: string) => receiver(Callable, thisValue, `Function.prototype.${method}`)
methods(ctx.builtins, ctx.builtins.Function, [
["call", 1, (thisValue, args) => ctx.call(target(thisValue, "call"), args[0], args.slice(1))],
["apply", 2, (thisValue, args) => ctx.call(target(thisValue, "apply"), args[0], listFromArrayLike(args[1]))],
[
"bind",
1,
(thisValue, args) => {
const fn = target(thisValue, "bind")
const bound = args.slice(1)
return native<R>(ctx.builtins, {
name: `bound ${coerceToString(get(fn, "name"))}`,
length: Math.max(0, fn.length - bound.length),
call: (_, rest) => ctx.call(fn, args[0], [...bound, ...rest]),
})
},
],
])
return constructor<R>(ctx.builtins, ctx.builtins.Function, {
name: "Function",
length: 1,
@@ -39,6 +58,16 @@ const functionGlobal = <R>(ctx: Interpreter<R>) => {
})
}
// CreateListFromArrayLike: `apply` reads `length` and the indexed properties of any object.
const listFromArrayLike = (value: Value): Array<Value> => {
if (value === undefined || value === null) return []
if (value instanceof Arr) return [...value.items]
if (!(value instanceof Obj)) throw typeError("Function.prototype.apply expects an array-like argument list.")
const length = Math.max(0, coerceToInteger(get(value, "length")))
checkArrayLength(length)
return Array.from({ length }, (_, index) => get(value, String(index)))
}
const symbolGlobal = <R>(ctx: Interpreter<R>) => {
const symbol = native<R>(ctx.builtins, {
name: "Symbol",
@@ -1,4 +1,5 @@
import type {
AnyNode,
ArrayExpression,
ArrayPattern,
AssignmentPattern,
@@ -81,6 +82,7 @@ import {
keys,
Native,
parseArrayIndex,
Arguments,
Arr,
Fn,
GeneratorObj,
@@ -178,6 +180,24 @@ const collectPatternNames = (pattern: Pattern, out: Array<string> = []): Array<s
return out
}
// Whether a function body (or a parameter default) reads `arguments`, looking through arrows but not nested
// functions, which own theirs. Memoized so the object is only built for calls that can observe it.
const argumentsUse = new WeakMap<Fn["body"], boolean>()
const usesArguments = (fn: Fn): boolean => {
const cached = argumentsUse.get(fn.body)
if (cached !== undefined) return cached
const found = [...fn.parameters, fn.body].some(function visit(node: AnyNode | null): boolean {
if (node === null || typeof node !== "object") return false
if (node.type === "Identifier") return node.name === "arguments"
if (node.type === "FunctionDeclaration" || node.type === "FunctionExpression") return false
return Object.values(node).some((child) =>
Array.isArray(child) ? child.some((item) => visit(item)) : visit(child as AnyNode | null),
)
})
argumentsUse.set(fn.body, found)
return found
}
// `var` names declared anywhere in a function body except inside nested functions, which own theirs.
// Memoized per body: a function's var names never change, and hoisting runs on every call.
const varNames = new WeakMap<ReadonlyArray<Statement | ModuleDeclaration>, ReadonlyArray<string>>()
@@ -287,7 +307,8 @@ export class Interpreter<R> {
this.pending = options.pending
this.builtins = options.builtins
this.logs = options.logs ?? []
const globalScope = new Map<string, Binding>()
// Program code has no receiver: top-level `this` is undefined, as in a module.
const globalScope = new Map<string, Binding>([["this", { mutable: false, value: undefined }]])
// Calling back into the program never reads frame state, so any frame serves; the root is always alive.
this.root = new Frame(this, new ScopeStack([globalScope]))
for (const [name, value] of [...globals(this), ...(options.globals?.(this) ?? [])]) {
@@ -468,6 +489,7 @@ class Frame<R> {
this.scopes.capture(),
node.async,
node.generator,
node.type === "ArrowFunctionExpression",
)
// Each generator function gets its own prototype, so `g() instanceof g` holds as in JS.
if (node.generator)
@@ -1257,6 +1279,8 @@ class Frame<R> {
}
case "Identifier":
return Effect.sync(() => this.scopes.get(node.name, node))
case "ThisExpression":
return Effect.sync(() => this.scopes.get("this", node))
case "BinaryExpression":
return this.evaluateBinaryExpression(node)
case "LogicalExpression":
@@ -1622,7 +1646,7 @@ class Frame<R> {
}
return yield* self.createToolCallPromise(callable.path, args)
}
if (callable instanceof Fn) return yield* self.invokeFunction(callable, args, node)
if (callable instanceof Fn) return yield* self.invokeFunction(callable, thisValue, args, node)
if (callable instanceof Native) {
return yield* self.native(() => (callable as Native<R>).call(thisValue, args), node)
}
@@ -1664,14 +1688,24 @@ class Frame<R> {
}
// A callback invoked by a built-in runs below the call that invoked the built-in, so the deeper of the two counts.
invokeFunction(fn: Fn, args: Array<Value>, node?: AstNode): Effect.Effect<Value, unknown, R> {
invokeFunction(fn: Fn, thisValue: Value, args: Array<Value>, node?: AstNode): Effect.Effect<Value, unknown, R> {
const self = this
return Effect.flatMap(CallSite, (site) => {
const depth = Math.max(self.depth, site.depth) + 1
if (depth > MAX_CALL_DEPTH) throw rangeError("Maximum call stack size exceeded", node)
const invocation = new Frame(this.ctx, new ScopeStack([...fn.capturedScopes, new Map()]), depth)
// Seed all parameters first so defaults cannot fall through to same-named outer bindings.
const paramScope = invocation.scopes.current()
// `this` and `arguments` are scope bindings so arrows resolve them lexically; a parameter named
// `arguments` shadows the object, as in JS.
if (!fn.arrow) paramScope.set("this", { mutable: false, value: thisValue, initialized: true })
if (!fn.arrow && usesArguments(fn)) {
paramScope.set("arguments", {
mutable: true,
value: new Arguments(self.ctx.builtins.Object, args),
initialized: true,
})
}
// Seed all parameters first so defaults cannot fall through to same-named outer bindings.
for (const parameter of fn.parameters) {
for (const name of collectPatternNames(parameter)) {
paramScope.set(name, { mutable: true, value: undefined, initialized: false })
+1 -1
View File
@@ -72,7 +72,7 @@ export const uriError = failure("URIError")
// Orient the agent rather than enumerate JavaScript; interpreter-support.md is the full matrix.
export const supportedSyntaxMessage =
"This is a restricted JavaScript-like language. Supported: plain and async functions, data literals, destructuring, standard control flow, await and Promise, and built-ins such as Array, Object, Math, JSON, Date, RegExp, Map, Set, and URL. Unsupported: classes, this, getters/setters, BigInt, and custom Symbols. Use plain functions and data objects instead."
"This is a restricted JavaScript-like language. Supported: plain and async functions, data literals, destructuring, standard control flow, await and Promise, and built-ins such as Array, Object, Math, JSON, Date, RegExp, Map, Set, and URL. Unsupported: classes, getters/setters, BigInt, and custom Symbols. Use plain functions and data objects instead."
export const unsupportedSyntax = (kind: string, node: AstNode): PendingThrow =>
new PendingThrow(
+22 -1
View File
@@ -149,7 +149,11 @@ export abstract class Opaque extends Obj {
export abstract class Callable extends Opaque {
override readonly tag = "Function"
constructor(proto: Obj, name: string, length: number) {
constructor(
proto: Obj,
name: string,
readonly length: number,
) {
super(proto)
define(this, "length", length, readonly)
define(this, "name", name, readonly)
@@ -168,12 +172,29 @@ export class Fn extends Callable {
readonly capturedScopes: Array<Map<string, Binding>>,
readonly async: boolean,
readonly generator: boolean,
/** Arrows have no `this` or `arguments` of their own; they read the enclosing function's. */
readonly arrow: boolean,
) {
const optional = parameters.findIndex((p) => p.type === "AssignmentPattern" || p.type === "RestElement")
super(proto, name, optional === -1 ? parameters.length : optional)
}
}
/** The strict `arguments` object: an ordinary object with indexed own properties and a hidden `length`. */
export class Arguments extends Obj {
override readonly tag = "Arguments"
constructor(proto: Obj, args: Array<Value>) {
super(proto)
args.forEach((arg, index) => define(this, String(index), arg))
define(this, "length", args.length, hidden)
}
override iterator() {
return keys(this)
.map((key) => get(this, key))
.values()
}
}
export type NativeCall<R> = (thisValue: Value, args: Array<Value>) => Effect.Effect<Value, unknown, R>
export type NativeConstruct<R> = (args: Array<Value>, newTarget: Callable) => Effect.Effect<Value, unknown, R>
+1 -1
View File
@@ -40,7 +40,7 @@ const parse = <R>(ctx: Interpreter<R>, args: Array<Value>): Effect.Effect<Value,
else set(value, name, revived)
}
}
return yield* apply([key, value])
return yield* apply([key, value], holder)
})
return visit(record(ctx.builtins.Object, { "": parsed }), "")
}
@@ -210,11 +210,14 @@ describe("Test262 JSON.stringify replacer adaptations", () => {
})
describe("CodeMode JSON callback boundaries", () => {
test("this remains unsupported rather than exposing callback holders", async () => {
const result = await Effect.runPromise(
CodeMode.execute({ code: `return JSON.parse("1", function (key, item) { return this })`, tools: {} }),
)
expect(result).toMatchObject({ ok: false, error: { kind: "UnsupportedSyntax" } })
test("revivers and replacers see the holder as this", async () => {
expect(
await value(`
const revived = JSON.parse('{"a":{"b":1}}', function (key, item) { return key === "b" ? this.b + 1 : item })
const text = JSON.stringify({ a: 1, b: 2 }, function (key, item) { return key === "a" ? this.b : item })
return [revived, text]
`),
).toEqual([{ a: { b: 2 } }, '{"a":2,"b":2}'])
})
test("prototype-named keys parse as own data and reach the reviver", async () => {
@@ -77,6 +77,6 @@ describe("new on a non-constructible callee", () => {
expect(failure.message).toStartWith(
"SyntaxError: Syntax 'ClassDeclaration' is not supported. This is a restricted JavaScript-like language. Supported: ",
)
expect(failure.message).toContain("Unsupported: classes, this, getters/setters, BigInt, and custom Symbols.")
expect(failure.message).toContain("Unsupported: classes, getters/setters, BigInt, and custom Symbols.")
})
})
+110
View File
@@ -1461,3 +1461,113 @@ describe("structuredClone", () => {
expect((await error(`structuredClone()`)).message).toContain("structuredClone requires 1 argument")
})
})
describe("error constructor prototype chain", () => {
test("derived error constructors extend Error and inherit its statics", async () => {
expect(
await value(`
const derived = [TypeError, RangeError, SyntaxError, ReferenceError, EvalError, URIError, AggregateError]
return [
derived.every((ctor) => Object.getPrototypeOf(ctor) === Error),
Object.getPrototypeOf(Error) === Function.prototype,
TypeError.isError(new RangeError("x")),
new TypeError("x") instanceof Error,
]
`),
).toEqual([true, true, true, true])
})
})
describe("this, arguments, and Function.prototype.call/apply/bind", () => {
test("this is the call receiver for non-arrow functions and lexical for arrows", async () => {
expect(
await value(`
const o = { n: 1, m() { return this.n }, a() { return (() => this.n)() }, bare() { return this } }
const detached = o.bare
function f() { return this }
return [o.m(), o["m"](), o?.m(), (o.m)(), o.a(), o.bare() === o, detached(), f(), (0, o.bare)(), this, (() => this)()]
`),
).toEqual([1, 1, 1, 1, 1, true, null, null, null, null, null])
})
test("this reaches generator and async methods and plain-function callbacks", async () => {
expect(
await value(`
const o = { n: 2, *g() { yield this.n }, async m() { return this.n }, xs: [1, 2], go() { return this.xs.map(function (x) { return [x, this] }) } }
return [[...o.g()], await o.m(), o.go()]
`),
).toEqual([
[2],
2,
[
[1, null],
[2, null],
],
])
})
test("arguments is an unmapped array-like that arrows and parameters interact with as in JS", async () => {
expect(
await value(`
function f(a) { arguments[0] = 9; return [arguments.length, arguments[1], a, [...arguments], Array.isArray(arguments), JSON.stringify(arguments), typeof arguments.map, (() => arguments[1])()] }
function shadow(arguments) { return arguments }
function hoisted() { var arguments; return arguments.length }
let outer
try { outer = arguments } catch (error) { outer = error.name }
return [f(1, 2), shadow(7), hoisted(1, 2, 3), outer]
`),
).toEqual([[2, 2, 1, [9, 2], false, '{"0":9,"1":2}', "undefined", 2], 7, 3, "ReferenceError"])
})
test("call, apply, and bind set this and arguments on program functions and built-ins", async () => {
expect(
await value(`
function f(a, b, c) { return [this, a, b, c] }
const g = f.bind({ k: 1 }, "A")
const arr = [1]
Array.prototype.push.call(arr, 2, 3)
return [
f.call("t", 1, 2),
f.apply({ k: 2 }, [1, 2]),
f.apply(null, { length: 2, 0: "x", 1: "y" }),
f.apply(null).length,
g("B", "C"), g.name, g.length,
f.bind(1).bind(2)()[0],
arr,
Math.max.apply(null, [1, 5, 3]),
Math.max.bind(null, 10)(3),
[1, 2].map(f.bind(null, 0)).map((r) => r[1]),
]
`),
).toEqual([
["t", 1, 2, null],
[{ k: 2 }, 1, 2, null],
[null, "x", "y", null],
4,
[{ k: 1 }, "A", "B", "C"],
"bound f",
2,
1,
[1, 2, 3],
5,
10,
[0, 0],
])
})
test("call and apply reject non-callable receivers and non-array-like argument lists", async () => {
expect((await error(`Function.prototype.call.call(1)`)).message).toContain(
"Function.prototype.call called on incompatible receiver",
)
expect((await error(`(() => 1).apply(null, 5)`)).message).toContain("expects an array-like argument list")
expect((await error(`(() => 1).apply(null, { length: 1e9 })`)).message).toContain("Invalid array length")
expect(
await value(
`function f() { return arguments.length } return [f.apply(null, { length: -5 }), f.apply(null, { length: "2" })]`,
),
).toEqual([0, 2])
expect((await error(`function f(n) { return f.call(null, n + 1) } f(0)`)).message).toContain(
"Maximum call stack size exceeded",
)
})
})
+3 -1
View File
@@ -1416,7 +1416,9 @@ describe("built-in iterators", () => {
const logged = await run(`console.log([1].keys()); return null`)
expect(logged.logs?.[0]).toBe("[opaque reference]")
expect((await error(`return [1].keys() + ""`)).message).toContain("Binary operators require data values")
expect((await error(`return [1].keys().next.call({})`)).message).toContain("is not a function")
expect((await error(`return [1].keys().next.call({})`)).message).toContain(
"Iterator.prototype.next called on incompatible receiver a data object",
)
expect((await error(`const it = [1].keys(); const next = it.next; return next()`)).message).toContain(
"Iterator.prototype.next called on incompatible receiver undefined",
)
+3 -3
View File
@@ -24,9 +24,9 @@ Without them the runner registers no tests, so CI is unaffected. Licensed under
`script/sync-test262.ts` skips a file when its frontmatter declares a `flags`, `features`, or `includes` value the
manifest marks unsupported, or when its code matches one of the manifest's `boundaries` patterns. The sync checks the
checkout is at the pinned revision, so every machine runs the same 4595 files. Boundaries are
intentional limits of the interpreter, not compatibility work: classes, `this`, `arguments`, prototype objects,
property descriptors, accessors, boxed primitives, sloppy mode, `eval`, `Symbol()`, and the `$262` host API. If one
checkout is at the pinned revision, so every machine runs the same files. Boundaries are
intentional limits of the interpreter, not compatibility work: classes, prototype objects, property descriptors,
accessors, boxed primitives, sloppy mode, `eval`, `Symbol()`, and the `$262` host API. If one
of those decisions changes, delete its entry and re-sync; the tests are upstream, not lost.
## Commands
+5 -3
View File
@@ -2,6 +2,9 @@
"revision": "250f204f23a9249ff204be2baec29600faae7b75",
"directories": [
"built-ins/Array/prototype",
"built-ins/Function/prototype/apply",
"built-ins/Function/prototype/bind",
"built-ins/Function/prototype/call",
"built-ins/Iterator",
"built-ins/Object/freeze",
"built-ins/Object/getPrototypeOf",
@@ -11,18 +14,17 @@
"built-ins/Object/isSealed",
"built-ins/Object/preventExtensions",
"built-ins/String/raw",
"language/arguments-object",
"language/expressions/does-not-equals",
"language/expressions/equals",
"language/expressions/tagged-template",
"language/expressions/this",
"language/statements"
],
"harness": ["assert.js", "sta.js", "compareArray.js", "doneprintHandle.js"],
"flags": ["module", "raw", "noStrict"],
"boundaries": {
"class": "\\bclass\\s*[A-Za-z_${]",
"this": "\\bthis\\b",
"arguments": "\\barguments\\b",
"call/apply/bind": "\\.(call|apply|bind)\\s*\\(",
"accessor properties": "\\b(get|set)\\s+[\\w$\\[][^\\n(]*\\(",
"property descriptors": "Object\\.(defineProperty|defineProperties|getOwnPropertyDescriptors?|getOwnPropertyNames|setPrototypeOf)\\b",
"boxed primitives": "\\bnew\\s+(String|Number|Boolean)\\s*\\(",
File diff suppressed because it is too large Load Diff
+12 -4
View File
@@ -151,8 +151,12 @@ type Overlay = {
function options(replacement: string, modelID: string | undefined, settings: Legacy): Overlay {
const converse = replacement === "@opencode/ai/providers/amazon-bedrock" && modelID !== undefined
const kept = Struct.omit(settings, ["headers", "extraBody", "useCompletionUrls", ...OPENROUTER_KEYS])
const thinking = converse ? bedrockThinking(modelID, settings) : undefined
return {
settings: replacement.startsWith("@opencode/ai/providers/amazon-bedrock") ? bedrockSettings(kept, converse) : kept,
settings: {
...(replacement.startsWith("@opencode/ai/providers/amazon-bedrock") ? bedrockSettings(kept, converse) : kept),
...(thinking === undefined ? {} : { thinking }),
},
...(settings.headers === undefined ? {} : { headers: settings.headers }),
...(settings.extraBody === undefined ? {} : { body: settings.extraBody }),
...(converse ? bedrockRequest(modelID, settings) : {}),
@@ -196,6 +200,13 @@ function bedrockSettings(settings: Legacy, converse: boolean) {
}
}
// Claude's enabled budget is a typed setting so the protocol can fit it under the output limit.
function bedrockThinking(modelID: string | undefined, settings: Legacy) {
const reasoning = settings.reasoningConfig
if (!modelID?.includes("anthropic") || reasoning?.type !== "enabled" || reasoning.budgetTokens === undefined) return
return { type: "enabled", budgetTokens: reasoning.budgetTokens }
}
function bedrockRequest(modelID: string | undefined, settings: Legacy): Pick<Overlay, "body"> {
const additional = settings.additionalModelRequestFields ?? {}
const reasoning = settings.reasoningConfig
@@ -210,9 +221,6 @@ function bedrockRequest(modelID: string | undefined, settings: Legacy): Pick<Ove
const betas = settings.anthropicBeta ?? []
const fields = Provider.mergeOverlay(additional, {
...(betas.length > 0 ? { anthropic_beta: [...(additional.anthropic_beta ?? []), ...betas] } : {}),
...(anthropic && type === "enabled" && budget !== undefined
? { thinking: { type: "enabled", budget_tokens: budget } }
: {}),
...(anthropic && type === "adaptive"
? { thinking: { type: "adaptive", ...(display === undefined ? {} : { display }) } }
: {}),
+20 -8
View File
@@ -24,7 +24,6 @@ import {
ProviderMetadata,
TransportError,
ToolResultValue,
UnknownProviderError,
type ContentPart,
type LLMRequest,
type Media,
@@ -367,6 +366,8 @@ function modelFromLanguage(info: RuntimeInfo, language: LanguageModelV3) {
provider: ProviderID.make(providerID),
providerMetadataKey: optionKey,
protocol: "ai-sdk",
// AI SDK providers convert tool schemas themselves, so model-family sanitizers stay off here.
sanitizer: "none",
endpoint: Endpoint.path("/", { baseURL: "https://ai-sdk.local" }),
auth: Auth.none,
transport: {
@@ -936,15 +937,18 @@ function llmError(error: unknown, operation: "request" | "read") {
code: network.code,
}),
})
return new AIError({
reason: new UnknownProviderError({
message: unknownErrorMessage(error),
body: errorBody(error),
cause: error,
}),
return RequestExecutor.httpFailure({
message: unknownErrorMessage(error),
data: errorValue(error) ?? error,
responseBody: errorBody(error),
cause: error,
})
}
// AI SDK stream errors can arrive as plain objects. A gateway's type validation error keeps the provider's error
// response in `value`, which carries the message and codes.
const errorValue = (error: unknown) => (ProviderShared.isRecord(error) ? error.value : undefined)
// Runtime-generated network failure shapes. The codes mirror the AI SDK's own
// Bun network error list in handleFetchError; the messages are undici's fetch
// TypeError and stream termination strings plus our SSE chunk timeout error.
@@ -1032,7 +1036,15 @@ const decodeProviderError = Schema.decodeUnknownOption(
)
function unknownErrorMessage(error: unknown) {
const message = error instanceof Error ? error.message : String(error)
const message =
error instanceof Error
? error.message
: typeof error === "string"
? error
: ([error, errorValue(error)]
.map((value) => Option.getOrUndefined(decodeProviderError(value)))
.flatMap((decoded) => [decoded?.error?.message, decoded?.message])
.find((value) => value?.trim()) ?? "")
return message.trim() === "" ? "Provider request failed" : message
}
+2
View File
@@ -101,6 +101,7 @@ import { VcsHgPlugin } from "./vcs/hg.js"
import { ToolInputRepairPlugin } from "./tool-input-repair.js"
import { OptimizePlugin } from "./optimize.js"
import { VcsGitPlugin } from "./vcs/git.js"
import { VerbosityPlugin } from "./verbosity.js"
import { WarmingPlugin } from "./warming.js"
import { WellKnownPlugin } from "../wellknown/plugin.js"
@@ -230,6 +231,7 @@ const pre = [
PatchTool.Plugin,
// Render model prompts after the patch plugin selects the available editing tools.
...OptimizePlugin.Plugins,
VerbosityPlugin.Plugin,
IdentityPlugin.Plugin,
EditTool.Plugin,
GlobTool.Plugin,
+54
View File
@@ -0,0 +1,54 @@
export * as VerbosityPlugin from "./verbosity.js"
import { define } from "@opencode/plugin/effect/plugin"
import type { SessionRequest } from "@opencode/plugin/effect/session"
import { Effect } from "effect"
import { Model } from "../model.js"
import { Provider } from "../provider.js"
import type { PluginInternal } from "./internal.js"
const direct = new Set([
"@opencode/ai/providers/openai",
"@opencode/ai/providers/openai/responses",
"@opencode/ai/providers/azure",
"@opencode/ai/providers/azure/responses",
])
const gateways = new Set(["@opencode/ai/providers/cloudflare-ai-gateway", Provider.aisdk("@ai-sdk/gateway")])
export const Plugin = define({
id: "opencode.prompt.verbosity",
effect: Effect.fn("VerbosityPlugin")(function* (ctx) {
const models = yield* Model.Service
const hook = (event: SessionRequest) =>
Effect.gen(function* () {
if (event.options.textVerbosity !== undefined) return
const model = yield* models.get(event.model.providerID, event.model.id)
if (!model) return
const id = openAIModelID(model)
if (!id || !supportsVerbosity(id)) return
if (model.settings?.textVerbosity !== undefined) return
const variant = model.variants.find((item) => item.id === event.model.variant)
if (variant?.settings?.textVerbosity !== undefined) return
event.options.textVerbosity = "low"
})
yield* ctx.session.hook("context", hook)
yield* ctx.session.hook("compaction", hook)
yield* ctx.session.hook("generate", hook)
yield* ctx.session.hook("title", hook)
}),
} satisfies PluginInternal.InternalPlugin)
function supportsVerbosity(id: string) {
if (id.includes("gpt-6")) return true
if (id.includes("-chat") || id.includes("-image")) return false
// New GPT-5 minor versions remain unset until their support is known.
return /(?:^|[/.])gpt-5\.[1-6](?:[.:-]|$)/.test(id) || /(?:^|[/.])gpt-5(?:-(?:mini|nano)(?:[.:-]|$)|$)/.test(id)
}
function openAIModelID(model: Model.Info) {
const id = model.modelID.toLowerCase()
if (direct.has(model.package ?? "")) return id
if (model.package === "@opencode/ai/providers/amazon-bedrock/mantle/responses" && id.startsWith("openai."))
return id.slice("openai.".length)
if (gateways.has(model.package ?? "") && id.startsWith("openai/")) return id.slice("openai/".length)
}
+6 -9
View File
@@ -37,9 +37,8 @@ import { toLLMMessages } from "./runner/to-llm-message.js"
import type { AgentNotFoundError } from "./error.js"
import type { Instructions } from "../instructions/index.js"
const DEFAULT_BUFFER = 20_000
const AUTO_THRESHOLD = 0.85
const DEFAULT_KEEP_TOKENS = 15_000
const OUTPUT_TOKEN_MAX = 32_000
const TOOL_OUTPUT_MAX_CHARS = 2_000
const IMAGE_TOKEN_ESTIMATE = 1_500
const PDF_TOKEN_ESTIMATE = 2_000
@@ -89,7 +88,7 @@ const LEGACY_HEADING = "## Additional Context"
export type Settings = {
auto: boolean
buffer: number
buffer?: number
tokens: number
}
@@ -401,7 +400,7 @@ export const layer = Layer.effect(
const state = State.create<Settings & { readonly native: NativeStrategy[] }, Editor>({
name: "session-compaction",
initial: () => ({ auto: true, buffer: DEFAULT_BUFFER, tokens: DEFAULT_KEEP_TOKENS, native: [] }),
initial: () => ({ auto: true, tokens: DEFAULT_KEEP_TOKENS, native: [] }),
editor: (editor) => ({
configure: (settings) => {
if (settings.auto !== undefined) editor.auto = settings.auto
@@ -754,11 +753,9 @@ export const layer = Layer.effect(
const limit = input.resolved.limit
const context = limit.context
if (context <= 0) return false
const output = Math.min(limit.output, OUTPUT_TOKEN_MAX)
const promptCeiling = Math.min(
limit.input === undefined ? Number.POSITIVE_INFINITY : limit.input - config.buffer,
context - Math.max(output, config.buffer),
)
const usable = Math.min(context, limit.input ?? context)
const promptCeiling =
config.buffer === undefined ? Math.floor(usable * AUTO_THRESHOLD) : usable - config.buffer
return estimateTokens(input) >= promptCeiling
}
const compactManual = Effect.fn("SessionCompaction.compactManual")(function* (input: ManualInput) {
+9 -2
View File
@@ -43,6 +43,7 @@ type Active = {
info: Info
file: string
size: number
newlines: number
// Resolves with the terminal Info once the command exits, times out, or is killed. A wait
// started after termination resolves immediately from the already-completed deferred.
done: Deferred.Deferred<Info, NotFoundError>
@@ -240,10 +241,12 @@ const layer = () =>
if (page.output.endsWith("\n")) lines.pop()
const truncated = latest.size > maxBytes || lines.length > maxLines
const text = lines.length > maxLines ? lines.slice(-maxLines).join("\n") : page.output
const total = (yield* require(info.id)).newlines + (page.output.endsWith("\n") ? 0 : 1)
const shown = Math.min(lines.length, maxLines)
const notice = truncated
? `${text ? "\n\n" : ""}[full output saved to ${info.file}]`
? `\n\n[showing lines ${total - shown + 1}-${total} of ${total}; full output saved to ${info.file}]`
: ""
return { output: `${text}${notice}`, truncated }
return { output: `${text || "(no output)"}${notice}`, truncated }
}).pipe(Effect.catchTag("Shell.NotFoundError", () => Effect.succeed(undefined)))
return { info, capture }
})
@@ -312,6 +315,7 @@ const layer = () =>
}),
file,
size: 0,
newlines: 0,
done: Deferred.makeUnsafe<Info, NotFoundError>(),
}
commands.set(id, command)
@@ -323,6 +327,9 @@ const layer = () =>
Effect.sync(() => {
stream.write(chunk)
command.size += chunk.length
// Count while streaming so truncation notices never rescan the output file.
for (let index = chunk.indexOf(10); index !== -1; index = chunk.indexOf(10, index + 1))
command.newlines++
}),
),
)
+3 -9
View File
@@ -75,23 +75,17 @@ const layer = Layer.effect(
const kept: string[] = []
let bytes = 0
let hitBytes = false
for (const line of lines.slice(0, limits.maxLines)) {
const size = Buffer.byteLength(line, "utf-8") + (kept.length > 0 ? 1 : 0)
if (bytes + size > limits.maxBytes) {
hitBytes = true
break
}
if (bytes + size > limits.maxBytes) break
kept.push(line)
bytes += size
}
if (!hitBytes && kept.length === lines.length && totalBytes > bytes) hitBytes = true
const removed = hitBytes ? totalBytes - bytes : lines.length - kept.length
const unit = hitBytes ? (removed === 1 ? "byte" : "bytes") : removed === 1 ? "line" : "lines"
const file = path.join(directory, Identifier.ascending("tool"))
yield* fs.ensureDir(directory).pipe(Effect.orDie)
yield* fs.writeFileString(file, text).pipe(Effect.orDie)
const marker = `... ${removed} ${unit} truncated; full content saved to ${file} ...`
const shown = kept.length > 0 ? `lines 1-${kept.length}` : "0 lines"
const marker = `[showing ${shown} of ${lines.length}; full output saved to ${file}]`
const bounded: Tool.Content[] = []
let remaining = kept.join("\n").length
let seenText = false
+1 -1
View File
@@ -308,7 +308,7 @@ function registrationError(tool: Tool.Info) {
if (error) return error
}
const name = normalizedName(tool)
if (!/^[A-Za-z0-9_-]{1,64}$/.test(name)) return new RegistrationError({ name, message: `Invalid tool name: ${name}` })
if (!/^[A-Za-z0-9_-]{1,128}$/.test(name)) return new RegistrationError({ name, message: `Invalid tool name: ${name}` })
const id = effectiveName(tool)
if (tool.options?.codemode === false && id === "execute")
return new RegistrationError({ name: id, message: 'Tool name "execute" is reserved for CodeMode' })
+22 -8
View File
@@ -28,6 +28,8 @@ const EFFORTS = ["low", "medium", "high"]
const ENCRYPTED_REASONING = ["reasoning.encrypted_content"]
const ADAPTIVE_THINKING = { type: "adaptive", display: "summarized" }
const ANTHROPIC_OUTPUT_TOKEN_MAX = 32_000
// Alibaba thinking budget variants stay under 64k instead of reaching the model's whole output limit.
const ALIBABA_THINKING_BUDGET_MAX = 64_000
const variant = (id: string, overlay: Overlay): Variants[number] => ({ id: Model.VariantID.make(id), ...overlay })
@@ -224,9 +226,12 @@ const alibabaChat: Protocol = (model, support) => {
case "toggle":
return toggle({ settings: { enableThinking: false } }, { settings: { enableThinking: true } })
case "budget_tokens":
return budgets(model, support, (tokens) => ({
settings: { enableThinking: true, thinkingBudget: tokens },
}))
return budgets(
model,
support,
(tokens) => ({ settings: { enableThinking: true, thinkingBudget: tokens } }),
ALIBABA_THINKING_BUDGET_MAX,
)
}
}
@@ -310,9 +315,12 @@ const alibabaMessages: Protocol = (model, support) => {
case "toggle":
return toggle({ settings: { thinking: { type: "disabled" } } }, { settings: { thinking: { type: "enabled" } } })
case "budget_tokens":
return budgets(model, support, (tokens) => ({
settings: { thinking: { type: "enabled", budgetTokens: tokens } },
}))
return budgets(
model,
support,
(tokens) => ({ settings: { thinking: { type: "enabled", budgetTokens: tokens } } }),
ALIBABA_THINKING_BUDGET_MAX,
)
}
}
@@ -390,8 +398,9 @@ const bedrockConverse: Protocol = (model, support) => {
model,
support,
(tokens) =>
// Claude's budget is a typed setting so the protocol can fit it under the output limit.
claude
? fields({ thinking: { type: "enabled", budget_tokens: tokens } })
? { settings: { thinking: { type: "enabled", budgetTokens: tokens } } }
: fields({ reasoningConfig: { type: "enabled", budgetTokens: tokens } }),
claude ? ANTHROPIC_OUTPUT_TOKEN_MAX : model.limit.output,
)
@@ -405,7 +414,12 @@ const alibabaAISDK: Protocol = (model, support) => {
case "toggle":
return toggle({ settings: { enableThinking: false } }, { settings: { enableThinking: true } })
case "budget_tokens":
return budgets(model, support, (tokens) => ({ settings: { enableThinking: true, thinkingBudget: tokens } }))
return budgets(
model,
support,
(tokens) => ({ settings: { enableThinking: true, thinkingBudget: tokens } }),
ALIBABA_THINKING_BUDGET_MAX,
)
}
}
+11
View File
@@ -247,6 +247,17 @@ describe("AISDKNative", () => {
},
})
expect(
map(
"@ai-sdk/amazon-bedrock",
{ reasoningConfig: { type: "enabled", budgetTokens: 12_000 } },
"anthropic.claude-sonnet-4-5-20250929-v1:0",
),
).toEqual({
package: "@opencode/ai/providers/amazon-bedrock",
settings: { thinking: { type: "enabled", budgetTokens: 12_000 } },
})
// gpt-oss (Harmony) keeps the flat chat-completions field.
expect(
map("@ai-sdk/amazon-bedrock", { reasoningConfig: { maxReasoningEffort: "high" } }, "openai.gpt-oss-120b-1:0")
+30
View File
@@ -13,6 +13,7 @@ import {
CompactionPart,
ProviderID,
HttpContext,
InvalidRequestError,
LLMEvent,
Message,
RateLimitError,
@@ -374,6 +375,18 @@ it.effect("routes AI Gateway model options by upstream prefix", () =>
bedrock: { reasoningConfig: { type: "enabled" } },
})
const openai = yield* aisdk.model({
...model("@ai-sdk/gateway", { gateway: { order: ["openai"] } }),
modelID: Model.ID.make("openai/gpt-5.5"),
})
const openaiPrepared = yield* compileRequest(
LLM.request({ model: openai, prompt: "Hello", providerOptions: { textVerbosity: "low" } }),
)
expect(openaiPrepared.body.providerOptions).toEqual({
gateway: { order: ["openai"] },
openai: { textVerbosity: "low" },
})
const fallback = yield* aisdk.model({
...model("@ai-sdk/gateway", { reasoningEffort: "high" }),
modelID: Model.ID.make("deepseek/deepseek-v4"),
@@ -883,6 +896,23 @@ Object.values({
)
})
// Shapes the Vercel AI Gateway streams when the upstream rejects a request.
Object.entries({
"type validation": {
name: "AI_TypeValidationError",
value: { error: { type: "invalid_request_error", message: "Bad max_tokens" } },
},
invalid_request: { code: "invalid_request", message: "Bad max_tokens" },
}).forEach(([shape, failure]) =>
it.effect(`reads gateway ${shape} stream errors as invalid requests`, () =>
Effect.gen(function* () {
const error = yield* streamFailure(failure, true)
expect(error.message).toBe("Bad max_tokens")
expect(error.reason).toBeInstanceOf(InvalidRequestError)
}),
),
)
it.effect("does not copy Error request internals into the provider body", () =>
Effect.gen(function* () {
const cause = Object.assign(new Error("Connection failed"), {
+1 -1
View File
@@ -172,5 +172,5 @@ const input = (tokens: number) => {
},
}
}
const bufferedInput = input(85_000)
const bufferedInput = input(82_000)
const nearInput = input(95_000)
+2 -2
View File
@@ -2117,7 +2117,7 @@ testEffect(Layer.empty).live("isolates invalid MCP tools and preserves plugin tr
}) satisfies Mcp.Tool
const healthy = [tool("demo", "search"), tool("other", "lookup")]
const namespace = tool("x".repeat(65), "lookup")
const catalog = yield* Ref.make([tool("demo", "x".repeat(65)), ...healthy, namespace])
const catalog = yield* Ref.make([tool("demo", "x".repeat(129)), ...healthy, namespace])
yield* Effect.gen(function* () {
const registry = yield* Tool.Service
@@ -2146,7 +2146,7 @@ testEffect(Layer.empty).live("isolates invalid MCP tools and preserves plugin tr
editor.remove("repaired_lookup")
})
yield* Ref.set(catalog, [tool("demo", "y".repeat(65)), ...healthy, tool("demo", "added"), namespace])
yield* Ref.set(catalog, [tool("demo", "y".repeat(129)), ...healthy, tool("demo", "added"), namespace])
yield* bus.publish(McpEvent.ToolsChanged, { server: "demo" })
yield* waitForTool(registry, "demo_added")
expect((yield* toolDefinitions(registry)).map((tool) => tool.name)).toEqual([
+163
View File
@@ -0,0 +1,163 @@
import { expect } from "bun:test"
import { Effect } from "effect"
import { Agent } from "@opencode/core/agent"
import { Model } from "@opencode/core/model"
import { Plugin } from "@opencode/core/plugin"
import { PluginHooks } from "@opencode/core/plugin/hooks"
import { PluginHost } from "@opencode/core/plugin/host"
import { VerbosityPlugin } from "@opencode/core/plugin/verbosity"
import { Provider } from "@opencode/core/provider"
import { Session } from "@opencode/core/session"
import type { SessionHooks } from "@opencode/plugin/effect/session"
import { testEffect } from "../lib/effect"
import { PluginTestLayer } from "./fixture"
const it = testEffect(PluginTestLayer)
const ref = (providerID: string, id: string, variant?: string) =>
Model.Ref.make({
providerID: Provider.ID.make(providerID),
id: Model.ID.make(id),
...(variant ? { variant: Model.VariantID.make(variant) } : {}),
})
const request = (model: Model.Ref, options: SessionHooks["context"]["options"] = {}): SessionHooks["context"] => ({
sessionID: Session.ID.make("ses_verbosity"),
agent: Agent.ID.make("build"),
model,
system: [],
messages: [],
tools: {},
options,
})
it.effect("sets known OpenAI Responses defaults without overriding configured or unknown models", () =>
Effect.gen(function* () {
const providers = yield* Provider.Service
const plugins = yield* Plugin.Service
const hooks = yield* PluginHooks.Service
yield* providers.transform((editor) => {
editor.add({
info: { ...Provider.Info.empty(Provider.ID.make("openai")), package: "@opencode/ai/providers/openai" },
models: [
{ ...Model.Info.default(Provider.ID.make("openai"), Model.ID.make("gpt-5.5")) },
{ ...Model.Info.default(Provider.ID.make("openai"), Model.ID.make("gpt-5.6-luna-fast")) },
{ ...Model.Info.default(Provider.ID.make("openai"), Model.ID.make("gpt-5.2-codex")) },
{ ...Model.Info.default(Provider.ID.make("openai"), Model.ID.make("gpt-5-mini-fast")) },
{ ...Model.Info.default(Provider.ID.make("openai"), Model.ID.make("gpt-6-astra-pro")) },
{ ...Model.Info.default(Provider.ID.make("openai"), Model.ID.make("gpt-4o")) },
{ ...Model.Info.default(Provider.ID.make("openai"), Model.ID.make("gpt-7")) },
{ ...Model.Info.default(Provider.ID.make("openai"), Model.ID.make("gpt-5.7")) },
{ ...Model.Info.default(Provider.ID.make("openai"), Model.ID.make("gpt-5.5-chat")) },
{ ...Model.Info.default(Provider.ID.make("openai"), Model.ID.make("gpt-5.4-image-2")) },
{
...Model.Info.default(Provider.ID.make("openai"), Model.ID.make("gpt-6-astra")),
variants: [{ id: Model.VariantID.make("quiet"), settings: { textVerbosity: null } }],
},
{
...Model.Info.default(Provider.ID.make("openai"), Model.ID.make("chat")),
modelID: Model.ID.make("gpt-5.5"),
package: "@opencode/ai/providers/openai/chat",
},
],
})
editor.add({
info: {
...Provider.Info.empty(Provider.ID.make("opencode")),
package: "@opencode/ai/providers/openai-compatible",
},
models: [
{
...Model.Info.default(Provider.ID.make("opencode"), Model.ID.make("astra-alias")),
modelID: Model.ID.make("gpt-6-astra"),
package: "@opencode/ai/providers/openai/responses",
},
{
...Model.Info.default(Provider.ID.make("opencode"), Model.ID.make("no-default")),
modelID: Model.ID.make("gpt-6-astra"),
package: "@opencode/ai/providers/openai/responses",
settings: { textVerbosity: null },
},
],
})
editor.add({
info: { ...Provider.Info.empty(Provider.ID.make("openrouter")), package: "@opencode/ai/providers/openrouter" },
models: [Model.Info.default(Provider.ID.make("openrouter"), Model.ID.make("gpt-5.5"))],
})
editor.add({
info: {
...Provider.Info.empty(Provider.ID.make("configured")),
package: "@opencode/ai/providers/openai",
settings: { textVerbosity: "medium" },
},
models: [Model.Info.default(Provider.ID.make("configured"), Model.ID.make("gpt-5.5"))],
})
for (const [providerID, packageName, modelID] of [
["azure", "@opencode/ai/providers/azure/responses", "gpt-5.5"],
["bedrock-mantle", "@opencode/ai/providers/amazon-bedrock/mantle/responses", "openai.gpt-6-sol"],
["cloudflare", "@opencode/ai/providers/cloudflare-ai-gateway", "openai/gpt-5.6-sol"],
["vercel", Provider.aisdk("@ai-sdk/gateway"), "openai/gpt-6-astra-fast"],
["azure-chat", "@opencode/ai/providers/azure/chat", "gpt-5.5"],
["bedrock-converse", "@opencode/ai/providers/amazon-bedrock", "global.openai.gpt-6-sol"],
["cloudflare-chat", "@opencode/ai/providers/cloudflare-ai-gateway", "workers-ai/gpt-5.5"],
["vercel-other", Provider.aisdk("@ai-sdk/gateway"), "anthropic/gpt-5.5"],
] as const) {
editor.add({
info: { ...Provider.Info.empty(Provider.ID.make(providerID)), package: packageName },
models: [
{
...Model.Info.default(Provider.ID.make(providerID), Model.ID.make("selected")),
modelID: Model.ID.make(modelID),
},
],
})
}
})
yield* VerbosityPlugin.Plugin.effect(yield* PluginHost.make(plugins))
for (const kind of ["context", "compaction", "generate", "title"] as const) {
const event = request(ref("openai", "gpt-5.5"))
yield* hooks.trigger("session", kind, event)
expect(event.options.textVerbosity).toBe("low")
}
for (const id of ["gpt-5.6-luna-fast", "gpt-5.2-codex", "gpt-5-mini-fast", "gpt-6-astra-pro"]) {
const event = request(ref("openai", id))
yield* hooks.trigger("session", "context", event)
expect(event.options.textVerbosity).toBe("low")
}
for (const model of [
ref("openai", "gpt-4o"),
ref("openai", "gpt-7"),
ref("openai", "gpt-5.7"),
ref("openai", "gpt-5.5-chat"),
ref("openai", "gpt-5.4-image-2"),
ref("openai", "chat"),
ref("openrouter", "gpt-5.5"),
ref("configured", "gpt-5.5"),
ref("azure-chat", "selected"),
ref("bedrock-converse", "selected"),
ref("cloudflare-chat", "selected"),
ref("vercel-other", "selected"),
ref("openai", "gpt-6-astra", "quiet"),
ref("opencode", "no-default"),
]) {
const event = request(model)
yield* hooks.trigger("session", "context", event)
expect(event.options.textVerbosity).toBeUndefined()
}
const alias = request(ref("opencode", "astra-alias"))
yield* hooks.trigger("session", "context", alias)
expect(alias.options.textVerbosity).toBe("low")
for (const providerID of ["azure", "bedrock-mantle", "cloudflare", "vercel"]) {
const event = request(ref(providerID, "selected"))
yield* hooks.trigger("session", "context", event)
expect(event.options.textVerbosity).toBe("low")
}
const overridden = request(ref("openai", "gpt-5.5"), { textVerbosity: "high" })
yield* hooks.trigger("session", "context", overridden)
expect(overridden.options.textVerbosity).toBe("high")
}),
)
+23 -10
View File
@@ -153,7 +153,7 @@ test("compaction prompts prohibit task execution", () => {
expect(SessionCompaction.buildPrompt(update)).toContain("Do not continue the task or call tools")
})
it.effect("auto compaction estimates current content against the buffered prompt ceiling", () =>
it.effect("auto compaction uses 85% by default and a configured buffer instead", () =>
Effect.gen(function* () {
const compaction = yield* SessionCompaction.Service
const session = Session.Info.make({
@@ -205,23 +205,27 @@ it.effect("auto compaction estimates current content against the buffered prompt
}
const inputLimited = { context: 400_000, input: 272_000, output: 128_000 }
expect(compaction.required(input(251_999, inputLimited))).toBe(false)
expect(compaction.required(input(252_000, inputLimited))).toBe(true)
expect(compaction.required(input(231_199, inputLimited))).toBe(false)
expect(compaction.required(input(231_200, inputLimited))).toBe(true)
const native = (tokens: number, limit: { context: number; input?: number; output: number } = inputLimited) => {
const selected = input(tokens, limit)
return { ...selected, resolved: { ...selected.resolved, compaction: { type: "native" as const } } }
}
expect(compaction.required(native(251_999))).toBe(false)
expect(compaction.required(native(252_000))).toBe(true)
expect(compaction.required(native(231_199))).toBe(false)
expect(compaction.required(native(231_200))).toBe(true)
expect(compaction.required(native(1_000_000, { context: 0, input: undefined, output: 0 }))).toBe(false)
const contextLimited = { context: 100_000, output: 10_000 }
expect(compaction.required(input(79_999, contextLimited))).toBe(false)
expect(compaction.required(input(80_000, contextLimited))).toBe(true)
expect(compaction.required(input(84_999, contextLimited))).toBe(false)
expect(compaction.required(input(85_000, contextLimited))).toBe(true)
const outputLimited = { context: 100_000, output: 30_000 }
expect(compaction.required(input(69_999, outputLimited))).toBe(false)
expect(compaction.required(input(70_000, outputLimited))).toBe(true)
expect(compaction.required(input(84_999, outputLimited))).toBe(false)
expect(compaction.required(input(85_000, outputLimited))).toBe(true)
const smallWindow = { context: 32_000, output: 32_000 }
expect(compaction.required(input(27_199, smallWindow))).toBe(false)
expect(compaction.required(input(27_200, smallWindow))).toBe(true)
const assistant = input(79_000, contextLimited).messages[0]
const tool = SessionMessage.AssistantTool.make({
@@ -233,7 +237,9 @@ it.effect("auto compaction estimates current content against the buffered prompt
})
const grown = { ...input(79_000, contextLimited), messages: [{ ...assistant, content: [tool] }] }
expect(SessionCompaction.estimateTokens(grown)).toBe(80_000)
expect(compaction.required(grown)).toBe(true)
expect(compaction.required(grown)).toBe(false)
const near = input(84_000, contextLimited)
expect(compaction.required({ ...near, messages: [{ ...near.messages[0], content: [tool] }] })).toBe(true)
const interrupted = { ...assistant, id: SessionMessage.ID.create(), tokens: undefined }
expect(SessionCompaction.estimateTokens({ ...grown, messages: [...grown.messages, interrupted] })).toBe(80_001)
@@ -285,6 +291,13 @@ it.effect("auto compaction estimates current content against the buffered prompt
time: { created: 0, completed: 0 },
})
expect(compaction.required({ ...grown, messages: [checkpoint] })).toBe(false)
yield* compaction.transform((editor) => editor.configure({ buffer: 10_000 }))
expect(compaction.required(input(89_999, contextLimited))).toBe(false)
expect(compaction.required(input(90_000, contextLimited))).toBe(true)
yield* compaction.transform((editor) => editor.configure({ buffer: 0 }))
expect(compaction.required(input(99_999, contextLimited))).toBe(false)
expect(compaction.required(input(100_000, contextLimited))).toBe(true)
}),
)
@@ -362,6 +362,7 @@ it.live("manual and automatic endpoint compaction keep the provider replacement
expect(JSON.stringify(replacement)).not.toContain("Original user")
expect(fixture.state.calls).toBe(2)
expect(fixture.headers[0]?.get("x-http-hook")).toBe("compaction")
expect(fixture.bodies[0]).toMatchObject({ tools: [expect.objectContaining({ name: "read" })] })
expect(fixture.bodies[0]).not.toHaveProperty("context_management")
}),
)
+3 -3
View File
@@ -2883,7 +2883,7 @@ describe("SessionRunnerLLM", () => {
agent.steps = 2
}),
)
yield* s.llm.push(TestLLM.textWithUsage("Earlier answer", "before-native", 10_000))
yield* s.llm.push(TestLLM.textWithUsage("Earlier answer", "before-native", 36_000))
yield* s.runPrompt("First real request")
const checkpoint = (encrypted: string) =>
CompactionCheckpointResponse.make({
@@ -2903,7 +2903,7 @@ describe("SessionRunnerLLM", () => {
const installed = (yield* s.messages).filter((message) => message.type === "compaction")
expect(installed).toMatchObject([{ status: "completed", reason: "auto", providerContext: { version: 1 } }])
// New input without a post-checkpoint usage anchor must not retrigger compaction.
yield* s.llm.push(TestLLM.textWithUsage("Measured", "measured", 10_000))
yield* s.llm.push(TestLLM.textWithUsage("Measured", "measured", 36_000))
yield* s.runPrompt("Third real request")
expect(s.requests).toHaveLength(5)
yield* s.llm.push(checkpoint("second"), TestLLM.textWithUsage("Continued", "continued", 10_000))
@@ -2929,7 +2929,7 @@ describe("SessionRunnerLLM", () => {
s.currentModel = LanguageModel.make({ id: "native", provider: "openai", route: OpenAIResponses.route })
modelLimits.set("native", { context: 42_000, output: 32_000 })
s.compaction = { type: "native" }
yield* s.llm.push(TestLLM.textWithUsage("Earlier answer", "before-native", 10_000))
yield* s.llm.push(TestLLM.textWithUsage("Earlier answer", "before-native", 36_000))
yield* s.runPrompt("Original durable request")
yield* s.llm.push(
CompactionCheckpointResponse.make({
+5 -5
View File
@@ -46,14 +46,14 @@ describe("ToolOutput", () => {
expect(yield* fs.readFileString(outputPath)).toBe("one\ntwo\nthree")
expect(result.content).toEqual([
{ type: "text", text: "one\ntwo" },
{ type: "text", text: `... 1 line truncated; full content saved to ${outputPath} ...` },
{ type: "text", text: `[showing lines 1-2 of 3; full output saved to ${outputPath}]` },
])
}),
{ maxLines: 2, maxBytes: 1_000 },
),
)
it.live("reports bytes omitted by the byte limit", () =>
it.live("reports lines shown under the byte limit", () =>
withStore(
(output) =>
Effect.gen(function* () {
@@ -62,7 +62,7 @@ describe("ToolOutput", () => {
{ type: "text", text: "one" },
{
type: "text",
text: expect.stringMatching(/^\.\.\. 4 bytes truncated; full content saved to .+ \.\.\.$/),
text: expect.stringMatching(/^\[showing lines 1-1 of 2; full output saved to .+\]$/),
},
])
}),
@@ -82,7 +82,7 @@ describe("ToolOutput", () => {
{ type: "text", text: "before" },
file,
{ type: "text", text: "after" },
{ type: "text", text: expect.stringMatching(/^\.\.\. 1 line truncated; full content saved to /) },
{ type: "text", text: expect.stringMatching(/^\[showing lines 1-2 of 3; full output saved to /) },
])
}),
{ maxLines: 2, maxBytes: 1_000 },
@@ -130,7 +130,7 @@ describe("ToolOutput", () => {
const result = yield* output.truncate({ content: [{ type: "text", text: "one\n" }] })
expect(result.content).toEqual([
{ type: "text", text: "one" },
{ type: "text", text: expect.stringMatching(/^\.\.\. 1 byte truncated; full content saved to /) },
{ type: "text", text: expect.stringMatching(/^\[showing lines 1-1 of 1; full output saved to /) },
])
}),
{ maxLines: 2, maxBytes: 3 },
+22 -1
View File
@@ -329,7 +329,7 @@ describe("Tool", () => {
{
before: make(),
"": make(),
["x".repeat(65)]: make(),
["x".repeat(129)]: make(),
"echo.tool": constant("first"),
echo_tool: constant("last"),
execute: make(),
@@ -346,6 +346,27 @@ describe("Tool", () => {
}),
)
it.effect("registers 128-character MCP tool names in Code Mode", () =>
Effect.gen(function* () {
const service = yield* Tool.Service
const name = "x".repeat(128)
yield* transform(service, { [name]: make(), ["x".repeat(129)]: make() }, { namespace: "cloudflare" })
const snapshot = yield* service.snapshot()
expect(codeModeListings(snapshot.codeModeCatalog!).map((tool) => tool.path)).toEqual([`cloudflare.${name}`])
const result = yield* snapshot.execute({
...call("execute"),
call: {
type: "tool-call",
id: "call-long-mcp-name",
name: "execute",
input: { code: `return (await tools.cloudflare[${JSON.stringify(name)}]({ text: "hello" })).text` },
},
})
expect(result.content).toEqual([{ type: "text", text: "hello" }])
}),
)
it.effect("executes native tools without requiring letter-leading names or namespace segments", () =>
Effect.gen(function* () {
const service = yield* Tool.Service
+1 -1
View File
@@ -1555,7 +1555,7 @@ describe("ShellTool", () => {
{
id: settled.metadata?.shellID,
status: "completed",
output: "Exited with code 7",
output: "(no output)\n\nExited with code 7",
},
])
}),
+32 -2
View File
@@ -175,8 +175,8 @@ test("spells Chat Completions variants for direct providers", () => {
]),
).toEqual([
{ id: "none", settings: { enableThinking: false } },
{ id: "high", settings: { enableThinking: true, thinkingBudget: 131_072 } },
{ id: "max", settings: { enableThinking: true, thinkingBudget: 262_144 } },
{ id: "high", settings: { enableThinking: true, thinkingBudget: 32_000 } },
{ id: "max", settings: { enableThinking: true, thinkingBudget: 63_999 } },
])
expect(
@@ -196,6 +196,17 @@ test("spells Chat Completions variants for direct providers", () => {
])
})
test("spells Bedrock Converse Claude budgets as a thinking setting", () => {
expect(
resolve(model("@opencode/ai/providers/amazon-bedrock", "us.anthropic.claude-haiku-4-5-20251001-v1:0", 64_000), [
{ type: "budget_tokens", min: 1024 },
]),
).toEqual([
{ id: "high", settings: { thinking: { type: "enabled", budgetTokens: 16_000 } } },
{ id: "max", settings: { thinking: { type: "enabled", budgetTokens: 31_999 } } },
])
})
test("spells Bedrock Converse effort for Grok and Nova", () => {
const supports: Variant.Support[] = [{ type: "effort", values: ["low", "xhigh"] }]
expect(resolve(model("@opencode/ai/providers/amazon-bedrock", "us.xai.grok-4.6"), supports)).toEqual([
@@ -214,6 +225,25 @@ test("spells Bedrock Converse effort for Grok and Nova", () => {
])
})
test("caps Alibaba thinking budget variants at 64k", () => {
const supports: Variant.Support[] = [{ type: "toggle" }, { type: "budget_tokens" }]
expect(resolve(model("@opencode/ai/providers/alibaba/chat", "kimi-k2.6", 262_144), supports)).toEqual([
{ id: "none", settings: { enableThinking: false } },
{ id: "high", settings: { enableThinking: true, thinkingBudget: 32_000 } },
{ id: "max", settings: { enableThinking: true, thinkingBudget: 63_999 } },
])
expect(resolve(model("@opencode/ai/providers/alibaba/messages", "kimi-k2.6", 262_144), supports)).toEqual([
{ id: "none", settings: { thinking: { type: "disabled" } } },
{ id: "high", settings: { thinking: { type: "enabled", budgetTokens: 32_000 } } },
{ id: "max", settings: { thinking: { type: "enabled", budgetTokens: 63_999 } } },
])
expect(resolve(model("@opencode/ai/providers/alibaba/chat", "kimi-k2.5", 32_768), supports)).toEqual([
{ id: "none", settings: { enableThinking: false } },
{ id: "high", settings: { enableThinking: true, thinkingBudget: 16_384 } },
{ id: "max", settings: { enableThinking: true, thinkingBudget: 32_767 } },
])
})
test("spells Chat Completions variants for hosting providers", () => {
expect(
resolve(model("@opencode/ai/providers/openai-compatible", "deepseek-ai/deepseek-v4-pro", undefined, "nvidia"), [

Some files were not shown because too many files have changed in this diff Show More