mirror of
https://github.com/anomalyco/opencode.git
synced 2026-09-25 18:17:35 +00:00
Compare commits
15
Commits
tiny-leftovers
..
v2
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6585bb7105 | ||
|
|
f954688fbb | ||
|
|
1463dabde9 | ||
|
|
29ea6ee05b | ||
|
|
b170904731 | ||
|
|
88e1fa9304 | ||
|
|
ff1bf315ed | ||
|
|
144ce00e00 | ||
|
|
ad504094f0 | ||
|
|
bad6834a3e | ||
|
|
0c4bbc3cd1 | ||
|
|
ae7dd82126 | ||
|
|
65d5123ead | ||
|
|
4eb46a8885 | ||
|
|
1986e92842 |
@@ -12,9 +12,9 @@
|
||||
|
||||
Per-type constructors live on the type, not as top-level re-exports. Use `Message.system(...)`, `Message.user(...)`, `Message.assistant(...)`, `Message.tool(...)`, `Message.media(...)`, `LanguageModel.make(...)`, `ToolDefinition.make(...)`, `ToolCallPart.make(...)`, `ToolResultPart.make(...)`, `ToolChoice.make(...)`, `ToolChoice.named(...)`, `SystemPart.make(...)`, and `GenerationOptions.make(...)` directly. The top-level `LLM` namespace is reserved for request-shaped call APIs: `LLM.request`, `LLM.generate`, `LLM.stream`, and `LLM.generateObject`. `LLM.generate`/`LLM.stream` and Promise `ai.llm.generate`/`ai.llm.stream` accept ergonomic input or an `LLMRequest`; both paths use the same canonical request. Core still builds, logs, replays, and updates that durable `LLMRequest` boundary. Use `LLMRequest.update(...)` when deriving canonical request data; do not add a duplicate `LLM.updateRequest(...)` path.
|
||||
|
||||
Modality namespaces mirror `LLM` exactly: `Image.request`, `Image.generate`, `Image.stream` (later `Video`, `Speech`, `Transcription`). Common request fields (`images`, `mask`, `n`, `size`, `aspectRatio`, `seed`, `format`) lower natively or fail with a typed `AIError`; provider-native controls always live under `providerOptions`, never under a modality-specific `options` key.
|
||||
Modality namespaces mirror `LLM` exactly: `Image.request`, `Image.generate`, `Image.stream`, and the same for `Video`, `Speech`, and `Transcription`. Common request fields (`images`, `mask`, `n`, `size`, `aspectRatio`, `seed`, `format`) lower natively or fail with a typed `AIError`; provider-native controls always live under `providerOptions`, never under a modality-specific `options` key.
|
||||
|
||||
Media payloads are always `Media.Asset` (`src/media.ts`). Construct them with `Media.bytes`, `Media.base64`, `Media.url`, `Media.ref`, `Media.fromDataUrl`, or `Media.file`; never introduce a parallel `data: string | Uint8Array` shape. `MediaPart.media`, `ImageRequest.images`/`mask`, `ImageResponse.images`, and the `media` `LLMEvent` all share it. Protocols branch on `asset.source.type` and `asset.kind` and use `ProviderShared.inlineMedia` / `requireInlineMedia` / `mediaUrl` / `MediaInput.refID` rather than re-deriving base64 or URL handling.
|
||||
Media payloads are always `Media.Asset` (`src/media.ts`). Construct them with `Media.bytes`, `Media.base64`, `Media.url`, `Media.ref`, `Media.fromDataUrl`, or `Media.file`; never introduce a parallel `data: string | Uint8Array` shape. `MediaPart.media`, `ImageRequest.images`/`mask`, `ImageResponse.images`, and the `media` `LLMEvent` all share it. Protocols branch on `asset.source.type` and `asset.kind` and use `ProviderShared.requireInlineMedia` / `inlineRequired` / `mediaUrl` / `mediaReference` and `MediaInput.inlineBytes` / `refID` rather than re-deriving base64 or URL handling.
|
||||
|
||||
`schema/messages.ts → media.ts → route/executor-service.ts` is an accepted runtime dependency from the schema layer on the executor service tag: `Media.Asset.bytes()` must be able to download `url` sources, and the tag lives in that leaf module precisely so the schema barrel never imports the executor implementation (which imports the schema barrel back). Do not move the tag into `route/executor.ts` or import `route/executor.ts` from `src/schema/*` or `src/media.ts`.
|
||||
|
||||
@@ -98,7 +98,7 @@ When a provider supports multiple physical transports, selection remains executi
|
||||
|
||||
Media does not fit the SSE-frames-to-event-state-machine LLM route. `MediaRoute.inline(...)` / `queued(...)` / `stream(...)` (`src/route/media.ts`) compose a `MediaProtocol` kind with `Endpoint` and `Auth` and own the transport plumbing: `http` option merging, URL/query rendering, auth headers, JSON vs multipart encoding, and handing the response back to the protocol. `MediaProtocol.inline` (`src/route/media-protocol.ts`) is `body.from(request)` plus `response.decode(response, context)`; each protocol declares `const route = MediaProtocol.identity({ id, name, provider })` once and decodes through `route.decodeJson` / `route.text` / `route.decodeStarted` so decode failures retain the raw body and HTTP context, raising `route.unsupported(operation, message)` for requests it cannot lower, and passes `route` as the first argument to `MediaProtocol.inline` / `queued` / `stream`. `Generation` (`src/generation.ts`) is the provider-neutral handle for a queued generation over a `GenerationRoute` (`status`, `result`, `cancel`). Image protocol files follow the same section order as LLM protocols and declare unsupported common fields once through the protocol's `unsupported` list.
|
||||
|
||||
`MediaProtocol.queued` is the submit-then-poll kind every video route uses: `start` (body + decode into `{ token, snapshot }`), `status`, `result`, and optional `cancel`, each addressed by a route-owned `token` whose `Schema.Codec` makes it serializable. `MediaRoute.inline` and `MediaRoute.queued` compose the two kinds with `Endpoint` and `Auth`; the queued route decodes the token once at the boundary (`start` output or `resume` input) and closes over it in a token-free `GenerationRoute` (`status`/`result`/`cancel` are plain Effects), so `Generation` never sees the token's shape and only carries the encoded JSON for persistence. Polls reuse the route's auth and deployment headers plus the request's `http` overlay after `start`, and resolve relative paths against the route base URL (provider-issued absolute URLs such as fal's `status_url` pass through). `result` is always its own GET even when the provider returns output inside the status document, so `Generation.await` behaves the same after `start` and after `resume`. `PollContext.auth` carries only what `Auth` added or changed so protocols can hand download credentials to output assets as transient `Media.Asset.headers` (Veo) — never part of `source` or JSON. Status strings map through a per-protocol `STATUS` table via `MediaProtocol.status`; terminal generations without output fail through `output.ended` / `output.contentPolicy` with the provider document on `reason.body`. `GenerationAwaitOptions` (`AwaitOptions` in `src/generation.ts`, `{ poll?: Poll }`) is the one options type for `await`, `events`, `Video.generate`, and `Video.stream`.
|
||||
`MediaProtocol.queued` is the submit-then-poll kind every video route uses: `start` (body + decode into `{ token, snapshot }`), `status`, `result`, and optional `cancel` (with `activeOnly` when the provider's cancel endpoint deletes finished work, as Runway's does: the route refreshes status first and skips terminal generations), each addressed by a route-owned `token` whose `Schema.Codec` makes it serializable. `MediaRoute.inline` and `MediaRoute.queued` compose the two kinds with `Endpoint` and `Auth`; the queued route decodes the token once at the boundary (`start` output or `resume` input) and closes over it in a token-free `GenerationRoute` (`status`/`result`/`cancel` are plain Effects), so `Generation` never sees the token's shape and only carries the encoded JSON for persistence. Polls reuse the route's auth and deployment headers plus the request's `http` overlay after `start`, and resolve relative paths against the route base URL (provider-issued absolute URLs such as fal's `status_url` pass through). `result` is always its own GET even when the provider returns output inside the status document, so `Generation.await` behaves the same after `start` and after `resume`. `PollContext.auth` carries only what `Auth` added or changed so protocols can hand download credentials to output assets as transient `Media.Asset.headers` (Veo) — never part of `source` or JSON. Status strings map through a per-protocol `STATUS` table via `MediaProtocol.status`; terminal generations without output fail through `output.ended` / `output.contentPolicy` with the provider document on `reason.body`. `GenerationAwaitOptions` (`AwaitOptions` in `src/generation.ts`, `{ poll?: Poll }`) is the one options type for `await`, `events`, `Video.generate`, and `Video.stream`.
|
||||
|
||||
`MediaProtocol.stream` is the incremental kind every speech route uses, with the same discipline as LLM protocols. `MediaRoute.stream` submits the caller's request as `MediaProtocol.Addressed<Request>` (`{ ...request, mode }`, `mode: "generate" | "stream"`), so one provider stays one protocol: `body.from`, the endpoint path, and `frames` read `request.mode` to pick the body, path, and framing. `frames(bytes, context)` returns frames — `Framing.sse`, `Framing.lines`, `Framing.document` (a single-document response shaped like a streamed record), or the raw `bytes` for chunked audio. `initial()` is fresh per-response parser state; `step` folds each frame into it and emits modality events; `finish(state, context)` runs once after the last frame with the request, body, and observed `http` (header-only usage lives there) and emits exactly one terminal event or fails with `route.incomplete()`. Keep parser state to real accumulators and derive anything the request or body determines in `finish`. `generate` runs the same stream and folds it with the modality's `collect`. Request-derived URL parameters go on the body's `query` (array values repeat the parameter), applied before route and caller `http.query`. Decode frames with `route.decodeFrame` and raise stream-time failures with `route.frameError` (the frame stays on `reason.body`); protocols never thread HTTP context, because the route fills `reason.http` on stream errors that lack it. Speech protocols share `protocols/utils/speech-stream.ts` for deltas, timestamps, voice ids, PCM and container descriptions, and the terminal asset.
|
||||
|
||||
@@ -112,7 +112,7 @@ For providers where the URL is derived from typed inputs (Azure resource name, B
|
||||
|
||||
### Provider Facades
|
||||
|
||||
Provider-facing APIs are configured facades over route values. Endpoint/auth/resource/API-version setup happens before model selection, and model selectors accept only a model or deployment id. Media models use per-modality selectors on the same facade (`openai.image(id)`, later `.video` / `.speech` / `.transcription`) that mirror `openai.responses(id)`; the one-word overlap with the request namespace is accepted over a second construction path:
|
||||
Provider-facing APIs are configured facades over route values. Endpoint/auth/resource/API-version setup happens before model selection, and model selectors accept only a model or deployment id. Media models use per-modality selectors on the same facade (`openai.image(id)`, `.speech(id)`, `.transcription(id)`, `google.video(id)`) that mirror `openai.responses(id)`; the one-word overlap with the request namespace is accepted over a second construction path:
|
||||
|
||||
```ts
|
||||
const openai = OpenAI.configure({ apiKey, baseURL })
|
||||
|
||||
+19
-18
@@ -475,18 +475,18 @@ const program = Effect.gen(function* () {
|
||||
Common fields are portable in shape, not in support. Unsupported fields fail with a typed `AIError` before any network
|
||||
call rather than being dropped, so check this table before swapping only the `model`:
|
||||
|
||||
| Provider | `n` | `size` | `aspectRatio` | `seed` | `format` | `images` | `mask` |
|
||||
| --------------------- | --- | --------- | ------------- | ------ | -------- | ------------------------- | ------------------- |
|
||||
| OpenAI | ✓¹ | ✓ | ✗ | ✗ | ✓ | ✓ | ✓ |
|
||||
| Google (Gemini) | 1 | ✗ | ✓ | ✓ | ✗ | ✓ (no public URLs) | ✗ |
|
||||
| xAI | ✓ | ✗ | ✓ | ✗ | ✗ | ✓ | ✗ |
|
||||
| Z.ai | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ |
|
||||
| Meta | ✓ | ✓ (hint) | ✗ | ✗ | ✓ | ✓ | ✗ |
|
||||
| Black Forest Labs | 1 | per model | per model | ✓ | ✓ | per model (1–8) | `flux-pro-1.0-fill` |
|
||||
| fal | ✓ | per model | per model | ✓ | ✓ | 1 (several on `/edit`) | ✓ |
|
||||
| Replicate | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ (use `providerOptions`) | ✗ |
|
||||
| Stability `image` | 1 | ✗ | ✓ | ✓ | ✓ | 1 (not on `core`) | ✗ |
|
||||
| Stability `upscale()` | ✗ | ✗ | ✗ | ✓ | ✓ | exactly 1 (required) | ✗ |
|
||||
| Provider | `n` | `size` | `aspectRatio` | `seed` | `format` | `images` | `mask` |
|
||||
| --------------------- | --- | --------- | ------------- | ------ | -------- | -------------------------------- | ------------------- |
|
||||
| OpenAI | ✓¹ | ✓ | ✗ | ✗ | ✓ | ✓ | ✓ |
|
||||
| Google (Gemini) | 1 | ✗ | ✓ | ✓ | ✗ | ✓ (no public URLs) | ✗ |
|
||||
| xAI | ✓ | ✗ | ✓ | ✗ | ✗ | ✓ | ✗ |
|
||||
| Z.ai | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ |
|
||||
| Meta | ✓ | ✓ (hint) | ✗ | ✗ | ✓ | ✓ | ✗ |
|
||||
| Black Forest Labs | 1 | per model | per model | ✓ | ✓ | per model (1–8) | `flux-pro-1.0-fill` |
|
||||
| fal | ✓ | per model | per model | ✓ | ✓ | 1 (several on `/edit`, `/multi`) | ✓ |
|
||||
| Replicate | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ (use `providerOptions`) | ✗ |
|
||||
| Stability `image` | 1 | ✗ | ✓ | ✓ | ✓ | 1 (not on `core`) | ✗ |
|
||||
| Stability `upscale()` | ✗ | ✗ | ✗ | ✓ | ✓ | exactly 1 (required) | ✗ |
|
||||
|
||||
✓ lowers natively; ✗ fails whenever the field is set (including `n: 1`); `1` means `n > 1` fails. ¹ `Image.stream` on OpenAI generates one image. fal
|
||||
rejects `size` and `aspectRatio` together; which one a fal or BFL model takes depends on the model.
|
||||
@@ -621,8 +621,7 @@ persist the bytes promptly if they must remain available.
|
||||
### Partial images
|
||||
|
||||
OpenAI's GPT image models stream previews. `Image.stream` sends `stream: true` with `partialImages` (0–3, default 2)
|
||||
and emits `image-partial` events before each final `image`; `Image.generate` keeps the plain JSON request.
|
||||
`dall-e-*` models do not stream and fail typed:
|
||||
and emits `image-partial` events before each final `image`; `Image.generate` keeps the plain JSON request:
|
||||
|
||||
```ts
|
||||
import { Stream } from "effect"
|
||||
@@ -699,7 +698,7 @@ const program = Effect.gen(function* () {
|
||||
})
|
||||
```
|
||||
|
||||
The hosted result is represented as a provider-executed tool call and tool result, and the generated image is also emitted as a first-class `media` `LLMEvent` (`response.message` then carries a `media` part). Gemini image-capable models emit the same `media` event for inline image output. Retaining `response.message` preserves the generated image for continuation on both routes.
|
||||
The hosted result is represented as a provider-executed tool call and a tool result whose content carries the generated image as a file. Gemini image-capable models instead emit a first-class `media` `LLMEvent` for inline image output (`response.message` then carries a `media` part). Retaining `response.message` preserves the generated image for continuation on both routes.
|
||||
|
||||
## Video generation
|
||||
|
||||
@@ -840,9 +839,10 @@ Provider notes:
|
||||
- **OpenAI** streams over SSE (`stream_format: "sse"`), which is also the only place it reports token usage; `tts-1`
|
||||
and `tts-1-hd` do not support SSE and stream the raw audio body instead. `pcm` is 24 kHz 16-bit mono. `language`
|
||||
and `timestamps` are not supported.
|
||||
- **Gemini TTS** returns raw 16-bit PCM only (`audio/L16;codec=pcm;rate=24000`), so any `format` other than `pcm`
|
||||
fails typed; wrap the samples yourself. Style is directed in the text, so `instructions` and `speed` fail typed.
|
||||
Only `gemini-3.1-flash-tts-preview` and later support streaming. Two-speaker audio goes through
|
||||
- **Gemini TTS** returns the provider's default output: WAV for Gemini 3.8 TTS `generate`, raw 16-bit PCM
|
||||
(`audio/L16;codec=pcm;rate=24000`) otherwise. `pcm` is the only explicit `format` it accepts, and it fails typed on
|
||||
Gemini 3.8 `generate`; the route never wraps PCM as WAV. Style is directed in the text, so `instructions` and
|
||||
`speed` fail typed. Only `gemini-3.1-flash-tts-preview` and later support streaming. Two-speaker audio goes through
|
||||
`providerOptions.speechConfig.multiSpeakerVoiceConfig`.
|
||||
- **ElevenLabs** requires `voice` (the path voice id) and authenticates with `xi-api-key`. `format` maps to the
|
||||
`output_format` query parameter (`mp3_44100_128`, `pcm_24000`, `wav_24000`, `opus_48000_64`);
|
||||
@@ -944,6 +944,7 @@ const transcript = await generation.await({ poll: { interval: 3_000 } })
|
||||
- **`ImageClient`** — Effect service and layer for image execution, parallel to `LLMClient`.
|
||||
- **`Media`** — the shared asset type (`Media.Asset`, `Media.Source`) and constructors used by messages, tool results, and media requests.
|
||||
- **`Generation`** — provider-neutral handle for an in-flight media generation (`await`, `refresh`, `cancel`, `events`) used by queued media routes.
|
||||
- **`Video.request` / `generate` / `stream` / `start` / `resume`** — queued video generation through a provider-neutral request; `VideoClient` is its Effect service and layer.
|
||||
- **`Speech.request` / `Speech.generate` / `Speech.stream`** — text-to-speech through a provider-neutral request; `SpeechClient` is its Effect service and layer.
|
||||
- **`Transcription.request` / `generate` / `stream` / `start` / `resume`** — speech-to-text over inline, streaming, and queued routes; `TranscriptionClient` is its Effect service and layer.
|
||||
- **`AIClient.layer` / `AIClient.layerWith(executor)`** — every modality client plus the request executor in one layer.
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
# Media generation in `@opencode/ai` — public API direction
|
||||
|
||||
Status: phases 1–4 implemented (through Image queued routes and partial images); phase 5 proposal.
|
||||
Status: phases 1–4 implemented (through Image queued routes and partial images; ElevenLabs Scribe transcription
|
||||
pending); phase 5 proposal.
|
||||
|
||||
## Goal
|
||||
|
||||
@@ -40,7 +41,7 @@ The design below is derived from a survey of the raw provider APIs (OpenAI, Gemi
|
||||
|
||||
### Model selection
|
||||
|
||||
A model value is built as `OpenAI.configure({ apiKey }).responses("gpt-5")` or `.image("gpt-image-2")`: `configure` fixes credentials, endpoint, and defaults; the selector fixes which of the provider's APIs to hit and binds the typed `providerOptions` generic. Media follows the same shape with one selector per modality — `openai.image(id)` today, `.video(id)` / `.speech(id)` / `.transcription(id)` as those modalities land — mirroring `openai.responses(id)`. `Image.request` accepts `ImageModel` only, exactly as `LLM.request` accepts `LanguageModel`.
|
||||
A model value is built as `OpenAI.configure({ apiKey }).responses("gpt-5")` or `.image("gpt-image-2")`: `configure` fixes credentials, endpoint, and defaults; the selector fixes which of the provider's APIs to hit and binds the typed `providerOptions` generic. Media follows the same shape with one selector per modality — `.image(id)`, `.video(id)`, `.speech(id)`, `.transcription(id)` on the facades that offer each — mirroring `openai.responses(id)`. `Image.request` accepts `ImageModel` only, exactly as `LLM.request` accepts `LanguageModel`.
|
||||
|
||||
```ts
|
||||
import { OpenAI, Google } from "@opencode/ai/providers"
|
||||
@@ -135,8 +136,8 @@ Effect.gen(function* () {
|
||||
})
|
||||
```
|
||||
|
||||
`size` and `aspectRatio` are not interchangeable; each route rejects fields it cannot lower — see the README's Image
|
||||
portability matrix.
|
||||
`size` and `aspectRatio` are not interchangeable; each route rejects fields it cannot lower — see the portability table
|
||||
in the README's Image generation section.
|
||||
|
||||
Editing is not a separate function; `images`/`mask` on the request select the edit path in the route (OpenAI `/images/edits`, Gemini multimodal parts, xAI `/images/edits`). Routes that cannot honor `mask` fail with `Unsupported`.
|
||||
|
||||
@@ -166,8 +167,8 @@ Effect.gen(function* () {
|
||||
|
||||
// Simple: wait for it.
|
||||
const response = yield* Video.generate(request, { poll: { interval: "10 seconds", timeout: "10 minutes" } })
|
||||
response.video // Media.Asset: url with expiresAt (+ transient `headers` for Veo downloads)
|
||||
response.usage // credits on Runway; the other three report none
|
||||
response.video // Media.Asset: url (expiresAt on Veo and Runway; transient `headers` for Veo downloads)
|
||||
response.usage // credits on Runway; the other three report none (xAI's usage.cost_in_usd_ticks is not decoded)
|
||||
response.notices // Veo raiMediaFilteredReasons → filtered, xAI respect_moderation → moderated
|
||||
yield* response.video.materialize() // pull bytes before the URL expires
|
||||
|
||||
@@ -175,7 +176,7 @@ Effect.gen(function* () {
|
||||
const generation = yield* Video.start(request) // Generation<VideoResponse>
|
||||
generation.id; generation.status; generation.progress; generation.position; generation.token
|
||||
yield* generation.await({ poll }) // VideoResponse
|
||||
yield* generation.cancel() // fal PUT cancel_url, Runway DELETE /tasks/{id}; no-op for Veo and xAI
|
||||
yield* generation.cancel() // fal PUT cancel_url, Runway DELETE /tasks/{id}; Veo and xAI succeed without a request
|
||||
|
||||
// Resume from another process. The token is validated against the route's codec and refreshed once. It carries no
|
||||
// route identity, so persist the provider and model ID alongside it: `resume` needs the model.
|
||||
@@ -188,10 +189,11 @@ Effect.gen(function* () {
|
||||
|
||||
Tokens are route-owned JSON: Veo `{ operation }`, xAI `{ requestID }`, Runway `{ taskID }`, fal
|
||||
`{ requestID, statusURL, responseURL, cancelURL }` (fal's follow-up URLs are authoritative and absolute). Common-field
|
||||
lowering per provider: Veo takes inline media only and rejects `audio: false` and `n > 1`; xAI rejects `seed` and
|
||||
`negativePrompt` and routes a `video` input to edits or (`providerOptions.mode: "extend"`) extensions; fal rejects
|
||||
`durationSeconds`, `references`, and `frames.last` because the field names and enums differ per model; Runway passes
|
||||
`aspectRatio` through as its pixel `ratio` and rejects `n`.
|
||||
lowering per provider: Veo takes inline media only, rejects `audio: false` and `n > 1`, and requires `frames.first`
|
||||
when `frames.last` is set; xAI rejects `n`, `seed`, and `negativePrompt` and routes a `video` input to edits or
|
||||
(`providerOptions.mode: "extend"`) extensions; fal rejects `n`, plus `durationSeconds`, `references`, and `frames.last`
|
||||
because the field names and enums differ per model; Runway passes `aspectRatio` through as its pixel `ratio` and
|
||||
rejects `n`.
|
||||
|
||||
Deferred: `Video.complete(model, token, webhook)` (finish from a webhook payload without polling) and provider poll
|
||||
hints (none of the four providers emit one). Later providers: Luma, Kling, MiniMax, Replicate.
|
||||
@@ -241,10 +243,12 @@ name→id resolution. Multi-speaker (Gemini `speechConfig.multiSpeakerVoiceConfi
|
||||
`opus_48000_64`, Cartesia `{ container, encoding, sample_rate }`, Deepgram `encoding`+`container`) and declares the
|
||||
asset's media type rather than sniffing, because headerless PCM can look like an MPEG frame sync. Headerless PCM
|
||||
always carries `info.encoding`, `info.sampleRate`, and `info.channels`; its media type is the provider's declaration
|
||||
(Gemini `audio/L16;codec=pcm;rate=24000`, Deepgram's `content-type`) or `audio/pcm`. Gemini returns PCM only, so any
|
||||
other `format` is rejected rather than wrapped as WAV by the route. Every `format` value a route cannot produce (unknown
|
||||
to it, a container on Cartesia SSE, WAV on an ElevenLabs stream, anything but PCM on Gemini) fails the same way as an
|
||||
unsupported field: `UnsupportedOperation` with `operation: "media.format"`.
|
||||
(Gemini `audio/L16;codec=pcm;rate=24000`, Deepgram's `content-type`) or `audio/pcm`. Gemini's asset follows the
|
||||
provider's declared type: WAV for Gemini 3.8 TTS `generate`, headerless PCM otherwise. The route never wraps PCM as WAV,
|
||||
so `pcm` is the only explicit `format` it accepts, and not on Gemini 3.8 `generate`. Every `format` value a route cannot
|
||||
produce (unknown to it, a container on Cartesia SSE, WAV on an ElevenLabs stream, anything but `pcm` on Gemini, `pcm` on
|
||||
Gemini 3.8 `generate`) fails the same way as an unsupported field: `UnsupportedOperation` with
|
||||
`operation: "media.format"`.
|
||||
|
||||
**Timestamps.** `timestamps: true` on the request asks for alignment. ElevenLabs selects the `with-timestamps`
|
||||
endpoints (character-level, NDJSON when streaming); Cartesia sets `add_timestamps` on `/tts/sse` (word-level; a
|
||||
@@ -277,7 +281,7 @@ const request = Transcription.request({
|
||||
language: "en", // provider-native passthrough
|
||||
timestamps: "segment", // none | segment | word
|
||||
diarize: true,
|
||||
speakers: 2, // expected count, hint only (AssemblyAI)
|
||||
speakers: 2, // exact speaker count (AssemblyAI only)
|
||||
providerOptions: { known_speaker_names: ["agent"] },
|
||||
})
|
||||
|
||||
@@ -291,10 +295,11 @@ yield* Transcription.resume(model, token)
|
||||
Transcription is the first modality whose providers span all three protocol kinds, and it needed no fourth kind.
|
||||
Every `MediaRoute` now carries its `kind`; `TranscriptionRoute` is the union of the inline, stream, and queued routes;
|
||||
`TranscriptionModel.fromRoute` is overloaded per protocol kind (arity picks the overload: `<Options>`,
|
||||
`<Options, Frame, State>`, `<Options, Token>`) and composes through `MediaRoute.inline` / `stream` / `queued`; and
|
||||
`TranscriptionClient` dispatches on `route.kind`. `generate` on a queued route is `start` then `await`; `stream` on an
|
||||
inline route is the response as a single `finish`, and on a queued route it is the status observations followed by
|
||||
`finish`. `start` / `resume` on a non-queued route fail with `UnsupportedOperation` (`transcription.start`). The
|
||||
`<Options, Frame, State>`, `<Options, Token>`) and composes through the shared `composeRoute` (`src/media-model.ts`),
|
||||
which picks `MediaRoute.inline` / `stream` / `queued`; and `TranscriptionClient`, like every modality client, is
|
||||
`MediaClient.make` (`src/media-client.ts`), which dispatches on `route.kind`. `generate` on a queued route is `start`
|
||||
then `await`; `stream` on an inline route is the response as a single `finish`, and on a queued route it is the status
|
||||
observations followed by `finish`. `start` / `resume` on a non-queued route fail with `UnsupportedOperation` (`transcription.start`). The
|
||||
`finish` event carries the whole transcript (text, segments, words, language, duration, usage), so the stream route's
|
||||
`collect` is just "take `finish`".
|
||||
|
||||
@@ -309,11 +314,12 @@ Settled rules:
|
||||
word offsets, so segment timestamps and diarization also request word offsets there.
|
||||
- **Diarization.** `diarize` means segments (and words, where the provider labels them) carry `speaker`. Labels are
|
||||
provider-native strings — OpenAI `A` or a known speaker name, Deepgram `0`, Gemini `spk:0`, AssemblyAI `A` — with no
|
||||
cross-provider speaker model. `speakers` is a hint; only AssemblyAI (`speakers_expected`) accepts it.
|
||||
cross-provider speaker model. `speakers` is the exact number of speakers to label, which AssemblyAI (`speakers_expected`, the only route that
|
||||
accepts it) treats as a constraint rather than a hint.
|
||||
- **Language** is passed through (`language`, OpenAI `gpt-transcribe` `languages[]`, Gemini `languageCodes`,
|
||||
AssemblyAI `language_code`). `response.language` is the provider's own value, lowercased but not normalized: an
|
||||
ISO code on most routes, `english` from whisper-1, `en_us` from AssemblyAI. Deepgram and AssemblyAI assume English
|
||||
unless asked to detect, so a missing `language` enables their detection.
|
||||
ISO code on most routes (AssemblyAI's detection returns `en`), `english` from whisper-1. Deepgram and AssemblyAI
|
||||
assume English unless asked to detect, so a missing `language` enables their detection.
|
||||
- **Gemini** requires a transcribe model; other model ids fail with `UnsupportedOperation` before the call, because
|
||||
general models ignore `audioTranscriptionConfig` and answer conversationally. Streamed chunks carry whole speaker
|
||||
turns (one part per turn), which join with a space.
|
||||
@@ -323,7 +329,7 @@ Settled rules:
|
||||
|
||||
| Provider | Kind | Audio input | `timestamps` | `diarize` | Unsupported | Usage |
|
||||
|---|---|---|---|---|---|---|
|
||||
| OpenAI | stream (`stream: true` in `stream` mode) | multipart `file` (inline only) | `whisper-1` (`verbose_json`); diarize model: `segment` | `gpt-4o-transcribe-diarize` (`diarized_json`) | `speakers`; `prompt` on the diarize model; streaming on `whisper-1` | `tokens` or `seconds` |
|
||||
| OpenAI | stream (`stream: true` in `stream` mode; `whisper-1` ignores `stream`, so it emits only `finish`) | multipart `file` (inline only) | `whisper-1` (`verbose_json`); diarize model: `segment` | `gpt-4o-transcribe-diarize` (`diarized_json`) | `speakers`; `prompt` on the diarize model | `tokens` or `seconds` |
|
||||
| Gemini | stream (`generateContent` / `streamGenerateContent`) | `inlineData` or Gemini Files `fileData` | `audioTranscriptionConfig.wordTimestamp` | `audioTranscriptionConfig.diarization` | `prompt`, `speakers` | `tokens` |
|
||||
| Deepgram | inline | raw body, or JSON `{ url }` | words always; `segment` → `utterances` | `diarize_model=latest` + `utterances` | `prompt`, `speakers` | `seconds` (`metadata.duration`) |
|
||||
| AssemblyAI | queued (upload → submit → poll) | `/v2/upload` then `audio_url`, or a URL | words always; `segment` → `speaker_labels` | `speaker_labels` | — | `seconds` (`audio_duration`) |
|
||||
@@ -353,7 +359,7 @@ GenerationAwaitOptions = { poll?: Poll }
|
||||
Poll = { interval?: Duration; timeout?: Duration }
|
||||
```
|
||||
|
||||
`Generation` is not video-specific. Image routes on BFL, fal, and Replicate are queued; `Image.start` exists for them. A route declares itself `inline` or `queued`; `generate` on a queued route is `start` then `await`.
|
||||
`Generation` is not video-specific. Image routes on BFL, fal, Replicate, and Stability `upscale()` are queued; `Image.start` exists for them. A route declares itself `inline` or `queued`; `generate` on a queued route is `start` then `await`.
|
||||
|
||||
### Usage
|
||||
|
||||
@@ -400,14 +406,15 @@ Streams become `AsyncIterable` via `Stream.toAsyncIterable`. `AIError` is thrown
|
||||
|
||||
### Providers
|
||||
|
||||
Existing facades gain per-modality selectors; the modality routes each facade provides:
|
||||
Existing facades gain per-modality selectors; the modality routes each facade provides (*italics* are not
|
||||
implemented):
|
||||
|
||||
| Facade | llm | image | video | speech | transcription | other |
|
||||
|---|---|---|---|---|---|---|
|
||||
| `OpenAI` | responses (default), chat | Images API (stream) | Sora (deprecated 2026-09-24) | ✓ | ✓ | |
|
||||
| `OpenAI` | responses (default), chat | Images API (stream) | *Sora skipped (decision 8)* | ✓ | ✓ | |
|
||||
| `Google` | Gemini | Gemini-native | Veo | Gemini TTS | `gemini-3.5-transcribe` | |
|
||||
| `XAI` | ✓ | ✓ | ✓ | | | |
|
||||
| `ElevenLabs` | | | | ✓ | Scribe | soundEffect, music |
|
||||
| `ElevenLabs` | | | | ✓ | *Scribe (pending)* | *soundEffect, music (phase 5)* |
|
||||
| `Cartesia` | | | | ✓ | | |
|
||||
| `Deepgram` | | | | Aura | ✓ | |
|
||||
| `Fal` | | ✓ (queued) | ✓ | | | |
|
||||
@@ -416,7 +423,7 @@ Existing facades gain per-modality selectors; the modality routes each facade pr
|
||||
| `Replicate` | | ✓ (queued) | | | | |
|
||||
| `Stability` | | `image` (inline), `upscale()` (queued) | | | | |
|
||||
| `Runway` | | | ✓ | | | |
|
||||
| `Luma`, `Kling`, `MiniMax` | | per provider | | | | |
|
||||
| `Luma`, `Kling`, `MiniMax` | | *deferred* | *deferred* | | | |
|
||||
|
||||
New facades follow the existing one-file-per-provider rule. The facade selector is the public path for media models; modality-specific package entrypoints (for example `@opencode/ai/providers/openai/images`) are deferred until Core has a modality-aware model resolver.
|
||||
|
||||
|
||||
@@ -53,6 +53,8 @@ export type Event = Observation | { readonly type: "generation-finished"; readon
|
||||
|
||||
const TERMINAL: ReadonlySet<Status> = new Set(["completed", "failed", "cancelled", "expired"])
|
||||
|
||||
export const isTerminal = (status: Status) => TERMINAL.has(status)
|
||||
|
||||
export class Generation<Response> {
|
||||
readonly id: string
|
||||
readonly status: Status
|
||||
@@ -81,7 +83,7 @@ export class Generation<Response> {
|
||||
}
|
||||
|
||||
get terminal() {
|
||||
return TERMINAL.has(this.status)
|
||||
return isTerminal(this.status)
|
||||
}
|
||||
|
||||
refresh(): Effect.Effect<Generation<Response>, AIError> {
|
||||
@@ -109,9 +111,10 @@ export class Generation<Response> {
|
||||
}
|
||||
|
||||
/**
|
||||
* Status observations as a stream, ending after the first terminal observation. Each poll is bounded by the time
|
||||
* remaining until `poll.timeout`, so a hung status request fails the stream instead of stalling it. (`Stream.interruptWhen`
|
||||
* would express this directly but deadlocks under `TestClock` when the source completes while the timer sleeps.)
|
||||
* Status observations as a stream, ending after the first terminal observation. Each poll and each sleep between polls
|
||||
* is bounded by the time remaining until `poll.timeout`, so a hung status request or a long interval fails the stream at
|
||||
* the deadline instead of stalling it. (`Stream.interruptWhen` would express this directly but deadlocks under
|
||||
* `TestClock` when the source completes while the timer sleeps.)
|
||||
*/
|
||||
events(options?: AwaitOptions): Stream.Stream<Event, AIError> {
|
||||
if (this.terminal) return Stream.make(this.event())
|
||||
@@ -120,17 +123,26 @@ export class Generation<Response> {
|
||||
Clock.currentTimeMillis.pipe(
|
||||
Effect.map((start) => {
|
||||
const deadline = start + Duration.toMillis(timeout)
|
||||
// Fail before polling once the deadline has passed: a fast status request could otherwise win the zero-budget
|
||||
// race and schedule another zero-delay poll.
|
||||
const refresh = Clock.currentTimeMillis.pipe(
|
||||
Effect.flatMap((now) =>
|
||||
this.refresh().pipe(
|
||||
Effect.timeoutOrElse({
|
||||
duration: Duration.millis(Math.max(0, deadline - now)),
|
||||
orElse: () => this.timeoutError(timeout),
|
||||
}),
|
||||
),
|
||||
now >= deadline
|
||||
? this.timeoutError(timeout)
|
||||
: this.refresh().pipe(
|
||||
Effect.timeoutOrElse({
|
||||
duration: Duration.millis(deadline - now),
|
||||
orElse: () => this.timeoutError(timeout),
|
||||
}),
|
||||
),
|
||||
),
|
||||
)
|
||||
return Stream.fromEffectSchedule(refresh, this.schedule(options?.poll)).pipe(
|
||||
const schedule = this.schedule(options?.poll).pipe(
|
||||
Schedule.modifyDelay((meta) =>
|
||||
Effect.succeed(Duration.min(meta.duration, Duration.millis(Math.max(0, deadline - meta.now)))),
|
||||
),
|
||||
)
|
||||
return Stream.fromEffectSchedule(refresh, schedule).pipe(
|
||||
Stream.takeUntil((generation) => generation.terminal),
|
||||
Stream.map((generation) => generation.event()),
|
||||
)
|
||||
|
||||
@@ -110,8 +110,11 @@ const fromRequest = Effect.fn("AssemblyAITranscription.fromRequest")(function* (
|
||||
language_code: request.language,
|
||||
language_detection: request.language === undefined ? true : undefined,
|
||||
prompt: request.prompt,
|
||||
// Turn-level `utterances`, the only segments AssemblyAI returns, require speaker labels.
|
||||
speaker_labels: request.diarize === true || request.timestamps === "segment" ? true : undefined,
|
||||
// Turn-level `utterances`, the only segments AssemblyAI returns, and `speakers_expected` require speaker labels.
|
||||
speaker_labels:
|
||||
request.diarize === true || request.timestamps === "segment" || request.speakers !== undefined
|
||||
? true
|
||||
: undefined,
|
||||
speakers_expected: request.speakers,
|
||||
},
|
||||
request.providerOptions,
|
||||
@@ -155,8 +158,7 @@ const decodeResult = Effect.fn("AssemblyAITranscription.decodeResult")(function*
|
||||
const error = transcript.error ?? undefined
|
||||
if (status === "failed")
|
||||
return yield* output.ended("failed", `${route.name} transcription failed${error === undefined ? "" : `: ${error}`}`)
|
||||
if (status !== "completed")
|
||||
return yield* output.invalid(`${route.name} transcript ${context.token.transcriptID} has not finished`)
|
||||
if (status !== "completed") return yield* output.pending(context.token.transcriptID)
|
||||
const duration = transcript.audio_duration ?? undefined
|
||||
return new TranscriptionResponse({
|
||||
text: transcript.text ?? "",
|
||||
|
||||
@@ -31,13 +31,21 @@ export type Request = ImageRequestFor<BlackForestLabsImageOptions>
|
||||
// 2. Token and response schemas
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** Regional clusters answer on different hosts, so the returned `polling_url` is followed verbatim. */
|
||||
export const Token = Schema.Struct({ id: Schema.String, pollingURL: Schema.String })
|
||||
/**
|
||||
* Regional clusters answer on different hosts, so the returned `polling_url` is followed verbatim. BFL reports the
|
||||
* credit cost on submit, so it rides on the token; it is optional so tokens persisted before it existed still decode.
|
||||
*/
|
||||
export const Token = Schema.Struct({
|
||||
id: Schema.String,
|
||||
pollingURL: Schema.String,
|
||||
cost: Schema.optionalKey(Schema.Number),
|
||||
})
|
||||
export type Token = Schema.Schema.Type<typeof Token>
|
||||
|
||||
const StartResponse = Schema.Struct({
|
||||
id: Schema.String,
|
||||
polling_url: Schema.String,
|
||||
cost: optionalNull(Schema.Number),
|
||||
})
|
||||
|
||||
const Result = Schema.Struct({
|
||||
@@ -145,7 +153,11 @@ const fromRequest = Effect.fn("BlackForestLabsImages.fromRequest")(function* (re
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const decodeStart = route.decodeStarted(StartResponse, (value) => ({
|
||||
token: { id: value.id, pollingURL: value.polling_url },
|
||||
token: {
|
||||
id: value.id,
|
||||
pollingURL: value.polling_url,
|
||||
...(value.cost === undefined || value.cost === null ? {} : { cost: value.cost }),
|
||||
},
|
||||
snapshot: { id: value.id, status: "queued" },
|
||||
}))
|
||||
|
||||
@@ -169,14 +181,16 @@ const decodeResult = Effect.fn("BlackForestLabsImages.decodeResult")(function* (
|
||||
if (isModerated(document.status)) return yield* output.contentPolicy(`${route.name} moderated the generation`)
|
||||
if (status === "failed" || status === "expired")
|
||||
return yield* output.ended(status, `${route.name} generation ${context.token.id} ended with ${document.status}`)
|
||||
if (status !== "completed" || document.result === undefined || document.result === null)
|
||||
if (status !== "completed") return yield* output.pending(context.token.id)
|
||||
if (document.result === undefined || document.result === null)
|
||||
return yield* output.invalid(`${route.name} generation ${context.token.id} has no result`)
|
||||
const { sample, seed, prompt, ...rest } = document.result
|
||||
// A settled `cost` on the result supersedes the submit-time cost carried on the token.
|
||||
const cost = document.cost ?? context.token.cost
|
||||
return new ImageResponse({
|
||||
// `sample` is a signed URL that expires 10 minutes after the result is ready, so it is downloaded now.
|
||||
images: [yield* context.materialize(Media.url(sample))],
|
||||
usage:
|
||||
document.cost === undefined || document.cost === null ? undefined : { type: "credits", credits: document.cost },
|
||||
usage: cost === undefined ? undefined : { type: "credits", credits: cost },
|
||||
providerMetadata: {
|
||||
bfl: { id: context.token.id, seed: seed ?? undefined, prompt: prompt ?? undefined, ...rest },
|
||||
},
|
||||
|
||||
@@ -67,6 +67,9 @@ const queryParameters = (request: Request) => {
|
||||
}
|
||||
|
||||
const fromRequest = Effect.fn("DeepgramSpeech.fromRequest")(function* (request: Request) {
|
||||
// Not in `unsupported`: that list would also reject `timestamps: false`, which asks for nothing.
|
||||
if (request.timestamps === true)
|
||||
return yield* route.unsupported("media.timestamps", `${route.name} does not return timestamps`)
|
||||
if (
|
||||
request.format !== undefined &&
|
||||
FORMATS[request.format] === undefined &&
|
||||
@@ -117,7 +120,7 @@ const finish = (state: State, context: MediaProtocol.ResponseContext<Request>) =
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export const protocol = MediaProtocol.stream<Request, SpeechEvent, Uint8Array, State>(route, {
|
||||
unsupported: ["voice", "language", "instructions", "timestamps"],
|
||||
unsupported: ["voice", "language", "instructions"],
|
||||
body: { from: fromRequest },
|
||||
frames: (bytes) => bytes,
|
||||
initial: () => ({ chunks: [] }),
|
||||
|
||||
@@ -49,7 +49,7 @@ const QueueResult = Schema.StructWithRest(
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const sizing = (model: string) => {
|
||||
if (/^fal-ai\/(nano-banana|flux-pro\/v1\.1-ultra)/.test(model)) return "aspect_ratio"
|
||||
if (/^fal-ai\/(nano-banana|flux-pro\/(v1\.1-ultra|kontext))/.test(model)) return "aspect_ratio"
|
||||
if (model.startsWith("fal-ai/flux")) return "image_size"
|
||||
return undefined
|
||||
}
|
||||
@@ -63,20 +63,24 @@ const validate = (request: Request) => {
|
||||
return Effect.fail(route.unsupported("media.size", `${id} sizes by aspectRatio`))
|
||||
if (request.aspectRatio !== undefined && field === "image_size")
|
||||
return Effect.fail(route.unsupported("media.aspectRatio", `${id} sizes by size (image_size)`))
|
||||
if ((request.images?.length ?? 0) > 1 && !isEdit(id))
|
||||
if ((request.images?.length ?? 0) > 1 && !takesImageList(id))
|
||||
return Effect.fail(
|
||||
route.unsupported("media.images", `${id} takes one image_url; use an /edit endpoint for several images`),
|
||||
route.unsupported(
|
||||
"media.images",
|
||||
`${id} takes one image_url; use an /edit or /multi endpoint for several images`,
|
||||
),
|
||||
)
|
||||
return Effect.void
|
||||
}
|
||||
|
||||
// `/edit` endpoints take an `image_urls` list; image-to-image, fill, and Ultra take one `image_url` (beside `mask_url`).
|
||||
const isEdit = (model: string) => model.endsWith("/edit")
|
||||
// `/edit` and `/multi` (Kontext) endpoints take an `image_urls` list; image-to-image, fill, and Ultra take one
|
||||
// `image_url` (beside `mask_url`).
|
||||
const takesImageList = (model: string) => model.endsWith("/edit") || model.endsWith("/multi")
|
||||
|
||||
const fromRequest = Effect.fn("FalImages.fromRequest")(function* (request: Request) {
|
||||
yield* validate(request)
|
||||
const images = yield* Effect.forEach(request.images ?? [], (image) => FalQueue.mediaUrl(image, route.name))
|
||||
const edit = isEdit(request.model.id)
|
||||
const list = takesImageList(request.model.id)
|
||||
return MediaProtocol.json(
|
||||
mergeJsonRecords(
|
||||
{
|
||||
@@ -86,8 +90,8 @@ const fromRequest = Effect.fn("FalImages.fromRequest")(function* (request: Reque
|
||||
image_size: request.size === undefined ? undefined : MediaInput.dimensions(request.size),
|
||||
aspect_ratio: request.aspectRatio,
|
||||
output_format: request.format,
|
||||
image_urls: edit && images.length > 0 ? images : undefined,
|
||||
image_url: edit ? undefined : images[0],
|
||||
image_urls: list && images.length > 0 ? images : undefined,
|
||||
image_url: list ? undefined : images[0],
|
||||
mask_url: request.mask === undefined ? undefined : yield* FalQueue.mediaUrl(request.mask, route.name),
|
||||
},
|
||||
request.providerOptions,
|
||||
@@ -112,12 +116,14 @@ const decodeResult = Effect.fn("FalImages.decodeResult")(function* (
|
||||
// With the safety checker on, flagged images come back blacked out rather than omitted.
|
||||
const flagged = (has_nsfw_concepts ?? []).flatMap((value, index) => (value ? [index] : []))
|
||||
return new ImageResponse({
|
||||
images: images.map((image) =>
|
||||
Media.url(image.url, {
|
||||
mediaType: image.content_type ?? undefined,
|
||||
info: { width: image.width ?? undefined, height: image.height ?? undefined },
|
||||
}),
|
||||
),
|
||||
images: images.map((image) => {
|
||||
const info = { width: image.width ?? undefined, height: image.height ?? undefined }
|
||||
// `sync_mode: true` returns data URIs instead of hosted URLs.
|
||||
return (
|
||||
Media.parseDataUrl(image.url, { info }) ??
|
||||
Media.url(image.url, { mediaType: image.content_type ?? undefined, info })
|
||||
)
|
||||
}),
|
||||
notices:
|
||||
flagged.length === 0
|
||||
? undefined
|
||||
|
||||
@@ -101,7 +101,7 @@ const generationConfig = (request: Request) => {
|
||||
const fromRequest = Effect.fn("GoogleImages.fromRequest")(function* (request: Request) {
|
||||
if (request.n !== undefined && request.n > 1)
|
||||
return yield* route.unsupported(
|
||||
"image.n",
|
||||
"media.n",
|
||||
`${route.name} generates one image per request; call it once per image instead of n=${request.n}`,
|
||||
)
|
||||
const parts = yield* Effect.forEach(request.images ?? [], (image) =>
|
||||
|
||||
@@ -56,6 +56,9 @@ interface State extends SpeechStream.Audio, GeminiGenerateContent.Metadata {
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const fromRequest = Effect.fn("GoogleSpeech.fromRequest")(function* (request: MediaProtocol.Addressed<Request>) {
|
||||
// Not in `unsupported`: that list would also reject `timestamps: false`, which asks for nothing.
|
||||
if (request.timestamps === true)
|
||||
return yield* route.unsupported("media.timestamps", `${route.name} does not return timestamps`)
|
||||
if (request.format === "pcm" && request.mode === "generate" && /^gemini-3\.8-.*-tts(?:-|$)/.test(request.model.id))
|
||||
return yield* route.unsupported(
|
||||
"media.format",
|
||||
@@ -125,7 +128,7 @@ const finish = (state: State, context: MediaProtocol.ResponseContext<Request>) =
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export const protocol = MediaProtocol.stream<Request, SpeechEvent, string, State>(route, {
|
||||
unsupported: ["instructions", "speed", "timestamps"],
|
||||
unsupported: ["instructions", "speed"],
|
||||
body: { from: fromRequest },
|
||||
frames: (bytes, context) => GeminiGenerateContent.frames(bytes, context.request.mode),
|
||||
initial: () => ({ chunks: [] }),
|
||||
|
||||
@@ -149,8 +149,7 @@ const decodeResult = Effect.fn("GoogleVideo.decodeResult")(function* (
|
||||
const output = yield* decodeOperation(response)
|
||||
const operation = output.value
|
||||
const status = statusOf(operation)
|
||||
if (status === "running")
|
||||
return yield* output.invalid(`${route.name} operation ${context.token.operation} has not finished`)
|
||||
if (status === "running") return yield* output.pending(context.token.operation)
|
||||
if (status === "failed")
|
||||
return yield* output.ended(
|
||||
"failed",
|
||||
|
||||
@@ -48,15 +48,17 @@ const Usage = Schema.Struct({
|
||||
output_tokens_details: Schema.optional(Schema.Record(Schema.String, Schema.Unknown)),
|
||||
})
|
||||
|
||||
const OpenAIImageResponse = Schema.Struct({
|
||||
data: Schema.Array(
|
||||
Schema.Struct({
|
||||
b64_json: Schema.optional(Schema.String),
|
||||
url: Schema.optional(Schema.String),
|
||||
revised_prompt: Schema.optional(Schema.String),
|
||||
}),
|
||||
),
|
||||
/** What the provider actually rendered; it can differ from the request when `auto` or a default applied. */
|
||||
const Settings = {
|
||||
output_format: Schema.optional(Schema.String),
|
||||
size: Schema.optional(Schema.String),
|
||||
quality: Schema.optional(Schema.String),
|
||||
background: Schema.optional(Schema.String),
|
||||
}
|
||||
|
||||
const OpenAIImageResponse = Schema.Struct({
|
||||
data: Schema.Array(Schema.Struct({ b64_json: Schema.String })),
|
||||
...Settings,
|
||||
usage: Schema.optional(Usage),
|
||||
})
|
||||
|
||||
@@ -69,11 +71,13 @@ const StreamEvent = Schema.Union([
|
||||
type: Schema.Literals(["image_generation.partial_image", "image_edit.partial_image"]),
|
||||
b64_json: Schema.String,
|
||||
partial_image_index: Schema.Number,
|
||||
...Settings,
|
||||
output_format: Schema.String,
|
||||
}),
|
||||
Schema.Struct({
|
||||
type: Schema.Literals(["image_generation.completed", "image_edit.completed"]),
|
||||
b64_json: Schema.String,
|
||||
...Settings,
|
||||
output_format: Schema.String,
|
||||
usage: Schema.optional(Usage),
|
||||
}),
|
||||
@@ -92,6 +96,9 @@ type Frame = string | { readonly document: string; readonly requested: string |
|
||||
interface State {
|
||||
readonly completed: number
|
||||
readonly format?: string
|
||||
readonly size?: string
|
||||
readonly quality?: string
|
||||
readonly background?: string
|
||||
readonly usage?: MediaUsage
|
||||
}
|
||||
|
||||
@@ -110,10 +117,6 @@ const nativeOptions = (options: OpenAIImageOptions | undefined) => {
|
||||
|
||||
const streamOptions = (request: MediaProtocol.Addressed<Request>) => {
|
||||
if (request.mode !== "stream") return Effect.succeed(undefined)
|
||||
if (request.model.id.startsWith("dall-e"))
|
||||
return Effect.fail(
|
||||
route.unsupported("media.stream", `${request.model.id} does not stream; use Image.generate or a GPT image model`),
|
||||
)
|
||||
if (request.n !== undefined && request.n > 1)
|
||||
return Effect.fail(
|
||||
route.unsupported("media.n", `${route.name} streams one image; use Image.generate for n=${request.n}`),
|
||||
@@ -194,21 +197,34 @@ const usage = (value: Schema.Schema.Type<typeof Usage> | undefined): MediaUsage
|
||||
details: { openai: value },
|
||||
}
|
||||
|
||||
const eventImage = (frame: string, label: string, data: string, format: string) =>
|
||||
/** `size` echoes the rendered `WIDTHxHEIGHT`; `auto` or any other value leaves the dimensions unknown. */
|
||||
const info = (format: string, size: string | undefined): Media.Info => {
|
||||
const match = size?.match(/^(\d+)x(\d+)$/)
|
||||
return match ? { format, width: Number(match[1]), height: Number(match[2]) } : { format }
|
||||
}
|
||||
|
||||
const eventImage = (frame: string, label: string, data: string, format: string, size: string | undefined) =>
|
||||
MediaInput.decodedAsset((message, cause) => route.frameError(message, frame, cause), label, data, `image/${format}`, {
|
||||
info: { format },
|
||||
info: info(format, size),
|
||||
})
|
||||
|
||||
const onEvent = Effect.fn("OpenAIImages.onEvent")(function* (state: State, frame: string) {
|
||||
const event = yield* decodeEvent(frame)
|
||||
const format = event.output_format
|
||||
if ("partial_image_index" in event) {
|
||||
const image = yield* eventImage(frame, `${route.name} partial image`, event.b64_json, format)
|
||||
const image = yield* eventImage(frame, `${route.name} partial image`, event.b64_json, format, event.size)
|
||||
return [state, [ImagePartialEvent.make({ index: event.partial_image_index, image })]] as const
|
||||
}
|
||||
const image = yield* eventImage(frame, `${route.name} result ${state.completed}`, event.b64_json, format)
|
||||
const image = yield* eventImage(frame, `${route.name} result ${state.completed}`, event.b64_json, format, event.size)
|
||||
return [
|
||||
{ ...state, completed: state.completed + 1, format, usage: usage(event.usage) },
|
||||
{
|
||||
completed: state.completed + 1,
|
||||
format,
|
||||
size: event.size,
|
||||
quality: event.quality,
|
||||
background: event.background,
|
||||
usage: usage(event.usage),
|
||||
},
|
||||
[ImageOutputEvent.make({ index: state.completed, image })],
|
||||
] as const
|
||||
})
|
||||
@@ -219,16 +235,20 @@ const onDocument = Effect.fn("OpenAIImages.onDocument")(function* (frame: Exclud
|
||||
Effect.mapError((cause) => invalid(`${route.name} returned an invalid response`, cause)),
|
||||
)
|
||||
const format = decoded.output_format ?? frame.requested ?? "png"
|
||||
const mediaType = `image/${format}`
|
||||
const images = yield* Effect.forEach(decoded.data, (item, index) =>
|
||||
MediaInput.imageOutput(invalid, `${route.name} result ${index}`, item, mediaType, {
|
||||
info: { format },
|
||||
providerMetadata:
|
||||
item.revised_prompt === undefined ? undefined : { openai: { revisedPrompt: item.revised_prompt } },
|
||||
MediaInput.decodedAsset(invalid, `${route.name} result ${index}`, item.b64_json, `image/${format}`, {
|
||||
info: info(format, decoded.size),
|
||||
}),
|
||||
)
|
||||
if (images.length === 0) return yield* invalid(`${route.name} returned no images`)
|
||||
const state: State = { completed: images.length, format, usage: usage(decoded.usage) }
|
||||
const state: State = {
|
||||
completed: images.length,
|
||||
format,
|
||||
size: decoded.size,
|
||||
quality: decoded.quality,
|
||||
background: decoded.background,
|
||||
usage: usage(decoded.usage),
|
||||
}
|
||||
return [state, images.map((image, index) => ImageOutputEvent.make({ index, image }))] as const
|
||||
})
|
||||
|
||||
@@ -237,7 +257,17 @@ const step = (state: State, frame: Frame) => (typeof frame === "string" ? onEven
|
||||
const finish = (state: State) => {
|
||||
if (state.completed === 0) return Effect.fail(route.incomplete())
|
||||
return Effect.succeed([
|
||||
ImageFinishEvent.make({ usage: state.usage, providerMetadata: { openai: { outputFormat: state.format } } }),
|
||||
ImageFinishEvent.make({
|
||||
usage: state.usage,
|
||||
providerMetadata: {
|
||||
openai: {
|
||||
outputFormat: state.format,
|
||||
size: state.size,
|
||||
quality: state.quality,
|
||||
background: state.background,
|
||||
},
|
||||
},
|
||||
}),
|
||||
])
|
||||
}
|
||||
|
||||
|
||||
@@ -143,12 +143,14 @@ const adapter = {
|
||||
restoreHostedToolItem: (item: unknown) => (Schema.is(OpenAIResponsesHostedToolItem)(item) ? item : undefined),
|
||||
} satisfies OpenResponses.ProviderAdapter
|
||||
|
||||
// Only GPT-6 Astra accepts `configuration_update`, and never alongside automatic `context_management` compaction.
|
||||
// GPT-6 Astra, Sol, and Luna accept `configuration_update` only in standard mode (not `reasoning.mode: "pro"` or
|
||||
// `-pro` slugs), and never alongside automatic `context_management` compaction.
|
||||
const supportsEffortUpdates = (request: LLMRequest) => {
|
||||
if (request.providerOptions?.contextManagement !== undefined) return false
|
||||
if (Schema.is(Schema.Struct({ mode: Schema.Literal("pro") }))(request.http?.body?.reasoning)) return false
|
||||
const override = request.model.compatibility?.supportsEffortUpdates
|
||||
if (override !== undefined) return override
|
||||
return /(?:^|\/)gpt-6-astra$/i.test(request.model.id)
|
||||
return /(?:^|\/)gpt-6-(?:astra|sol|luna)$/i.test(request.model.id)
|
||||
}
|
||||
|
||||
const nativeImageToolInput = (tool: ToolDefinition) => {
|
||||
|
||||
@@ -61,6 +61,9 @@ interface State extends SpeechStream.Audio {
|
||||
const supportsSse = (model: string) => !/^tts-1(-hd)?(-|$)/.test(model)
|
||||
|
||||
const fromRequest = Effect.fn("OpenAISpeech.fromRequest")(function* (request: MediaProtocol.Addressed<Request>) {
|
||||
// Not in `unsupported`: that list would also reject `timestamps: false`, which asks for nothing.
|
||||
if (request.timestamps === true)
|
||||
return yield* route.unsupported("media.timestamps", `${route.name} does not return timestamps`)
|
||||
return MediaProtocol.json(
|
||||
mergeJsonRecords(
|
||||
{
|
||||
@@ -121,7 +124,7 @@ const finish = (state: State, context: MediaProtocol.ResponseContext<Request>) =
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export const protocol = MediaProtocol.stream<Request, SpeechEvent, string | Uint8Array, State>(route, {
|
||||
unsupported: ["language", "timestamps"],
|
||||
unsupported: ["language"],
|
||||
body: { from: fromRequest },
|
||||
frames: (bytes, context) => (isSse(context.body) ? Framing.sse.frame(bytes) : bytes),
|
||||
initial: () => ({ chunks: [], done: false }),
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
import { Effect, Schema, Stream } from "effect"
|
||||
import { classifyProviderFailure } from "../provider-error.js"
|
||||
import { Framing } from "../route/framing.js"
|
||||
import { MediaProtocol } from "../route/media-protocol.js"
|
||||
import { MediaRoute } from "../route/media.js"
|
||||
import { mergeJsonRecords, type MediaUsage } from "../schema/index.js"
|
||||
import { AIError, mergeJsonRecords, type MediaUsage } from "../schema/index.js"
|
||||
import {
|
||||
TranscriptionFinishEvent,
|
||||
TranscriptionModel,
|
||||
@@ -59,6 +60,9 @@ const Usage = Schema.Union([
|
||||
input_tokens: Schema.optional(Schema.Number),
|
||||
output_tokens: Schema.optional(Schema.Number),
|
||||
total_tokens: Schema.optional(Schema.Number),
|
||||
input_token_details: Schema.optional(
|
||||
Schema.Struct({ audio_tokens: Schema.optional(Schema.Number), text_tokens: Schema.optional(Schema.Number) }),
|
||||
),
|
||||
}),
|
||||
Schema.Struct({ type: Schema.Literal("duration"), seconds: Schema.Number }),
|
||||
])
|
||||
@@ -75,14 +79,23 @@ const transcriptFields = {
|
||||
usage: Schema.optional(Usage),
|
||||
}
|
||||
|
||||
/** OpenAI may add stream event types; frames outside `EVENT_TYPES` are ignored. */
|
||||
const EventType = Schema.Struct({ type: Schema.String })
|
||||
const Event = Schema.Union([
|
||||
Schema.Struct({ type: Schema.Literal("transcript.text.delta"), delta: Schema.String }),
|
||||
Schema.Struct({ type: Schema.Literal("transcript.text.segment"), ...Segment.fields }),
|
||||
Schema.Struct({ type: Schema.Literal("transcript.text.done"), ...transcriptFields }),
|
||||
Schema.Struct({
|
||||
type: Schema.Literal("error"),
|
||||
message: Schema.optional(Schema.String),
|
||||
error: Schema.optional(Schema.Struct({ message: Schema.optional(Schema.String) })),
|
||||
}),
|
||||
])
|
||||
const EVENT_TYPES = new Set(["transcript.text.delta", "transcript.text.segment", "transcript.text.done", "error"])
|
||||
const Transcript = Schema.Struct(transcriptFields)
|
||||
type Transcript = Schema.Schema.Type<typeof Transcript>
|
||||
|
||||
const decodeEventType = route.decodeFrame(EventType)
|
||||
const decodeEvent = route.decodeFrame(Event)
|
||||
const decodeTranscript = route.decodeFrame(Transcript)
|
||||
|
||||
@@ -118,10 +131,12 @@ const capabilities = (model: string): Capabilities => {
|
||||
return TRANSCRIBE
|
||||
}
|
||||
|
||||
/** whisper-1 ignores `stream`, so its `stream` mode sends a plain request and emits only `finish`. */
|
||||
const streamsEvents = (request: MediaProtocol.Addressed<Request>) =>
|
||||
request.mode === "stream" && capabilities(request.model.id).stream
|
||||
|
||||
const validate = (request: MediaProtocol.Addressed<Request>, model: Capabilities) => {
|
||||
const id = request.model.id
|
||||
if (request.mode === "stream" && !model.stream)
|
||||
return Effect.fail(route.unsupported("media.stream", `${id} does not stream; use Transcription.generate`))
|
||||
if (request.diarize === true && !model.diarize)
|
||||
return Effect.fail(route.unsupported("media.diarize", `${id} does not diarize; use gpt-4o-transcribe-diarize`))
|
||||
if (request.prompt !== undefined && model.diarize)
|
||||
@@ -173,7 +188,7 @@ const fromRequest = Effect.fn("OpenAITranscription.fromRequest")(function* (requ
|
||||
timestamp_granularities: responseFormat === "verbose_json" ? [request.timestamps] : undefined,
|
||||
// Diarizing audio longer than 30 seconds requires a chunking strategy.
|
||||
chunking_strategy: model.diarize ? "auto" : undefined,
|
||||
stream: request.mode === "stream" ? true : undefined,
|
||||
stream: streamsEvents(request) ? true : undefined,
|
||||
},
|
||||
{
|
||||
overlay: mergeJsonRecords(request.providerOptions, request.http?.body),
|
||||
@@ -196,7 +211,15 @@ const segment = (value: Schema.Schema.Type<typeof Segment>): TranscriptionSegmen
|
||||
})
|
||||
|
||||
const onEvent = Effect.fn("OpenAITranscription.onEvent")(function* (state: State, frame: string) {
|
||||
if (!EVENT_TYPES.has((yield* decodeEventType(frame)).type)) return [state, []] as const
|
||||
const event = yield* decodeEvent(frame)
|
||||
if (event.type === "error")
|
||||
return yield* new AIError({
|
||||
reason: classifyProviderFailure({
|
||||
message: `${route.name} stream failed: ${event.message ?? event.error?.message ?? "unknown error"}`,
|
||||
rawBody: frame,
|
||||
}),
|
||||
})
|
||||
if (event.type === "transcript.text.done") return [{ ...state, transcript: event }, []] as const
|
||||
if (event.type === "transcript.text.delta")
|
||||
return [state, event.delta.length === 0 ? [] : [TranscriptionTextDeltaEvent.make({ delta: event.delta })]] as const
|
||||
@@ -246,7 +269,7 @@ export const protocol = MediaProtocol.stream<Request, TranscriptionEvent, Frame,
|
||||
unsupported: ["speakers"],
|
||||
body: { from: fromRequest },
|
||||
frames: (bytes, context) =>
|
||||
context.request.mode === "stream"
|
||||
streamsEvents(context.request)
|
||||
? Framing.sse.frame(bytes)
|
||||
: Framing.document.frame(bytes).pipe(Stream.map((document) => ({ document }))),
|
||||
initial: () => ({ segments: [] }),
|
||||
|
||||
@@ -132,8 +132,7 @@ const decodeResult = Effect.fn("ReplicateImages.decodeResult")(function* (
|
||||
status,
|
||||
`${route.name} prediction ${context.token.id} ${prediction.status}${typeof prediction.error === "string" ? `: ${prediction.error}` : ""}`,
|
||||
)
|
||||
if (status !== "completed")
|
||||
return yield* output.invalid(`${route.name} prediction ${context.token.id} has not finished`)
|
||||
if (status !== "completed") return yield* output.pending(context.token.id)
|
||||
if (prediction.data_removed === true)
|
||||
return yield* output.ended("expired", `${route.name} removed the output of prediction ${context.token.id}`)
|
||||
if (!isOutput(prediction.output))
|
||||
|
||||
@@ -141,8 +141,7 @@ const decodeResult = Effect.fn("RunwayVideo.decodeResult")(function* (
|
||||
}
|
||||
if (status === "cancelled")
|
||||
return yield* output.ended("cancelled", `${route.name} task ${context.token.taskID} was cancelled`)
|
||||
if (status !== "completed")
|
||||
return yield* output.invalid(`${route.name} task ${context.token.taskID} has not finished`)
|
||||
if (status !== "completed") return yield* output.pending(context.token.taskID)
|
||||
const urls = task.output ?? []
|
||||
if (urls.length === 0) return yield* output.invalid(`${route.name} task succeeded without any output`)
|
||||
return new VideoResponse({
|
||||
@@ -171,7 +170,7 @@ export const protocol = MediaProtocol.queued<Request, VideoResponse, Token>(rout
|
||||
start: { body: { from: fromRequest }, decode: decodeStart },
|
||||
status: { path: taskPath, decode: decodeStatus },
|
||||
result: { path: taskPath, decode: decodeResult },
|
||||
cancel: { method: "DELETE", path: taskPath },
|
||||
cancel: { method: "DELETE", path: taskPath, activeOnly: true },
|
||||
})
|
||||
|
||||
const startPath = (request: Request) => {
|
||||
|
||||
@@ -175,7 +175,7 @@ const decodeUpscaleResult = Effect.fn("StabilityImages.decodeUpscaleResult")(fun
|
||||
) {
|
||||
if (response.status === 202) {
|
||||
const output = yield* upscaleRoute.text(response)
|
||||
return yield* output.invalid(`${upscaleRoute.name} upscale ${context.token.id} has not finished`)
|
||||
return yield* output.pending(context.token.id)
|
||||
}
|
||||
return yield* decodeUpscaleImage(response)
|
||||
})
|
||||
|
||||
@@ -101,7 +101,8 @@ const decodeResponse = Effect.fn("XAIImages.decodeResponse")(function* (
|
||||
)
|
||||
if (images.length === 0) return yield* output.invalid(`${route.name} returned no images`)
|
||||
const usage = ProviderShared.isRecord(decoded.usage) ? decoded.usage : undefined
|
||||
// xAI reports image counts rather than tokens, seconds, or credits; the raw record stays in provider metadata.
|
||||
// xAI reports a USD cost (`cost_in_usd_ticks`) rather than tokens, seconds, or credits; the raw record stays in
|
||||
// provider metadata.
|
||||
return new ImageResponse({
|
||||
images,
|
||||
providerMetadata: usage === undefined ? undefined : { xai: { usage } },
|
||||
|
||||
@@ -136,8 +136,7 @@ const decodeResult = Effect.fn("XAIVideo.decodeResult")(function* (
|
||||
const output = yield* decodeVideoStatus(response)
|
||||
const decoded = output.value
|
||||
const status = yield* MediaProtocol.status(STATUS, decoded.status, output)
|
||||
if (status === "running")
|
||||
return yield* output.invalid(`${route.name} request ${context.token.requestID} has not finished`)
|
||||
if (status === "running") return yield* output.pending(context.token.requestID)
|
||||
if (status === "failed") {
|
||||
const code = decoded.error?.code ?? undefined
|
||||
const message = decoded.error?.message ?? undefined
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
import { Effect, Schema } from "effect"
|
||||
import { Duration, Effect, Schema } from "effect"
|
||||
import type { HttpClientResponse } from "effect/unstable/http"
|
||||
import { ImageModel, ImageResponse, type ImageRequestFor } from "../image.js"
|
||||
import { Media } from "../media.js"
|
||||
import { MediaProtocol } from "../route/media-protocol.js"
|
||||
import { MediaRoute } from "../route/media.js"
|
||||
import { mergeJsonRecords, type OpenString } from "../schema/index.js"
|
||||
@@ -9,6 +8,7 @@ import { mergeJsonRecords, type OpenString } from "../schema/index.js"
|
||||
const route = MediaProtocol.identity({ id: "zai-images", name: "Z.ai Images", provider: "zai" })
|
||||
export const DEFAULT_BASE_URL = "https://api.z.ai/api/paas/v4"
|
||||
export const PATH = "/images/generations"
|
||||
const OUTPUT_RETENTION = Duration.days(30)
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 1. Public model input
|
||||
@@ -76,7 +76,7 @@ const decodeResponse = Effect.fn("ZAIImages.decodeResponse")(function* (
|
||||
const filters = decoded.content_filter ?? []
|
||||
return new ImageResponse({
|
||||
// Z.ai returns only URLs and no content type; the media type resolves when the asset is materialized.
|
||||
images: decoded.data.map((item) => Media.url(item.url)),
|
||||
images: yield* Effect.forEach(decoded.data, (item) => MediaProtocol.expiringUrl(item.url, OUTPUT_RETENTION)),
|
||||
// Z.ai reports applied content filters alongside a successful result; surface them instead of dropping them.
|
||||
notices:
|
||||
filters.length === 0
|
||||
|
||||
@@ -137,6 +137,11 @@ export interface Queued<Request, Response, Token> {
|
||||
readonly cancel?: {
|
||||
readonly method: AuthInput["method"]
|
||||
readonly path: (token: Token) => string
|
||||
/**
|
||||
* Fetch a fresh status first and skip the call for terminal generations, for providers whose cancel endpoint
|
||||
* destroys finished work (Runway's `DELETE /v1/tasks/{id}` deletes completed tasks and their outputs).
|
||||
*/
|
||||
readonly activeOnly?: boolean
|
||||
}
|
||||
}
|
||||
|
||||
@@ -195,7 +200,8 @@ export const identity = (input: { readonly id: string; readonly name: string; re
|
||||
/**
|
||||
* Read a text body while retaining the original payload and HTTP context on every downstream error. `invalid` is a
|
||||
* malformed provider document; `ended` is a generation that reached a terminal status without output (`failed` is
|
||||
* provider-side, `cancelled`/`expired` mean the result will never exist); `contentPolicy` is a moderated result.
|
||||
* provider-side, `cancelled`/`expired` mean the result will never exist); `pending` is a `result()` read before the
|
||||
* generation finished, which is caller misuse; `contentPolicy` is a moderated result.
|
||||
*/
|
||||
const text = Effect.fn("MediaProtocol.text")(function* (response: HttpClientResponse.HttpClientResponse) {
|
||||
const http = context(response)
|
||||
@@ -224,6 +230,14 @@ export const identity = (input: { readonly id: string; readonly name: string; re
|
||||
? new ProviderInternalError({ message, body, http })
|
||||
: new InvalidRequestError({ message, body, http }),
|
||||
}),
|
||||
pending: (id: string) =>
|
||||
new AIError({
|
||||
reason: new InvalidRequestError({
|
||||
message: `${input.name} generation ${id} has not finished; await it before reading the result`,
|
||||
body,
|
||||
http,
|
||||
}),
|
||||
}),
|
||||
contentPolicy: (message: string) => new AIError({ reason: new ContentPolicyError({ message, body, http }) }),
|
||||
}
|
||||
})
|
||||
@@ -285,9 +299,8 @@ export const status = <Table extends Record<string, Status>>(
|
||||
raw: string,
|
||||
output: Output,
|
||||
): Effect.Effect<Status, AIError> => {
|
||||
const normalized: Status | undefined = table[raw]
|
||||
if (normalized === undefined) return Effect.fail(output.invalid(`Unknown generation status "${raw}"`))
|
||||
return Effect.succeed(normalized)
|
||||
if (!Object.hasOwn(table, raw)) return Effect.fail(output.invalid(`Unknown generation status "${raw}"`))
|
||||
return Effect.succeed(table[raw])
|
||||
}
|
||||
|
||||
/** A `url` asset whose provider-declared retention window starts now. */
|
||||
|
||||
@@ -5,7 +5,7 @@ import { Endpoint } from "./endpoint.js"
|
||||
import { RequestExecutorService, type Interface } from "./executor-service.js"
|
||||
import { RequestExecutor } from "./executor.js"
|
||||
import { MediaProtocol } from "./media-protocol.js"
|
||||
import { Generation } from "../generation.js"
|
||||
import { Generation, isTerminal } from "../generation.js"
|
||||
import type { Media } from "../media.js"
|
||||
import {
|
||||
AIError,
|
||||
@@ -164,14 +164,19 @@ export const queued = <Request extends MediaRequest, Response, Token>(
|
||||
transport
|
||||
.call("GET", operation.path(token), http, execute)
|
||||
.pipe(Effect.flatMap((sent) => operation.decode(sent.response, { token, auth: sent.auth, materialize })))
|
||||
const status = poll(protocol.status)
|
||||
const cancel = protocol.cancel
|
||||
const send =
|
||||
cancel === undefined
|
||||
? undefined
|
||||
: transport.call(cancel.method, cancel.path(token), http, execute).pipe(Effect.asVoid)
|
||||
return {
|
||||
status: poll(protocol.status),
|
||||
status,
|
||||
result: poll(protocol.result),
|
||||
cancel:
|
||||
cancel === undefined
|
||||
? undefined
|
||||
: transport.call(cancel.method, cancel.path(token), http, execute).pipe(Effect.asVoid),
|
||||
send !== undefined && cancel?.activeOnly
|
||||
? status.pipe(Effect.flatMap((snapshot) => (isTerminal(snapshot.status) ? Effect.void : send)))
|
||||
: send,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -380,7 +385,11 @@ const encode = (body: MediaProtocol.Body | undefined, headers: Headers.Headers)
|
||||
}
|
||||
}
|
||||
|
||||
/** Common fields are never silently dropped: a present field the protocol declared unsupported fails typed. */
|
||||
/**
|
||||
* Common fields are never silently dropped: a present field the protocol declared unsupported fails typed. `false`
|
||||
* counts as present because some booleans mean something when false (video `audio`); protocols reject opt-in
|
||||
* booleans such as speech `timestamps` with `=== true` in `body.from` instead of listing them.
|
||||
*/
|
||||
const rejectUnsupported = <Request extends object>(
|
||||
route: string,
|
||||
provider: ProviderID,
|
||||
|
||||
@@ -351,10 +351,34 @@ describe("OpenAI Responses effort updates", () => {
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("strips markers when the body overlay selects pro reasoning mode", () =>
|
||||
Effect.gen(function* () {
|
||||
const prepared = yield* compileRequest(
|
||||
LLM.request({
|
||||
model: OpenAI.configure({ apiKey: "fixture", http: { body: { reasoning: { mode: "pro" } } } }).responses(
|
||||
"gpt-6-sol",
|
||||
),
|
||||
messages: conversation,
|
||||
providerOptions: { reasoningEffort: "low" },
|
||||
}),
|
||||
)
|
||||
|
||||
expect(updates(prepared.body)).toEqual([])
|
||||
expect(prepared.body.reasoning).toEqual({ effort: "low" })
|
||||
}),
|
||||
)
|
||||
|
||||
for (const [id, supported] of [
|
||||
["gpt-6-astra", true],
|
||||
["openai/gpt-6-astra", true],
|
||||
["gpt-6-sol", true],
|
||||
["openai/gpt-6-sol", true],
|
||||
["gpt-6-luna", true],
|
||||
["openai/gpt-6-luna", true],
|
||||
["gpt-6-astra-2026-09-01", false],
|
||||
["gpt-6-sol-pro", false],
|
||||
["gpt-6-luna-pro", false],
|
||||
["gpt-6-sol-fast", false],
|
||||
["gpt-5.6-sol", false],
|
||||
] as const) {
|
||||
it.effect(`${supported ? "lowers" : "strips"} markers for ${id}`, () =>
|
||||
|
||||
@@ -23,6 +23,7 @@ import { Provider as ProviderSubpath } from "@opencode/ai/provider"
|
||||
import {
|
||||
AssemblyAI,
|
||||
Baseten,
|
||||
BlackForestLabs,
|
||||
Cartesia,
|
||||
CloudflareAIGateway,
|
||||
CloudflareWorkersAI,
|
||||
@@ -32,14 +33,18 @@ import {
|
||||
Fal,
|
||||
Fireworks,
|
||||
Google,
|
||||
Meta,
|
||||
OpenCodeZen,
|
||||
OpenAI,
|
||||
OpenAICompatible,
|
||||
OpenRouter,
|
||||
Replicate,
|
||||
Runway,
|
||||
Stability,
|
||||
TypeSafeAI,
|
||||
VercelAIGateway,
|
||||
XAI,
|
||||
ZAI,
|
||||
} from "@opencode/ai/providers"
|
||||
import {
|
||||
OpenAIChat,
|
||||
@@ -151,8 +156,34 @@ describe("public exports", () => {
|
||||
expect(XAI.provider.chat).toBe(XAI.chat)
|
||||
expect(XAI.configure({ apiKey: "fixture" }).responses("grok-4.3").route.id).toBe("openai-responses")
|
||||
expect(XAI.configure({ apiKey: "fixture" }).chat("grok-4.3").route.id).toBe("openai-compatible-chat")
|
||||
expect(OpenAI.configure({ apiKey: "fixture" }).image("gpt-image-2").route.id).toBe("openai-images")
|
||||
expect(OpenAI.provider.image).toBe(OpenAI.image)
|
||||
expect(Google.configure({ apiKey: "fixture" }).image("imagen-4.0-generate-001").route.id).toBe("google-images")
|
||||
expect(Google.provider.image).toBe(Google.image)
|
||||
expect(XAI.configure({ apiKey: "fixture" }).image("grok-imagine-image").route.id).toBe("xai-images")
|
||||
expect(XAI.provider.image).toBe(XAI.image)
|
||||
expect(Fal.configure({ apiKey: "fixture" }).image("fal-ai/flux/dev").route.id).toBe("fal-images")
|
||||
expect(Fal.provider.image).toBe(Fal.image)
|
||||
expect(BlackForestLabs.configure({ apiKey: "fixture" }).image("flux-2-pro").route.id).toBe("bfl-images")
|
||||
expect(BlackForestLabs.provider.image).toBe(BlackForestLabs.image)
|
||||
expect(Replicate.configure({ apiKey: "fixture" }).image("black-forest-labs/flux-schnell").route.id).toBe(
|
||||
"replicate-images",
|
||||
)
|
||||
expect(Replicate.provider.image).toBe(Replicate.image)
|
||||
expect(Stability.configure({ apiKey: "fixture" }).image("sd3.5-large").route.id).toBe("stability-images")
|
||||
expect(Stability.provider.image).toBe(Stability.image)
|
||||
expect(Stability.configure({ apiKey: "fixture" }).upscale().route.id).toBe("stability-upscale")
|
||||
expect(Stability.provider.upscale).toBe(Stability.upscale)
|
||||
expect(Meta.configure({ apiKey: "fixture" }).image("muse-image").route.id).toBe("meta-images")
|
||||
expect(Meta.provider.image).toBe(Meta.image)
|
||||
expect(ZAI.configure({ apiKey: "fixture" }).image("glm-image").route.id).toBe("zai-images")
|
||||
expect(ZAI.provider.image).toBe(ZAI.image)
|
||||
expect(XAI.configure({ apiKey: "fixture" }).video("grok-imagine-video-1.5").route.id).toBe("xai-video")
|
||||
expect(XAI.provider.video).toBe(XAI.video)
|
||||
expect(Google.configure({ apiKey: "fixture" }).video("veo-3.1-generate-preview").route.id).toBe("google-video")
|
||||
expect(Google.provider.video).toBe(Google.video)
|
||||
expect(Fal.configure({ apiKey: "fixture" }).video("fal-ai/veo3.1").route.id).toBe("fal-video")
|
||||
expect(Fal.provider.video).toBe(Fal.video)
|
||||
expect(Runway.configure({ apiKey: "fixture" }).video("gen4.5").route.id).toBe("runway-video")
|
||||
expect(Runway.provider.video).toBe(Runway.video)
|
||||
expect(OpenAI.configure({ apiKey: "fixture" }).speech("gpt-4o-mini-tts").route.id).toBe("openai-speech")
|
||||
|
||||
@@ -97,6 +97,26 @@ describe("Generation", () => {
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("fails an event stream at the deadline when the poll interval is longer than the timeout", () =>
|
||||
Effect.gen(function* () {
|
||||
const scripted = yield* scriptedRoute(["running"], "never")
|
||||
const generation = new Generation(scripted.route, "t", { id: "gen_1", status: "queued" })
|
||||
|
||||
const fiber = yield* Effect.forkChild(
|
||||
generation
|
||||
.events({ poll: { interval: "30 seconds", timeout: "10 seconds" } })
|
||||
.pipe(Stream.runCollect, Effect.flip),
|
||||
)
|
||||
yield* TestClock.adjust("9 seconds")
|
||||
expect(fiber.pollUnsafe()).toBeUndefined()
|
||||
yield* TestClock.adjust("1 second")
|
||||
const error = yield* Fiber.join(fiber)
|
||||
|
||||
expect(error.reason._tag).toBe("Timeout")
|
||||
expect(yield* Ref.get(scripted.polls)).toBe(1)
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("surfaces the route failure body for failed generations", () =>
|
||||
Effect.gen(function* () {
|
||||
const scripted = yield* scriptedRoute(["running", "failed"], "unused")
|
||||
|
||||
@@ -78,8 +78,11 @@ describe("Image", () => {
|
||||
mediaType: "image/webp",
|
||||
})
|
||||
expect(yield* response.image.bytes()).toEqual(Uint8Array.from([1, 2, 3]))
|
||||
expect(response.image.providerMetadata).toEqual({ openai: { revisedPrompt: "A precise robot" } })
|
||||
expect(response.image.info).toEqual({ format: "webp", width: 2048, height: 2048 })
|
||||
expect(response.usage).toMatchObject({ type: "tokens", total: 12 })
|
||||
expect(response.providerMetadata).toEqual({
|
||||
openai: { outputFormat: "webp", size: "2048x2048", quality: "high", background: "opaque" },
|
||||
})
|
||||
}).pipe(
|
||||
Effect.provide(
|
||||
ImageClient.layer.pipe(
|
||||
@@ -107,8 +110,11 @@ describe("Image", () => {
|
||||
})
|
||||
return input.respond(
|
||||
JSON.stringify({
|
||||
data: [{ b64_json: "AQID", revised_prompt: "A precise robot" }, { b64_json: "BAUG" }],
|
||||
data: [{ b64_json: "AQID" }, { b64_json: "BAUG" }],
|
||||
output_format: "webp",
|
||||
size: "2048x2048",
|
||||
quality: "high",
|
||||
background: "opaque",
|
||||
usage: { input_tokens: 4, output_tokens: 8, total_tokens: 12 },
|
||||
}),
|
||||
{ headers: { "content-type": "application/json" } },
|
||||
@@ -144,6 +150,7 @@ describe("Image", () => {
|
||||
),
|
||||
)
|
||||
expect(response.image.source).toEqual({ type: "bytes", data: Uint8Array.from([1, 2, 3]), mediaType: "image/png" })
|
||||
expect(response.image.info).toEqual({ format: "png" })
|
||||
}),
|
||||
)
|
||||
|
||||
@@ -725,6 +732,7 @@ describe("Image", () => {
|
||||
const errors = yield* Effect.all(
|
||||
[
|
||||
Image.start({ model: Google.configure({ apiKey: "test" }).image("gemini-3.1-flash-image"), prompt }),
|
||||
Image.generate({ model: Google.configure({ apiKey: "test" }).image("gemini-3.1-flash-image"), prompt, n: 2 }),
|
||||
Image.start({
|
||||
model: BlackForestLabs.configure({ apiKey: "test" }).image("flux-2-pro"),
|
||||
prompt,
|
||||
@@ -735,7 +743,6 @@ describe("Image", () => {
|
||||
prompt,
|
||||
size: "512x512",
|
||||
}),
|
||||
Stream.runCollect(Image.stream({ model: openai.image("dall-e-3"), prompt })),
|
||||
Stream.runCollect(Image.stream({ model: openai.image("gpt-image-2"), prompt, n: 2 })),
|
||||
Image.start({ model: replicate, prompt, seed: 7 }),
|
||||
Image.start({
|
||||
@@ -749,9 +756,9 @@ describe("Image", () => {
|
||||
expect(errors.map((error) => [error.reason._tag, "operation" in error.reason && error.reason.operation])).toEqual(
|
||||
[
|
||||
["UnsupportedOperation", "image.start"],
|
||||
["UnsupportedOperation", "media.n"],
|
||||
["UnsupportedOperation", "media.aspectRatio"],
|
||||
["UnsupportedOperation", "media.size"],
|
||||
["UnsupportedOperation", "media.stream"],
|
||||
["UnsupportedOperation", "media.n"],
|
||||
["UnsupportedOperation", "media.seed"],
|
||||
["InvalidRequest", false],
|
||||
@@ -761,6 +768,80 @@ describe("Image", () => {
|
||||
}).pipe(Effect.provide(layer(() => Effect.die("an unsupported request reached the network")))),
|
||||
)
|
||||
|
||||
const falToken = {
|
||||
requestID: "r1",
|
||||
statusURL: "https://queue.fal.test/fal-ai/flux/requests/r1/status",
|
||||
responseURL: "https://queue.fal.test/fal-ai/flux/requests/r1",
|
||||
cancelURL: "https://queue.fal.test/fal-ai/flux/requests/r1/cancel",
|
||||
}
|
||||
const falSubmitted = {
|
||||
request_id: falToken.requestID,
|
||||
status_url: falToken.statusURL,
|
||||
response_url: falToken.responseURL,
|
||||
cancel_url: falToken.cancelURL,
|
||||
}
|
||||
const bodies: Array<unknown> = []
|
||||
it.effect("sizes fal Kontext by aspect ratio and sends several images to /multi", () =>
|
||||
Effect.gen(function* () {
|
||||
const fal = Fal.configure({ apiKey: "test", baseURL: "https://queue.fal.test" })
|
||||
const images = [Media.url("https://example.test/a.png"), Media.url("https://example.test/b.png")]
|
||||
const rejected = yield* Image.start({
|
||||
model: fal.image("fal-ai/flux-pro/kontext"),
|
||||
prompt: "A lighthouse",
|
||||
size: "512x512",
|
||||
}).pipe(Effect.flip)
|
||||
yield* Image.start({
|
||||
model: fal.image("fal-ai/flux-pro/kontext"),
|
||||
prompt: "A lighthouse",
|
||||
images: images.slice(0, 1),
|
||||
aspectRatio: "16:9",
|
||||
})
|
||||
yield* Image.start({ model: fal.image("fal-ai/flux-pro/kontext/max/multi"), prompt: "A lighthouse", images })
|
||||
|
||||
expect(rejected.reason).toMatchObject({ _tag: "UnsupportedOperation", operation: "media.size" })
|
||||
expect(bodies).toEqual([
|
||||
{ prompt: "A lighthouse", aspect_ratio: "16:9", image_url: "https://example.test/a.png" },
|
||||
{ prompt: "A lighthouse", image_urls: ["https://example.test/a.png", "https://example.test/b.png"] },
|
||||
])
|
||||
}).pipe(
|
||||
Effect.provide(
|
||||
layer((input) => {
|
||||
bodies.push(JSON.parse(input.text))
|
||||
return Effect.succeed(json(input, falSubmitted))
|
||||
}),
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
it.effect("decodes fal sync_mode data URIs as inline images", () =>
|
||||
Effect.gen(function* () {
|
||||
const generation = yield* Image.resume(Fal.configure({ apiKey: "test" }).image("fal-ai/flux/schnell"), falToken)
|
||||
const response = yield* generation.await()
|
||||
|
||||
expect(response.images.map((image) => image.source)).toEqual([
|
||||
{ type: "base64", data: "AQID", mediaType: "image/png" },
|
||||
{ type: "url", url: "https://v3.fal.media/out.jpg", mediaType: "image/jpeg" },
|
||||
])
|
||||
expect(response.image.info).toEqual({ width: 512, height: 512 })
|
||||
expect(yield* response.image.bytes()).toEqual(Uint8Array.from([1, 2, 3]))
|
||||
}).pipe(
|
||||
Effect.provide(
|
||||
layer((input) =>
|
||||
Effect.succeed(
|
||||
input.request.url === falToken.statusURL
|
||||
? json(input, { status: "COMPLETED" })
|
||||
: json(input, {
|
||||
images: [
|
||||
{ url: "data:image/png;base64,AQID", width: 512, height: 512, content_type: "image/png" },
|
||||
{ url: "https://v3.fal.media/out.jpg", width: 512, height: 512, content_type: "image/jpeg" },
|
||||
],
|
||||
}),
|
||||
),
|
||||
),
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
const moderated = { id: "req_1", status: "Content Moderated" }
|
||||
const prediction = {
|
||||
id: "p_1",
|
||||
@@ -768,6 +849,56 @@ describe("Image", () => {
|
||||
output: { text: "not an image" },
|
||||
urls: { get: "https://replicate.test/p_1", cancel: "https://replicate.test/p_1/cancel" },
|
||||
}
|
||||
for (const pending of [
|
||||
{
|
||||
model: BlackForestLabs.configure({ apiKey: "test" }).image("flux-2-pro"),
|
||||
token: { id: "req_1", pollingURL: "https://bfl.test/v1/get_result?id=req_1" },
|
||||
status: 200,
|
||||
body: { id: "req_1", status: "Pending" },
|
||||
message: "Black Forest Labs generation req_1",
|
||||
},
|
||||
{
|
||||
model: Replicate.configure({ apiKey: "test" }).image("owner/model"),
|
||||
token: { id: "p_1", getURL: "https://replicate.test/p_1", cancelURL: "https://replicate.test/p_1/cancel" },
|
||||
status: 200,
|
||||
body: {
|
||||
id: "p_1",
|
||||
status: "processing",
|
||||
urls: { get: "https://replicate.test/p_1", cancel: "https://replicate.test/p_1/cancel" },
|
||||
},
|
||||
message: "Replicate generation p_1",
|
||||
},
|
||||
{
|
||||
model: Stability.configure({ apiKey: "test", baseURL: "https://stability.test" }).upscale(),
|
||||
token: { id: "up_1" },
|
||||
status: 202,
|
||||
body: { id: "up_1", status: "in-progress" },
|
||||
message: "Stability AI generation up_1",
|
||||
},
|
||||
]) {
|
||||
it.effect(`rejects reading a ${pending.model.provider} result before the generation finishes`, () =>
|
||||
Effect.gen(function* () {
|
||||
const generation = yield* Image.resume(pending.model, pending.token)
|
||||
const error = yield* generation.result().pipe(Effect.flip)
|
||||
expect(error.reason._tag).toBe("InvalidRequest")
|
||||
expect(error.message).toBe(`${pending.message} has not finished; await it before reading the result`)
|
||||
expect(error.reason.body).toBe(JSON.stringify(pending.body))
|
||||
expect(error.reason.http?.status).toBe(pending.status)
|
||||
}).pipe(
|
||||
Effect.provide(
|
||||
layer((input) =>
|
||||
Effect.succeed(
|
||||
input.respond(JSON.stringify(pending.body), {
|
||||
status: pending.status,
|
||||
headers: { "content-type": "application/json" },
|
||||
}),
|
||||
),
|
||||
),
|
||||
),
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
it.effect("classifies terminal outcomes the recordings never saw", () =>
|
||||
Effect.gen(function* () {
|
||||
const bfl = yield* Image.resume(BlackForestLabs.configure({ apiKey: "test" }).image("flux-2-pro"), {
|
||||
|
||||
@@ -3,7 +3,7 @@ import { NodeFileSystem } from "@effect/platform-node"
|
||||
import { Effect, Ref, Schema } from "effect"
|
||||
import { FileSystem } from "effect"
|
||||
import { HttpClientRequest } from "effect/unstable/http"
|
||||
import { Media, Message } from "../src/index.js"
|
||||
import { AIError, Media, Message } from "../src/index.js"
|
||||
import { it } from "./lib/effect.js"
|
||||
import { dynamicResponse, scriptedResponses } from "./lib/http.js"
|
||||
|
||||
@@ -161,6 +161,56 @@ describe("Media", () => {
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("keeps transient url download headers out of toJSON and AssetSchema encoding", () =>
|
||||
Effect.sync(() => {
|
||||
const asset = Media.url("https://cdn.example.test/video.mp4", {
|
||||
mediaType: "video/mp4",
|
||||
expiresAt: 42,
|
||||
headers: { "x-goog-api-key": "secret" },
|
||||
})
|
||||
expect(asset.headers).toEqual({ "x-goog-api-key": "secret" })
|
||||
const source = { type: "url", url: "https://cdn.example.test/video.mp4", mediaType: "video/mp4", expiresAt: 42 }
|
||||
|
||||
expect(asset.toJSON()).not.toHaveProperty("headers")
|
||||
expect(JSON.stringify(asset)).not.toContain("secret")
|
||||
expect(asset.toJSON().source).toEqual(source)
|
||||
|
||||
const encoded = Schema.encodeSync(Media.AssetSchema)(asset)
|
||||
expect(encoded).not.toHaveProperty("headers")
|
||||
expect(encoded.source).toEqual(source)
|
||||
|
||||
const codec = Schema.fromJsonString(Media.AssetSchema)
|
||||
const json = Schema.encodeSync(codec)(asset)
|
||||
expect(json).not.toContain("secret")
|
||||
const restored = Schema.decodeSync(codec)(json)
|
||||
expect(restored).toBeInstanceOf(Media.Asset)
|
||||
expect(restored.source).toEqual(source)
|
||||
expect(restored.expiresAt).toBe(42)
|
||||
expect(restored.headers).toBeUndefined()
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("fails url downloads with non-2xx status as a typed AIError keeping http and body", () =>
|
||||
Effect.gen(function* () {
|
||||
const body = JSON.stringify({ error: { message: "file expired" } })
|
||||
const error = yield* Media.url("https://cdn.example.test/expired.png")
|
||||
.bytes()
|
||||
.pipe(
|
||||
Effect.flip,
|
||||
Effect.provide(
|
||||
dynamicResponse((input) =>
|
||||
Effect.succeed(input.respond(body, { status: 404, headers: { "content-type": "application/json" } })),
|
||||
),
|
||||
),
|
||||
)
|
||||
expect(error).toBeInstanceOf(AIError)
|
||||
expect(error.message).toContain("file expired")
|
||||
expect(error.reason.http?.status).toBe(404)
|
||||
expect(error.reason.http?.url).toBe("https://cdn.example.test/expired.png")
|
||||
expect(error.reason.body).toBe(body)
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("reads files with sniffed media types and writes materialized assets", () =>
|
||||
Effect.gen(function* () {
|
||||
const fs = yield* FileSystem.FileSystem
|
||||
|
||||
@@ -33,6 +33,8 @@ describe("Black Forest Labs Images recorded", () => {
|
||||
|
||||
expect(response.image.source.type).toBe("bytes")
|
||||
expect(dimensions(yield* response.image.bytes())).toEqual({ width: 512, height: 512 })
|
||||
// BFL reports cost on submit only; the Ready result omits it.
|
||||
expect(response.usage).toEqual({ type: "credits", credits: 1.4000000000000001 })
|
||||
}),
|
||||
{ timeout: 15 * 60 * 1000 },
|
||||
)
|
||||
|
||||
@@ -29,7 +29,11 @@ describe("OpenAI Images recorded", () => {
|
||||
|
||||
expect(response.images).toHaveLength(1)
|
||||
expect(response.image.mediaType).toBe("image/jpeg")
|
||||
expect(response.image.info).toEqual({ format: "jpeg", width: 1024, height: 1024 })
|
||||
expect((yield* response.image.bytes()).length).toBeGreaterThan(0)
|
||||
expect(response.providerMetadata).toEqual({
|
||||
openai: { outputFormat: "jpeg", size: "1024x1024", quality: "low", background: "opaque" },
|
||||
})
|
||||
}),
|
||||
)
|
||||
|
||||
@@ -76,8 +80,13 @@ describe("OpenAI Images recorded", () => {
|
||||
expect(events.map((event) => event.type)).toEqual(["image-partial", "image", "finish"])
|
||||
const image = events.find(ImageEvent.is.image)
|
||||
expect(image?.image.mediaType).toBe("image/jpeg")
|
||||
expect(image?.image.info).toEqual({ format: "jpeg", width: 1024, height: 1024 })
|
||||
expect(dimensions(yield* image!.image.bytes())).toEqual({ width: 1024, height: 1024 })
|
||||
expect(events.find(ImageEvent.is.finish)?.usage).toMatchObject({ type: "tokens" })
|
||||
const finish = events.find(ImageEvent.is.finish)
|
||||
expect(finish?.usage).toMatchObject({ type: "tokens" })
|
||||
expect(finish?.providerMetadata).toEqual({
|
||||
openai: { outputFormat: "jpeg", size: "1024x1024", quality: "low", background: "opaque" },
|
||||
})
|
||||
}),
|
||||
)
|
||||
})
|
||||
|
||||
@@ -44,7 +44,12 @@ describe("OpenAI Transcription recorded", () => {
|
||||
expect(deltas.length).toBeGreaterThan(1)
|
||||
expect(deltas.join("")).toBe(finish.text)
|
||||
expect(finish.text).toMatch(TRANSCRIPT)
|
||||
expect(finish.usage).toMatchObject({ type: "tokens", input: expect.any(Number), output: expect.any(Number) })
|
||||
expect(finish.usage).toMatchObject({
|
||||
type: "tokens",
|
||||
input: expect.any(Number),
|
||||
output: expect.any(Number),
|
||||
details: { openai: { input_token_details: { audio_tokens: expect.any(Number) } } },
|
||||
})
|
||||
}),
|
||||
)
|
||||
|
||||
|
||||
@@ -32,7 +32,12 @@ describe("Z.ai Images", () => {
|
||||
|
||||
expect(response.images).toHaveLength(1)
|
||||
expect(response.image.mediaType).toBe("application/octet-stream")
|
||||
expect(response.image.source).toEqual({ type: "url", url: "https://cdn.z.ai/generated.png" })
|
||||
// Z.ai documents that output URLs expire 30 days after generation; the test clock starts at 0.
|
||||
expect(response.image.source).toEqual({
|
||||
type: "url",
|
||||
url: "https://cdn.z.ai/generated.png",
|
||||
expiresAt: 30 * 24 * 60 * 60 * 1000,
|
||||
})
|
||||
expect(response.notices).toEqual([
|
||||
{
|
||||
type: "moderated",
|
||||
|
||||
@@ -97,6 +97,42 @@ describe("Speech", () => {
|
||||
}).pipe(Effect.provide(layer(() => Effect.die("an unsupported request reached the network")))),
|
||||
)
|
||||
|
||||
it.effect("treats timestamps: false as not asking for timestamps on routes that cannot return them", () =>
|
||||
Effect.gen(function* () {
|
||||
const bytes = Uint8Array.from([1, 2, 3])
|
||||
const gemini = JSON.stringify({
|
||||
candidates: [
|
||||
{ content: { parts: [{ inlineData: { mimeType: "audio/L16;codec=pcm;rate=24000", data: "AQID" } }] } },
|
||||
],
|
||||
})
|
||||
const responses = yield* Effect.all([
|
||||
Speech.generate({ model: openai, text: "Hi", timestamps: false }).pipe(
|
||||
Effect.provide(respond(new Blob([bytes]).stream(), "audio/mpeg")),
|
||||
),
|
||||
Speech.generate({ model: google, text: "Hi", timestamps: false }).pipe(
|
||||
Effect.provide(respond(gemini, "application/json")),
|
||||
),
|
||||
Speech.generate({ model: deepgram, text: "Hi", timestamps: false }).pipe(
|
||||
Effect.provide(respond(new Blob([bytes]).stream(), "audio/mpeg")),
|
||||
),
|
||||
])
|
||||
for (const response of responses) expect(yield* response.audio.bytes()).toEqual(bytes)
|
||||
|
||||
const errors = yield* Effect.all(
|
||||
[openai, google, deepgram].map((model) =>
|
||||
Speech.generate({ model, text: "Hi", timestamps: true }).pipe(Effect.flip),
|
||||
),
|
||||
).pipe(Effect.provide(layer(() => Effect.die("an unsupported request reached the network"))))
|
||||
expect(errors.map((error) => [error.reason._tag, "operation" in error.reason && error.reason.operation])).toEqual(
|
||||
[
|
||||
["UnsupportedOperation", "media.timestamps"],
|
||||
["UnsupportedOperation", "media.timestamps"],
|
||||
["UnsupportedOperation", "media.timestamps"],
|
||||
],
|
||||
)
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("classifies stream failures and keeps the provider payload and HTTP context", () =>
|
||||
Effect.gen(function* () {
|
||||
const badFrame = JSON.stringify({ type: "speech.audio.delta", audio: "not base64!" })
|
||||
|
||||
@@ -5,7 +5,7 @@ import { HttpClientRequest } from "effect/unstable/http"
|
||||
import { Media, Transcription, TranscriptionClient } from "../src/index.js"
|
||||
import { AssemblyAI, Deepgram, Google, OpenAI } from "../src/providers.js"
|
||||
import { it } from "./lib/effect.js"
|
||||
import { dynamicResponse } from "./lib/http.js"
|
||||
import { dynamicResponse, json, observe, type Call } from "./lib/http.js"
|
||||
|
||||
const layer = (handler: Parameters<typeof dynamicResponse>[0]) =>
|
||||
TranscriptionClient.layer.pipe(Layer.provideMerge(dynamicResponse(handler)))
|
||||
@@ -25,7 +25,6 @@ describe("Transcription", () => {
|
||||
Effect.gen(function* () {
|
||||
const errors = yield* Effect.all(
|
||||
[
|
||||
Stream.runCollect(Transcription.stream({ model: openai.transcription("whisper-1"), audio })),
|
||||
Transcription.generate({ model: openai.transcription("gpt-4o-mini-transcribe"), audio, diarize: true }),
|
||||
Transcription.generate({ model: openai.transcription("gpt-4o-mini-transcribe"), audio, timestamps: "word" }),
|
||||
Transcription.generate({ model: openai.transcription("gpt-4o-transcribe-diarize"), audio, prompt: "Names" }),
|
||||
@@ -53,7 +52,6 @@ describe("Transcription", () => {
|
||||
)
|
||||
expect(errors.map((error) => [error.reason._tag, "operation" in error.reason && error.reason.operation])).toEqual(
|
||||
[
|
||||
["UnsupportedOperation", "media.stream"],
|
||||
["UnsupportedOperation", "media.diarize"],
|
||||
["UnsupportedOperation", "media.timestamps"],
|
||||
["UnsupportedOperation", "media.prompt"],
|
||||
@@ -70,6 +68,67 @@ describe("Transcription", () => {
|
||||
}).pipe(Effect.provide(layer(() => Effect.die("an unsupported request reached the network")))),
|
||||
)
|
||||
|
||||
it.effect("ignores unknown OpenAI stream events and fails on an error event with the frame", () =>
|
||||
Effect.gen(function* () {
|
||||
const sse = (...frames: ReadonlyArray<string>) => frames.map((frame) => `data: ${frame}\n\n`).join("")
|
||||
const failure = `{"type":"error","error":{"type":"server_error","code":"server_error","message":"The server had an error"}}`
|
||||
const bodies = [
|
||||
sse(
|
||||
`{"type":"transcript.text.delta","delta":"Hi"}`,
|
||||
`{"type":"transcript.text.future","payload":1}`,
|
||||
`{"type":"transcript.text.done","text":"Hi"}`,
|
||||
"[DONE]",
|
||||
),
|
||||
sse(`{"type":"transcript.text.delta","delta":"Hi"}`, failure),
|
||||
]
|
||||
const model = openai.transcription("gpt-4o-mini-transcribe")
|
||||
const program = Effect.gen(function* () {
|
||||
const events = Array.from(yield* Stream.runCollect(Transcription.stream({ model, audio })))
|
||||
const error = yield* Stream.runCollect(Transcription.stream({ model, audio })).pipe(Effect.flip)
|
||||
return { events, error }
|
||||
})
|
||||
const { events, error } = yield* program.pipe(
|
||||
Effect.provide(
|
||||
layer((input) =>
|
||||
Effect.sync(() =>
|
||||
input.respond(bodies.shift() ?? "", { headers: { "content-type": "text/event-stream" } }),
|
||||
),
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
expect(events.map((event) => event.type)).toEqual(["text-delta", "finish"])
|
||||
expect(error.reason).toMatchObject({ _tag: "ProviderInternal", body: failure })
|
||||
expect(error.message).toContain("The server had an error")
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("streams whisper-1 as a single finish from a plain request", () =>
|
||||
Effect.gen(function* () {
|
||||
const bodies: Array<string> = []
|
||||
const events = Array.from(
|
||||
yield* Stream.runCollect(Transcription.stream({ model: openai.transcription("whisper-1"), audio })).pipe(
|
||||
Effect.provide(
|
||||
layer((input) =>
|
||||
Effect.sync(() => {
|
||||
bodies.push(input.text)
|
||||
return input.respond(
|
||||
JSON.stringify({ text: "Hello there.", usage: { type: "duration", seconds: 2 } }),
|
||||
{ headers: { "content-type": "application/json" } },
|
||||
)
|
||||
}),
|
||||
),
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
expect(bodies[0]).not.toContain('name="stream"')
|
||||
expect(events).toEqual([
|
||||
expect.objectContaining({ type: "finish", text: "Hello there.", usage: { type: "seconds", seconds: 2 } }),
|
||||
])
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect(
|
||||
"uploads inline audio to AssemblyAI, resumes polling from a persisted token, and surfaces failed transcripts",
|
||||
() =>
|
||||
@@ -171,4 +230,45 @@ describe("Transcription", () => {
|
||||
expect(failure.reason).toMatchObject({ _tag: "ProviderInternal", body: failed })
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("enables AssemblyAI speaker labels when only an expected speaker count is given", () =>
|
||||
Effect.gen(function* () {
|
||||
const calls: Array<Call> = []
|
||||
yield* Transcription.start({ model: assemblyai, audio: Media.url("https://a.test/call.mp3"), speakers: 2 }).pipe(
|
||||
Effect.provide(
|
||||
layer((input) => observe(calls, input).pipe(Effect.as(json(input, { id: "tr_1", status: "queued" })))),
|
||||
),
|
||||
)
|
||||
expect(calls.map((call) => JSON.parse(call.body))).toEqual([
|
||||
{
|
||||
audio_url: "https://a.test/call.mp3",
|
||||
speech_models: ["universal-3-5-pro"],
|
||||
language_detection: true,
|
||||
speaker_labels: true,
|
||||
speakers_expected: 2,
|
||||
},
|
||||
])
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("rejects reading an AssemblyAI result before the transcript finishes", () =>
|
||||
Effect.gen(function* () {
|
||||
const generation = yield* Transcription.resume(assemblyai, { transcriptID: "tr_1" })
|
||||
const error = yield* generation.result().pipe(Effect.flip)
|
||||
expect(error.reason._tag).toBe("InvalidRequest")
|
||||
expect(error.message).toBe("AssemblyAI generation tr_1 has not finished; await it before reading the result")
|
||||
expect(error.reason.body).toBe(JSON.stringify({ id: "tr_1", status: "processing" }))
|
||||
expect(error.reason.http?.status).toBe(200)
|
||||
}).pipe(
|
||||
Effect.provide(
|
||||
layer((input) =>
|
||||
Effect.succeed(
|
||||
input.respond(JSON.stringify({ id: "tr_1", status: "processing" }), {
|
||||
headers: { "content-type": "application/json" },
|
||||
}),
|
||||
),
|
||||
),
|
||||
),
|
||||
),
|
||||
)
|
||||
})
|
||||
|
||||
@@ -576,7 +576,7 @@ describe("Video / Runway", () => {
|
||||
const model = runway.video("gen4.5")
|
||||
const taskUrl = "https://runway.test/v1/tasks/task_1"
|
||||
|
||||
it.effect("submits image_to_video with the API version header, polls the task, and reports credits", () =>
|
||||
it.effect("submits image_to_video, polls the task, reports credits, and keeps the finished task on cancel", () =>
|
||||
Effect.gen(function* () {
|
||||
const calls: Array<Call> = []
|
||||
const program = Effect.gen(function* () {
|
||||
@@ -624,7 +624,7 @@ describe("Video / Runway", () => {
|
||||
return json(input, { id: "task_1", estimatedCost: { credits: 25 } })
|
||||
}
|
||||
expect(call.url).toBe(taskUrl)
|
||||
if (call.method === "DELETE") return input.respond(null, { status: 204 })
|
||||
if (call.method === "DELETE") return yield* Effect.die("cancel deleted a finished Runway task")
|
||||
if (nth === 1) return json(input, { id: "task_1", status: "PENDING", estimatedCost: { credits: 25 } })
|
||||
if (nth === 2) return json(input, { id: "task_1", status: "THROTTLED", estimatedCost: { credits: 25 } })
|
||||
if (nth === 3) return json(input, { id: "task_1", status: "RUNNING", progress: 0.5 })
|
||||
@@ -653,6 +653,32 @@ describe("Video / Runway", () => {
|
||||
`GET ${taskUrl}`,
|
||||
`GET ${taskUrl}`,
|
||||
`GET ${taskUrl}`,
|
||||
`GET ${taskUrl}`,
|
||||
])
|
||||
}),
|
||||
)
|
||||
|
||||
it.effect("cancels a task that is still running", () =>
|
||||
Effect.gen(function* () {
|
||||
const calls: Array<Call> = []
|
||||
yield* Effect.gen(function* () {
|
||||
const generation = yield* Video.start({ model, prompt: "x" })
|
||||
yield* generation.cancel()
|
||||
}).pipe(
|
||||
Effect.provide(
|
||||
layer((input) =>
|
||||
Effect.gen(function* () {
|
||||
const { call } = yield* observe(calls, input)
|
||||
if (call.method === "POST") return json(input, { id: "task_1" })
|
||||
if (call.method === "DELETE") return input.respond(null, { status: 204 })
|
||||
return json(input, { id: "task_1", status: "RUNNING", progress: 0.2 })
|
||||
}),
|
||||
),
|
||||
),
|
||||
)
|
||||
expect(calls.map((call) => `${call.method} ${call.url}`)).toEqual([
|
||||
"POST https://runway.test/v1/text_to_video",
|
||||
`GET ${taskUrl}`,
|
||||
`DELETE ${taskUrl}`,
|
||||
])
|
||||
}),
|
||||
@@ -812,3 +838,55 @@ describe("Video / Runway", () => {
|
||||
}),
|
||||
)
|
||||
})
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Shared queued behavior
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe("Video / queued result", () => {
|
||||
for (const pending of [
|
||||
{
|
||||
model: Google.configure({ apiKey: "test", baseURL: "https://google.test/v1beta" }).video("veo-3.1"),
|
||||
token: { operation: "models/veo-3.1/operations/op_1" },
|
||||
body: { name: "models/veo-3.1/operations/op_1", done: false },
|
||||
name: "Google Veo",
|
||||
},
|
||||
{
|
||||
model: XAI.configure({ apiKey: "test", baseURL: "https://xai.test/v1" }).video("grok-imagine-video-1.5"),
|
||||
token: { requestID: "req_1" },
|
||||
body: { status: "pending", progress: 40 },
|
||||
name: "xAI Video",
|
||||
},
|
||||
{
|
||||
model: Runway.configure({ apiKey: "test", baseURL: "https://runway.test/v1" }).video("gen4.5"),
|
||||
token: { taskID: "task_1" },
|
||||
body: { status: "RUNNING", progress: 0.5 },
|
||||
name: "Runway",
|
||||
},
|
||||
]) {
|
||||
it.effect(`rejects reading a ${pending.model.provider} result before the generation finishes`, () =>
|
||||
Effect.gen(function* () {
|
||||
const generation = yield* Video.resume(pending.model, pending.token)
|
||||
const error = yield* generation.result().pipe(Effect.flip)
|
||||
expect(error.reason._tag).toBe("InvalidRequest")
|
||||
expect(error.message).toBe(
|
||||
`${pending.name} generation ${generation.id} has not finished; await it before reading the result`,
|
||||
)
|
||||
expect(error.reason.body).toBe(JSON.stringify(pending.body))
|
||||
expect(error.reason.http?.status).toBe(200)
|
||||
}).pipe(Effect.provide(layer((input) => Effect.succeed(json(input, pending.body))))),
|
||||
)
|
||||
}
|
||||
|
||||
it.effect("rejects a status that only matches an inherited property", () =>
|
||||
Effect.gen(function* () {
|
||||
const error = yield* Video.resume(
|
||||
XAI.configure({ apiKey: "test", baseURL: "https://xai.test/v1" }).video("grok-imagine-video-1.5"),
|
||||
{ requestID: "req_1" },
|
||||
).pipe(Effect.flip)
|
||||
expect(error.reason._tag).toBe("InvalidProviderOutput")
|
||||
expect(error.message).toBe('Unknown generation status "constructor"')
|
||||
expect(error.reason.body).toBe(JSON.stringify({ status: "constructor" }))
|
||||
}).pipe(Effect.provide(layer((input) => Effect.succeed(json(input, { status: "constructor" }))))),
|
||||
)
|
||||
})
|
||||
|
||||
@@ -55,7 +55,7 @@ ultimate source of truth. Upstream test262 files run verbatim from `test/test262
|
||||
- [x] Tagged templates: a tag applied to a template literal is called as `tag(strings, ...values)`, with the tag read
|
||||
like a callee so a member tag keeps its receiver. `strings` is an array of the cooked text with a read-only `raw`
|
||||
array of the source text; an invalid escape such as `\unicode` cooks to `undefined`. One template object per
|
||||
site and both arrays are frozen, as in JS: `strings[0] = "x"` throws a `TypeError`.
|
||||
site, as in JS, but it is not frozen: `strings[0] = "x"` succeeds here where JS throws.
|
||||
- [x] Regular-expression literals.
|
||||
- [x] `NaN` and `Infinity` globals.
|
||||
- [ ] BigInt literals and in-interpreter BigInt arithmetic; BigInt remains invalid at JSON-like host boundaries.
|
||||
@@ -101,11 +101,9 @@ ultimate source of truth. Upstream test262 files run verbatim from `test/test262
|
||||
- [x] `for`, `while`, and `do...while`.
|
||||
- [x] `for...of` over arrays, strings, Maps, Sets, URLSearchParams, Headers, Uint8Arrays, built-in iterators, custom
|
||||
synchronous iterators, and confined synchronous generators. Abrupt completion invokes the iterator's optional `return()`.
|
||||
- [x] `for...in` over the enumerable keys of plain objects, arrays, strings, and tool references, following the
|
||||
prototype chain like JS (`for (k in Object.create({ a: 1 }))` visits `a`; built-in prototype methods are
|
||||
non-enumerable so `for (k in [])` visits nothing). A key deleted before its turn is skipped and keys added during
|
||||
the loop are not visited. `null`, `undefined`, and other non-objects iterate nothing. An un-awaited promise
|
||||
throws rather than iterating.
|
||||
- [x] `for...in` over own keys of plain objects, arrays, strings, and tool references. `null`, `undefined`, and other
|
||||
non-objects iterate nothing. An un-awaited promise throws rather than iterating.
|
||||
- [ ] `for...in` over inherited enumerable keys (`Object.create(proto)`), and skipping keys deleted during the loop.
|
||||
- [x] Unlabeled `break` and `continue`.
|
||||
- [x] `try`, `catch`, optional catch bindings, and `finally`.
|
||||
- [x] `throw` with arbitrary values.
|
||||
@@ -172,11 +170,8 @@ Math.floor)` is `"3"`). A detached method loses its receiver, as in JS: `values.
|
||||
- [x] Redeclaring a function in the same scope, or alongside a `var`, is allowed: the last declaration wins.
|
||||
- [x] Generator functions have their own `prototype` (inheriting the shared generator prototype), so
|
||||
`g() instanceof g` holds. Plain functions have none, since they cannot construct.
|
||||
- [x] Generator functions inherit from `GeneratorFunction.prototype` (async ones from
|
||||
`AsyncGeneratorFunction.prototype`), an ordinary non-callable object under `Function.prototype` whose `prototype`
|
||||
is the shared generator prototype and vice versa (`Object.getPrototypeOf(g).prototype.constructor`). Neither is
|
||||
a global; async non-generator functions still inherit from `Function.prototype` directly. A generator whose
|
||||
`prototype` was replaced by a non-object creates from the shared generator prototype.
|
||||
- [ ] `GeneratorFunction.prototype`: every function, generator or not, inherits directly from `Function.prototype`,
|
||||
and a generator whose `prototype` was replaced by a non-object still creates from the shared generator prototype.
|
||||
- [x] Generator and async generator functions bind parameters (defaults, destructuring) at the call and defer only the
|
||||
body to the first `next()`, so a bad argument throws synchronously from the call site, as in JS.
|
||||
- [x] Synchronous and async generator declarations/expressions, `yield`, and `yield*`, including lazy bodies,
|
||||
@@ -252,13 +247,9 @@ Math.floor)` is `"3"`). A detached method loses its receiver, as in JS: `values.
|
||||
hint (`"abc".indexOf({ toString() { return "b" } })` is `1`, `(255).toString({ valueOf() { return 16 } })` is
|
||||
`"ff"`, `String.prototype.trim.call({ toString() { return " a " } })` is `"a"`). Only consumed positions
|
||||
convert; a RegExp pattern is used as is, and `includes`/`startsWith`/`endsWith` reject one before converting.
|
||||
- [x] `Error.prototype.toString` converts an object `name` or `message` through its own `toString` (`String(e)` with
|
||||
`e.message = { toString() { return "m" } }` is `"Error: m"`); an uncaught error's report at the result boundary
|
||||
still uses the built-in form.
|
||||
- [x] `Array.prototype.toString` calls `this.join`, so `arr.join = () => "j"` makes `arr + ""`, `String(arr)`, and
|
||||
`${arr}` all `"j"`; a non-callable `join` gives `"[object Array]"`.
|
||||
- [ ] ToPrimitive elsewhere: numeric arguments of the Array and Uint8Array methods (`at`, `indexOf` start, `slice`)
|
||||
still use the built-in form (`NaN`) and ignore own methods.
|
||||
- [ ] ToPrimitive elsewhere: `Error.prototype.toString` on an object `message` and numeric arguments of the Array and
|
||||
Uint8Array methods (`at`, `indexOf` start, `slice`) still use the built-in form (`NaN`, `"[object Object]"`) and
|
||||
ignore own methods.
|
||||
- [x] Property keys follow ToPropertyKey: `x[null]` and `x[true]` become string keys, and a data object key
|
||||
converts through its own `toString`/`valueOf` (string hint) exactly once per access, in reads, writes,
|
||||
compound assignment, `++`, `delete`, `in`, object literals, and destructuring:
|
||||
@@ -268,8 +259,6 @@ Math.floor)` is `"3"`). A detached method loses its receiver, as in JS: `values.
|
||||
## Promises and tools
|
||||
|
||||
- [x] Tool calls start eagerly and return supervised, run-once CodeMode promises.
|
||||
- [x] Tool references have identity: `tools.x === tools.x` and `tools.ns === tools.ns`, so they work as `Map`/`Set`
|
||||
members and `switch` cases like any other object.
|
||||
- [x] Direct `await`, repeated awaits, and recursive thenable assimilation when a promise or thenable is returned from
|
||||
a function/program.
|
||||
- [x] `Promise.resolve` and `Promise.reject`.
|
||||
@@ -346,7 +335,7 @@ reject }` object.
|
||||
`TypeError`.
|
||||
- [x] `Object.create(proto)` with an object or `null` prototype; any other prototype is a `TypeError`
|
||||
(`Object prototype may only be an Object or null`). Inherited reads, `in`, `hasOwnProperty`, and own-only
|
||||
`Object.keys` follow the chain as in JS, and `for...in` enumerates inherited keys too. A second `properties`
|
||||
`Object.keys` follow the chain as in JS, but `for...in` still enumerates own keys only. A second `properties`
|
||||
argument other than `undefined` throws a `TypeError`: property descriptors are not supported (there is no
|
||||
`Object.defineProperty` either).
|
||||
- [x] `Object.freeze`, `Object.seal`, and `Object.preventExtensions`, with `isFrozen`, `isSealed`, and `isExtensible`.
|
||||
|
||||
@@ -20,7 +20,7 @@ import {
|
||||
type Value,
|
||||
} from "./objects.js"
|
||||
import type { Interpreter } from "./interpreter.js"
|
||||
import { toPrimitiveString, withPrimitives } from "./callback.js"
|
||||
import { toPrimitiveString } from "./callback.js"
|
||||
import { formatValue } from "../stdlib/console.js"
|
||||
|
||||
export const normalizeError = (error: unknown): Diagnostic => {
|
||||
@@ -48,9 +48,7 @@ export const normalizeError = (error: unknown): Diagnostic => {
|
||||
if (error instanceof Throw) {
|
||||
const value = error.value
|
||||
if (value instanceof ErrorObj) {
|
||||
return value.host
|
||||
? normalizeError(value.host)
|
||||
: { kind: "ExecutionFailure", message: errorToString(get(value, "name"), get(value, "message")) }
|
||||
return value.host ? normalizeError(value.host) : { kind: "ExecutionFailure", message: errorToString(value) }
|
||||
}
|
||||
let message: string
|
||||
if (containsRuntimeReference(value)) {
|
||||
@@ -115,7 +113,9 @@ export const materialize = <R>(ctx: Interpreter<R>, thrown: unknown): Value => {
|
||||
}
|
||||
|
||||
/** Error.prototype.toString: `name: message`, omitting whichever side is empty. */
|
||||
const errorToString = (name: Value, message: Value): string => {
|
||||
const errorToString = (self: Obj): string => {
|
||||
const name = get(self, "name")
|
||||
const message = get(self, "message")
|
||||
const shownName = name === undefined ? "Error" : coerceToString(name)
|
||||
const shownMessage = message === undefined ? "" : coerceToString(message)
|
||||
if (shownMessage === "") return shownName
|
||||
@@ -179,16 +179,7 @@ export const errorGlobal = <R>(type: ErrorType, ctx: Interpreter<R>) => {
|
||||
})
|
||||
if (type === "Error") {
|
||||
methods(builtins, prototype, [
|
||||
[
|
||||
"toString",
|
||||
0,
|
||||
(thisValue) => {
|
||||
const self = receiver(Obj, thisValue, "Error.prototype.toString")
|
||||
return withPrimitives(ctx, "string", [get(self, "name"), get(self, "message")], ([name, message]) =>
|
||||
errorToString(name, message),
|
||||
)
|
||||
},
|
||||
],
|
||||
["toString", 0, (thisValue) => errorToString(receiver(Obj, thisValue, "Error.prototype.toString"))],
|
||||
])
|
||||
methods(builtins, ctor, [["isError", 1, (_, args) => args[0] instanceof ErrorObj]])
|
||||
return ctor
|
||||
|
||||
@@ -79,8 +79,7 @@ import {
|
||||
has,
|
||||
hidden,
|
||||
hasPrototype,
|
||||
ownKeys,
|
||||
enumerable,
|
||||
keys,
|
||||
Native,
|
||||
parseArrayIndex,
|
||||
Arguments,
|
||||
@@ -101,7 +100,7 @@ import { Pending, resolvePromise, resolvePromiseValue } from "./promises.js"
|
||||
import { describeValue, isOpaque, rejectCircularInsertion, typeofValue } from "./references.js"
|
||||
import { ScopeStack } from "./scope.js"
|
||||
import { constructRegExp } from "../stdlib/regexp.js"
|
||||
import { enumerableSource, restrict } from "../stdlib/object.js"
|
||||
import { enumerableSource } from "../stdlib/object.js"
|
||||
import { compoundOperators } from "../stdlib/value.js"
|
||||
|
||||
/** The binary operators that convert object operands through ToPrimitive before acting on primitives. */
|
||||
@@ -507,7 +506,7 @@ class Frame<R> {
|
||||
): Fn {
|
||||
const builtins = this.ctx.builtins
|
||||
const fn = new Fn(
|
||||
node.generator ? (node.async ? builtins.AsyncGeneratorFunction : builtins.GeneratorFunction) : builtins.Function,
|
||||
builtins.Function,
|
||||
name,
|
||||
node.params,
|
||||
node.body,
|
||||
@@ -956,24 +955,10 @@ class Frame<R> {
|
||||
}
|
||||
|
||||
// for...in over null/undefined iterates nothing, like JS.
|
||||
// EnumerateObjectProperties: own keys, then each prototype's, visiting a shadowed key once.
|
||||
private enumerableKeys(value: Value, node: AstNode): Array<string> {
|
||||
if (value instanceof ToolReference) return [...this.ctx.tools.keys(value.path)]
|
||||
if (value === null || value === undefined) return []
|
||||
const seen = new Set<string>()
|
||||
const result: Array<string> = []
|
||||
for (
|
||||
let current: Obj | null = enumerableSource(this.ctx, "for...in", value, node);
|
||||
current !== null;
|
||||
current = current.proto
|
||||
) {
|
||||
for (const key of ownKeys(current)) {
|
||||
if (typeof key !== "string" || seen.has(key)) continue
|
||||
seen.add(key)
|
||||
if (enumerable(current, key)) result.push(key)
|
||||
}
|
||||
}
|
||||
return result
|
||||
return keys(enumerableSource(this.ctx, "for...in", value, node))
|
||||
}
|
||||
|
||||
private evaluateForInStatement(
|
||||
@@ -997,8 +982,6 @@ class Frame<R> {
|
||||
const assignment = left.type === "VariableDeclaration" ? undefined : left
|
||||
|
||||
for (const key of keys) {
|
||||
// A key deleted before its turn is skipped, as in JS.
|
||||
if (right instanceof Obj && !has(right, key)) continue
|
||||
const result = yield* Effect.gen(function* () {
|
||||
if (declared?.lexical) {
|
||||
self.scopes.push()
|
||||
@@ -1727,15 +1710,12 @@ class Frame<R> {
|
||||
})
|
||||
}
|
||||
|
||||
// Built-ins throw without a location, synchronously or inside their Effect; the call site supplies it. A built-in
|
||||
// reached through `ctx.call` runs on the root frame, so the deeper of the frame and the enclosing site counts.
|
||||
// Built-ins throw without a location, synchronously or inside their Effect; the call site supplies it.
|
||||
private native(body: () => Effect.Effect<Value, unknown, R>, node?: AstNode): Effect.Effect<Value, unknown, R> {
|
||||
return Effect.flatMap(CallSite, (site) =>
|
||||
Effect.provideService(
|
||||
Effect.catchDefect(Effect.suspend(body), (defect) => Effect.die(locate(defect, node))),
|
||||
CallSite,
|
||||
{ node, depth: Math.max(this.depth, site.depth) },
|
||||
),
|
||||
return Effect.provideService(
|
||||
Effect.catchDefect(Effect.suspend(body), (defect) => Effect.die(locate(defect, node))),
|
||||
CallSite,
|
||||
{ node, depth: this.depth },
|
||||
)
|
||||
}
|
||||
|
||||
@@ -2175,16 +2155,12 @@ class Frame<R> {
|
||||
define(
|
||||
strings,
|
||||
"raw",
|
||||
restrict(
|
||||
"freeze",
|
||||
new Arr(
|
||||
array,
|
||||
node.quasi.quasis.map((quasi) => quasi.value.raw),
|
||||
),
|
||||
new Arr(
|
||||
array,
|
||||
node.quasi.quasis.map((quasi) => quasi.value.raw),
|
||||
),
|
||||
frozen,
|
||||
)
|
||||
restrict("freeze", strings)
|
||||
this.ctx.templates.set(node, strings)
|
||||
return strings
|
||||
}
|
||||
@@ -2232,7 +2208,7 @@ class Frame<R> {
|
||||
if (typeof key !== "string") {
|
||||
throw typeError("Tool paths must use string property names.", propertyNode)
|
||||
}
|
||||
return objectValue.child(key)
|
||||
return new ToolReference([...objectValue.path, key])
|
||||
}
|
||||
|
||||
if (objectValue instanceof Obj) return { target: objectValue, key, receiver: objectValue }
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { Effect } from "effect"
|
||||
import { Arr, define, ErrorObj, hidden, Native, Obj, readOnly, type Value } from "./objects.js"
|
||||
import { define, hidden, Native, Arr, ErrorObj, Obj, type Value } from "./objects.js"
|
||||
|
||||
export const errorTypes = [
|
||||
"Error",
|
||||
@@ -41,8 +41,6 @@ const builtins = [
|
||||
"AsyncIterator",
|
||||
"Generator",
|
||||
"AsyncGenerator",
|
||||
"GeneratorFunction",
|
||||
"AsyncGeneratorFunction",
|
||||
] as const
|
||||
|
||||
/**
|
||||
@@ -81,15 +79,6 @@ export const createBuiltins = (): Builtins => {
|
||||
}
|
||||
const iterator = plain()
|
||||
const asyncIterator = plain()
|
||||
// %GeneratorFunction.prototype% is an ordinary object linked both ways with %GeneratorPrototype%.
|
||||
const generatorFunction = (generator: Obj) => {
|
||||
const proto = new Obj(fn)
|
||||
define(proto, "prototype", generator, readOnly)
|
||||
define(generator, "constructor", proto, readOnly)
|
||||
return proto
|
||||
}
|
||||
const generator = new Obj(iterator)
|
||||
const asyncGenerator = new Obj(asyncIterator)
|
||||
return {
|
||||
Object: object,
|
||||
Function: fn,
|
||||
@@ -113,10 +102,8 @@ export const createBuiltins = (): Builtins => {
|
||||
Iterator: iterator,
|
||||
IteratorHelper: new Obj(iterator),
|
||||
AsyncIterator: asyncIterator,
|
||||
Generator: generator,
|
||||
AsyncGenerator: asyncGenerator,
|
||||
GeneratorFunction: generatorFunction(generator),
|
||||
AsyncGeneratorFunction: generatorFunction(asyncGenerator),
|
||||
Generator: new Obj(iterator),
|
||||
AsyncGenerator: new Obj(asyncIterator),
|
||||
Error: error,
|
||||
TypeError: derived("TypeError"),
|
||||
RangeError: derived("RangeError"),
|
||||
|
||||
@@ -36,7 +36,6 @@ export const hidden: Attributes = { writable: true, enumerable: false, configura
|
||||
export const readonly: Attributes = { writable: false, enumerable: false, configurable: true }
|
||||
/** Constants such as `Math.PI` and a constructor's `prototype`. */
|
||||
export const frozen: Attributes = { writable: false, enumerable: false, configurable: false }
|
||||
export const readOnly: Attributes = { writable: false, enumerable: false, configurable: true }
|
||||
|
||||
/**
|
||||
* An object owned by the program: own properties plus a prototype link. Subclasses answer, in one place, how a
|
||||
@@ -646,7 +645,7 @@ export const ownKeys = (target: Obj): Array<string | symbol> => {
|
||||
]
|
||||
}
|
||||
|
||||
export const enumerable = (target: Obj, key: string | symbol): boolean => own(target, key)?.enumerable === true
|
||||
const enumerable = (target: Obj, key: string | symbol): boolean => own(target, key)?.enumerable === true
|
||||
|
||||
/** Own enumerable keys, including the iterator symbols; what spread and `Object.assign` copy. */
|
||||
export const enumerableKeys = (target: Obj): Array<string | symbol> =>
|
||||
|
||||
@@ -5,7 +5,6 @@ import { invalidData, IteratorSymbol, rangeError, typeError } from "../interpret
|
||||
import {
|
||||
define,
|
||||
get,
|
||||
Callable,
|
||||
hidden,
|
||||
Arr,
|
||||
GeneratorObj,
|
||||
@@ -166,12 +165,13 @@ export const arrayGlobal = <R>(ctx: Interpreter<R>) => {
|
||||
[
|
||||
"toString",
|
||||
0,
|
||||
(thisValue) => {
|
||||
// Spec: delegate to this.join, so an overridden join shows up in `arr + ""` and String(arr).
|
||||
const target = self(thisValue, "toString")
|
||||
const join = get(target, "join")
|
||||
return join instanceof Callable ? ctx.call(join, target, []) : `[object ${target.tag}]`
|
||||
},
|
||||
(thisValue) =>
|
||||
withPrimitives(
|
||||
ctx,
|
||||
"string",
|
||||
Array.from(self(thisValue, "toString").items, (item) => item ?? ""),
|
||||
(items) => items.map(coerceToString).join(","),
|
||||
),
|
||||
],
|
||||
[
|
||||
"includes",
|
||||
|
||||
@@ -104,7 +104,7 @@ const propertyKey = (value: Value): PropertyKey =>
|
||||
|
||||
// SetIntegrityLevel: primitives pass through. A typed array's bytes cannot carry attributes, so JS throws after
|
||||
// already making it non-extensible.
|
||||
export const restrict = (level: "freeze" | "seal" | "preventExtensions", value: Value): Value => {
|
||||
const restrict = (level: "freeze" | "seal" | "preventExtensions", value: Value): Value => {
|
||||
if (!(value instanceof Obj)) return value
|
||||
value.extensible = false
|
||||
if (level === "preventExtensions") return value
|
||||
|
||||
@@ -113,16 +113,7 @@ export const toolExpression = (path: string) =>
|
||||
.join("")
|
||||
|
||||
export class ToolReference {
|
||||
private readonly children = new Map<string, ToolReference>()
|
||||
constructor(readonly path: ReadonlyArray<string>) {}
|
||||
/** One reference per path, so `tools.a === tools.a` holds like any other member read. */
|
||||
child(key: string): ToolReference {
|
||||
const existing = this.children.get(key)
|
||||
if (existing !== undefined) return existing
|
||||
const created = new ToolReference([...this.path, key])
|
||||
this.children.set(key, created)
|
||||
return created
|
||||
}
|
||||
}
|
||||
|
||||
// Dots in tool names are namespace separators; the last tool for a canonical path wins.
|
||||
|
||||
@@ -178,25 +178,6 @@ describe("call depth", () => {
|
||||
expect(Date.now() - started).toBeLessThan(2000)
|
||||
})
|
||||
|
||||
test("recursion routed through nested built-ins keeps counting depth", async () => {
|
||||
const started = Date.now()
|
||||
expect(
|
||||
await value(`
|
||||
const a = [1]
|
||||
a.join = () => a + ""
|
||||
const o = { toString() { return [o].map(String)[0] } }
|
||||
const e = new Error()
|
||||
e.message = { toString() { return String(e) } }
|
||||
const names = []
|
||||
for (const run of [() => String(a), () => String(o), () => String(e)]) {
|
||||
try { run() } catch (error) { names.push(error.name) }
|
||||
}
|
||||
return names
|
||||
`),
|
||||
).toEqual(["RangeError", "RangeError", "RangeError"])
|
||||
expect(Date.now() - started).toBeLessThan(2000)
|
||||
})
|
||||
|
||||
test("uncaught overflow reports the call that overflowed", async () => {
|
||||
const failure = await error(`const f = (n) => f(n + 1); return f(0)`)
|
||||
expect(failure.kind).toBe("ExecutionFailure")
|
||||
|
||||
@@ -1382,7 +1382,7 @@ describe("Object.getPrototypeOf and Object.create", () => {
|
||||
expect((await error(`Object.getPrototypeOf(Symbol.iterator)`)).message).toContain("cannot convert a symbol")
|
||||
})
|
||||
|
||||
test("Object.create links the prototype: inherited reads, in, own-only keys, and for...in", async () => {
|
||||
test("Object.create links the prototype: inherited reads, in, and own-only keys", async () => {
|
||||
expect(
|
||||
await value(`
|
||||
const p = { greet(name) { return "hi " + name }, a: 1 }
|
||||
@@ -1392,7 +1392,7 @@ describe("Object.getPrototypeOf and Object.create", () => {
|
||||
for (const key in c) seen.push(key)
|
||||
return ["greet" in c, Object.keys(c), c.hasOwnProperty("a"), c.a, c.greet(c.name), Object.getPrototypeOf(c) === p, Object.getPrototypeOf(Object.create(null)), seen]
|
||||
`),
|
||||
).toEqual([true, ["name"], false, 1, "hi x", true, null, ["name", "greet", "a"]])
|
||||
).toEqual([true, ["name"], false, 1, "hi x", true, null, ["name"]])
|
||||
})
|
||||
|
||||
test("Object.create rejects non-object prototypes and property descriptors", async () => {
|
||||
@@ -1984,79 +1984,3 @@ describe("WeakMap and WeakSet", () => {
|
||||
expect((await error(`structuredClone(new WeakSet())`)).message).toContain("DataCloneError")
|
||||
})
|
||||
})
|
||||
|
||||
describe("small language leftovers", () => {
|
||||
test("for...in walks the prototype chain and skips keys deleted before their turn", async () => {
|
||||
expect(
|
||||
await value(`
|
||||
const o = Object.create({ a: 1, shadowed: 1 })
|
||||
o.b = 2
|
||||
o.shadowed = 3
|
||||
const keys = []
|
||||
for (const k in o) keys.push(k)
|
||||
const live = { a: 1, b: 2, c: 3 }
|
||||
const seen = []
|
||||
for (const k in live) { seen.push(k); delete live.b; live.z = 1 }
|
||||
const none = []
|
||||
for (const k in []) none.push(k)
|
||||
for (const k in new TypeError("x")) none.push(k)
|
||||
return [keys, seen, none]
|
||||
`),
|
||||
).toEqual([["b", "shadowed", "a"], ["a", "c"], []])
|
||||
})
|
||||
|
||||
test("tagged template objects are frozen", async () => {
|
||||
expect(
|
||||
await value(`
|
||||
const tag = (s) => s
|
||||
const f = () => tag\`a\${1}b\`
|
||||
return [f() === f(), Object.isFrozen(f()), Object.isFrozen(f().raw)]
|
||||
`),
|
||||
).toEqual([true, true, true])
|
||||
expect((await error("const tag = (s) => s; tag`a`[0] = 'x'")).message).toContain("read only")
|
||||
})
|
||||
|
||||
test("Array.prototype.toString delegates to join", async () => {
|
||||
expect(
|
||||
await value(`
|
||||
const a = [1, 2]
|
||||
a.join = () => "j"
|
||||
const b = [1]
|
||||
b.join = 5
|
||||
return [a + "", String(a), \`\${a}\`, b.toString(), Array.prototype.toString.call([3, [4]])]
|
||||
`),
|
||||
).toEqual(["j", "j", "j", "[object Array]", "3,4"])
|
||||
})
|
||||
|
||||
test("Error.prototype.toString converts object name and message", async () => {
|
||||
expect(
|
||||
await value(`
|
||||
const e = new Error("m")
|
||||
e.message = { toString() { return "obj" } }
|
||||
e.name = { valueOf() { return "N" }, toString() { return "T" } }
|
||||
return [String(e), Error.prototype.toString.call({ name: "", message: "m" }), Error.prototype.toString.call({})]
|
||||
`),
|
||||
).toEqual(["T: obj", "m", "Error"])
|
||||
expect(
|
||||
(await error(`const e = new Error(); e.message = { toString() { throw new RangeError("r") } }; String(e)`))
|
||||
.message,
|
||||
).toContain("r")
|
||||
})
|
||||
|
||||
test("generator functions inherit from GeneratorFunction.prototype", async () => {
|
||||
expect(
|
||||
await value(`
|
||||
function* g() {}
|
||||
async function* ag() {}
|
||||
const GFP = Object.getPrototypeOf(g)
|
||||
g.prototype = null
|
||||
return [
|
||||
typeof GFP, GFP === Function.prototype, Object.getPrototypeOf(GFP) === Function.prototype,
|
||||
GFP.prototype.constructor === GFP, Object.getPrototypeOf(ag) === GFP, typeof g.bind,
|
||||
Object.getPrototypeOf(g()) === GFP.prototype,
|
||||
]
|
||||
`),
|
||||
).toEqual(["object", false, true, true, false, "function", true])
|
||||
expect((await error(`function* g() {} Object.getPrototypeOf(g)()`)).message).toContain("not a function")
|
||||
})
|
||||
})
|
||||
|
||||
@@ -884,6 +884,7 @@ built-ins/AsyncGeneratorFunction/invoked-as-constructor-no-arguments.js # TypeE
|
||||
built-ins/AsyncGeneratorFunction/invoked-as-function-multiple-arguments.js # TypeError: The Function constructor is not supported; write the function inline.
|
||||
built-ins/AsyncGeneratorFunction/invoked-as-function-no-arguments.js # TypeError: The Function constructor is not supported; write the function inline.
|
||||
built-ins/AsyncGeneratorFunction/invoked-as-function-single-argument.js # TypeError: The Function constructor is not supported; write the function inline.
|
||||
built-ins/AsyncGeneratorFunction/prototype/not-callable.js # Expected SameValue(«"function"», «"object"») to be true
|
||||
built-ins/AsyncGeneratorPrototype/next/this-val-not-async-generator.js # TypeError: Cannot read properties of undefined (reading '…').
|
||||
built-ins/AsyncGeneratorPrototype/return/this-val-not-async-generator.js # TypeError: Cannot read properties of undefined (reading '…').
|
||||
built-ins/AsyncGeneratorPrototype/throw/this-val-not-async-generator.js # TypeError: Cannot read properties of undefined (reading '…').
|
||||
@@ -899,6 +900,7 @@ built-ins/Date/prototype/toJSON/invoke-result.js # TypeError: Date.prototype.to
|
||||
built-ins/Date/prototype/toJSON/to-primitive-value-of.js # TypeError: Date.prototype.toJSON called on incompatible receiver a data object.
|
||||
built-ins/Date/prototype/toString/format.js # Expected SameValue(«null», «null») to be false
|
||||
built-ins/Date/prototype/toString/negative-year.js # Date.prototype.toString serializes year -1 to "-0001" Expected SameValue(«undefined», «"-0001"») to
|
||||
built-ins/Error/prototype/toString/tostring-message-throws-toprimitive.js # ToPrimitive(msg) called by ToString(msg) throws TypeError Expected a TypeError to be thrown but no e
|
||||
built-ins/Function/15.3.5-1gs.js # Expected a TypeError to be thrown but no exception was thrown at all
|
||||
built-ins/Function/15.3.5-2gs.js # Expected a TypeError to be thrown but no exception was thrown at all
|
||||
built-ins/Function/15.3.5.4_2-1gs.js # Expected a TypeError to be thrown but no exception was thrown at all
|
||||
@@ -929,6 +931,7 @@ built-ins/GeneratorFunction/invoked-as-constructor-no-arguments.js # TypeError:
|
||||
built-ins/GeneratorFunction/invoked-as-function-multiple-arguments.js # TypeError: The Function constructor is not supported; write the function inline.
|
||||
built-ins/GeneratorFunction/invoked-as-function-no-arguments.js # TypeError: The Function constructor is not supported; write the function inline.
|
||||
built-ins/GeneratorFunction/invoked-as-function-single-argument.js # TypeError: The Function constructor is not supported; write the function inline.
|
||||
built-ins/GeneratorFunction/prototype/not-callable.js # Expected SameValue(«"function"», «"object"») to be true
|
||||
built-ins/GeneratorPrototype/throw/from-state-completed.js # Expected a E but got a TypeError
|
||||
built-ins/GeneratorPrototype/throw/from-state-suspended-start.js # Expected a E but got a TypeError
|
||||
built-ins/Iterator/from/return-method-returns-iterator-result.js # Iterator next must be a function.
|
||||
@@ -1465,10 +1468,12 @@ language/expressions/function/dstr/ary-init-iter-get-err-array-prototype.js # E
|
||||
language/expressions/function/dstr/ary-ptrn-elem-id-iter-val-array-prototype.js # Expected SameValue(«3», «42») to be true
|
||||
language/expressions/function/dstr/dflt-ary-init-iter-get-err-array-prototype.js # Expected a TypeError to be thrown but no exception was thrown at all
|
||||
language/expressions/function/dstr/dflt-ary-ptrn-elem-id-iter-val-array-prototype.js # Expected SameValue(«3», «42») to be true
|
||||
language/expressions/generators/default-proto.js # Expected SameValue(«object», «undefined») to be true
|
||||
language/expressions/generators/dstr/ary-init-iter-get-err-array-prototype.js # Expected a TypeError to be thrown but no exception was thrown at all
|
||||
language/expressions/generators/dstr/ary-ptrn-elem-id-iter-val-array-prototype.js # Expected SameValue(«3», «42») to be true
|
||||
language/expressions/generators/dstr/dflt-ary-init-iter-get-err-array-prototype.js # Expected a TypeError to be thrown but no exception was thrown at all
|
||||
language/expressions/generators/dstr/dflt-ary-ptrn-elem-id-iter-val-array-prototype.js # Expected SameValue(«3», «42») to be true
|
||||
language/expressions/generators/prototype-relation-to-function.js # Expected SameValue(«object», «») to be true
|
||||
language/expressions/greater-than-or-equal/S11.8.4_A3.2_T1.2.js # TypeError: Binary operators require data values.
|
||||
language/expressions/greater-than/S11.8.2_A3.2_T1.2.js # TypeError: Binary operators require data values.
|
||||
language/expressions/in/S8.12.6_A2_T2.js # TypeError: Robin cannot be constructed: user-defined constructors and classes are not supported. Cal
|
||||
@@ -1554,6 +1559,7 @@ language/expressions/super/prop-expr-obj-key-err.js # unsupported syntax Super
|
||||
language/expressions/super/prop-expr-obj-ref-strict.js # unsupported syntax Super
|
||||
language/expressions/super/prop-expr-obj-unresolvable.js # Expected SameValue(«SyntaxError», «ReferenceError») to be true
|
||||
language/expressions/tagged-template/constructor-invocation.js # The called value cannot be constructed: user-defined constructors and classes are not supported.
|
||||
language/expressions/tagged-template/template-object-frozen-strict.js # Expected a TypeError to be thrown but no exception was thrown at all
|
||||
language/expressions/this/11.1.1-1.js # Expected SameValue(«undefined», «undefined») to be false
|
||||
language/expressions/unary-minus/S11.4.7_A3_T5.js # TypeError: Unary operators require data values.
|
||||
language/expressions/unary-plus/S11.4.6_A3_T5.js # TypeError: Unary operators require data values.
|
||||
@@ -1582,6 +1588,7 @@ language/statements/async-generator/dstr/dflt-ary-ptrn-elem-id-iter-val-array-pr
|
||||
language/statements/async-generator/return-undefined-implicit-and-explicit.js # Actual ["tick 1", "tick 2", "g1 ret", "g2 ret", "g3 ret", "g4 ret"] and expected ["tick 1", "g1 ret"
|
||||
language/statements/const/dstr/ary-init-iter-get-err-array-prototype.js # Expected a TypeError to be thrown but no exception was thrown at all
|
||||
language/statements/const/dstr/ary-ptrn-elem-id-iter-val-array-prototype.js # Expected SameValue(«3», «42») to be true
|
||||
language/statements/for-in/S12.6.4_A7_T2.js # for...in visits keys deleted during the loop
|
||||
language/statements/for-in/order-enumerable-shadowed.js # Object.create property descriptors are not supported; assign the fields after creating the object.
|
||||
language/statements/for-in/S12.6.4_A6.1.js # TypeError: FACTORY cannot be constructed: user-defined constructors and classes are not supported. C
|
||||
language/statements/for-in/S12.6.4_A6.js # TypeError: FACTORY cannot be constructed: user-defined constructors and classes are not supported. C
|
||||
@@ -1653,10 +1660,12 @@ language/statements/function/dstr/ary-init-iter-get-err-array-prototype.js # Ex
|
||||
language/statements/function/dstr/ary-ptrn-elem-id-iter-val-array-prototype.js # Expected SameValue(«3», «42») to be true
|
||||
language/statements/function/dstr/dflt-ary-init-iter-get-err-array-prototype.js # Expected a TypeError to be thrown but no exception was thrown at all
|
||||
language/statements/function/dstr/dflt-ary-ptrn-elem-id-iter-val-array-prototype.js # Expected SameValue(«3», «42») to be true
|
||||
language/statements/generators/default-proto.js # generator functions have no GeneratorFunction.prototype
|
||||
language/statements/generators/dstr/ary-init-iter-get-err-array-prototype.js # Expected a TypeError to be thrown but no exception was thrown at all
|
||||
language/statements/generators/dstr/ary-ptrn-elem-id-iter-val-array-prototype.js # Expected SameValue(«3», «42») to be true
|
||||
language/statements/generators/dstr/dflt-ary-init-iter-get-err-array-prototype.js # Expected a TypeError to be thrown but no exception was thrown at all
|
||||
language/statements/generators/dstr/dflt-ary-ptrn-elem-id-iter-val-array-prototype.js # Expected SameValue(«3», «42») to be true
|
||||
language/statements/generators/prototype-relation-to-function.js # generator functions have no GeneratorFunction.prototype
|
||||
language/statements/generators/restricted-properties.js # Expected a TypeError to be thrown but no exception was thrown at all
|
||||
language/statements/labeled/value-await-non-module.js # Failed to parse TypeScript: Expression expected.
|
||||
language/statements/let/dstr/ary-init-iter-get-err-array-prototype.js # Expected a TypeError to be thrown but no exception was thrown at all
|
||||
|
||||
@@ -368,16 +368,3 @@ describe("tool references under ==", () => {
|
||||
).toEqual([false, false, false, 0])
|
||||
})
|
||||
})
|
||||
|
||||
describe("tool reference identity", () => {
|
||||
test("repeated member reads yield the same reference", async () => {
|
||||
const runtime = CodeMode.make({ tools: { probe: echo("Probe", "ok"), "ns.inner": echo("Inner", "in") } })
|
||||
expect(
|
||||
await value(
|
||||
runtime,
|
||||
`return [tools.probe === tools.probe, tools.ns.inner === tools.ns.inner, tools.ns === tools.ns, tools["probe"] === tools.probe,
|
||||
tools.probe === tools.ns.inner, new Set([tools.probe, tools.probe]).size, await tools.ns.inner({})]`,
|
||||
),
|
||||
).toEqual([true, true, true, true, false, 1, "in"])
|
||||
})
|
||||
})
|
||||
|
||||
@@ -54,8 +54,7 @@ const modelID = (model: Model.Info) => model.modelID ?? model.id
|
||||
|
||||
function claudeInfo(model: Model.Info) {
|
||||
const id = modelID(model)
|
||||
// Versions are at most two digits so snapshot dates such as `-20260901` are not read as versions.
|
||||
const familyFirst = /(?:claude-)?(opus|sonnet|haiku|fable|mythos)-(\d{1,2})(?:[.-](\d{1,2}))?(?!\d)/i.exec(id)
|
||||
const familyFirst = /(?:claude-)?(opus|sonnet|haiku|fable|mythos)-(\d+)(?:[.-](\d+))?/i.exec(id)
|
||||
const versionFirst = /claude-(\d+)(?:[.-](\d+))?-(opus|sonnet|haiku|fable|mythos)/i.exec(id)
|
||||
const family = (familyFirst?.[1] ?? versionFirst?.[3])?.toLowerCase()
|
||||
const major = Number(familyFirst?.[2] ?? versionFirst?.[1])
|
||||
@@ -65,12 +64,7 @@ function claudeInfo(model: Model.Info) {
|
||||
major,
|
||||
minor,
|
||||
manual: (major === 3 && minor === 7) || (major === 4 && minor < 6),
|
||||
// Opus 5.5 and later reject disabled thinking.
|
||||
always:
|
||||
family === "fable" ||
|
||||
family === "mythos" ||
|
||||
id.toLowerCase().includes("mythos-preview") ||
|
||||
(family === "opus" && (major > 5 || (major === 5 && minor >= 5))),
|
||||
always: family === "fable" || family === "mythos" || id.toLowerCase().includes("mythos-preview"),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -396,7 +390,6 @@ const bedrockConverse: Protocol = (model, support) => {
|
||||
return fields({ reasoningConfig: { type: "enabled", maxReasoningEffort: effort } })
|
||||
})
|
||||
case "toggle":
|
||||
if (claude && claudeInfo(model).always) return []
|
||||
return claude
|
||||
? toggle(fields({ thinking: { type: "disabled" } }), fields({ thinking: ADAPTIVE_THINKING }))
|
||||
: toggle(fields({ reasoningConfig: { type: "disabled" } }), fields({ reasoningConfig: { type: "enabled" } }))
|
||||
@@ -456,7 +449,6 @@ const bedrockAISDK: Protocol = (model, support) => {
|
||||
: { reasoningConfig: { type: "enabled", maxReasoningEffort: effort } },
|
||||
}))
|
||||
case "toggle":
|
||||
if (claude && claudeInfo(model).always) return []
|
||||
return claude
|
||||
? toggle(
|
||||
{ settings: { additionalModelRequestFields: { thinking: { type: "disabled" } } } },
|
||||
@@ -515,7 +507,6 @@ const sapAICore: Protocol = (model, support) => {
|
||||
sap({ additionalModelRequestFields: { thinking: { type: "disabled" } } }),
|
||||
sap({ additionalModelRequestFields: { thinking: { type: "enabled" } } }),
|
||||
)
|
||||
if (id.includes("anthropic") && claudeInfo(model).always) return []
|
||||
if (id.includes("anthropic"))
|
||||
return toggle(
|
||||
sap({ additionalModelRequestFields: { thinking: { type: "disabled" } } }),
|
||||
|
||||
@@ -104,30 +104,6 @@ test("recognizes Claude version spellings and future models", () => {
|
||||
])
|
||||
})
|
||||
|
||||
test("keeps thinking on for Claude Opus 5.5 and later", () => {
|
||||
const supports: Variant.Support[] = [{ type: "toggle" }, { type: "effort", values: ["low", "high"] }]
|
||||
const adaptive = ["low", "high"].map((effort) => ({
|
||||
id: effort,
|
||||
settings: { effort, thinking: { type: "adaptive", display: "summarized" } },
|
||||
}))
|
||||
for (const input of [
|
||||
model("@opencode/ai/providers/cloudflare-ai-gateway", "anthropic/claude-opus-5.5"),
|
||||
model("@opencode/ai/providers/anthropic", "claude-opus-5-5"),
|
||||
model("@opencode/ai/providers/anthropic", "claude-opus-6"),
|
||||
])
|
||||
expect(resolve(input, supports)).toEqual(adaptive)
|
||||
|
||||
expect(
|
||||
resolve(model("@opencode/ai/providers/amazon-bedrock", "us.anthropic.claude-opus-5-5-v1:0"), [{ type: "toggle" }]),
|
||||
).toEqual([])
|
||||
|
||||
for (const id of ["claude-opus-5", "claude-opus-5-20260901"])
|
||||
expect(resolve(model("@opencode/ai/providers/anthropic", id), supports)).toEqual([
|
||||
{ id: "none", settings: { thinking: { type: "disabled" } } },
|
||||
...adaptive,
|
||||
])
|
||||
})
|
||||
|
||||
test("spells Cloudflare AI Gateway variants for their upstream routes", () => {
|
||||
const pkg = "@opencode/ai/providers/cloudflare-ai-gateway"
|
||||
expect(resolve(model(pkg, "openai/gpt-5.4"), [{ type: "effort", values: ["low", "xhigh"] }])).toEqual([
|
||||
|
||||
@@ -128,5 +128,4 @@ workspace with `/connect`, and `/models` lists their enabled models.
|
||||
## Usage and limits
|
||||
|
||||
The provider bills you for the tokens. Console records every request and, using the pricing configured on the model,
|
||||
counts it toward workspace and member monthly limits. In [usage exports](/console/usage), BYOK rows have
|
||||
`billing_source` set to `byok` and a zero `cost_micro_cents`.
|
||||
counts it toward workspace and member monthly limits.
|
||||
|
||||
@@ -149,8 +149,8 @@ Token prices are per 1M tokens.
|
||||
| Qwen3.7 Plus (> 256K tokens) | $1.20 | $4.80 | $0.12 | $1.50 | **$60** |
|
||||
| Qwen3.6 Plus (≤ 256K tokens) | $0.50 | $3.00 | $0.05 | $0.625 | **$60** |
|
||||
| Qwen3.6 Plus (> 256K tokens) | $2.00 | $6.00 | $0.20 | $2.50 | **$60** |
|
||||
| DeepSeek V4.1 Flash (Off-Peak) | $0.15 | $0.60 | $0.003 | - | ~~$15~~ **$60**<br /><small>4x · Ends Sep 27</small> |
|
||||
| DeepSeek V4.1 Flash (Peak) | $0.30 | $1.20 | $0.006 | - | ~~$15~~ **$60**<br /><small>4x · Ends Sep 27</small> |
|
||||
| DeepSeek V4.1 Flash (Off-Peak) | $0.15 | $0.60 | $0.003 | - | **$60** |
|
||||
| DeepSeek V4.1 Flash (Peak) | $0.30 | $1.20 | $0.006 | - | **$60** |
|
||||
| DeepSeek V4 Pro (Off-Peak) | $0.66 | $1.98 | $0.022 | - | **$15** |
|
||||
| DeepSeek V4 Pro (Peak) | $1.32 | $3.96 | $0.044 | - | **$15** |
|
||||
| DeepSeek V4 Flash (Off-Peak) | $0.15 | $0.60 | $0.003 | - | **$30** |
|
||||
@@ -206,7 +206,7 @@ The table below provides an estimated request count based on typical Go usage pa
|
||||
| Qwen3.7 Max | 170 | 420 | 840 |
|
||||
| Qwen3.7 Plus | 4,300 | 10,800 | 21,600 |
|
||||
| Qwen3.6 Plus | 3,300 | 8,200 | 16,300 |
|
||||
| DeepSeek V4.1 Flash<br /><small>4x · Ends Sep 27</small> | ~~6,500~~<br />**26,000** | ~~16,250~~<br />**65,000** | ~~32,500~~<br />**130,000** |
|
||||
| DeepSeek V4.1 Flash | 26,000 | 65,000 | 130,000 |
|
||||
| DeepSeek V4 Pro | 1,050 | 2,600 | 5,200 |
|
||||
| DeepSeek V4 Flash | 13,000 | 32,500 | 65,000 |
|
||||
| DeepSeek V4 Flash Vision Exp | 6,500 | 16,250 | 32,500 |
|
||||
|
||||
@@ -1,146 +0,0 @@
|
||||
---
|
||||
title: "Usage API"
|
||||
description: "Export usage records as CSV for a workspace, member, service account, or model."
|
||||
---
|
||||
|
||||
Export usage records as CSV for a workspace, member, service account, or model. Use the versioned endpoint in scripts,
|
||||
reporting jobs, and billing integrations.
|
||||
|
||||
## Quick start
|
||||
|
||||
Create a service account API key in the [Console](https://opencode.ai/console). This endpoint accepts service account
|
||||
keys only; user session tokens are rejected.
|
||||
|
||||
```bash
|
||||
export CONSOLE_URL="https://opencode.ai/console"
|
||||
export SERVICE_API_KEY="oc_sk_..."
|
||||
```
|
||||
|
||||
Export the last seven days of workspace usage:
|
||||
|
||||
```bash
|
||||
curl --fail-with-body --get \
|
||||
"${CONSOLE_URL}/api/v1/usage/export" \
|
||||
--header "Authorization: Bearer ${SERVICE_API_KEY}" \
|
||||
--header "Accept: text/csv" \
|
||||
--data-urlencode "scope=organization" \
|
||||
--data-urlencode "range=7d" \
|
||||
--output "usage-organization-7d.csv"
|
||||
```
|
||||
|
||||
A successful request writes `usage-organization-7d.csv`. Remove `--output` to print the CSV in your terminal instead.
|
||||
|
||||
## Endpoint and scopes
|
||||
|
||||
```text
|
||||
GET /api/v1/usage/export
|
||||
```
|
||||
|
||||
Every request requires `scope` and `range`. Supported ranges are `24h`, `7d`, and `30d`. They start at midnight UTC
|
||||
rather than being rolling windows. The full range is returned as a single streamed CSV file, regardless of size.
|
||||
|
||||
| Scope | Additional parameters | Result |
|
||||
| ----------------- | ---------------------- | ---------------------------------------- |
|
||||
| `organization` | None | All usage in the workspace. |
|
||||
| `member` | `user_email` | Usage attributed to one member. |
|
||||
| `service_account` | `service_account_id` | Usage attributed to one service account. |
|
||||
| `model` | `provider` and `model` | Usage for one provider and model pair. |
|
||||
|
||||
<Callout title="Use only the parameters for your scope">
|
||||
Member, service account, and model filters are mutually exclusive. Extra filters return HTTP 400 instead of being
|
||||
silently ignored.
|
||||
</Callout>
|
||||
|
||||
## Export examples
|
||||
|
||||
One member:
|
||||
|
||||
```bash
|
||||
curl --fail-with-body --get \
|
||||
"${CONSOLE_URL}/api/v1/usage/export" \
|
||||
--header "Authorization: Bearer ${SERVICE_API_KEY}" \
|
||||
--header "Accept: text/csv" \
|
||||
--data-urlencode "scope=member" \
|
||||
--data-urlencode "range=24h" \
|
||||
--data-urlencode "user_email=alice@example.com" \
|
||||
--output "usage-member-24h.csv"
|
||||
```
|
||||
|
||||
One service account:
|
||||
|
||||
```bash
|
||||
curl --fail-with-body --get \
|
||||
"${CONSOLE_URL}/api/v1/usage/export" \
|
||||
--header "Authorization: Bearer ${SERVICE_API_KEY}" \
|
||||
--header "Accept: text/csv" \
|
||||
--data-urlencode "scope=service_account" \
|
||||
--data-urlencode "range=30d" \
|
||||
--data-urlencode "service_account_id=svcacct_..." \
|
||||
--output "usage-service-account-30d.csv"
|
||||
```
|
||||
|
||||
One provider and model:
|
||||
|
||||
```bash
|
||||
curl --fail-with-body --get \
|
||||
"${CONSOLE_URL}/api/v1/usage/export" \
|
||||
--header "Authorization: Bearer ${SERVICE_API_KEY}" \
|
||||
--header "Accept: text/csv" \
|
||||
--data-urlencode "scope=model" \
|
||||
--data-urlencode "range=7d" \
|
||||
--data-urlencode "provider=anthropic" \
|
||||
--data-urlencode "model=claude-sonnet-4-5" \
|
||||
--output "usage-model-7d.csv"
|
||||
```
|
||||
|
||||
Provider is the provider catalog key, such as `anthropic`, `deepseek`, or `opencode`. A valid filter with no matching
|
||||
records still returns HTTP 200 with a header-only CSV file.
|
||||
|
||||
## CSV response
|
||||
|
||||
Results are newest first. Empty optional values are blank cells. `cost_micro_cents` is the amount charged by Console
|
||||
in microcents, where 100,000,000 equals one dollar. Managed-inference usage carries its charge; free, BYOK, and
|
||||
unclassified legacy usage have a zero charge.
|
||||
|
||||
Organization and member exports include [Websearch](/console/websearch) charges. Search rows identify `web-search` in
|
||||
the `service` column and leave provider, model, and token fields blank. Model and provider filters select inference
|
||||
records only.
|
||||
|
||||
<div class="docs-table-scroll" role="region" aria-label="Usage export CSV fields" tabIndex={0}>
|
||||
|
||||
| Field | Description |
|
||||
| ------------------------- | -------------------------------------------------------------------------------------------------------- |
|
||||
| `id` | Unique record ID. New service records use a `service:` prefix; inference and historical IDs are numeric. |
|
||||
| `user_email` | Current email of the member that generated the usage, when applicable. |
|
||||
| `service_account_name` | Current name of the service account that generated the usage, when applicable. |
|
||||
| `app` | Client app title, or its referrer when no title was reported. |
|
||||
| `provider` | Provider catalog key, such as `openai`, `anthropic`, `deepseek`, or `opencode`. |
|
||||
| `model` | Model identifier reported for the request. |
|
||||
| `input_tokens` | Number of input tokens consumed. |
|
||||
| `output_tokens` | Number of output tokens generated. |
|
||||
| `reasoning_tokens` | Number of reasoning tokens reported by the provider. |
|
||||
| `cache_read_tokens` | Number of input tokens read from the provider cache. |
|
||||
| `cache_write_5m_tokens` | Number of tokens written to a five-minute cache. |
|
||||
| `cache_write_1h_tokens` | Number of tokens written to a one-hour cache. |
|
||||
| `reasoning_mode` | Reasoning mode used, such as `effort`, `adaptive`, `budget`, or `disabled`. |
|
||||
| `reasoning_effort` | Provider-specific reasoning effort value, when supplied. |
|
||||
| `reasoning_budget_tokens` | Requested reasoning-token budget, when supplied. |
|
||||
| `reasoning_source` | Source from which the reasoning configuration was derived. |
|
||||
| `billing_source` | How the request was funded, such as `managed-inference`, `credit`, `byok`, or `free`. |
|
||||
| `cost_micro_cents` | Amount charged by Console in microcents. Divide by 100,000,000 to convert to USD. |
|
||||
| `created_at` | When the usage was recorded, as an ISO 8601 UTC timestamp. |
|
||||
| `service` | Service identifier, currently `web-search`. Blank for model requests. |
|
||||
| `quantity` | Number of billable operations represented by the record; normally one. |
|
||||
|
||||
</div>
|
||||
|
||||
## Errors
|
||||
|
||||
| Status | Meaning |
|
||||
| ------ | ------------------------------------------------------------------------------------ |
|
||||
| `400` | Missing parameters, invalid values, or filters that do not match the selected scope. |
|
||||
| `401` | Missing, invalid, expired, or revoked service API key. |
|
||||
| `403` | The authenticated service account is not allowed to read usage. |
|
||||
|
||||
Curl uses `--fail-with-body` in these examples so authentication and validation failures produce a non-zero exit code
|
||||
while preserving any error response.
|
||||
@@ -146,7 +146,6 @@ export const docsSections: DocsSection[] = [
|
||||
items: [
|
||||
{ title: "Inference", slug: "console/inference" },
|
||||
{ title: "BYOK", slug: "console/byok" },
|
||||
{ title: "Usage", slug: "console/usage" },
|
||||
{ title: "Budgets", slug: "console/budgets" },
|
||||
],
|
||||
},
|
||||
|
||||
Reference in New Issue
Block a user