opencode/packages/ai

@opencode-ai/ai

Schema-first AI primitives for opencode. Provider quirks live in adapters, not in calling code.

import { Effect } from "effect"
import { LLM, LLMClient } from "@opencode-ai/ai"
import { OpenAI } from "@opencode-ai/ai/providers"

const model = OpenAI.configure({ apiKey: process.env.OPENAI_API_KEY }).responses("gpt-4o-mini")

const request = LLM.request({
  model,
  system: "You are concise.",
  prompt: "Say hello in one short sentence.",
  generation: { maxTokens: 40 },
})

const program = Effect.gen(function* () {
  const response = yield* LLMClient.generate(request)
  console.log(response.text)
})

Run LLMClient.stream(request) instead of generate when you want incremental LLMEvents. The event stream is provider-neutral — same shape across OpenAI Chat, OpenAI Responses, Anthropic Messages, Gemini, Bedrock Converse, and any OpenAI-compatible deployment.

Image generation

Use Image.generate with an image model for direct asset generation:

import { Image, ImageInput } from "@opencode-ai/ai"
import { OpenAI } from "@opencode-ai/ai/providers"

const program = Effect.gen(function* () {
  const response = yield* Image.generate({
    model: OpenAI.configure({ apiKey: process.env.OPENAI_API_KEY }).image("gpt-image-2"),
    prompt: "A robot tending a rooftop garden",
    options: {
      n: 2,
      size: "1024x1024",
      quality: "high", // inferred from the OpenAI image model
      outputFormat: "webp",
      future_option: true, // unknown native options pass through unchanged
    },
  })

  return response.images // GeneratedImage[] with owned bytes or a provider URL
})

Pass ordered image inputs to the same method for editing, composition, or image-conditioned generation:

const response =
  yield *
  Image.generate({
    model,
    prompt: "Combine these product photos into one studio scene",
    images: [
      ImageInput.bytes(firstBytes, "image/png"),
      ImageInput.url("https://example.com/second.webp"),
      ImageInput.file("file_123"),
    ],
    options,
    http,
  })

ImageInput.fileUri(uri, mediaType) represents provider file URIs such as Gemini Files. Raw strings are not accepted as image inputs, avoiding ambiguity between base64, URLs, and provider IDs. Empty or omitted images uses text-to-image generation; a non-empty array selects the provider's edit behavior without enforcing provider image-count limits locally. images is the only common image-editing field. OpenAI uses multipart for byte/data-URL edits and its JSON reference body for URL or file-ID edits. Its provider-specific options.mask accepts an ImageInput for inpainting:

yield *
  Image.generate({
    model: OpenAI.configure({ apiKey }).image("gpt-image-2"),
    prompt,
    images: [ImageInput.bytes(sourceBytes, "image/png")],
    options: { mask: ImageInput.bytes(maskBytes, "image/png") },
  })

The OpenAI adapter extracts this helper value into the edit request's native mask field rather than passing the tagged ImageInput object through as an ordinary option. On multipart requests, http.body can override option fields but not structural model, prompt, image[], or mask fields, and the transport owns the multipart Content-Type boundary. For JSON requests, http.body remains the final raw-native overlay. Gemini does not fetch public HTTP URLs, and hosted Z.ai image generation does not accept image inputs. These cases fail with InvalidRequest before network I/O.

Provider-native image options belong to each request. Raw http.body fields have final precedence over them:

const model = OpenAI.configure({ apiKey }).image("gpt-image-2")

yield *
  Image.generate({
    model,
    prompt,
    options: { quality: "medium" },
    http,
  })

xAI image models use the same request API with xAI-native controls:

yield *
  Image.generate({
    model: XAI.configure({ apiKey }).image("any-model-id"),
    prompt,
    options: {
      n: 2,
      aspectRatio: "16:9",
      resolution: "1k",
      responseFormat: "b64_json",
      future_option: true,
    },
    http,
  })

Google's current Gemini image models use the same direct API:

import { Google } from "@opencode-ai/ai/providers"

const googleProgram = Effect.gen(function* () {
  const response = yield* Image.generate({
    model: Google.configure({ apiKey }).image("any-model-id"),
    prompt: "A robot tending a rooftop garden",
    options: {
      aspectRatio: "16:9",
      imageSize: "2K",
      seed: 42,
      thinkingLevel: "HIGH",
      includeThoughts: true,
      futureOption: true,
    },
    http,
  })

  return response.images
})

Google image options are request-scoped and inferred from the selected model. Known fields autocomplete while future string values and arbitrary native Gemini generationConfig fields remain available. Native fields override their mapped aliases, and http.body is the final deep overlay. The selected model ID is sent to Gemini generateContent without a local allowlist.

Z.ai image models infer open Z.ai-native options from the selected model:

yield *
  Image.generate({
    model: ZAI.configure({ apiKey }).image("any-model-id"),
    prompt,
    options: {
      quality: "hd",
      userID: "user-123",
      future_option: true,
    },
    http,
  })

Z.ai does not include trustworthy MIME metadata for output URLs, so generated images use application/octet-stream. Output URLs expire after 30 days; download and persist them promptly if they must remain available.

Conversational image generation remains part of the LLM interaction. OpenAI Responses exposes it through its hosted image tool:

const program = Effect.gen(function* () {
  const response = yield* LLM.generate(
    LLM.request({
      model: OpenAI.configure({ apiKey }).responses("gpt-5"),
      prompt: "Design a solarpunk rooftop garden, then show me.",
      tools: [OpenAI.imageGeneration({ quality: "high" })],
    }),
  )

  return response.message
})

The hosted result is represented as a provider-executed tool call and tool result. Its image is a file content item with a data URI, so retaining response.message preserves the generated image for continuation.

Public API

• LLM.request({...}) — build a provider-neutral LLMRequest. Accepts ergonomic inputs (system: string, prompt: string) that normalize into the canonical Schema classes.
• LLM.generate / LLM.stream — re-exported from LLMClient for one-import use.
• Message.user(...) / Message.assistant(...) / Message.tool(...) — message constructors from the canonical schema model.
• Model.make(...) / ToolCallPart.make(...) / ToolResultPart.make(...) / ToolDefinition.make(...) — model and tool-related constructors from the canonical schema model.
• LLMClient.prepare(request) — compile a request through protocol body construction, validation, and HTTP preparation without sending. Useful for inspection and testing.
• LLMEvent.is.* — typed guards (is.textDelta, is.toolCall, is.finish, …) for filtering streams.
• Image.generate({...}) — generate images through a provider-neutral image request and response model.
• ImageClient — Effect service and layer for image execution, parallel to LLMClient.

Caching

Prompt caching is on by default. Every LLMRequest resolves to cache: "auto" unless the caller opts out with cache: "none". Each protocol translates CacheHints to its wire format (cache_control on Anthropic, cachePoint on Bedrock; OpenAI and Gemini do implicit caching server-side and don't need inline markers — auto is a no-op there).

Auto placement

"auto" places three breakpoints — last tool definition, last system part, latest user message. The last-user-message boundary is the load-bearing detail: in a tool-use loop, a single user turn expands into many assistant/tool round-trips, all sharing that prefix. Caching at that boundary lets every intra-turn API call hit.

The math justifies the default: Anthropic's 5-minute cache write is 1.25× base, read is 0.1×, so a single reuse within 5 minutes already wins. One-shot completions below the per-model minimum-cacheable-token threshold silently no-op on the wire, so the worst case is harmless.

Opting out

LLM.request({
  model,
  system,
  prompt: "one-off question",
  cache: "none",
})

Granular policy

cache: {
  tools?: boolean,
  system?: boolean,
  messages?: "latest-user-message" | "latest-assistant" | { tail: number },
  ttlSeconds?: number,         // ≥ 3600 → 1h on Anthropic/Bedrock; else 5m
}

Manual hints

Inline CacheHint on any text / system / tool / tool-result part overrides automatic placement. The auto policy preserves manual hints; it only fills gaps.

LLM.request({
  model,
  system: [
    { type: "text", text: "stable system prompt", cache: { type: "ephemeral" } },
  ],
  ...
})

Provider behavior table

Protocol	cache: "auto"
Anthropic Messages	emits up to 3 cache_control markers (4-breakpoint cap enforced)
Bedrock Converse	emits up to 3 cachePoint blocks (4-breakpoint cap enforced)
OpenAI Chat / Responses	no-op (implicit caching above 1024 tokens)
Gemini	no-op (implicit caching on 2.5+; explicit CachedContent is out-of-band)

Normalized cache usage is read back into response.usage.cacheReadInputTokens and cacheWriteInputTokens across every provider.

Providers

Provider facades configure endpoint/auth/deployment details first, then expose model selectors that take only a model or deployment id. The selected model carries the executable route value used at runtime.

import { OpenAI, CloudflareAIGateway } from "@opencode-ai/ai/providers"

const openai = OpenAI.configure({ apiKey: process.env.OPENAI_API_KEY }).responses("gpt-4o-mini")
const gateway = CloudflareAIGateway.configure({
  accountId: process.env.CLOUDFLARE_ACCOUNT_ID,
  gatewayApiKey: process.env.CLOUDFLARE_API_TOKEN,
}).model("workers-ai/@cf/meta/llama-3.1-8b-instruct")

Included providers: OpenAI, Anthropic, Google (Gemini), Google Vertex Gemini and Anthropic, Amazon Bedrock, Azure OpenAI, Cloudflare AI Gateway, Cloudflare Workers AI, GitHub Copilot, OpenRouter, xAI, Z.ai, plus generic OpenAI-compatible Chat and Responses entrypoints and an Anthropic Messages-compatible entrypoint.

Package-like entrypoints

Native catalog integrations load provider behavior through package-like entrypoints. These are export paths from the same @opencode-ai/ai npm package, not independently published packages. Each entrypoint exports the same model(modelID, settings) contract, and settings contains serializable provider configuration plus common headers, body, and limits overlays.

import { model } from "@opencode-ai/ai/providers/openai/responses"

const selected = model("gpt-5", {
  apiKey: process.env.OPENAI_API_KEY,
  transport: "websocket",
  headers: { "x-application": "opencode" },
  limits: { context: 200_000, output: 64_000 },
})

OpenAI Chat and OpenAI Responses are separate semantic entrypoints:

• @opencode-ai/ai/providers/openai/chat
• @opencode-ai/ai/providers/openai/responses
• @opencode-ai/ai/providers/openai-compatible/responses
• @opencode-ai/ai/providers/anthropic-compatible
• @opencode-ai/ai/providers/google-vertex/gemini
• @opencode-ai/ai/providers/google-vertex/chat
• @opencode-ai/ai/providers/google-vertex/responses
• @opencode-ai/ai/providers/google-vertex/messages

Responses HTTP versus WebSocket is a scoped transport setting on the OpenAI Responses entrypoint, not another entrypoint. Azure follows the same Chat/Responses split at providers/azure/chat and providers/azure/responses. Generic OpenAI-compatible Chat remains at providers/openai-compatible; compatible Responses is separate at providers/openai-compatible/responses. Generic Anthropic Messages-compatible providers use providers/anthropic-compatible, which the named Anthropic provider composes. Google Gemini and Amazon Bedrock expose their single native API through their existing provider paths.

Vertex Gemini, Vertex Chat, Vertex Responses, and Vertex Messages are separate API entrypoints. All accept project, location, and an optional accessToken; when no explicit token or auth override is supplied they lazily use Google Application Default Credentials. Vertex Gemini instead selects express mode when apiKey or GOOGLE_VERTEX_API_KEY is present. Vertex Chat targets MaaS models through the OpenAI-compatible Chat Completions endpoint, while Vertex Responses targets Grok models and defaults store to false as required by Vertex. providers/google-vertex remains the default alias for providers/google-vertex/gemini.

Tuned Vertex Gemini deployments use model ids shaped like endpoints/1234567890 and require OAuth or ADC; Vertex express-mode API keys support publisher models only.

import { model } from "@opencode-ai/ai/providers/google-vertex/gemini"

model("gemini-3.5-flash", { project: "my-project", location: "global" })

import { model } from "@opencode-ai/ai/providers/google-vertex/chat"

model("deepseek-ai/deepseek-v3.2-maas", { project: "my-project", location: "global" })

import { model } from "@opencode-ai/ai/providers/google-vertex/responses"

model("xai/grok-4.20-reasoning", { project: "my-project", location: "global" })

import { model } from "@opencode-ai/ai/providers/google-vertex/messages"

model("claude-sonnet-4-6", { project: "my-project", location: "global" })

Provider facades such as OpenAI.configure(...).responses(...) remain the direct application API. Package-like entrypoints are the self-similar loading contract used when a catalog selects behavior by export path.

Other provider exports listed above remain direct facades until they explicitly implement the package-like contract. Exporting a provider facade does not implicitly make it a catalog-loadable provider package.

Provider options & HTTP overlays

Three escape hatches in order of stability:

1. generation — portable knobs (maxTokens, temperature, topP, topK, penalties, seed, stop).
2. providerOptions: { <provider>: {...} } — typed-at-the-facade provider-specific knobs (OpenAI promptCacheKey, Anthropic thinking, Gemini thinkingConfig, OpenRouter routing).
3. http: { body, headers, query } — last-resort serializable overlays merged into the final HTTP request. Reach for this only when a stable typed path doesn't yet exist.

Route/provider defaults are overridden by request-level values for each axis.

Routes

Adding a new model or deployment is usually 5-15 lines using Route.make({ protocol, endpoint, auth, framing, ... }). The route owns endpoint/auth/framing and the protocol owns body construction plus stream parsing. Transports are reusable IO templates that receive route endpoint/auth at compile time. Capability/catalog metadata lives outside this low-level package; unsupported request shapes fail during protocol lowering. See AGENTS.md for the architectural detail.

Effect

This package is built on Effect. Public methods return Effect or Stream; provide LLMClient.layer for LLM dispatch and ImageClient.layer for image dispatch, then import the provider/protocol modules for the routes you use. The example at example/tutorial.ts is a runnable walkthrough.

See also

• AGENTS.md — architecture, route construction, contributor guide
• STATUS.md — native provider parity status and AI SDK migration gaps
• example/tutorial.ts — runnable end-to-end walkthrough
• test/provider/*.test.ts — fixture-first protocol tests; *.recorded.test.ts files cover live cassettes