Compare commits

...
19 changed files with 1501 additions and 1039 deletions
+1 -1
View File
@@ -184,7 +184,7 @@ const table = sqliteTable("session", {
- Keep `SessionRunner`, model resolution, tool registry, permissions, and filesystem Location-scoped. Omitted `Location.workspaceID` means implicit-local placement; explicit workspace identity remains reserved for future placement semantics.
- Preserve one explicit `llm.stream(request)` call per Physical Attempt and reload projected history before durable continuation. A logical Step may use generic pre-output retries, one full-context retry after continuation rejection, incomplete-stream continuation, or one overflow-compaction rebuild. Generic retries retain the logical step number and do not consume another agent-step allowance. Do not delegate orchestration to an in-memory tool loop.
- Keep local Session drains process-local until clustering is implemented. `SessionRunCoordinator` joins explicit same-Session resumes, coalesces prompt wakeups, and allows different Sessions to run concurrently. A write-ahead execution claim marks a process-local busy period for restart recovery: terminal completion, failure, or user interruption releases it, while shutdown interruption and process death preserve it. Startup recovery resumes claimed top-level Sessions with durable per-execution attempt accounting. The claim is a recovery marker, not clustered ownership, fencing, or an exactly-once guarantee.
- Keep native compaction mechanisms out of `SessionCompaction`. Plugins register `native` strategies through the `SessionCompaction` editor that turn a prepared request into a replacement window (the built-in `NativeCompactionPlugin` handles `@opencode/ai` compaction operations); later registrations win. Core owns the provider-mode decision, route provenance, the retry policy, overflow recovery, interruption, usage accounting, and checkpoint persistence.
- Keep provider-specific native compaction mechanisms in `@opencode/ai` behind `LLMClient.compact`. `SessionCompaction` chooses a summary or native compaction from the model's `compaction` setting and owns route provenance, request shrinking, the retry policy, interruption, usage accounting, and checkpoint persistence.
- Keep delivery vocabulary explicit. Prompts steer by default. At safe step boundaries, steered compaction takes priority up to the first steered move control; other steers retain enqueue order. At an idle boundary, steers take priority; otherwise exactly one queued item delivers before the runner reevaluates continuation. Inbox items may be cancelled or changed between queue and steer before delivery. Promoting new user input resets the selected agent's step allowance; a batch of steers resets it once.
- One step is one logical LLM call; its durable record covers only the model-visible span. Do not write "provider turn", and do not use bare "turn" for a single call: "turn" is reserved for the future assistant-turn unit containing all steps from prompt promotion until the session would go idle.
- Keep event replay ownership separate from clustered Session execution ownership.
@@ -18,7 +18,7 @@ export const Plugin = define({
editor.configure({
...(entry.info.compaction.auto === undefined ? {} : { auto: entry.info.compaction.auto }),
...(entry.info.compaction.buffer === undefined ? {} : { buffer: entry.info.compaction.buffer }),
...(entry.info.compaction.keep?.tokens === undefined ? {} : { tokens: entry.info.compaction.keep.tokens }),
...(entry.info.compaction.keep?.tokens === undefined ? {} : { keep: entry.info.compaction.keep.tokens }),
})
}
})
-29
View File
@@ -1,29 +0,0 @@
export * as NativeCompactionPlugin from "./compaction.js"
import { LLMClient, Message } from "@opencode/ai"
import { define } from "@opencode/plugin/effect/plugin"
import { Effect } from "effect"
import { SessionCompaction } from "../session/compaction.js"
import type { PluginInternal } from "./internal.js"
export const Plugin = define({
id: "opencode.compaction.native",
effect: Effect.fn("NativeCompactionPlugin")(function* () {
const llm = yield* LLMClient.Service
const compaction = yield* SessionCompaction.Service
yield* compaction.transform((editor) => {
editor.native((input) => {
const request = input.request
if (LLMClient.canCompact(request, { mechanism: "trigger" }))
return Effect.gen(function* () {
const retained = yield* input.retained
const result = yield* llm.compact(request, { ...input.options, mechanism: "trigger" })
return { replacement: [...retained, Message.assistant(result.checkpoint)], usage: result.usage }
})
if (LLMClient.canCompact(request))
return llm.compact(request, { mechanism: "endpoint", http: input.options.http })
return undefined
})
})
}),
} satisfies PluginInternal.InternalPlugin)
-2
View File
@@ -88,7 +88,6 @@ import { WriteTool } from "../tool/plugin/write.js"
import { AgentPlugin } from "./agent.js"
import BrowserPlugin from "@opencode/plugin-browser"
import { CommandPlugin } from "./command.js"
import { NativeCompactionPlugin } from "./compaction.js"
import { IdentityPlugin } from "./identity.js"
import { PlanPlugin } from "./plan.js"
import { ModelsDevPlugin } from "./models-dev.js"
@@ -225,7 +224,6 @@ const pre = [
SkillPlugin.Plugin,
VcsHgPlugin.Plugin,
ModelsDevPlugin,
NativeCompactionPlugin.Plugin,
...ProviderPlugins,
...WebSearchPlugins,
PatchTool.Plugin,
@@ -1,5 +1,6 @@
import type { IntegrationOAuthMethodRegistration } from "@opencode/plugin/effect/integration"
import { define } from "@opencode/plugin/effect/plugin"
import type { SessionRequest } from "@opencode/plugin/effect/session"
import { Deferred, Effect, Option, Schema, Semaphore, Stream } from "effect"
import type { Server } from "node:http"
import { App } from "../../app.js"
@@ -307,6 +308,13 @@ export const OpenAIPlugin = define({
}),
{ providerID: Provider.ID.openai },
)
// The ChatGPT backend rejects a requested output limit, and OpenAI counts one against rate limits.
const omitOutputLimit = (evt: SessionRequest) =>
Effect.sync(() => {
delete evt.options.maxTokens
})
for (const name of ["context", "compaction"] as const)
yield* ctx.session.hook(name, omitOutputLimit, { providerID: Provider.ID.openai })
const refresh = () => loading.withPermit(load().pipe(Effect.andThen(ctx.provider.reload())))
yield* bus.subscribe(Credential.Event.Switched).pipe(
Stream.filter((event) => event.data.integrationID === Integration.ID.make("openai")),
File diff suppressed because it is too large Load Diff
+27 -1
View File
@@ -44,6 +44,14 @@ const IMAGE_BYTES_TARGET = 15 * 1024 * 1024 // 15 MiB
const IMAGE_REMOVED =
"[This image was removed to reduce the request size and is no longer visible. Do not make claims about its contents from memory. If needed, retrieve it again with an available tool or ask the user to attach it again.]"
const GENERATION_KEYS = new Set(Object.keys(GenerationOptions.fields))
// Default output limit caps per request kind. Titles and generate have none and keep the provider default, because
// their reasoning is hard to budget.
const OUTPUT_TOKEN_CAPS: Partial<Record<SessionRequestKind, number>> = { primary: 256_000, compaction: 32_000 }
// Used when the catalog has no output limit for the model.
const OUTPUT_TOKEN_FALLBACK = 32_000
// Prompt text is estimated at about 4 characters per token, which can run low on dense text such as code.
const ESTIMATE_ERROR = 0.15
const OUTPUT_TOKEN_MIN = 1_024
/** Tool errors, plus the user declining a permission or dismissing a question. */
export type ExecuteError = Tool.Error | Permission.DeclinedError | QuestionTool.CancelledError
@@ -69,6 +77,16 @@ export interface Input {
readonly toolChoice?: LLM.RequestInput["toolChoice"]
/** Only the durable runner may use a stateful WebSocket. */
readonly webSocket?: "session"
/** Prompt size, measured by the provider or estimated. The default output limit leaves room for it. */
readonly inputTokens?: { readonly measured: number; readonly estimated: number }
}
/** The default output limit: the catalog limit, capped, and fitted to the room the prompt leaves in the context window. */
export const outputLimit = (limit: Model.Info["limit"], cap: number, inputTokens?: Input["inputTokens"]) => {
const requested = Math.min(limit.output > 0 ? limit.output : OUTPUT_TOKEN_FALLBACK, cap)
if (inputTokens === undefined || limit.context <= 0) return requested
const room = limit.context - inputTokens.measured - Math.ceil(inputTokens.estimated * (1 + ESTIMATE_ERROR))
return Math.min(requested, Math.max(OUTPUT_TOKEN_MIN, room))
}
export const baseTranscript = (input: {
@@ -218,8 +236,16 @@ export const layer = Layer.effect(
const given = new Map(
tools.definitions.map((t) => [{ description: t.description, input: { ...t.inputSchema } }, t] as const),
)
// Hooks see the default output limit and may change or remove it.
const cap = OUTPUT_TOKEN_CAPS[kind]
const shaped = yield* shape(
{ sessionID: session.id, model: model.ref, system: input.system, messages: input.messages, options: {} },
{
sessionID: session.id,
model: model.ref,
system: input.system,
messages: input.messages,
options: cap === undefined ? {} : { maxTokens: outputLimit(model.limit, cap, input.inputTokens) },
},
Object.fromEntries(Array.from(given, ([d, t]) => [t.name, d])),
)
// Match by identity first, then by key. Entries matching neither were invented by a
+36 -39
View File
@@ -20,6 +20,7 @@ import { SessionSchema } from "../schema.js"
import { SessionStore } from "../store.js"
import { SessionMessageTable } from "../sql.js"
import { SessionTitle } from "../title.js"
import { toSessionError } from "../to-session-error.js"
import { DrainResult, Service, type Interface } from "./index.js"
import { Snapshot } from "../../snapshot.js"
import { makeLocationNode } from "@opencode/util/effect/app-node"
@@ -109,39 +110,39 @@ const layer = Layer.effect(
if (pending?.type === "move")
return DrainResult.Moved({ continuation: continuing ? { step } : undefined })
if (pending?.type === "compaction") {
const session = yield* store.get(sessionID)
if (!session) return yield* Effect.die(new Error(`Session not found: ${sessionID}`))
const compacted = yield* restore(
Effect.gen(function* () {
return yield* compaction.compactManual({
session,
resolveContext: (session) =>
Effect.gen(function* () {
const selected = yield* context.select(session.id)
const model = yield* context.resolveModel(selected.session)
// Preview updates without admitting them after the already-delivered compaction marker.
const history = yield* SessionHistory.preview(
db,
session.id,
selected.instructions,
SessionProviderContext.provenance(model) ?? "local",
)
return {
session: selected.session,
agent: selected.agent,
tools: selected.tools,
model,
initial: history.initial,
messages: history.messages,
instructionUpdate: history.instructionUpdate,
}
}),
prepare: context.request.compaction,
messages: yield* store.context(sessionID),
const selected = yield* context.select(sessionID)
const model = yield* context.resolveModel(selected.session)
// Preview updates without admitting them after the already-delivered compaction marker.
const history = yield* SessionHistory.preview(
db,
sessionID,
selected.instructions,
SessionProviderContext.provenance(model) ?? "local",
)
return yield* compaction.compact({
reason: "manual",
inputID: pending.id,
started: true,
context: {
session: selected.session,
agent: selected.agent,
tools: selected.tools,
model,
initial: history.initial,
messages: history.messages,
},
})
}),
}).pipe(
Effect.catch((error) =>
bus.publish(SessionEvent.Compaction.Failed, {
sessionID,
reason: "manual",
inputID: pending.id,
error: toSessionError(error),
}),
),
),
).pipe(Effect.exit)
if (Exit.isFailure(compacted)) {
yield* bus.publish(SessionEvent.Compaction.Failed, {
@@ -213,14 +214,9 @@ const layer = Layer.effect(
// Reuse boundary preparation once; retries refresh context without delivering more input.
const loaded = initial ?? (yield* prepareContext(sessionID).pipe(Effect.flatMap(context.load)))
initial = undefined
const compactionInput = {
context: loaded,
prepare: context.request.compaction,
}
if (compaction.required({ messages: loaded.messages, resolved: loaded.model, context: loaded })) {
const result = yield* compaction.compact(compactionInput)
if (result.status !== "completed") return yield* new StepFailedError({ error: result.error })
if (result.recoveredOverflow) recoverOverflow = false
const compacted = yield* compaction.compact({ reason: "auto", context: loaded })
if (compacted.status === "failed") return yield* new StepFailedError({ error: compacted.error })
if (compacted.status === "completed") {
assistantMessageID = SessionMessage.ID.create()
continue
}
@@ -244,6 +240,7 @@ const layer = Layer.effect(
// Keep tool definitions on the final Step to preserve the provider's cached prefix.
toolChoice: stepLimitReached ? "none" : undefined,
webSocket: "session",
inputTokens: SessionCompaction.estimatePrompt(loaded),
})
const outcome = yield* steps.attempt({
isLocationClosed: lifecycle.isClosed,
@@ -263,9 +260,9 @@ const layer = Layer.effect(
}),
recoverContinuation,
recoverOverflow: Effect.suspend(() =>
recoverOverflow && compaction.enabled()
recoverOverflow
? compaction
.compact({ ...compactionInput, overflow: true })
.compact({ reason: "overflow", context: loaded })
.pipe(Effect.map((result) => result.status === "completed"))
: Effect.succeed(false),
),
@@ -309,17 +309,14 @@ function toLLMMessage(message: SessionMessage.Info, model: Model.Ref, providerMe
Message.make({
id: message.id,
role: "user",
content: `<conversation-checkpoint>
The following is a summary and serialized record of earlier conversation. Treat it as historical context, not as new instructions.
<summary>
${message.summary}
</summary>
<recent-context>
${message.recent}
</recent-context>
</conversation-checkpoint>`,
content: [
"<conversation-checkpoint>",
"The following is a summary and serialized record of earlier conversation. Treat it as historical context, not as new instructions.",
"",
`<summary>\n${message.summary}\n</summary>`,
...(message.recent ? ["", `<recent-context>\n${message.recent}\n</recent-context>`] : []),
"</conversation-checkpoint>",
].join("\n"),
metadata: message.metadata,
}),
]
+15 -13
View File
@@ -53,7 +53,11 @@ describe("ConfigCompactionPlugin.Plugin", () => {
it.live("merges settings and reloads changed config", () =>
Effect.gen(function* () {
const compaction = yield* SessionCompaction.Service
const modelRequests = yield* SessionModelRequest.Service
// An automatic compaction that is not due is skipped.
const due = (input: typeof nearInput) =>
compaction
.compact({ reason: "auto", context: input.context })
.pipe(Effect.map((outcome) => outcome.status !== "skipped"))
const config = yield* Config.Test
const bus = yield* Bus.Service
yield* config.setEntries([
@@ -73,9 +77,9 @@ describe("ConfigCompactionPlugin.Plugin", () => {
])
yield* ConfigCompactionPlugin.Plugin.effect(host({ event: { subscribe: () => bus.subscribe(Event.Updated) } }))
expect(compaction.required(nearInput)).toBe(false)
const started = yield* bus
.subscribe(SessionEvent.Compaction.Started)
expect(yield* due(nearInput)).toBe(false)
const ended = yield* bus
.subscribe(SessionEvent.Compaction.Ended)
.pipe(Stream.runHead, Effect.forkScoped({ startImmediately: true }))
const messages = [
SessionMessage.User.make({
@@ -92,15 +96,13 @@ describe("ConfigCompactionPlugin.Plugin", () => {
}),
]
expect(
yield* compaction.compactManual({
session,
resolveContext: () => Effect.succeed({ ...nearInput.context, messages, instructionUpdate: "" }),
prepare: modelRequests.compaction,
messages,
yield* compaction.compact({
reason: "manual",
context: { ...nearInput.context, messages },
inputID: SessionMessage.ID.make("msg_compaction_manual"),
}),
).toEqual({ status: "completed" })
expect(Option.getOrThrow(yield* Fiber.join(started)).data.recent).toContain("Recent context")
expect(Option.getOrThrow(yield* Fiber.join(ended)).data.recent).toContain("Recent context")
yield* config.setEntries([
new Document({
@@ -115,12 +117,12 @@ describe("ConfigCompactionPlugin.Plugin", () => {
yield* bus.publish(Event.Updated, {})
yield* Effect.gen(function* () {
for (let attempt = 0; attempt < 200; attempt++) {
if (compaction.required(nearInput)) return
if (yield* due(nearInput)) return
yield* Effect.sleep("10 millis")
}
yield* Effect.die(new Error("Timed out waiting for compaction config reload"))
})
expect(compaction.required(bufferedInput)).toBe(false)
expect(yield* due(bufferedInput)).toBe(false)
yield* config.setEntries([
new Document({
@@ -130,7 +132,7 @@ describe("ConfigCompactionPlugin.Plugin", () => {
])
yield* bus.publish(Event.Updated, {})
for (let attempt = 0; attempt < 200; attempt++) {
if (compaction.required(bufferedInput)) return
if (yield* due(bufferedInput)) return
yield* Effect.sleep("10 millis")
}
yield* Effect.die(new Error("Timed out waiting for compaction config reload"))
@@ -216,6 +216,31 @@ describe("OpenAIPlugin", () => {
}),
)
it.effect("omits the default output limit from OpenAI steps and compaction", () =>
Effect.gen(function* () {
yield* addPlugin()
const hooks = yield* PluginHooks.Service
const maxTokens = (providerID: Provider.ID) =>
Effect.gen(function* () {
const draft = {
sessionID: Session.ID.make("ses_test"),
model: Model.Ref.make({ providerID, id: Model.ID.make("gpt-5.5") }),
system: [],
messages: [],
options: { maxTokens: 128_000 },
}
const events = [
yield* hooks.trigger("session", "context", { ...draft, agent: Agent.ID.make("build"), tools: {} }),
yield* hooks.trigger("session", "compaction", { ...draft, agent: Agent.ID.make("build"), tools: {} }),
]
return events.map((event) => event.options.maxTokens)
})
expect(yield* maxTokens(Provider.ID.openai)).toEqual([undefined, undefined])
expect(yield* maxTokens(Provider.ID.azure)).toEqual([128_000, 128_000])
}),
)
it.effect("selects WebSocket only from explicit policy", () =>
Effect.gen(function* () {
const credentials = yield* Credential.Service
+166 -155
View File
@@ -1,5 +1,5 @@
import { expect, test } from "bun:test"
import { LLMClient, LLMEvent, LanguageModel, ToolDefinition, type LLMRequest } from "@opencode/ai"
import { GenerationOptions, LLMClient, LLMEvent, LanguageModel, ToolDefinition, type LLMRequest } from "@opencode/ai"
import { OpenAIChat } from "@opencode/ai/protocols"
import { Database } from "@opencode/core/database/database"
import { AppNodeBuilder } from "@opencode/core/effect/app-node-builder"
@@ -8,6 +8,7 @@ import { LayerNode } from "@opencode/util/effect/layer-node"
import { Bus } from "@opencode/core/bus"
import { EventTable } from "@opencode/core/event/sql"
import { SessionCompaction } from "@opencode/core/session/compaction"
import type { SessionContext } from "@opencode/core/session/context"
import { SessionEvent } from "@opencode/core/session/event"
import { SessionMessage } from "@opencode/core/session/message"
import { SessionModelRequest } from "@opencode/core/session/model-request"
@@ -21,9 +22,7 @@ import { Project } from "@opencode/core/project"
import { ProjectTable } from "@opencode/core/project/sql"
import { App } from "@opencode/core/app"
import { Agent } from "@opencode/core/agent"
import { Model } from "@opencode/core/model"
import { Provider } from "@opencode/core/provider"
import { Location } from "@opencode/core/location"
import { AbsolutePath } from "@opencode/core/schema"
import { Money } from "@opencode/schema/money"
import { Skill } from "@opencode/schema/skill"
@@ -102,28 +101,38 @@ test("compaction prompt preserves detailed work state and relevant files", () =>
expect(prompt).toContain("## Relevant Files")
})
test("compaction describes tool media without embedding base64", () => {
const base64 = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAAB"
const serialized = SessionCompaction.serializeToolContent([
{ type: "text", text: "Image read successfully" },
{
type: "file",
uri: `data:image/png;base64,${base64}`,
mime: "image/png",
name: "pixel.png",
},
])
it.effect("compaction describes tool media without embedding base64", () =>
Effect.gen(function* () {
const base64 = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAAB"
const recent = yield* recentWithToolOutput(Session.ID.make("ses_tool_media"), [
{ type: "text", text: "Image read successfully" },
{
type: "file",
uri: `data:image/png;base64,${base64}`,
mime: "image/png",
name: "pixel.png",
},
])
expect(serialized).toBe("Image read successfully\n[Attached image/png: pixel.png]")
expect(serialized).not.toContain(base64)
})
expect(recent).toContain("[Tool result]: Image read successfully\n[Attached image/png: pixel.png]")
expect(recent).not.toContain(base64)
}),
)
test("compaction truncation does not split surrogate pairs", () => {
const prefix = "a".repeat(1_999)
it.effect("compaction truncation does not split surrogate pairs", () =>
Effect.gen(function* () {
const prefix = "a".repeat(1_249)
const split = yield* recentWithToolOutput(Session.ID.make("ses_truncate_split"), [
{ type: "text", text: `${prefix}😀suffix` },
])
const whole = yield* recentWithToolOutput(Session.ID.make("ses_truncate_whole"), [
{ type: "text", text: "😀".repeat(1_250) },
])
expect(SessionCompaction.truncateToolOutput(`${prefix}😀suffix`)).toBe(`${prefix}😀\n[truncated]`)
expect(SessionCompaction.truncateToolOutput("😀".repeat(2_000))).toBe("😀".repeat(2_000))
})
expect(split).toEndWith(`[Tool result]: ${prefix}😀\n[truncated]`)
expect(whole).toEndWith(`[Tool result]: ${"😀".repeat(1_250)}`)
}),
)
test("compaction prompt requires the checkpoint headings in order", () => {
const prompt = SessionCompaction.buildPrompt(false)
@@ -156,74 +165,63 @@ test("compaction prompts prohibit task execution", () => {
it.effect("auto compaction estimates current content against the buffered prompt ceiling", () =>
Effect.gen(function* () {
const compaction = yield* SessionCompaction.Service
const session = Session.Info.make({
id: Session.ID.make("ses_input_limit"),
projectID: Project.ID.global,
cost: Money.USD.zero,
tokens: { input: 0, output: 0, reasoning: 0, cache: { read: 0, write: 0 } },
time: { created: DateTime.makeUnsafe(0), updated: DateTime.makeUnsafe(0) },
location: Location.Ref.make({ directory: AbsolutePath.make("/tmp") }),
})
const input = (tokens: number, limit: { context: number; input?: number; output: number }) => {
const resolved = SessionRunnerModel.resolved(model, {
const session = yield* insertSession(Session.ID.make("ses_input_limit"))
const input = (tokens: number, limit: { context: number; input?: number; output: number }) => ({
session,
model: SessionRunnerModel.resolved(model, {
capabilities: { tools: true, input: ["text", "image", "pdf"], output: ["text"] },
cost: [],
limit,
})
const messages = [
}),
messages: [
Schema.decodeUnknownSync(SessionMessage.Assistant)({
id: SessionMessage.ID.make("msg_assistant"),
type: "assistant",
agent: Agent.defaultID,
model: { id: "test-model", providerID: "test-provider" },
model: { id: "summary-model", providerID: "test" },
content: [{ type: "text", text: "Done" }],
tokens: { input: tokens, output: 0, reasoning: 0, cache: { read: 0, write: 0 } },
time: { created: 0, completed: 0 },
}),
]
return {
session,
resolved,
messages,
context: {
session,
model: resolved,
messages,
agent: {
id: Agent.defaultID,
info: { ...Agent.Info.default(Agent.defaultID), system: "You are a helpful assistant." },
},
initial: "Project instructions.",
tools: {
definitions: [
ToolDefinition.make({ name: "read", description: "Read files", inputSchema: { type: "object" } }),
],
execute: () => Effect.die("unused"),
},
},
}
}
],
agent: {
id: Agent.defaultID,
info: { ...Agent.Info.default(Agent.defaultID), system: "You are a helpful assistant." },
},
initial: "Project instructions.",
tools: {
definitions: [
ToolDefinition.make({ name: "read", description: "Read files", inputSchema: { type: "object" } }),
],
execute: () => Effect.die("unused"),
},
})
// An automatic compaction that is not due is skipped.
const due = (context: SessionContext.Loaded) =>
compaction.compact({ reason: "auto", context }).pipe(Effect.map((outcome) => outcome.status !== "skipped"))
// 90% of the input limit, which takes precedence over the context window.
const inputLimited = { context: 400_000, input: 272_000, output: 128_000 }
expect(compaction.required(input(251_999, inputLimited))).toBe(false)
expect(compaction.required(input(252_000, inputLimited))).toBe(true)
expect(yield* due(input(244_799, inputLimited))).toBe(false)
expect(yield* due(input(244_800, inputLimited))).toBe(true)
const native = (tokens: number, limit: { context: number; input?: number; output: number } = inputLimited) => {
const selected = input(tokens, limit)
return { ...selected, resolved: { ...selected.resolved, compaction: { type: "native" as const } } }
return { ...selected, model: { ...selected.model, compaction: { type: "native" as const } } }
}
expect(compaction.required(native(251_999))).toBe(false)
expect(compaction.required(native(252_000))).toBe(true)
expect(compaction.required(native(1_000_000, { context: 0, input: undefined, output: 0 }))).toBe(false)
expect(yield* due(native(244_799))).toBe(false)
expect(yield* due(native(244_800))).toBe(true)
expect(yield* due(native(1_000_000, { context: 0, input: undefined, output: 0 }))).toBe(false)
const contextLimited = { context: 100_000, output: 10_000 }
expect(compaction.required(input(79_999, contextLimited))).toBe(false)
expect(compaction.required(input(80_000, contextLimited))).toBe(true)
expect(yield* due(input(89_999, contextLimited))).toBe(false)
expect(yield* due(input(90_000, contextLimited))).toBe(true)
// The reply limit does not lower the ceiling.
const outputLimited = { context: 100_000, output: 30_000 }
expect(compaction.required(input(69_999, outputLimited))).toBe(false)
expect(compaction.required(input(70_000, outputLimited))).toBe(true)
expect(yield* due(input(89_999, outputLimited))).toBe(false)
expect(yield* due(input(90_000, outputLimited))).toBe(true)
const assistant = input(79_000, contextLimited).messages[0]
const assistant = input(89_000, contextLimited).messages[0]
const tool = SessionMessage.AssistantTool.make({
type: "tool",
id: "call_read",
@@ -231,16 +229,19 @@ it.effect("auto compaction estimates current content against the buffered prompt
state: { status: "completed", input: {}, content: [{ type: "text", text: "x".repeat(4_000) }] },
time: { created: DateTime.makeUnsafe(0) },
})
const grown = { ...input(79_000, contextLimited), messages: [{ ...assistant, content: [tool] }] }
expect(SessionCompaction.estimateTokens(grown)).toBe(80_000)
expect(compaction.required(grown)).toBe(true)
const grown = { ...input(89_000, contextLimited), messages: [{ ...assistant, content: [tool] }] }
expect(SessionCompaction.estimateContext(grown)).toBe(90_000)
expect(yield* due(grown)).toBe(true)
const interrupted = { ...assistant, id: SessionMessage.ID.create(), tokens: undefined }
expect(SessionCompaction.estimateTokens({ ...grown, messages: [...grown.messages, interrupted] })).toBe(80_001)
expect(SessionCompaction.estimateContext({ ...grown, messages: [...grown.messages, interrupted] })).toBe(90_001)
// Without provider usage, include 20 tokens for the system prompt, instructions, and tool definition.
expect(SessionCompaction.estimateTokens({ ...grown, messages: [interrupted] })).toBe(21)
expect(SessionCompaction.estimateContext({ ...grown, messages: [interrupted] })).toBe(21)
// Another provider's usage is not trusted either.
const foreign = { ...assistant, model: { ...assistant.model, providerID: Provider.ID.make("other") } }
expect(SessionCompaction.estimateContext({ ...grown, messages: [foreign] })).toBe(21)
expect(
SessionCompaction.estimateTokens({
SessionCompaction.estimateContext({
...grown,
messages: [{ ...interrupted, tokens: input(0, contextLimited).messages[0].tokens }],
}),
@@ -253,7 +254,7 @@ it.effect("auto compaction estimates current content against the buffered prompt
const messages = [
{ ...assistant, content: [{ ...tool, state: { status: "completed" as const, input: {}, content: media } }] },
]
expect(SessionCompaction.estimateTokens({ ...grown, messages })).toBe(82_500)
expect(SessionCompaction.estimateContext({ ...grown, messages })).toBe(92_500)
const user = Schema.decodeUnknownSync(SessionMessage.User)({
id: SessionMessage.ID.create(),
type: "user",
@@ -261,18 +262,18 @@ it.effect("auto compaction estimates current content against the buffered prompt
files: media.map((file) => ({ mime: file.mime, data: "a".repeat(100_000), source: { type: "inline" } })),
time: { created: 0 },
})
expect(SessionCompaction.estimateTokens({ ...grown, messages: [...messages, user] })).toBe(86_000)
expect(SessionCompaction.estimateContext({ ...grown, messages: [...messages, user] })).toBe(96_000)
for (const [modalities, tokens, fallback] of [
[["text", "image"], 82_040, 1_520],
[["text", "pdf"], 83_042, 2_021],
[["text"], 79_082, 41],
[["text", "image"], 92_040, 1_520],
[["text", "pdf"], 93_042, 2_021],
[["text"], 89_082, 41],
] as const) {
const selected = {
...grown,
resolved: { ...grown.resolved, capabilities: { ...grown.resolved.capabilities, input: modalities } },
model: { ...grown.model, capabilities: { ...grown.model.capabilities, input: modalities } },
}
expect(SessionCompaction.estimateTokens({ ...selected, messages: [...messages, user] })).toBe(tokens)
expect(SessionCompaction.estimateTokens({ ...selected, messages: [user] })).toBe(fallback + 20)
expect(SessionCompaction.estimateContext({ ...selected, messages: [...messages, user] })).toBe(tokens)
expect(SessionCompaction.estimateContext({ ...selected, messages: [user] })).toBe(fallback + 20)
}
const checkpoint = Schema.decodeUnknownSync(SessionMessage.CompactionCompleted)({
@@ -284,7 +285,7 @@ it.effect("auto compaction estimates current content against the buffered prompt
recent: "",
time: { created: 0, completed: 0 },
})
expect(compaction.required({ ...grown, messages: [checkpoint] })).toBe(false)
expect(yield* due({ ...grown, messages: [checkpoint] })).toBe(false)
}),
)
@@ -323,15 +324,66 @@ const loaded = (session: Session.Info, messages: readonly SessionMessage.Info[])
model: resolved,
agent: { id: Agent.defaultID, info: Agent.Info.default(Agent.defaultID) },
initial: "Session instructions",
instructionUpdate: "",
tools: { definitions: [], execute: () => Effect.die("Compaction must not execute tools") },
})
/** Opens the compaction's message as the runner does when it delivers `/compact`, then compacts. */
const compactManually = (
session: Session.Info,
messages: readonly SessionMessage.Info[],
inputID = SessionMessage.ID.create(),
) =>
Effect.gen(function* () {
const bus = yield* Bus.Service
const compaction = yield* SessionCompaction.Service
yield* bus.publish(SessionEvent.Compaction.Started, {
sessionID: session.id,
reason: "manual",
recent: "",
inputID,
})
return yield* compaction.compact({ reason: "manual", context: loaded(session, messages), inputID })
})
/** The recent text a manual compaction keeps when the latest exchange is one tool call with this output. */
const recentWithToolOutput = (id: Session.ID, content: SessionMessage.ToolStateCompleted["content"]) =>
Effect.gen(function* () {
const session = yield* insertSession(id)
const user = (text: string) =>
SessionMessage.User.make({
id: SessionMessage.ID.create(),
type: "user",
text,
time: { created: DateTime.makeUnsafe(0) },
})
const assistant = Schema.decodeUnknownSync(SessionMessage.Assistant)({
id: SessionMessage.ID.create(),
type: "assistant",
agent: Agent.defaultID,
model: { id: "summary-model", providerID: "test" },
content: [
{
type: "tool",
id: "call_read",
name: "read",
state: { status: "completed", input: {}, content },
time: { created: 0 },
},
],
time: { created: 0, completed: 0 },
})
expect(yield* compactManually(session, [user("Earlier question"), user("Read it"), assistant])).toEqual({
status: "completed",
})
const store = yield* SessionStore.Service
const stored = (yield* store.context(id))[0]
return stored?.type === "compaction" && stored.status === "completed" ? stored.recent : ""
})
it.effect("manual compaction summarizes short context instead of no-op", () =>
Effect.gen(function* () {
requests = []
const db = (yield* Database.Service).db
const compaction = yield* SessionCompaction.Service
const bus = yield* Bus.Service
const store = yield* SessionStore.Service
const sessionID = Session.ID.make("ses_manual_compaction")
@@ -350,7 +402,6 @@ it.effect("manual compaction summarizes short context instead of no-op", () =>
time: { created: DateTime.makeUnsafe(0) },
}
const session = yield* insertSession(sessionID, { parent_id: parentID })
const modelRequests = yield* SessionModelRequest.Service
const hooks = yield* PluginHooks.Service
let hooked = 0
yield* hooks.register("session", "compaction", (event) =>
@@ -383,15 +434,9 @@ it.effect("manual compaction summarizes short context instead of no-op", () =>
.subscribe(SessionEvent.Compaction.Delta)
.pipe(Stream.take(1), Stream.runCollect, Effect.forkScoped)
yield* Effect.yieldNow
expect(
yield* compaction.compactManual({
session,
resolveContext: () => Effect.succeed(loaded(session, messages)),
prepare: modelRequests.compaction,
messages,
inputID: SessionMessage.ID.make("msg_manual_compaction"),
}),
).toEqual({ status: "completed" })
expect(yield* compactManually(session, messages, SessionMessage.ID.make("msg_manual_compaction"))).toEqual({
status: "completed",
})
expect(Array.from(yield* Fiber.join(delta)).map((event) => event.data.text)).toEqual([
"## Objective\n- manual summary",
])
@@ -407,7 +452,7 @@ it.effect("manual compaction summarizes short context instead of no-op", () =>
"x-opencode-session": sessionID,
"x-opencode-client": "opencode",
})
expect(requests[0]?.generation).toBeUndefined()
expect(requests[0]?.generation).toEqual(GenerationOptions.make({ maxTokens: 32_000 }))
expect(JSON.stringify(requests[0]?.messages)).toContain("Manual compaction should include this short conversation.")
expect(JSON.stringify(requests[0]?.messages)).toContain("Use Effect services and generators.")
expect(JSON.stringify(requests[0]?.messages)).toContain("User shell pwd completed: /project")
@@ -449,12 +494,10 @@ it.effect("compaction hooks can supply the summary instead of the model", () =>
Effect.gen(function* () {
requests = []
const db = (yield* Database.Service).db
const compaction = yield* SessionCompaction.Service
const hooks = yield* PluginHooks.Service
const store = yield* SessionStore.Service
const sessionID = Session.ID.make("ses_hooked_compaction")
const session = yield* insertSession(sessionID)
const modelRequests = yield* SessionModelRequest.Service
const messages = [
{
id: SessionMessage.ID.create(),
@@ -474,15 +517,9 @@ it.effect("compaction hooks can supply the summary instead of the model", () =>
}),
)
expect(
yield* compaction.compactManual({
session,
resolveContext: () => Effect.succeed(loaded(session, messages)),
prepare: modelRequests.compaction,
messages,
inputID: SessionMessage.ID.make("msg_hooked_compaction"),
}),
).toEqual({ status: "completed" })
expect(yield* compactManually(session, messages, SessionMessage.ID.make("msg_hooked_compaction"))).toEqual({
status: "completed",
})
expect(contexts).toBe(0)
expect(requests).toEqual([])
@@ -504,65 +541,45 @@ it.effect("compaction hooks can supply the summary instead of the model", () =>
}),
)
it.effect("manual compaction records model resolution failures without calling the model", () =>
it.effect("native compaction fails without a model call on a route that cannot compact", () =>
Effect.gen(function* () {
requests = []
const compaction = yield* SessionCompaction.Service
const store = yield* SessionStore.Service
const sessionID = Session.ID.make("ses_manual_resolution_failure")
const session = yield* insertSession(sessionID)
const modelRequests = yield* SessionModelRequest.Service
const inputID = SessionMessage.ID.make("msg_manual_resolution_failure")
const session = yield* insertSession(Session.ID.make("ses_native_unsupported"))
const messages = [
SessionMessage.User.make({
id: SessionMessage.ID.create(),
type: "user",
text: "Compact this natively.",
time: { created: DateTime.makeUnsafe(0) },
}),
]
expect(
yield* compaction.compactManual({
session,
resolveContext: () =>
Effect.fail(
new SessionRunnerModel.ModelUnavailableError({
providerID: Provider.ID.make("test"),
modelID: Model.ID.make("missing"),
}),
),
prepare: modelRequests.compaction,
messages: [
{
id: SessionMessage.ID.create(),
type: "user",
text: "Summarize this conversation.",
time: { created: DateTime.makeUnsafe(0) },
},
],
inputID,
yield* compaction.compact({
reason: "manual",
context: { ...loaded(session, messages), model: { ...resolved, compaction: { type: "native" } } },
inputID: SessionMessage.ID.create(),
}),
).toEqual({
status: "failed",
error: { type: "provider.no-route", message: "Model unavailable: test/missing" },
error: {
type: "provider.unsupported-operation",
message: "Native compaction is not supported for test/openai-chat",
},
})
expect(requests).toHaveLength(0)
expect(yield* store.context(sessionID)).toMatchObject([
{
id: inputID,
type: "compaction",
status: "failed",
reason: "manual",
error: { type: "provider.no-route", message: "Model unavailable: test/missing" },
},
])
}),
)
it.effect("forked session compaction reuses the fork root prompt cache key", () =>
Effect.gen(function* () {
requests = []
const compaction = yield* SessionCompaction.Service
const sessionID = Session.ID.make("ses_fork_compaction")
const rootID = Session.ID.make("ses_fork_compaction_root")
const session = yield* insertSession(sessionID, {
fork_session_id: rootID,
fork_boundary: { type: "before", messageID: SessionMessage.ID.create() },
})
const modelRequests = yield* SessionModelRequest.Service
const messages = [
SessionMessage.User.make({
id: SessionMessage.ID.create(),
@@ -571,15 +588,9 @@ it.effect("forked session compaction reuses the fork root prompt cache key", ()
time: { created: DateTime.makeUnsafe(0) },
}),
]
expect(
yield* compaction.compactManual({
session,
resolveContext: () => Effect.succeed(loaded(session, messages)),
prepare: modelRequests.compaction,
messages,
inputID: SessionMessage.ID.make("msg_fork_compaction"),
}),
).toEqual({ status: "completed" })
expect(yield* compactManually(session, messages, SessionMessage.ID.make("msg_fork_compaction"))).toEqual({
status: "completed",
})
expect(requests).toHaveLength(1)
expect(requests[0]?.promptCacheKey).toBe(rootID)
@@ -154,3 +154,61 @@ describe("SessionModelRequest HTTP hooks", () => {
}),
)
})
describe("SessionModelRequest output limit", () => {
const input = { session, agent: Agent.ID.make("build"), model, system: [], messages: [] }
it.effect("caps the default output limit per request kind", () =>
Effect.gen(function* () {
const requests = yield* SessionModelRequest.Service.pipe(Effect.provide(SessionModelRequest.layer))
const large = {
...input,
model: SessionRunnerModel.resolved(OpenAIChat.route.model({ id: "large-output", provider: "test" }), {
capabilities: { tools: true, input: ["text"], output: ["text"] },
cost: [],
limit: { context: 1_000_000, output: 384_000 },
}),
}
const maxTokens = (prepared: SessionModelRequest.Prepared<unknown>) => prepared.request.generation?.maxTokens
expect(maxTokens(yield* requests.primary(large))).toBe(256_000)
expect(maxTokens(yield* requests.compaction(large))).toBe(32_000)
const inputTokens = { measured: 170_000, estimated: 8_000 }
expect(maxTokens(yield* requests.primary({ ...input, inputTokens }))).toBe(20_800)
expect(maxTokens(yield* requests.compaction({ ...input, inputTokens }))).toBe(20_800)
}).pipe(Effect.provideService(SessionModelTransport.Service, transport)),
)
// Provider plugins that remove the default limit only hook `context` and `compaction`. If titles or generate get a
// default, also hook `title` and `generate` in: the OpenAI plugin (`omitOutputLimit`), whose ChatGPT backend
// rejects any requested limit.
it.effect("sends no output limit for titles and generate by default", () =>
Effect.gen(function* () {
const requests = yield* SessionModelRequest.Service.pipe(Effect.provide(SessionModelRequest.layer))
expect((yield* requests.title(input)).request.generation?.maxTokens).toBeUndefined()
expect((yield* requests.generate(input)).request.generation?.maxTokens).toBeUndefined()
}).pipe(Effect.provideService(SessionModelTransport.Service, transport)),
)
it.effect("lets hooks change or remove the default output limit", () =>
Effect.gen(function* () {
const hooks = yield* PluginHooks.Service
const seen: Array<number | undefined> = []
yield* hooks.register("session", "context", (event) =>
Effect.sync(() => {
seen.push(event.options.maxTokens)
delete event.options.maxTokens
}),
)
yield* hooks.register("session", "title", (event) =>
Effect.sync(() => {
event.options.maxTokens = 100
}),
)
const requests = yield* SessionModelRequest.Service.pipe(Effect.provide(SessionModelRequest.layer))
expect((yield* requests.primary(input)).request.generation).toBeUndefined()
expect((yield* requests.title(input)).request.generation?.maxTokens).toBe(100)
expect(seen).toEqual([32_000])
}).pipe(Effect.provideService(SessionModelTransport.Service, transport)),
)
})
@@ -1,9 +1,40 @@
import { describe, expect, test } from "bun:test"
import { Message, ToolResultPart, Media } from "@opencode/ai"
import { boundImages, unsupportedParts } from "@opencode/core/session/model-request"
import { boundImages, outputLimit, unsupportedParts } from "@opencode/core/session/model-request"
const capabilities = (input: string[]) => ({ tools: true, input, output: ["text"] })
describe("SessionModelRequest.outputLimit", () => {
test("requests the catalog output limit up to the cap", () => {
expect(outputLimit({ context: 1_000_000, output: 128_000 }, 256_000)).toBe(128_000)
expect(outputLimit({ context: 200_000, output: 64_000 }, 256_000)).toBe(64_000)
expect(outputLimit({ context: 1_048_576, output: 1_048_576 }, 256_000)).toBe(256_000)
expect(outputLimit({ context: 200_000, output: 64_000 }, 32_000)).toBe(32_000)
})
test("falls back to 32k when the catalog has no output limit", () => {
expect(outputLimit({ context: 200_000, output: 0 }, 256_000)).toBe(32_000)
})
test("fits the limit to the room the prompt leaves in the context window", () => {
const limit = { context: 1_000_000, output: 128_000 }
expect(outputLimit(limit, 256_000, { measured: 50_000, estimated: 0 })).toBe(128_000)
expect(outputLimit(limit, 256_000, { measured: 900_000, estimated: 0 })).toBe(100_000)
// Estimated text counts 15% extra, so 40k estimated takes 46k of the room.
expect(outputLimit(limit, 256_000, { measured: 900_000, estimated: 40_000 })).toBe(54_000)
})
test("keeps a minimum limit when the prompt nearly fills the context window", () => {
const prompt = { measured: 199_000, estimated: 0 }
expect(outputLimit({ context: 200_000, output: 64_000 }, 256_000, prompt)).toBe(1_024)
expect(outputLimit({ context: 200_000, output: 512 }, 256_000, prompt)).toBe(512)
})
test("ignores the prompt size when the context window is unknown", () => {
expect(outputLimit({ context: 0, output: 32_000 }, 256_000, { measured: 500_000, estimated: 0 })).toBe(32_000)
})
})
describe("SessionModelRequest.unsupportedParts", () => {
test("replaces unsupported user media with a visible error", () => {
const messages = unsupportedParts(
@@ -1,5 +1,5 @@
import { expect, test } from "bun:test"
import { LLMClient, LanguageModel, Message, ToolDefinition, Usage } from "@opencode/ai"
import { LLMClient, LanguageModel, Message, ToolDefinition } from "@opencode/ai"
import { OpenAI } from "@opencode/ai/providers"
import { Agent } from "@opencode/core/agent"
import { Bus } from "@opencode/core/bus"
@@ -7,7 +7,6 @@ import { Database } from "@opencode/core/database/database"
import { AppNodeBuilder } from "@opencode/core/effect/app-node-builder"
import { llmClient } from "@opencode/core/effect/app-node-platform"
import { Instructions } from "@opencode/core/instructions/index"
import { NativeCompactionPlugin } from "@opencode/core/plugin/compaction"
import { PluginHooks } from "@opencode/core/plugin/hooks"
import { Project } from "@opencode/core/project"
import { ProjectTable } from "@opencode/core/project/sql"
@@ -27,7 +26,6 @@ import { SessionStore } from "@opencode/core/session/store"
import { LayerNode } from "@opencode/util/effect/layer-node"
import { DateTime, Deferred, Effect, Fiber, Schema } from "effect"
import { testEffect } from "./lib/effect"
import { host } from "./plugin/host"
const it = testEffect(
AppNodeBuilder.build(
@@ -46,7 +44,7 @@ const it = testEffect(
),
)
const setup = Effect.fnUntraced(function* (options: { endpoint?: boolean; plugin?: boolean } = {}) {
const setup = Effect.fnUntraced(function* (options: { endpoint?: boolean } = {}) {
const endpoint = options.endpoint ?? false
const db = (yield* Database.Service).db
const bus = yield* Bus.Service
@@ -57,7 +55,7 @@ const setup = Effect.fnUntraced(function* (options: { endpoint?: boolean; plugin
const hooks = yield* PluginHooks.Service
const blocked = Deferred.makeUnsafe<void>()
const hanging = Promise.withResolvers<Response>()
const state = { failure: false, flaky: false, hang: false, overflow: false, localFailure: false, calls: 0 }
const state = { failure: false, flaky: false, hang: false, overflow: false, calls: 0 }
const bodies: Record<string, unknown>[] = []
const headers: Headers[] = []
const server = yield* Effect.acquireRelease(
@@ -87,7 +85,7 @@ const setup = Effect.fnUntraced(function* (options: { endpoint?: boolean; plugin
)
}
const trigger = JSON.stringify(bodies.at(-1)).includes("compaction_trigger")
if (state.overflow && (trigger || state.localFailure))
if (state.overflow && trigger)
return Response.json(
{
error: {
@@ -114,26 +112,8 @@ const setup = Effect.fnUntraced(function* (options: { endpoint?: boolean; plugin
usage: { input_tokens: 20, output_tokens: 4, total_tokens: 24 },
})
const output = trigger ? [checkpoint] : []
const summary = state.overflow
? [
{
type: "response.output_item.added",
output_index: 0,
item: { type: "message", id: "summary", role: "assistant", content: [] },
},
{
type: "response.output_text.delta",
item_id: "summary",
output_index: 0,
content_index: 0,
delta: "## Objective\n- Recovered locally",
},
]
.map((event) => `data: ${JSON.stringify(event)}\n\n`)
.join("")
: ""
return new Response(
`${summary}data: ${JSON.stringify({
`data: ${JSON.stringify({
type: "response.completed",
response: {
id: `resp_${state.calls}`,
@@ -188,7 +168,6 @@ const setup = Effect.fnUntraced(function* (options: { endpoint?: boolean; plugin
render: { initial: String, changed: (_previous, value) => value, removed: () => "removed" },
})
yield* InstructionState.prepare(db, bus, instructions, sessionID)
if (options.plugin !== false) yield* NativeCompactionPlugin.Plugin.effect(host())
yield* hooks.register("session", "model.request", (event) =>
Effect.sync(() => {
event.headers["x-test-hook"] = event.kind
@@ -218,7 +197,6 @@ const setup = Effect.fnUntraced(function* (options: { endpoint?: boolean; plugin
model,
initial: history.initial,
messages: history.messages,
instructionUpdate: history.instructionUpdate,
agent: { id: Agent.defaultID, info: Agent.Info.default(Agent.defaultID) },
tools: {
definitions: [
@@ -228,14 +206,11 @@ const setup = Effect.fnUntraced(function* (options: { endpoint?: boolean; plugin
},
}
})
// Opens the compaction's message as the runner does when it delivers `/compact`.
const compact = Effect.gen(function* () {
return yield* compaction.compactManual({
session,
messages: yield* store.context(sessionID),
inputID: SessionMessage.ID.create(),
resolveContext: () => load,
prepare: requests.compaction,
})
const inputID = SessionMessage.ID.create()
yield* bus.publish(SessionEvent.Compaction.Started, { sessionID, reason: "manual", recent: "", inputID })
return yield* compaction.compact({ reason: "manual", context: yield* load, inputID })
})
const checkpoint = Effect.gen(function* () {
const messages = (yield* load).messages
@@ -250,8 +225,9 @@ const setup = Effect.fnUntraced(function* (options: { endpoint?: boolean; plugin
})
return {
compact,
automatic: Effect.gen(function* () {
return yield* compaction.compact({ context: yield* load, prepare: requests.compaction })
// An automatic compaction that skips the "is it due" check, which a context this small never passes.
overflow: Effect.gen(function* () {
return yield* compaction.compact({ reason: "overflow", context: yield* load })
}),
checkpoint,
prompt,
@@ -356,7 +332,8 @@ it.live("manual and automatic endpoint compaction keep the provider replacement
const fixture = yield* setup({ endpoint: true })
yield* fixture.prompt("Original user")
expect(yield* fixture.compact).toEqual({ status: "completed" })
expect(yield* fixture.automatic).toEqual({ status: "completed" })
yield* fixture.prompt("Later user")
expect(yield* fixture.overflow).toEqual({ status: "completed" })
const replacement = SessionProviderContext.decode(yield* fixture.checkpoint)
expect(replacement[0]?.content).toEqual([Message.text("endpoint retained")])
expect(JSON.stringify(replacement)).not.toContain("Original user")
@@ -367,7 +344,7 @@ it.live("manual and automatic endpoint compaction keep the provider replacement
}),
)
it.live("only known automatic native overflow falls back locally and failed recovery retains the checkpoint", () =>
it.live("automatic native failures, interruptions, and overflows retain the checkpoint", () =>
Effect.gen(function* () {
const fixture = yield* setup()
yield* fixture.prompt("Original durable request")
@@ -375,12 +352,12 @@ it.live("only known automatic native overflow falls back locally and failed reco
const installed = yield* fixture.checkpoint
yield* fixture.prompt("Recent request")
fixture.state.failure = true
expect(yield* fixture.automatic).toMatchObject({ status: "failed", error: { type: "provider.rate-limit" } })
expect(yield* fixture.overflow).toMatchObject({ status: "failed", error: { type: "provider.rate-limit" } })
expect(fixture.state.calls).toBe(2)
expect(yield* fixture.checkpoint).toEqual(installed)
fixture.state.failure = false
fixture.state.hang = true
const pending = yield* fixture.automatic.pipe(Effect.forkScoped)
const pending = yield* fixture.overflow.pipe(Effect.forkScoped)
yield* Deferred.await(fixture.blocked)
yield* Fiber.interrupt(pending)
expect((yield* fixture.load).messages.at(-1)).toMatchObject({
@@ -390,19 +367,12 @@ it.live("only known automatic native overflow falls back locally and failed reco
})
expect(yield* fixture.checkpoint).toEqual(installed)
fixture.state.hang = false
// Overflow retries natively, with the provider's window, and gives up once nothing is left to shrink.
fixture.state.overflow = true
fixture.state.localFailure = true
expect(yield* fixture.automatic).toMatchObject({ status: "failed" })
expect(fixture.state.calls).toBe(5)
expect(yield* fixture.overflow).toMatchObject({ status: "failed", error: { type: "compaction.failed" } })
expect(fixture.state.calls).toBe(4)
expect(JSON.stringify(fixture.bodies[3])).toContain("encrypted_1")
expect(yield* fixture.checkpoint).toEqual(installed)
expect(JSON.stringify(fixture.bodies[4])).toContain("Original durable request")
expect(JSON.stringify(fixture.bodies[4])).not.toContain("encrypted_1")
fixture.state.localFailure = false
expect(yield* fixture.automatic).toEqual({ status: "completed", recoveredOverflow: true })
expect(fixture.state.calls).toBe(7)
expect((yield* fixture.load).messages).toContainEqual(
expect.objectContaining({ type: "compaction", summary: "## Objective\n- Recovered locally" }),
)
}),
)
@@ -451,31 +421,6 @@ it.live("rejects request-hook route rewrites before provider compaction", () =>
}),
)
it.live("provider compaction fails without a native strategy and persists a registered strategy's window", () =>
Effect.gen(function* () {
const fixture = yield* setup({ plugin: false })
yield* fixture.prompt("Original user")
expect(yield* fixture.compact).toMatchObject({
status: "failed",
error: { type: "provider.unsupported-operation", message: expect.stringContaining("openai/openai-responses") },
})
yield* fixture.compaction.transform((editor) => {
editor.native(() =>
Effect.succeed({
replacement: [Message.assistant("plugin window")],
usage: new Usage({ nonCachedInputTokens: 20, outputTokens: 4 }),
}),
)
})
expect(yield* fixture.compact).toEqual({ status: "completed" })
expect(fixture.state.calls).toBe(0)
const installed = yield* fixture.checkpoint
expect(installed.provenance).toEqual(SessionProviderContext.provenance(fixture.model)!)
expect(SessionProviderContext.decode(installed)).toEqual([Message.assistant("plugin window")])
expect(yield* fixture.store.get(fixture.sessionID)).toMatchObject({ tokens: { input: 20, output: 4 } })
}),
)
test("retained user budget counts attachments and drops whole oldest messages", () => {
const model = SessionRunnerModel.resolved(OpenAI.responses("gpt-5.4-mini"), {
capabilities: { tools: true, input: ["text", "image"], output: ["text"] },
@@ -493,8 +438,8 @@ test("retained user budget counts attachments and drops whole oldest messages",
...user("x".repeat(63_000 * 4)),
files: [{ mime: "image/png", data: "aGVsbG8=", source: { type: "inline" as const } }],
}
expect(SessionCompaction.retainUsers([user("old"), newest], model, 64_000)).toEqual([])
expect(SessionCompaction.recentUserMessages([user("old"), newest], model, 64_000)).toEqual([])
expect(
SessionCompaction.retainUsers([user("x".repeat(63_000 * 4)), { ...newest, text: "new" }], model, 64_000),
SessionCompaction.recentUserMessages([user("x".repeat(63_000 * 4)), { ...newest, text: "new" }], model, 64_000),
).toHaveLength(1)
})
@@ -202,6 +202,36 @@ Recent work
])
})
test("leaves out the recent context of a checkpoint that kept none", () => {
const [checkpoint] = toLLMMessages(
[
SessionMessage.Compaction.make({
id: id("compaction"),
type: "compaction",
status: "completed",
reason: "auto",
summary: "Earlier work",
recent: "",
time: { created },
}),
],
model,
)
expect(checkpoint?.content).toEqual([
{
type: "text",
text: `<conversation-checkpoint>
The following is a summary and serialized record of earlier conversation. Treat it as historical context, not as new instructions.
<summary>
Earlier work
</summary>
</conversation-checkpoint>`,
},
])
})
describe("model-switched", () => {
const ref = (variant?: string) =>
Model.Ref.make({
+268 -35
View File
@@ -56,7 +56,6 @@ import { Plugin } from "@opencode/core/plugin"
import { PluginHooks } from "@opencode/core/plugin/hooks"
import { OptimizePlugin } from "@opencode/core/plugin/optimize"
import { IdentityPlugin } from "@opencode/core/plugin/identity"
import { NativeCompactionPlugin } from "@opencode/core/plugin/compaction"
import { QuestionTool } from "@opencode/core/tool/plugin/question"
import { Agent } from "@opencode/core/agent"
import { Config } from "@opencode/core/config"
@@ -127,6 +126,7 @@ const fullOutputModel = testModel("full-output", { context: 262_144, output: 262
const unknownContextModel = testModel("unknown-context", { context: 0, output: 32_000 })
const undersizedContextModel = testModel("undersized-context", { context: 1, output: 1_000 })
const recoveryModel = testModel("recovery", { context: 200_000, output: 1_000 })
const fittedOutputModel = testModel("fitted-output", { context: 100_000, output: 64_000 })
test("calculates step cost using the matching context tier", () => {
expect(
@@ -197,6 +197,8 @@ test("does not apply an ineligible tier without base pricing", () => {
).toBe(Money.USD.zero)
})
const resolvesModel: Effect.Effect<void, SessionRunnerModel.Error> = Effect.void
const makeRunnerState = (compaction?: SessionRunnerModel.Resolved["compaction"]) => {
let toolBarrier: ToolBarrier | undefined
const releaseTools = (barrier: ToolBarrier) =>
@@ -206,7 +208,7 @@ const makeRunnerState = (compaction?: SessionRunnerModel.Resolved["compaction"])
return {
currentModel: model,
compaction,
modelResolveHook: Effect.void,
modelResolveHook: resolvesModel,
systemBaseline: "Initial context",
systemRemoved: false,
systemUnavailable: false,
@@ -531,7 +533,6 @@ const setup = Effect.gen(function* () {
discard: true,
})
yield* IdentityPlugin.Plugin.effect(pluginHost)
yield* NativeCompactionPlugin.Plugin.effect(pluginHost)
yield* agents.transform((editor) => {
editor.update(Agent.ID.make("build"), (agent) => {
agent.mode = "primary"
@@ -671,6 +672,11 @@ const invalidRequest = () =>
reason: new InvalidRequestError({ message: "Invalid request" }),
})
const payloadTooLarge = () =>
new AIError({
reason: new InvalidRequestError({ message: "Too large", classification: "payload-too-large" }),
})
const rateLimited = (retryAfterMs?: number) =>
new AIError({
reason: new RateLimitError({ message: "Rate limited", retryAfterMs }),
@@ -1296,7 +1302,7 @@ describe("SessionRunnerLLM", () => {
},
)
scenario("delivers controls without preflighting unavailable initial instructions", function* (s) {
scenario("settles compaction and delivers a move while initial instructions are unavailable", function* (s) {
const runner = yield* SessionRunner.Service
s.systemUnavailable = true
let reads = 0
@@ -1323,14 +1329,15 @@ describe("SessionRunnerLLM", () => {
expect(yield* runner.drain({ sessionID, force: false })).toEqual(SessionRunner.DrainResult.Moved({}))
expect(reads).toBe(0)
// Compaction needs the model and instructions, so it reads them and fails; the move does not.
expect(reads).toBe(1)
expect(s.requests).toHaveLength(0)
expect(yield* s.inbox).toEqual([])
expect((yield* s.session.get(sessionID)).location.directory).toBe(AbsolutePath.make("/moved"))
expect((yield* s.messages).find((message) => message.id === compaction.id)).toMatchObject({
type: "compaction",
status: "failed",
error: { type: "compaction.unavailable", message: "Nothing to compact yet" },
error: { message: "Instruction initialization blocked by unavailable sources: test/context" },
})
})
@@ -2096,7 +2103,7 @@ describe("SessionRunnerLLM", () => {
yield* s.llm.push(TestLLM.text("## Objective\n- summary", "epoch-summary"))
yield* s.session.compact({ sessionID })
yield* s.resume
expect(systemTexts(s.requests[1])).toEqual(["Changed before compaction"])
expect(systemTexts(s.requests[1])).toEqual([])
expect((yield* s.context).some((message) => message.type === "system")).toBe(false)
s.systemBaseline = "Replacement context"
yield* s.runPrompt("Second")
@@ -2373,7 +2380,6 @@ describe("SessionRunnerLLM", () => {
scenario("explains when manual compaction has no history", function* (s) {
const compaction = yield* s.session.compact({ sessionID })
s.modelResolveHook = Effect.die("model resolution should not run")
yield* s.resume
@@ -2749,22 +2755,16 @@ describe("SessionRunnerLLM", () => {
})
}
for (const response of ["length", "content-filter", "context overflow"] as const) {
for (const response of ["length", "content-filter"] as const) {
scenario(`rejects compaction ${response} without retrying or committing its draft`, function* (s) {
yield* s.llm.push(TestLLM.text("Earlier answer", "history"))
yield* s.runPrompt("Earlier question")
s.requests.length = 0
yield* s.llm.push(
response === "context overflow"
? Stream.fail(
new AIError({
reason: new InvalidRequestError({ message: "Too long", classification: "context-overflow" }),
}),
)
: TestLLM.complete(
{ reason: { normalized: response } },
LLMEvent.textDelta({ id: "truncated", text: "## Objective\n- Incomplete summary" }),
),
TestLLM.complete(
{ reason: { normalized: response } },
LLMEvent.textDelta({ id: "truncated", text: "## Objective\n- Incomplete summary" }),
),
)
const compaction = yield* s.session.compact({ sessionID })
yield* s.resume
@@ -2778,6 +2778,205 @@ describe("SessionRunnerLLM", () => {
})
}
scenario("stops after three smaller compaction inputs overflow", function* (s) {
// Large enough that the conversation, not the system prompt, is most of the request.
const filler = "context ".repeat(1_000)
yield* s.llm.push(...Array.from({ length: 8 }, (_, index) => TestLLM.text(`Answer ${index}`, `answer-${index}`)))
yield* Effect.forEach(
Array.from({ length: 8 }, (_, index) => index),
(index) => s.runPrompt(`Request ${index}: ${filler}`),
)
s.currentModel = unknownContextModel
s.requests.length = 0
const overflow = () =>
Stream.fail(
new AIError({ reason: new InvalidRequestError({ message: "Too long", classification: "context-overflow" }) }),
)
yield* s.llm.push(overflow(), overflow(), overflow(), overflow())
const compaction = yield* s.session.compact({ sessionID })
yield* s.resume
expect(s.requests).toHaveLength(4)
expect(userTexts(s.requests[2])[0].length).toBeLessThan(userTexts(s.requests[1])[0].length)
expect(userTexts(s.requests[3])[0].length).toBeLessThan(userTexts(s.requests[2])[0].length)
expect((yield* s.messages).find((message) => message.id === compaction.id)).toMatchObject({ status: "failed" })
yield* s.llm.push(TestLLM.text("Continued", "continued"))
yield* s.runPrompt("Continue")
expect(userTexts(s.requests[4])).toContain(`Request 0: ${filler}`)
})
scenario("resends compaction as text after payload too large, then shrinks later rejections", function* (s) {
const image = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII="
const filler = "context ".repeat(1_000)
yield* s.session.prompt({
sessionID,
text: "Request 0 with an image",
files: [{ uri: `data:image/png;base64,${image}` }],
resume: false,
})
yield* s.llm.push(...Array.from({ length: 7 }, (_, index) => TestLLM.text(`Answer ${index}`, `answer-${index}`)))
yield* s.resume
yield* Effect.forEach(
Array.from({ length: 6 }, (_, index) => index + 1),
(index) => s.runPrompt(`Request ${index}: ${filler}`),
)
s.currentModel = unknownContextModel
s.requests.length = 0
yield* s.llm.push(...Array.from({ length: 5 }, () => Stream.fail(payloadTooLarge())))
const compaction = yield* s.session.compact({ sessionID })
yield* s.resume
// The first rejection resends everything as text, which carries no media; later ones shrink like "too long".
expect(s.requests).toHaveLength(5)
expect(s.requests[0]?.messages.some((message) => message.content.some((part) => part.type === "media"))).toBeTrue()
expect(s.requests[1]?.messages.every((message) => message.role === "user")).toBeTrue()
expect(userTexts(s.requests[1])[0]).toContain("[image/png omitted]")
expect(userTexts(s.requests[1])[0]).not.toMatch(/older exchanges? omitted/)
expect(userTexts(s.requests[2])[0].length).toBeLessThan(userTexts(s.requests[1])[0].length)
expect(userTexts(s.requests[3])[0].length).toBeLessThan(userTexts(s.requests[2])[0].length)
expect(userTexts(s.requests[4])[0].length).toBeLessThan(userTexts(s.requests[3])[0].length)
expect((yield* s.messages).find((message) => message.id === compaction.id)).toMatchObject({ status: "failed" })
})
scenario("resends whole history as text after payload too large when it cannot be shortened", function* (s) {
const image = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII="
yield* s.session.prompt({
sessionID,
text: `Request 0: ${"x".repeat(40_000)}`,
files: [{ uri: `data:image/png;base64,${image}` }],
resume: false,
})
yield* s.llm.push(TestLLM.text("Answer 0", "answer-0"))
yield* s.resume
s.currentModel = testModel("smaller-history", { context: 7_000, output: 1_000 })
s.requests.length = 0
yield* s.llm.push(Stream.fail(payloadTooLarge()), TestLLM.text("## Objective\n- Recovered", "summary"))
const compaction = yield* s.session.compact({ sessionID })
yield* s.resume
// Even the latest exchange is over the estimated limit, so the first send is unchanged and the second keeps it all.
expect(s.requests).toHaveLength(2)
expect(s.requests[0]?.messages.some((message) => message.content.some((part) => part.type === "media"))).toBeTrue()
expect(s.requests[1]?.messages.every((message) => message.role === "user")).toBeTrue()
expect(userTexts(s.requests[1])[0]).toContain("[image/png omitted]")
expect(userTexts(s.requests[1])[0]).toContain(`Request 0: ${"x".repeat(40_000)}`)
expect((yield* s.messages).find((message) => message.id === compaction.id)).toMatchObject({ status: "completed" })
})
scenario("shrinks after payload too large on a compaction already sent as text", function* (s) {
const service = yield* SessionCompaction.Service
yield* service.transform((editor) => editor.configure({ buffer: 3_000 }))
s.currentModel = testModel("large-history", { context: 1_000_000, output: 32_000 })
yield* s.llm.push(...Array.from({ length: 6 }, (_, index) => TestLLM.text(`Answer ${index}`, `answer-${index}`)))
yield* Effect.forEach(
Array.from({ length: 6 }, (_, index) => index),
(index) => s.runPrompt(`Request ${index}: ${"x".repeat(8_000)}`),
)
s.currentModel = testModel("smaller-history", { context: 12_000, output: 1_000 })
s.requests.length = 0
yield* s.llm.push(Stream.fail(payloadTooLarge()), TestLLM.text("## Objective\n- Recovered", "summary"))
const compaction = yield* s.session.compact({ sessionID })
yield* s.resume
// The first send was already text, so there is no media left to drop; the rejection counts like "too long".
expect(s.requests).toHaveLength(2)
expect(s.requests[0]?.messages.every((message) => message.role === "user")).toBeTrue()
expect(userTexts(s.requests[1])[0].length).toBeLessThan(userTexts(s.requests[0])[0].length)
expect((yield* s.messages).find((message) => message.id === compaction.id)).toMatchObject({ status: "completed" })
})
scenario("aims the first overflow compaction below the rejected context", function* (s) {
const filler = "context ".repeat(1_000)
yield* s.llm.push(...Array.from({ length: 4 }, (_, index) => TestLLM.text(`Answer ${index}`, `answer-${index}`)))
yield* Effect.forEach(
Array.from({ length: 4 }, (_, index) => index),
(index) => s.runPrompt(`Request ${index}: ${filler}`),
)
s.currentModel = recoveryModel
s.requests.length = 0
yield* s.llm.push(
[LLMEvent.providerError({ message: "prompt too long", classification: "context-overflow" })],
TestLLM.text("## Objective\n- Recovered", "summary"),
TestLLM.text("Recovered", "recovered"),
)
yield* s.runPrompt("Continue")
// The history fits the model's window by estimate, but the provider just rejected it, so older exchanges go.
expect(s.requests).toHaveLength(3)
expect(userTexts(s.requests[1])[0]).toContain("older exchanges omitted")
expect(userTexts(s.requests[1])[0]).not.toContain("Request 0:")
expect(userTexts(s.requests[1])[0]).toContain("Request 3:")
})
scenario("serializes history and omits media after a summary input overflow", function* (s) {
const image = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII="
yield* s.session.prompt({
sessionID,
text: "Earlier question",
files: [{ uri: `data:image/png;base64,${image}` }],
resume: false,
})
yield* s.llm.push(
TestLLM.stop(
LLMEvent.toolCall({ id: "hosted", name: "web_search", input: { query: "earlier" }, providerExecuted: true }),
LLMEvent.toolResult({
id: "hosted",
name: "web_search",
result: { type: "text", value: "x".repeat(5_000) },
providerExecuted: true,
}),
LLMEvent.textStart({ id: "history" }),
LLMEvent.textDelta({ id: "history", text: "Earlier answer" }),
LLMEvent.textEnd({ id: "history" }),
),
)
yield* s.resume
s.requests.length = 0
yield* s.llm.push(
[LLMEvent.providerError({ message: "Too long", classification: "context-overflow" })],
TestLLM.text("## Objective\n- Recovered", "summary"),
)
const compaction = yield* s.session.compact({ sessionID })
yield* s.resume
expect(s.requests).toHaveLength(2)
expect(s.requests[0]?.messages.some((message) => message.content.some((part) => part.type === "media"))).toBeTrue()
expect(s.requests[1]?.messages.every((message) => message.role === "user")).toBeTrue()
expect(userTexts(s.requests[1])[0]).toContain("[image/png omitted]")
expect(userTexts(s.requests[1])[0]).toContain("[Tool result]:")
expect(userTexts(s.requests[1])[0]).toContain(`${"x".repeat(1_250)}\n[truncated]`)
expect(userTexts(s.requests[1])[0]).not.toContain("x".repeat(1_251))
expect(s.requests[1]?.system).toEqual(s.requests[0]?.system)
expect(s.requests[1]?.tools).toEqual(s.requests[0]?.tools)
expect((yield* s.messages).find((message) => message.id === compaction.id)).toMatchObject({ status: "completed" })
})
scenario("fits serialized compaction history by omitting oldest exchanges", function* (s) {
const service = yield* SessionCompaction.Service
yield* service.transform((editor) => editor.configure({ buffer: 3_000 }))
s.currentModel = testModel("large-history", { context: 1_000_000, output: 32_000 })
yield* s.llm.push(...Array.from({ length: 4 }, (_, index) => TestLLM.text(`Answer ${index}`, `answer-${index}`)))
yield* Effect.forEach(
Array.from({ length: 4 }, (_, index) => index),
(index) => s.runPrompt(`Request ${index}: ${"x".repeat(8_000)}`),
)
s.currentModel = testModel("smaller-history", { context: 7_000, output: 1_000 })
s.requests.length = 0
yield* s.llm.push(TestLLM.text("## Objective\n- Recovered", "summary"))
const compaction = yield* s.session.compact({ sessionID })
yield* s.resume
expect(s.requests).toHaveLength(1)
expect(userTexts(s.requests[0])[0]).toContain("older exchanges omitted")
expect(userTexts(s.requests[0])[0]).not.toContain("Request 0:")
expect(userTexts(s.requests[0])[0]).not.toContain("Request 1:")
expect(userTexts(s.requests[0])[0]).toContain("Request 2:")
expect((yield* s.messages).find((message) => message.id === compaction.id)).toMatchObject({
status: "completed",
summary: "## Objective\n- Recovered",
})
})
scenario("records cancelled manual compaction without surfacing an internal failure", function* (s) {
yield* s.llm.push(TestLLM.text("Earlier answer", "text-manual-interrupt-history"))
yield* s.runPrompt("Earlier question")
@@ -2827,6 +3026,30 @@ describe("SessionRunnerLLM", () => {
).toHaveLength(1)
})
scenario("records manual compaction model resolution failures without calling the model", function* (s) {
yield* s.llm.push(TestLLM.text("Earlier answer", "text-manual-unavailable-history"))
yield* s.runPrompt("Earlier question")
s.requests.length = 0
const compaction = yield* s.session.compact({ sessionID })
s.modelResolveHook = Effect.fail(
new SessionRunnerModel.ModelUnavailableError({
providerID: Provider.ID.make("test"),
modelID: Model.ID.make("missing"),
}),
)
yield* s.resume
expect(s.requests).toHaveLength(0)
expect(yield* SessionInbox.find(s.db, compaction.id)).toBeUndefined()
expect((yield* s.messages).find((message) => message.id === compaction.id)).toMatchObject({
type: "compaction",
status: "failed",
reason: "manual",
error: { type: "provider.no-route", message: "Model unavailable: test/missing" },
})
})
scenario("automatically compacts into a completed summary and retained recent turn", function* (s) {
const store = yield* SessionStore.Service
yield* s.llm.push(TestLLM.textWithUsage("Earlier answer", "text-first", 3_950))
@@ -2875,7 +3098,7 @@ describe("SessionRunnerLLM", () => {
scenario("automatically persists native windows, retains earlier users, and waits for fresh usage", function* (s) {
s.currentModel = LanguageModel.make({ id: "native", provider: "openai", route: OpenAIResponses.route })
modelLimits.set("native", { context: 42_000, output: 32_000 })
modelLimits.set("native", { context: 11_000, output: 1_000 })
s.compaction = { type: "native" }
const agents = yield* Agent.Service
yield* agents.transform((editor) =>
@@ -2925,34 +3148,32 @@ describe("SessionRunnerLLM", () => {
expect(JSON.stringify(s.requests[7].messages)).toContain('"encrypted":"second"')
})
scenario("recovers an overflowing native window locally from original durable history", function* (s) {
scenario("recovers an overflowing native window with another native compaction", function* (s) {
s.currentModel = LanguageModel.make({ id: "native", provider: "openai", route: OpenAIResponses.route })
modelLimits.set("native", { context: 42_000, output: 32_000 })
modelLimits.set("native", { context: 11_000, output: 1_000 })
s.compaction = { type: "native" }
const checkpoint = (encrypted: string) =>
CompactionCheckpointResponse.make({
responseID: `resp_${encrypted}`,
checkpoint: { type: "compaction", provider: s.currentModel.provider, encrypted },
})
yield* s.llm.push(TestLLM.textWithUsage("Earlier answer", "before-native", 10_000))
yield* s.runPrompt("Original durable request")
yield* s.llm.push(
CompactionCheckpointResponse.make({
responseID: "resp_native",
checkpoint: { type: "compaction", provider: s.currentModel.provider, encrypted: "native-window" },
}),
TestLLM.text("After native", "after-native"),
)
yield* s.llm.push(checkpoint("native-window"), TestLLM.text("After native", "after-native"))
yield* s.runPrompt("Before native checkpoint")
s.requests.length = 0
yield* s.llm.push(
[LLMEvent.providerError({ message: "prompt too long", classification: "context-overflow" })],
TestLLM.text("## Objective\n- Recovered original history", "local-recovery"),
checkpoint("recovered-window"),
TestLLM.text("Recovered", "recovered"),
)
yield* s.runPrompt("Overflow request")
expect(s.requests).toHaveLength(3)
expect(JSON.stringify(s.requests[0].messages)).toContain("native-window")
expect(JSON.stringify(s.requests[1].messages)).not.toContain("native-window")
expect(userTexts(s.requests[1])).toContain("Original durable request")
expect(userTexts(s.requests[1]).at(-1)).toBe(SessionCompaction.buildPrompt(false))
expect(JSON.stringify(s.requests[2].messages)).toContain("recovered-window")
expect(JSON.stringify(s.requests[2].messages)).not.toContain('"encrypted":"native-window"')
expect(yield* s.context).toMatchObject([
{ type: "compaction", summary: "## Objective\n- Recovered original history" },
{ type: "compaction", status: "completed", providerContext: { version: 1 } },
{ type: "assistant" },
])
})
@@ -2985,7 +3206,7 @@ describe("SessionRunnerLLM", () => {
expect(yield* Effect.exit(s.resume)).toMatchObject({ _tag: "Failure" })
expect(s.requests).toHaveLength(1)
expect(s.requests[0]?.generation).toBeUndefined()
expect(s.requests[0]?.generation?.maxTokens).toBe(50)
expect(yield* s.context).toContainEqual(
expect.objectContaining({
type: "compaction",
@@ -3190,6 +3411,18 @@ describe("SessionRunnerLLM", () => {
])
})
scenario("fits the output limit to the prompt size", function* (s) {
s.currentModel = fittedOutputModel
yield* s.llm.push(TestLLM.textWithUsage("Earlier answer", "text-fitted-first", 50_000))
yield* s.runPrompt("Earlier question")
yield* s.llm.push(TestLLM.text("Continued", "text-fitted-final"))
yield* s.runPrompt("Continue")
expect(s.requests[0]?.generation?.maxTokens).toBe(64_000)
expect(s.requests[1]?.generation?.maxTokens).toBeLessThan(100_000 - 50_000)
expect(s.requests[1]?.generation?.maxTokens).toBeGreaterThan(100_000 - 50_000 - 100)
})
scenario("publishes the original overflow when recovery summarization fails", function* (s) {
yield* setupOverflowRecovery(s)
yield* s.llm.push(
@@ -1012,7 +1012,7 @@ async function connected(
data.location.provider.sync(location),
])
toast.show({ variant: "success", message: `Connected ${integration.name}` })
if (onConnected) {
if (onConnected && integration.metadata?.source !== "mcp") {
onConnected(providerID(data, location, integration.id))
return
}
+3 -2
View File
@@ -85,7 +85,7 @@ Add `compaction` to any [OpenCode configuration file](/config):
| --- | ---: | --- |
| `auto` | `true` | Compact automatically near the context limit, and recover once when a provider rejects a request as too long. Manual compaction always works. |
| `keep.tokens` | `15000` | Approximate recent conversation kept beside the summary. Larger values preserve more detail but leave less room for new work. |
| `buffer` | `20000` | Tokens to reserve below the model's limit. Larger values start automatic compaction earlier. |
| `buffer` | 10% of the limit | Tokens to keep free below the model's limit. Larger values start automatic compaction earlier. |
`keep.tokens` and `buffer` accept non-negative integers.
@@ -127,7 +127,7 @@ after [ encrypted checkpoint + recent user messages ]
| Support | OpenAI Responses models. Support varies by deployment and model. |
| Threshold | Same `auto` and `buffer` settings as summary compaction. Manual requests work too. |
| Portability | An encrypted checkpoint only works with the same provider, model, and endpoint. If you switch models, the session continues from the original conversation instead. |
| Fallback | If the provider rejects an automatic compaction as too long, OpenCode writes a summary itself. |
| Too long | If the provider rejects a compaction as too long, OpenCode retries it with a smaller request. It does not switch to a summary. |
## Summaries
@@ -156,6 +156,7 @@ session's baseline; see [Instructions](/instructions).
| --- | --- |
| Model | Compaction uses the session's model. There is no separate compaction model. |
| History | Compaction needs older conversation to replace. It cannot create room when a request is mostly fixed instructions and tool schemas. |
| Size | If the conversation to compact is itself too long, OpenCode sends a shortened text version and may leave out the oldest exchanges. |
| Recovery | A request rejected as too long is compacted and retried once. A second rejection is returned as an error. |
| Storage | Earlier messages remain stored even when they are no longer sent to the model. |