mirror of
https://github.com/anomalyco/opencode.git
synced 2026-07-21 02:06:00 +00:00
fix(codemode): unify catalog signatures (#35452)
This commit is contained in:
Aiden Cline
+17
-10
@@ -179,14 +179,14 @@ Supported bearer, basic, header, and query authentication follows OpenAPI `secur
|
||||
|
||||
## Discovery
|
||||
|
||||
The agent-tool instructions use a budgeted catalog. Every tool namespace is always listed with its tool count regardless of budget, and as many complete tool signatures (each with a one-line description) as fit an estimated-token budget are inlined. Selection is round-robin across namespaces for fairness: in each round (namespaces alphabetical), every namespace still holding un-inlined tools attempts to place its next-cheapest signature line against the shared budget, and a namespace whose next line does not fit drops out while the others keep going - so every namespace gets some representation before any namespace gets everything. The instructions state exactly how comprehensive the list is, both overall (`COMPLETE list` vs `PARTIAL - N of M shown`) and per namespace (`(3 tools)`, `(3 tools, 1 shown)`, `(3 tools, none shown)`).
|
||||
The agent-tool instructions use a budgeted catalog. Every tool namespace is always listed with its tool count regardless of budget, and as many complete, JSDoc-annotated tool signatures (each with a one-line description) as fit an estimated-token budget are inlined. Schema field descriptions and tags are part of each signature's measured cost. Selection is round-robin across namespaces for fairness: in each round (namespaces alphabetical), every namespace still holding un-inlined tools attempts to place its next-cheapest signature against the shared budget, and a namespace whose next signature does not fit drops out while the others keep going - so every namespace gets some representation before any namespace gets everything. The instructions state exactly how comprehensive the list is, both overall (`COMPLETE list` vs `PARTIAL - N of M shown`) and per namespace (`(3 tools)`, `(3 tools, 1 shown)`, `(3 tools, none shown)`).
|
||||
|
||||
The default budget is 2,000 estimated tokens (characters / 4, the same heuristic OpenCode uses). Override it when constructing a runtime:
|
||||
The catalog-entry budget defaults to 2,000 estimated tokens (characters / 4, the same heuristic OpenCode uses). It applies only to full tool entries shown in the catalog; fixed instructions and namespace summaries are not counted. Override it when constructing a runtime:
|
||||
|
||||
```ts
|
||||
const runtime = CodeMode.make({
|
||||
tools,
|
||||
discovery: { maxInlineCatalogTokens: 6_000 },
|
||||
discovery: { catalogBudget: 6_000 },
|
||||
})
|
||||
```
|
||||
|
||||
@@ -199,30 +199,37 @@ const matches = await tools.$codemode.search({
|
||||
query: "order status",
|
||||
namespace: "orders", // optional: scope to one top-level namespace
|
||||
limit: 10,
|
||||
offset: 0,
|
||||
})
|
||||
```
|
||||
|
||||
`search` performs deterministic, additive field-weighted matching. The query is tokenized (camelCase boundaries split; every non-alphanumeric character is a separator; empties and `*` are dropped), and each term scores every tool: exact path or path-segment match (20), path substring (8), description substring (4), and searchable-text substring (2). Each term also carries naive singular variants (trailing `s`/`es` stripped), and a field check passes when the term or any variant matches - so a plural query term (`issues`) still finds a tool whose text only says `issue`, without changing the weights. The searchable text also includes the input schema's property names and their description strings, so a query naming a parameter finds its tool, and substring matching means partial words match. Scores sum across terms; matches are sorted by score (ties broken alphabetically by path) and capped at `limit` results (default 10).
|
||||
`search` performs deterministic, additive field-weighted matching. The query is tokenized (camelCase boundaries split; every non-alphanumeric character is a separator; empties and `*` are dropped), and each term scores every tool: exact path or path-segment match (20), path substring (8), description substring (4), and searchable-text substring (2). Each term also carries naive singular variants (trailing `s`/`es` stripped), and a field check passes when the term or any variant matches - so a plural query term (`issues`) still finds a tool whose text only says `issue`, without changing the weights. The searchable text also includes the input schema's property names and their description strings, so a query naming a parameter finds its tool, and substring matching means partial words match. Scores sum across terms; matches are sorted by score (ties broken alphabetically by path), then sliced from the zero-based `offset` (default 0) to the configured `limit` (default 10). `remaining` counts matches after the current page. `next` is `{ offset }` when another page exists and `null` on the final page; spread it into the original request to preserve its query, namespace, and limit.
|
||||
|
||||
Each result contains the path, description, and generated TypeScript signature, so no second lookup is needed. The result signature is the pretty, JSDoc-annotated multiline form: each described input/output field carries its schema `description` as a `/** ... */` comment, and constraints TypeScript cannot express ride along as tags (`@deprecated`, `@default`, `@format`, `@minItems`, `@maxItems`). The inline catalog in the instructions keeps the compact single-line form.
|
||||
```ts
|
||||
const request = { query: "order status", namespace: "orders", limit: 10 }
|
||||
const page = await tools.$codemode.search(request)
|
||||
const nextPage = page.next ? await tools.$codemode.search({ ...request, ...page.next }) : undefined
|
||||
```
|
||||
|
||||
Each result contains the path, description, and the same generated TypeScript signature used by the inline catalog, so no second lookup is needed. Signatures use the JSDoc-annotated multiline form: each described input/output field carries its schema `description` as a `/** ... */` comment, and constraints TypeScript cannot express ride along as tags (`@deprecated`, `@default`, `@format`, `@minItems`, `@maxItems`).
|
||||
|
||||
```ts
|
||||
tools.github.list_issues(input: {
|
||||
/** Repository owner */
|
||||
owner: string
|
||||
owner: string,
|
||||
/** Cursor from the previous response's pageInfo */
|
||||
after?: string
|
||||
after?: string,
|
||||
/**
|
||||
* Results per page
|
||||
* @default 30
|
||||
*/
|
||||
perPage?: number
|
||||
perPage?: number,
|
||||
}): Promise<unknown>
|
||||
```
|
||||
|
||||
Result paths are rendered as JavaScript expressions rooted at `tools` (`tools.orders.lookup`, or `tools.context7["resolve-library-id"]` for non-identifier segments), so each `path` is directly usable as the call site. An empty query browses the catalog alphabetically by path; combined with `namespace` (`{ query: "", namespace: "orders" }`) it lists everything in that namespace. A query that names one tool path exactly (canonical path, `tools.`-prefixed path, or rendered JavaScript expression) is treated as a lookup and returns that tool alone.
|
||||
|
||||
The instructions are structured markdown, ordered so the workflow sits at the top and the catalog at the bottom: a `## Workflow` section with numbered steps (find a tool via search when the catalog is partial, or pick from the inlined list when it is complete; call the exact path as-is; `JSON.parse` string results; return only the needed fields), a `## Rules` section holding only guidance the workflow does not already cover (only listed/search-result tools exist inside `tools`; filter and aggregate collections in code; treat `Promise<unknown>` results as shapeless until verified; run independent calls through `Promise.all`; enumerate `tools` with `Object.keys`/`for...in`; browse a namespace via search when it is advertised), a short `## Syntax` section that assumes standard JavaScript and names only what is unusual (TypeScript annotations stripped; the data-boundary serialization of Date/Map/Set/RegExp) or missing (classes, generators, `for await...of`, `.then`/`.catch`/`.finally`), and the budgeted `## Available tools` catalog. Example call forms use explicit `<namespace>.<tool>`/`<field>` placeholders - never a real or fabricated tool name.
|
||||
The instructions are structured markdown, ordered so the workflow sits at the top and the catalog at the bottom: a `## Workflow` section with numbered steps (find a tool via search when the catalog is partial, or pick from the inlined list when it is complete; call the exact path as-is; return only the needed fields), a `## Rules` section holding only guidance the workflow does not already cover (only listed/search-result Code Mode tools and internal runtime tools exist inside `tools`; filter and aggregate collections in code; narrow `Promise<unknown>` results at runtime; run independent calls through `Promise.all`; enumerate `tools` with `Object.keys`/`for...in`; browse a namespace and paginate search results when search is advertised), a short `## Language` section that identifies the runtime as a restricted JavaScript orchestration language and names its major unavailable capabilities, and the budgeted `## Available tools` catalog. Example call forms use explicit `<namespace>.<tool>`/`<field>` placeholders - never a real or fabricated tool name.
|
||||
|
||||
A host cannot define its own `$codemode` top-level namespace.
|
||||
|
||||
@@ -234,7 +241,7 @@ CodeMode executes a deliberately bounded JavaScript subset. It supports:
|
||||
- `if`, conditional expressions, `switch`, `for`, `for...of` (arrays, strings, Maps, Sets), `for...in` (own keys of plain objects, index strings of arrays, and namespace/tool names of `tools` references - anything else is an error suggesting `for...of` or `Object.keys`, rather than real JS's surprising behavior of indices for strings and zero iterations for Maps/Sets), `while`, and `do...while`.
|
||||
- Arrow functions and function declarations with closures, defaults, rest parameters, and destructuring.
|
||||
- Optional chaining, nullish coalescing, templates, spread (arrays, strings, Maps, Sets), and `try`/`catch`.
|
||||
- Common array, string, number, `Object`, `Math`, and `JSON` operations. Mutating array methods include `push`/`pop`/`shift`/`unshift`/`splice` (removes in place and returns the removed elements)/`fill`/`copyWithin`; array `keys`/`values`/`entries` return **arrays** (matching the Map/Set convention) and work with `for...of` and spread. String methods include `localeCompare` (locale/options arguments ignored), `normalize`, and the `trimLeft`/`trimRight` aliases. `Object.keys` also accepts arrays (index strings, as in JS) and tool references: `Object.keys(tools)` lists the top-level namespaces and `Object.keys(tools.ns)` the names at that node (a callable tool enumerates as `[]`; an unknown path is an `UnknownTool` diagnostic). `Object.values`/`Object.entries` on a tool reference fail with a pointer at `Object.keys(tools)` and `tools.$codemode.search`.
|
||||
- Common array, string, number, `Object`, `Math`, and `JSON` operations. Mutating array methods include `push`/`pop`/`shift`/`unshift`/`splice` (removes in place and returns the removed elements)/`fill`/`copyWithin`; array `keys`/`values`/`entries` return **arrays** (matching the Map/Set convention) and work with `for...of` and spread. String methods include `localeCompare` (locale/options arguments ignored), `normalize`, and the `trimLeft`/`trimRight` aliases. `Object.keys` also accepts arrays (index strings, as in JS) and tool references: `Object.keys(tools)` lists the top-level namespaces, including `$codemode`, and `Object.keys(tools.ns)` lists the names at that node (a callable tool enumerates as `[]`; an unknown path is an `UnknownTool` diagnostic). `Object.values`/`Object.entries` on a tool reference fail with a pointer at `Object.keys(tools)` and `tools.$codemode.search`.
|
||||
- `Date` - `Date.now()`/`Date.parse()`/`Date.UTC()`, `new Date(...)`, the getter methods, and date arithmetic/comparison via the time value. Dates stringify as ISO (`toString` included, for determinism across host timezones).
|
||||
- Regular expressions - `/literals/` and `new RegExp(...)` with `test`/`exec` (stateful `lastIndex` for `g`), plus string `match`/`matchAll`/`replace`/`replaceAll`/`split`/`search` with patterns. Match results are arrays carrying `index` and named `groups` as own properties (`input` is omitted). Invalid patterns, invalid flags, and missing-`g` calls fail with catchable errors that say what was wrong and how to fix it (escaping hints, the exact `/pattern/g` to write). Patterns run on the host engine, so pathological backtracking is bounded only by the execution timeout. Function replacers are not supported.
|
||||
- `Map` and `Set` - construction from entries/arrays/strings, `get`/`set`/`add`/`has`/`delete`/`clear`/`size`/`forEach`, and `keys`/`values`/`entries` returning **arrays** (not iterators).
|
||||
|
||||
@@ -65,17 +65,24 @@ From issue #34787 and design discussion. Do not relitigate these casually.
|
||||
### Discovery / search
|
||||
|
||||
- **Search only - no separate `describe`.** `tools.$codemode.search({ query?, namespace?,
|
||||
limit? })` over the final tool tree, owned by this package.
|
||||
- Search result item shape: `{ path, description, signature }` in an `{ items, total }`
|
||||
wrapper. The `signature` string embeds the full input/output TypeScript types - in search
|
||||
results it is the pretty, JSDoc-annotated multiline form (Fix 7), so per-field schema
|
||||
`description`s and constraints (`@default`, `@format`, `@deprecated`, `@minItems`,
|
||||
`@maxItems`) ride along as field comments. The original spec's separate `input`/`output`
|
||||
limit?, offset? })` over the final tool tree, owned by this package.
|
||||
- Search result item shape: `{ path, description, signature }` in an
|
||||
`{ items, remaining, next }`
|
||||
wrapper. The `signature` string embeds the full input/output TypeScript types and uses the
|
||||
same pretty, JSDoc-annotated multiline form in inline catalogs and search results, so
|
||||
per-field schema `description`s and constraints (`@default`, `@format`, `@deprecated`,
|
||||
`@minItems`, `@maxItems`) ride along as field comments. The original spec's separate `input`/`output`
|
||||
raw-schema fields are deliberately NOT added: shapes are already fully expressed in the
|
||||
TypeScript signature and schema annotations now arrive as JSDoc - intent satisfied, letter
|
||||
deviated. Result `path`s render a JavaScript expression rooted at `tools` (for example
|
||||
`tools.github.list_issues` or `tools.context7["resolve-library-id"]`) so each is directly
|
||||
usable as the call site; the internal `ToolDescription.path` stays unprefixed.
|
||||
- `offset` is zero-based and defaults to 0. `remaining` counts matches after the current page;
|
||||
`next` is `{ offset }` when another page exists and `null` on the final page.
|
||||
- Search is an internal `Tool.make` definition backed by Effect input/output schemas. Its
|
||||
validation, output checking, call observation, and TypeScript signature use the same path as
|
||||
host-provided schema tools. Host-only catalog preparation keeps internal tools out of their
|
||||
own search index; only conditional advertisement remains special.
|
||||
- Default limit: **10** (done). Exact-path lookup goes through search too: a query equal to a
|
||||
canonical tool path, `tools.`-prefixed path, or rendered JavaScript expression returns that
|
||||
tool alone (done).
|
||||
@@ -251,7 +258,7 @@ output limit; return a smaller value]`; logs keep leading lines within the remai
|
||||
truncation is now the only result-size mechanism.)
|
||||
- **Search polish**: default limit 12 -> **10** (`defaultSearchLimit`); exact-path lookup - a
|
||||
trimmed query equal to one tool path (optionally `tools.`-prefixed) returns that tool alone
|
||||
(`total: 1`), bypassing ranking. Tokenization/ranking/shape unchanged.
|
||||
(`remaining: 0`, `next: null`), bypassing ranking. Tokenization/ranking/shape unchanged.
|
||||
|
||||
### Wave 3 - OpenCode MCP adapter (done)
|
||||
|
||||
@@ -312,10 +319,10 @@ Instructions are now the budgeted-catalog + prompting-guidance form; verified e2
|
||||
real MCP config. Package still 101 tests / 0 fail; opencode adapter suites still 34 + 16; both
|
||||
packages typecheck clean.
|
||||
|
||||
- **Budgeted catalog** (`discoveryPlan` in `tool-runtime.ts`): the all-or-nothing
|
||||
- **Budgeted catalog** (`prepare` in `tool-runtime.ts`): the all-or-nothing
|
||||
inline/search modes are gone - `DiscoveryMode` deleted, `CodeMode.DiscoveryOptions` is just
|
||||
`{ maxInlineCatalogBytes? }` (default 16,000 UTF-8 bytes; later converted to
|
||||
`maxInlineCatalogTokens`, default 4,000 estimated tokens - see Post-wave fixes). Port of
|
||||
`catalogBudget`, default 4,000 estimated tokens - see Post-wave fixes). Port of
|
||||
the old opencode
|
||||
`describe()` `PREVIEW_BUDGET` algorithm, adapted to `ToolDescription`: every namespace is
|
||||
ALWAYS listed with its tool count; full signature lines
|
||||
@@ -439,9 +446,9 @@ adapter needed **no changes**.
|
||||
defeating discovery. Fixes, all in this package:
|
||||
- `ToolRuntime.make` now returns a `keys(path)` capability (`namespaceKeys` in
|
||||
`tool-runtime.ts`) threaded into the `Interpreter` alongside `invoke` - the interpreter
|
||||
still never holds the host tool tree. `Object.keys(tools)` yields the top-level namespace
|
||||
names (never `$codemode`, which is virtual - but `Object.keys(tools.$codemode)` yields
|
||||
`["search"]`), `Object.keys(tools.ns)` the names at that node; a callable tool leaf
|
||||
still never holds the callable tool tree. `Object.keys(tools)` yields the top-level namespace
|
||||
names, including the internally registered `$codemode`; `Object.keys(tools.$codemode)` yields
|
||||
`["search"]`, and `Object.keys(tools.ns)` yields names at that node; a callable tool leaf
|
||||
enumerates as `[]` (like `Object.keys` of a JS function); an unknown path throws an
|
||||
`UnknownTool` diagnostic suggesting `Object.keys(tools)` and `$codemode.search` (matching
|
||||
call-time unknown-tool behavior rather than silently returning `[]`).
|
||||
@@ -459,7 +466,7 @@ adapter needed **no changes**.
|
||||
- `supportedSyntaxMessage`, the instructions loops line, and README "Supported Programs"
|
||||
mention the new surface; tests in `test/enumeration.test.ts` (14, incl. the exact
|
||||
transcript program) plus one adapter-level assertion that `Object.keys(tools)` returns
|
||||
MCP server names.
|
||||
MCP server and CodeMode namespace names.
|
||||
|
||||
- **Search ranking, namespace scoping, prefixed result paths (done).**
|
||||
Motivation: the Wave 4 e2e run showed a model retrying calls because search-result paths
|
||||
@@ -480,7 +487,7 @@ adapter needed **no changes**.
|
||||
An empty query now browses ALPHABETICALLY by path (was declaration order). Kept:
|
||||
`{ path, description, signature }` result items, default limit 10, exact-path instant
|
||||
lookup, input validation errors.
|
||||
- **Namespace scoping**: `tools.$codemode.search({ query?, namespace?, limit? })` -
|
||||
- **Namespace scoping**: `tools.$codemode.search({ query?, namespace?, limit?, offset? })` -
|
||||
`namespace` (validated as a string when provided) filters `SearchEntry`s to one top-level
|
||||
namespace before ranking; `{ query: "", namespace: "github" }` lists that namespace
|
||||
alphabetically. `searchSignature` updated.
|
||||
@@ -489,7 +496,7 @@ adapter needed **no changes**.
|
||||
segments), directly usable as the call site. Internal `ToolDescription.path` stays
|
||||
unprefixed; only the search RESULT items are rendered this way. Exact-path queries accept
|
||||
canonical paths and rendered expressions.
|
||||
- **Instructions** (`discoveryPlan`): an explicit calling-convention line and a browse
|
||||
- **Instructions** (`prepare`): an explicit calling-convention line and a browse
|
||||
hint on the search advertisement (both since absorbed into the `## Rules` section by
|
||||
the instructions restructure below).
|
||||
- **Tests**: package search/discovery tests updated (prefixed paths, alphabetical browse)
|
||||
@@ -499,28 +506,23 @@ adapter needed **no changes**.
|
||||
|
||||
- **Instructions restructure: markdown sections, placeholder-only call forms (done).**
|
||||
The flat prose instructions (which mixed a real catalog tool with fabricated result
|
||||
fields in the worked example) are replaced by structured markdown in `discoveryPlan`,
|
||||
fields in the worked example) are replaced by structured markdown in `prepare`,
|
||||
ordered so the workflow sits at the top (the least likely part of a long description to
|
||||
be truncated or skimmed away) and the catalog at the bottom (the per-section content
|
||||
described here was later condensed by Fix 8 - Workflow/Rules deduped, Syntax inverted):
|
||||
- **Intro** (2 lines): "Write a CodeMode program... Return code only." + "Execute
|
||||
JavaScript in a confined runtime with access to the tools listed below under
|
||||
`tools.*`." (the second line drops the tools clause when the tree is empty).
|
||||
- **`## Workflow`**: numbered steps - find a tool via `tools.$codemode.search` -> read
|
||||
the `{ path, description, signature }` matches -> call by path -> `typeof res ===
|
||||
"string" ? JSON.parse(res) : res` -> return only the needed fields. When the catalog is
|
||||
COMPLETE the search/read steps collapse into "Pick a tool from the list under
|
||||
`## Available tools`" and the steps renumber (4 instead of 5).
|
||||
- **`## Rules`**: call-by-exact-path; TEXT-is-JSON -> JSON.parse; return small (never raw
|
||||
payloads); filter/aggregate large collections in code instead of per-item round-trips;
|
||||
described here was later condensed by Fix 8 and the language-accuracy pass):
|
||||
- **Intro**: identifies the language as restricted JavaScript for calling tools rather than
|
||||
a general-purpose runtime.
|
||||
- **`## Workflow`**: with a partial catalog, return search results from one execution, then
|
||||
copy a selected path into the next execution. With a complete catalog, pick and call an
|
||||
inlined signature, then return only the needed fields.
|
||||
- **`## Rules`**: narrow unknown results at runtime; filter/aggregate large collections in code instead of per-item round-trips;
|
||||
console.log/warn/error/dir/table for intermediates; `Promise.all` parallelism (no
|
||||
.then/.catch - await + try/catch); `Object.keys(tools)`/`for...in` enumeration;
|
||||
browse-one-namespace via search (PARTIAL only); and host-side media handling (files/
|
||||
images never enter the program; a media-only call yields a small text marker - wording
|
||||
verified against the adapter's `toSandboxResult`/`mediaMarker`).
|
||||
- **`## Syntax`**: the dense syntax lines unchanged, minus the Promise.all and console
|
||||
lines (moved into Rules) and the `for (const ns in tools)` fragment (redundant with
|
||||
the enumeration rule).
|
||||
- **`## Language`**: a concise positive capability summary plus the major unavailable
|
||||
runtime capabilities and the data-boundary serialization note.
|
||||
- **`## Available tools`**: the budgeted catalog unchanged, with the COMPLETE/PARTIAL
|
||||
header merged into the section heading (no trailing colon); the search-signature
|
||||
advertisement follows when PARTIAL (its description-reading and browse clauses moved
|
||||
@@ -532,7 +534,7 @@ adapter needed **no changes**.
|
||||
appear anywhere in the instructions. Zero tools keep "No tools are currently
|
||||
available." under minimal sections (intro + Syntax + Available tools).
|
||||
- **Tests**: the package worked-example test replaced by section-structure/placeholder
|
||||
assertions (section order; JSON.parse + return-small rules present; no
|
||||
assertions (section order; unknown-result + return-small rules present; no
|
||||
`total_count`/`list_issues`/real-tool example lines; browse hint only when PARTIAL;
|
||||
zero-tool minimal sections) - 156 pass / 0 fail; adapter suites gain the same
|
||||
assertions on the built description (still 35 + 16, green).
|
||||
@@ -542,9 +544,9 @@ budget; namespaces must always be present):
|
||||
|
||||
- `src/token.ts` added: copy of `@opencode-ai/core/util/token` (`round(chars / 4)`), so
|
||||
the package stays dependency-free; keep in sync if the core heuristic changes.
|
||||
- `CodeMode.DiscoveryOptions.maxInlineCatalogBytes` -> `maxInlineCatalogTokens` (default 4,000
|
||||
- `CodeMode.DiscoveryOptions.maxInlineCatalogBytes` -> `catalogBudget` (default 4,000
|
||||
estimated tokens ~ the old 16,000 bytes at 4 chars/token - behavior parity, not a size
|
||||
reduction). `discoveryPlan` charges `estimate(catalogLine(tool))` per line; cheapest-first
|
||||
reduction). `prepare` charges `estimate(catalogLine(tool))` per line; cheapest-first
|
||||
- stop-on-first-miss unchanged at the time (stop-on-first-miss replaced by round-robin in
|
||||
Fix 8). Namespace stub lines were and remain unbudgeted - every
|
||||
namespace always appears with its tool count, even at budget 0 (asserted in package and
|
||||
@@ -672,10 +674,8 @@ along). All in `tool-runtime.ts`; no interpreter changes.
|
||||
interfaces/type aliases are stripped and TS **enums actually work** (transpileModule
|
||||
compiles them to an IIFE the interpreter runs), hence enums deliberately unmentioned.
|
||||
`supportedSyntaxMessage` (the in-diagnostic text in `codemode.ts`) is untouched.
|
||||
- **Workflow/Rules deduped**: the call-by-exact-path, JSON.parse-string-results, and
|
||||
return-small content now lives ONLY in the numbered Workflow steps (with their
|
||||
compliance-driving justifications inline: "most tools return JSON as a string", "raw
|
||||
payloads get truncated and waste context"); Rules keeps only bullets adding new
|
||||
- **Workflow/Rules deduped**: the call-by-exact-path and return-small content now lives ONLY
|
||||
in the numbered Workflow steps; Rules keeps only bullets adding new
|
||||
content - filter/aggregate collections in code, console.\* intermediates (logs ride
|
||||
back), Promise.all parallelism, Object.keys/for...in enumeration, browse-namespace
|
||||
(PARTIAL only), and the media rule compressed to one line. The no-.then/.catch
|
||||
@@ -690,7 +690,7 @@ along). All in `tool-runtime.ts`; no interpreter changes.
|
||||
(332 -> 176), Syntax 453 -> 188 (419 -> 174); fixed prose total 1,005 -> 610 (927 -> 562),
|
||||
~ 40% reduction with no behavioral content dropped. Workflow grew slightly because it
|
||||
absorbed the deduped parse/return-small justifications.
|
||||
- **Round-robin namespace inlining** (`discoveryPlan`): the ported stop-on-first-miss
|
||||
- **Round-robin namespace inlining** (`prepare`): the ported stop-on-first-miss
|
||||
behavior (alphabetically-late namespaces starved to "none shown" while an early
|
||||
namespace inlines everything) is replaced by round-robin fairness - in each round
|
||||
(namespaces alphabetical), every namespace still holding un-inlined tools attempts to
|
||||
@@ -708,9 +708,9 @@ along). All in `tool-runtime.ts`; no interpreter changes.
|
||||
the four field checks passes when ANY form matches. Weights, exact-path lookup, and
|
||||
namespace scoping untouched. A true plural path match still outranks a singular-only
|
||||
description match (path substring 8 + searchable 2 > description 4 + searchable 2).
|
||||
- **Tests**: package instruction/structure assertions updated to the new text; new
|
||||
syntax-section test (leads with "Standard modern JavaScript works", names the
|
||||
verified not-supported list, keeps the data-boundary note); the budget-exhaustion
|
||||
- **Tests**: package instruction/structure assertions updated to the new text; the
|
||||
language-section test rejects full-runtime wording, names major unavailable capabilities,
|
||||
and keeps the data-boundary note; the budget-exhaustion
|
||||
test rewritten to assert the new fairness (alpha.expensive not fitting must NOT
|
||||
prevent beta.cheap from showing: PARTIAL 2 of 3, `- beta (1 tool)` fully shown); new
|
||||
plural/singular test (query "issues" finds a singular-only tool; ranking still
|
||||
@@ -724,7 +724,7 @@ along). All in `tool-runtime.ts`; no interpreter changes.
|
||||
**Fix 9 - prompting trims per user review of Fix 8** (user reviewed the condensed
|
||||
instructions and directed further cuts):
|
||||
|
||||
- Default `maxInlineCatalogTokens` 4,000 -> **2,000** (user wants ~2k tokens of signatures
|
||||
- Default `catalogBudget` 4,000 -> **2,000** (user wants ~2k tokens of signatures
|
||||
auto-inlined; round-robin fairness from Fix 8 spreads it across all namespaces).
|
||||
- Console rule and files/images rule DROPPED from `## Rules`. Replaced by a single
|
||||
`unknown`-treatment warning: "A result typed `Promise<unknown>` has no guaranteed
|
||||
@@ -733,9 +733,9 @@ instructions and directed further cuts):
|
||||
return the same data; the prompt stays console-neutral, neither for nor against.)
|
||||
The media-stripping MECHANISM is unchanged and still tested; only the prose about it
|
||||
is gone - the `[N images attached]` marker is self-explanatory in context.
|
||||
- Kept as-is per user: the JSON.parse workflow step (maps to the original motivating
|
||||
transcript failure; NOT copied from prior art - see section 5 note), the browse-namespace rule
|
||||
(undecided), no no-fetch/ambient-authority rule added (proposed, not approved).
|
||||
- Later revised: unconditional JSON parsing was removed because text results are not
|
||||
necessarily JSON. The browse-namespace rule remains; the language section now states that
|
||||
ambient `fetch` is unavailable and external operations go through Code Mode tools.
|
||||
- Explicitly REJECTED for now: auto-parsing JSON-looking text results at the adapter
|
||||
boundary ("could get weird" - type flips, program-sees vs tool-sent divergence). Logged
|
||||
as a next-iteration follow-up below.
|
||||
@@ -1004,11 +1004,19 @@ focused interpreter-surface pass rather than picked off piecemeal.
|
||||
- [x] Sandbox values nested inside logged containers print `[CodeMode reference]`
|
||||
(`console.log({ m: map })`) - could deep-format instead.
|
||||
|
||||
### Next iteration: optional search input boundary
|
||||
|
||||
- [ ] `SearchInput` uses Effect's exact `optionalKey`, so an omitted field is accepted but an
|
||||
explicitly present `undefined` field is rejected. The previous handwritten validator
|
||||
treated explicit `undefined` as omission. Decide whether search should preserve that
|
||||
convenience locally or whether all tool arguments should adopt JSON-style undefined
|
||||
normalization; do not broaden `copyOut` semantics solely to fix search.
|
||||
|
||||
### Next iteration: text-result handling (deliberate follow-up, user-directed)
|
||||
|
||||
- [ ] Revisit how MCP text results reach the program. Today: `structuredContent` when the
|
||||
server sends it, else joined text as a plain string (the program JSON.parses it,
|
||||
guided by a workflow step). Considered and deferred: (a) conservative boundary
|
||||
server sends it, else joined text as a plain string. Programs narrow unknown results
|
||||
before use; the prompt no longer recommends unconditional JSON parsing. Considered and deferred: (a) conservative boundary
|
||||
auto-parse (text starting with `{`/`[` that parses cleanly becomes an object) -
|
||||
rejected for now as potentially confusing (type flips; program sees something other
|
||||
than what the tool sent); (b) raw-envelope passthrough with the envelope shape
|
||||
|
||||
@@ -38,12 +38,12 @@ export type ExecutionLimits = {
|
||||
/** Controls how much of the tool catalog is inlined in agent instructions. */
|
||||
export type DiscoveryOptions = {
|
||||
/**
|
||||
* Estimated-token budget (chars/4, default 2000) for inlined full tool signatures in agent
|
||||
* instructions. Signatures that fit are inlined round-robin across namespaces; every
|
||||
* namespace is always listed with its tool count regardless of budget, and
|
||||
* Approximate budget, in estimated tokens (chars/4, default 2000), for full tool entries in
|
||||
* the instruction catalog. Tool entries are selected round-robin across namespaces. Fixed
|
||||
* instructions and namespace summaries do not count toward the budget, and
|
||||
* `tools.$codemode.search` is always registered.
|
||||
*/
|
||||
readonly maxInlineCatalogTokens?: number
|
||||
readonly catalogBudget?: number
|
||||
}
|
||||
|
||||
type ToolTree<R = never> = {
|
||||
@@ -3921,8 +3921,8 @@ const executeWithLimits = <const Tools extends Record<string, unknown>>(
|
||||
const tools = ToolRuntime.make(
|
||||
(options.tools ?? {}) as HostTools<Services<Tools>>,
|
||||
limits.maxToolCalls,
|
||||
hooks,
|
||||
searchIndex,
|
||||
hooks,
|
||||
)
|
||||
const logs: Array<string> = []
|
||||
const logged = () => (logs.length > 0 ? { logs: [...logs] } : {})
|
||||
@@ -4061,10 +4061,10 @@ export const make = <const Tools extends Record<string, unknown> = {}>(
|
||||
const tools = (options.tools ?? {}) as HostTools<Services<Tools>>
|
||||
ToolRuntime.assertValidTools(tools)
|
||||
const limits = resolveExecutionLimits(options.limits)
|
||||
const discovery = ToolRuntime.discoveryPlan(tools, options.discovery?.maxInlineCatalogTokens)
|
||||
const executeProgram = (code: string) => executeWithLimits<Tools>({ ...options, code }, limits, discovery.searchIndex)
|
||||
const catalog = discovery.catalog
|
||||
const instructions = discovery.instructions
|
||||
const prepared = ToolRuntime.prepare(tools, options.discovery?.catalogBudget)
|
||||
const executeProgram = (code: string) => executeWithLimits<Tools>({ ...options, code }, limits, prepared.searchIndex)
|
||||
const catalog = prepared.catalog
|
||||
const instructions = prepared.instructions
|
||||
|
||||
return {
|
||||
catalog: () => catalog,
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { Cause, Effect } from "effect"
|
||||
import { Cause, Effect, Schema } from "effect"
|
||||
import { ToolError, toolError } from "./tool-error.js"
|
||||
import {
|
||||
decodeInput as decodeToolInput,
|
||||
@@ -8,6 +8,7 @@ import {
|
||||
inputTypeScript,
|
||||
isDefinition as isToolDefinition,
|
||||
outputTypeScript,
|
||||
Tool,
|
||||
type Definition,
|
||||
} from "./tool.js"
|
||||
import { SandboxDate, SandboxMap, SandboxPromise, SandboxRegExp, SandboxSet } from "./values.js"
|
||||
@@ -76,10 +77,26 @@ export type ToolDescription = {
|
||||
export type SafeObject = Record<string, unknown>
|
||||
|
||||
const reservedNamespace = "$codemode"
|
||||
const defaultMaxInlineCatalogTokens = 2_000
|
||||
const defaultCatalogBudget = 2_000
|
||||
const defaultSearchLimit = 10
|
||||
const searchSignature =
|
||||
"tools.$codemode.search({ query?: string, namespace?: string, limit?: number }): Promise<{ items: Array<{ path: string; description: string; signature: string }>; total: number }>"
|
||||
const PositiveInt = Schema.Int.check(Schema.isGreaterThan(0))
|
||||
const NonNegativeInt = Schema.Int.check(Schema.isGreaterThanOrEqualTo(0))
|
||||
const SearchInput = Schema.Struct({
|
||||
query: Schema.optionalKey(Schema.String),
|
||||
namespace: Schema.optionalKey(Schema.String),
|
||||
limit: Schema.optionalKey(PositiveInt),
|
||||
offset: Schema.optionalKey(NonNegativeInt),
|
||||
})
|
||||
const SearchItem = Schema.Struct({
|
||||
path: Schema.String,
|
||||
description: Schema.String,
|
||||
signature: Schema.String,
|
||||
})
|
||||
const SearchOutput = Schema.Struct({
|
||||
items: Schema.Array(SearchItem),
|
||||
remaining: NonNegativeInt,
|
||||
next: Schema.NullOr(Schema.Struct({ offset: NonNegativeInt })),
|
||||
})
|
||||
const toolExpression = (path: string) =>
|
||||
"tools" +
|
||||
path
|
||||
@@ -295,15 +312,17 @@ const definitions = <R>(
|
||||
return entries
|
||||
}
|
||||
|
||||
const describeDefinition = <R>(path: string, definition: Definition<R>): ToolDescription => ({
|
||||
path,
|
||||
description: definition.description,
|
||||
signature: `${toolExpression(path)}(input: ${inputTypeScript(definition, true)}): Promise<${outputTypeScript(definition, true)}>`,
|
||||
})
|
||||
|
||||
const visibleDefinitions = <R>(tools: HostTools<R>) =>
|
||||
definitions(tools).map(({ path, definition }) => ({
|
||||
path,
|
||||
definition,
|
||||
description: {
|
||||
path,
|
||||
description: definition.description,
|
||||
signature: `${toolExpression(path)}(input: ${inputTypeScript(definition)}): Promise<${outputTypeScript(definition)}>`,
|
||||
},
|
||||
description: describeDefinition(path, definition),
|
||||
}))
|
||||
|
||||
export const catalog = <R>(tools: HostTools<R>): ReadonlyArray<ToolDescription> =>
|
||||
@@ -317,11 +336,6 @@ export type DiscoveryPlan = {
|
||||
|
||||
export type SearchEntry = {
|
||||
readonly description: ToolDescription
|
||||
/**
|
||||
* JSDoc-annotated multiline signature shown on search-result items; the compact
|
||||
* single-line form (inline catalog lines) stays in `description.signature`.
|
||||
*/
|
||||
readonly signature: string
|
||||
/** Top-level namespace (first path segment), matched by the search `namespace` option. */
|
||||
readonly namespace: string
|
||||
/** Lowercased path + description + input property names/descriptions, for substring matching. */
|
||||
@@ -355,8 +369,76 @@ const termForms = (term: string): Array<string> => {
|
||||
return forms
|
||||
}
|
||||
|
||||
const makeSearchTool = (searchIndex: ReadonlyArray<SearchEntry>) =>
|
||||
Tool.make({
|
||||
description: "Search available Code Mode tools",
|
||||
input: SearchInput,
|
||||
output: SearchOutput,
|
||||
run: (request) =>
|
||||
Effect.sync(() => {
|
||||
const query = request.query ?? ""
|
||||
const offset = request.offset ?? 0
|
||||
const scoped =
|
||||
request.namespace === undefined
|
||||
? searchIndex
|
||||
: searchIndex.filter((entry) => entry.namespace === request.namespace)
|
||||
// A query that names one tool path exactly (canonical path or rendered JavaScript
|
||||
// expression) is a lookup, not a search: return that tool alone.
|
||||
const trimmed = query.trim()
|
||||
const pathQuery = trimmed.startsWith("tools.") ? trimmed.slice("tools.".length) : trimmed
|
||||
const exact =
|
||||
pathQuery === ""
|
||||
? undefined
|
||||
: scoped.find(
|
||||
(entry) =>
|
||||
entry.description.path === pathQuery || toolExpression(entry.description.path) === trimmed,
|
||||
)
|
||||
const terms = tokenize(query).map(termForms)
|
||||
// Additive field-weighted scoring, summed across terms: exact path or path segment
|
||||
// (20) > path substring (8) > description substring (4) > any searchable text,
|
||||
// including input parameter names and descriptions (2).
|
||||
const ranked =
|
||||
exact !== undefined
|
||||
? [exact]
|
||||
: scoped
|
||||
.map((entry) => {
|
||||
const path = entry.description.path.toLowerCase()
|
||||
const description = entry.description.description.toLowerCase()
|
||||
const score = terms.reduce(
|
||||
(total, forms) =>
|
||||
total +
|
||||
(forms.some((form) => path === form || path.endsWith(`.${form}`)) ? 20 : 0) +
|
||||
(forms.some((form) => path.includes(form)) ? 8 : 0) +
|
||||
(forms.some((form) => description.includes(form)) ? 4 : 0) +
|
||||
(forms.some((form) => entry.searchText.includes(form)) ? 2 : 0),
|
||||
0,
|
||||
)
|
||||
return { entry, score }
|
||||
})
|
||||
.filter(({ score }) => terms.length === 0 || score > 0)
|
||||
.sort(
|
||||
(left, right) =>
|
||||
right.score - left.score ||
|
||||
left.entry.description.path.localeCompare(right.entry.description.path),
|
||||
)
|
||||
.map(({ entry }) => entry)
|
||||
const items = ranked.slice(offset, offset + (request.limit ?? defaultSearchLimit)).map(({ description }) => ({
|
||||
...description,
|
||||
path: toolExpression(description.path),
|
||||
}))
|
||||
const remaining = Math.max(0, ranked.length - offset - items.length)
|
||||
return {
|
||||
items,
|
||||
remaining,
|
||||
next: remaining > 0 ? { offset: offset + items.length } : null,
|
||||
}
|
||||
}),
|
||||
})
|
||||
|
||||
const searchDescription = describeDefinition(`${reservedNamespace}.search`, makeSearchTool([]))
|
||||
|
||||
const catalogLine = (tool: ToolDescription) => {
|
||||
// Inline catalog lines use only a compact first line; full text stays in search results.
|
||||
// Keep the tool description concise; the full schema documentation remains in the signature.
|
||||
const line = tool.description.split("\n", 1)[0]!.trim()
|
||||
const description = line.length > 120 ? line.slice(0, 119) + "..." : line
|
||||
return description === "" ? ` - ${tool.signature}` : ` - ${tool.signature} // ${description}`
|
||||
@@ -364,7 +446,6 @@ const catalogLine = (tool: ToolDescription) => {
|
||||
|
||||
const toSearchEntry = <R>(path: string, definition: Definition<R>, description: ToolDescription): SearchEntry => ({
|
||||
description,
|
||||
signature: `${toolExpression(path)}(input: ${inputTypeScript(definition, true)}): Promise<${outputTypeScript(definition, true)}>`,
|
||||
namespace: path.split(".", 1)[0]!,
|
||||
searchText: [
|
||||
path,
|
||||
@@ -389,7 +470,7 @@ export const assertValidTools = <R>(tools: HostTools<R>): void => {
|
||||
|
||||
/**
|
||||
* Budgeted catalog: every namespace is always listed with its tool count; full call
|
||||
* signatures are inlined against the `maxInlineCatalogTokens` budget (estimated tokens,
|
||||
* signatures are inlined against the `catalogBudget` (estimated tokens,
|
||||
* chars/4) round-robin across namespaces - in each round (namespaces alphabetical), every
|
||||
* namespace still holding un-inlined tools attempts to place its next-cheapest line, and
|
||||
* a namespace whose next line does not fit is done while the others keep going - so every
|
||||
@@ -398,12 +479,12 @@ export const assertValidTools = <R>(tools: HostTools<R>): void => {
|
||||
* namespace. Namespace stub lines are never budgeted: every namespace appears with its
|
||||
* tool count even at budget 0.
|
||||
*/
|
||||
export const discoveryPlan = <R>(
|
||||
export const prepare = <R>(
|
||||
tools: HostTools<R>,
|
||||
maxInlineCatalogTokens = defaultMaxInlineCatalogTokens,
|
||||
catalogBudget = defaultCatalogBudget,
|
||||
): DiscoveryPlan => {
|
||||
if (!Number.isSafeInteger(maxInlineCatalogTokens) || maxInlineCatalogTokens < 0) {
|
||||
throw new RangeError("discovery.maxInlineCatalogTokens must be a non-negative safe integer")
|
||||
if (!Number.isSafeInteger(catalogBudget) || catalogBudget < 0) {
|
||||
throw new RangeError("discovery.catalogBudget must be a non-negative safe integer")
|
||||
}
|
||||
const visible = visibleDefinitions(tools)
|
||||
const described = visible.map(({ description }) => description)
|
||||
@@ -438,7 +519,7 @@ export const discoveryPlan = <R>(
|
||||
for (const selection of active) {
|
||||
const tool = selection.queue[0]!
|
||||
const cost = estimateTokens(catalogLine(tool))
|
||||
if (used + cost > maxInlineCatalogTokens) continue
|
||||
if (used + cost > catalogBudget) continue
|
||||
selection.queue.shift()
|
||||
selection.picked.add(tool)
|
||||
used += cost
|
||||
@@ -459,12 +540,11 @@ export const discoveryPlan = <R>(
|
||||
// catalog at the bottom. Example call forms use placeholders - never a real or fabricated
|
||||
// tool name - and show both dot and bracket notation so non-identifier names are not normalized.
|
||||
const intro = [
|
||||
"Write a CodeMode program to answer the request. Return code only.",
|
||||
empty
|
||||
? "Execute JavaScript in a confined runtime."
|
||||
? "This is a restricted JavaScript language for calling tools, not a general-purpose runtime."
|
||||
: complete
|
||||
? "Execute JavaScript in a confined runtime. Inside this program, `tools` contains only the host-provided tools listed below; surrounding agent tools are not available unless listed here."
|
||||
: "Execute JavaScript in a confined runtime. Inside this program, `tools` contains only the host-provided tools listed or searchable below; surrounding agent tools are not available unless listed here.",
|
||||
? "This is a restricted JavaScript language for calling tools, not a general-purpose runtime. Inside the confined interpreter, `tools` contains the Code Mode tools listed below and internal runtime tools; surrounding agent tools are not available."
|
||||
: "This is a restricted JavaScript language for calling tools, not a general-purpose runtime. Inside the confined interpreter, `tools` contains the Code Mode tools listed or searchable below and internal runtime tools; surrounding agent tools are not available.",
|
||||
...(empty
|
||||
? []
|
||||
: ["Do not infer or normalize tool names; use only exact signatures shown below or returned by search."]),
|
||||
@@ -481,16 +561,12 @@ export const discoveryPlan = <R>(
|
||||
...(complete
|
||||
? [
|
||||
"1. Pick a tool from the list under `## Available tools` - each line is the exact call signature; use it as-is rather than guessing segments.",
|
||||
"2. Call it using the exact signature shown; bracket notation and quotes are part of the path.",
|
||||
'3. Parse text results: `const data = typeof res === "string" ? JSON.parse(res) : res` - most tools return JSON as a string.',
|
||||
"4. Return only the fields you need: `return { <field>: data.<field> }` - raw payloads get truncated and waste context.",
|
||||
"2. Call it using the exact signature shown: `const result = await tools.<namespace>.<tool>(input)`; bracket notation and quotes are part of the path.",
|
||||
"3. Return only the fields you need from structured results; narrow unknown results before reading fields, and avoid returning large raw payloads.",
|
||||
]
|
||||
: [
|
||||
'1. If the exact signature is not listed below, first search: `const { items } = await tools.$codemode.search({ query: "<intent + key nouns>" })`.',
|
||||
"2. Read the matches: each item is `{ path, description, signature }` - read the description before using an unfamiliar tool.",
|
||||
"3. Call the result's `path` as-is; bracket notation and quotes are part of the path.",
|
||||
'4. Parse text results: `const data = typeof res === "string" ? JSON.parse(res) : res` - most tools return JSON as a string.',
|
||||
"5. Return only the fields you need: `return { <field>: data.<field> }` - raw payloads get truncated and waste context.",
|
||||
'1. If needed, discover tools: `return await tools.$codemode.search({ query: "<intent + key nouns>" })`.',
|
||||
"2. In the next execution, copy a returned path exactly, call it, and return only the needed fields.",
|
||||
]),
|
||||
]
|
||||
|
||||
@@ -501,24 +577,26 @@ export const discoveryPlan = <R>(
|
||||
"## Rules",
|
||||
"",
|
||||
complete
|
||||
? "- Only tools listed here are available inside `tools`; tools from the surrounding agent/runtime are not implicitly exposed."
|
||||
: "- Only tools listed here or returned by `tools.$codemode.search` are available inside `tools`; tools from the surrounding agent/runtime are not implicitly exposed.",
|
||||
? "- Only Code Mode tools listed here and internal runtime tools are available; surrounding agent tools are not implicitly exposed."
|
||||
: "- Only Code Mode tools listed here or returned by `tools.$codemode.search` and internal runtime tools are available; surrounding agent tools are not implicitly exposed.",
|
||||
"- Filter, aggregate, and transform collections in code - never return them raw or call a tool per item across messages.",
|
||||
"- A result typed `Promise<unknown>` has no guaranteed shape - verify what actually came back before relying on its fields.",
|
||||
"- A result typed `Promise<unknown>` may be structured data or text. Before reading fields, check that it is a non-null object and not an array; otherwise handle the returned text or primitive directly.",
|
||||
'- Run independent calls in parallel: `await Promise.all(items.map((item) => tools.<namespace>.<tool>(item)))`, or use `tools.<namespace>["tool-name"](item)` when the listed signature uses bracket notation.',
|
||||
"- `Object.keys(tools)` lists namespaces; `Object.keys(tools.<namespace>)` lists its tools; `for...in` works on both.",
|
||||
...(complete
|
||||
? []
|
||||
: ['- Browse one namespace: `await tools.$codemode.search({ query: "", namespace: "<name>" })`.']),
|
||||
: [
|
||||
'- Browse one namespace: `await tools.$codemode.search({ query: "", namespace: "<name>" })`.',
|
||||
"- If search returns `next`, repeat the same search with `offset: next.offset`.",
|
||||
]),
|
||||
]
|
||||
|
||||
const syntax = [
|
||||
const language = [
|
||||
"",
|
||||
"## Syntax",
|
||||
"## Language",
|
||||
"",
|
||||
"Standard modern JavaScript works: functions/closures, destructuring, template literals, loops, try/catch, spread, optional chaining, the usual Array/String/Object/Math/JSON methods, plus Date, RegExp, Map, Set, and Promise.all/allSettled/race/resolve/reject.",
|
||||
"TypeScript type annotations are allowed and stripped before execution (decorators are not supported).",
|
||||
"Not supported (each fails with a message naming the alternative): classes, generators, for await...of, .then/.catch/.finally (use await with try/catch).",
|
||||
"Use common JavaScript data operations, functions, control flow, selected standard-library methods, and awaited tool calls.",
|
||||
"Modules/imports, classes, generators, timers, fetch, eval, prototype access, arbitrary methods, and promise chaining are unavailable. Use Code Mode tools for external operations. Use await with try/catch.",
|
||||
"Dates serialize to ISO strings at data boundaries; Map/Set/RegExp serialize to `{}`.",
|
||||
]
|
||||
|
||||
@@ -547,11 +625,11 @@ export const discoveryPlan = <R>(
|
||||
for (const tool of group) if (picked.has(tool)) toolSection.push(catalogLine(tool))
|
||||
}
|
||||
if (!complete) {
|
||||
toolSection.push("", "Search returns complete callable signatures:", `- ${searchSignature}`)
|
||||
toolSection.push("", "Search returns complete callable signatures:", `- ${searchDescription.signature}`)
|
||||
}
|
||||
}
|
||||
|
||||
const lines = [...intro, ...workflow, ...rules, ...syntax, ...toolSection]
|
||||
const lines = [...intro, ...workflow, ...rules, ...language, ...toolSection]
|
||||
return {
|
||||
catalog: described,
|
||||
instructions: lines.join("\n"),
|
||||
@@ -560,7 +638,7 @@ export const discoveryPlan = <R>(
|
||||
}
|
||||
|
||||
/**
|
||||
* The enumerable names at one node of the host tool tree - namespace names at the root,
|
||||
* The enumerable names at one node of the callable tool tree - namespace names at the root,
|
||||
* tool/namespace names below - powering `Object.keys(tools)` and `for...in` over tool
|
||||
* references. A callable tool is a leaf and enumerates as `[]` (like `Object.keys` of a
|
||||
* function in JS). An unknown path is an `UnknownTool` error pointing at the working
|
||||
@@ -569,11 +647,7 @@ export const discoveryPlan = <R>(
|
||||
const namespaceKeys = <R>(
|
||||
tools: HostTools<R>,
|
||||
path: ReadonlyArray<string>,
|
||||
searchEnabled: boolean,
|
||||
): ReadonlyArray<string> => {
|
||||
// The reserved discovery namespace is virtual (never present in the host tree); enumerate
|
||||
// it explicitly so `Object.keys(tools.$codemode)` matches the callable surface.
|
||||
if (searchEnabled && path.length === 1 && path[0] === reservedNamespace) return ["search"]
|
||||
let value: HostTool<R> | Definition<R> | HostTools<R> = tools
|
||||
for (const segment of path) {
|
||||
if (
|
||||
@@ -585,11 +659,9 @@ const namespaceKeys = <R>(
|
||||
throw new ToolRuntimeError(
|
||||
"UnknownTool",
|
||||
`Unknown tool namespace '${path.join(".")}'.`,
|
||||
searchEnabled
|
||||
? [
|
||||
"Object.keys(tools) lists the available namespaces; tools.$codemode.search({ query }) finds described tools.",
|
||||
]
|
||||
: ["Object.keys(tools) lists the available namespaces."],
|
||||
[
|
||||
"Object.keys(tools) lists the available namespaces; tools.$codemode.search({ query }) finds described tools.",
|
||||
],
|
||||
)
|
||||
}
|
||||
value = value[segment] as HostTool<R> | Definition<R> | HostTools<R>
|
||||
@@ -601,7 +673,6 @@ const namespaceKeys = <R>(
|
||||
const resolve = <R>(
|
||||
tools: HostTools<R>,
|
||||
path: ReadonlyArray<string>,
|
||||
searchEnabled: boolean,
|
||||
): HostTool<R> | Definition<R> => {
|
||||
let value: HostTool<R> | Definition<R> | HostTools<R> = tools
|
||||
|
||||
@@ -615,7 +686,7 @@ const resolve = <R>(
|
||||
throw new ToolRuntimeError(
|
||||
"UnknownTool",
|
||||
`Unknown tool '${path.join(".")}'.`,
|
||||
searchEnabled ? ["Use tools.$codemode.search({ query }) to find available described tools."] : [],
|
||||
["Use tools.$codemode.search({ query }) to find available described tools."],
|
||||
)
|
||||
}
|
||||
value = value[segment] as HostTool<R> | Definition<R> | HostTools<R>
|
||||
@@ -632,7 +703,7 @@ export type ToolRuntime<R = never> = {
|
||||
readonly root: ToolReference
|
||||
readonly calls: Array<ToolCall>
|
||||
readonly invoke: (path: ReadonlyArray<string>, args: Array<unknown>) => Effect.Effect<unknown, unknown, R>
|
||||
/** Enumerable namespace/tool names at one node of the host tool tree; see `namespaceKeys`. */
|
||||
/** Enumerable namespace/tool names at one node of the callable tool tree; see `namespaceKeys`. */
|
||||
readonly keys: (path: ReadonlyArray<string>) => ReadonlyArray<string>
|
||||
}
|
||||
|
||||
@@ -640,11 +711,14 @@ export const make = <R>(
|
||||
tools: HostTools<R>,
|
||||
/** Undefined means unlimited tool calls. */
|
||||
maxToolCalls: number | undefined,
|
||||
searchIndex: ReadonlyArray<SearchEntry>,
|
||||
hooks?: ToolCallHooks<R>,
|
||||
searchIndex?: ReadonlyArray<SearchEntry>,
|
||||
): ToolRuntime<R> => {
|
||||
const calls: Array<ToolCall> = []
|
||||
const searchEnabled = searchIndex !== undefined
|
||||
const callableTools = {
|
||||
...tools,
|
||||
[reservedNamespace]: { search: makeSearchTool(searchIndex) },
|
||||
}
|
||||
|
||||
// Wraps the settling portion of a tool call so onToolCallEnd observes success and failure
|
||||
// symmetrically. Interruption (e.g. the execution timeout) fires neither outcome.
|
||||
@@ -683,7 +757,7 @@ export const make = <R>(
|
||||
return {
|
||||
root: new ToolReference([]),
|
||||
calls,
|
||||
keys: (path) => namespaceKeys(tools, path, searchEnabled),
|
||||
keys: (path) => namespaceKeys(callableTools, path),
|
||||
invoke: (path, args) =>
|
||||
Effect.gen(function* () {
|
||||
const name = path.join(".")
|
||||
@@ -694,107 +768,7 @@ export const make = <R>(
|
||||
recordCall(call)
|
||||
return calls.length - 1
|
||||
}).pipe(Effect.tap((index) => hooks?.onToolCallStart?.({ index, name, input }) ?? Effect.void))
|
||||
if (name === "$codemode.search") {
|
||||
if (!searchEnabled) throw new ToolRuntimeError("UnknownTool", `Unknown tool '${name}'.`)
|
||||
const input = externalArgs[0]
|
||||
if (externalArgs.length !== 1 || input === null || typeof input !== "object" || Array.isArray(input)) {
|
||||
throw new ToolRuntimeError(
|
||||
"InvalidToolInput",
|
||||
"tools.$codemode.search expects { query?: string; namespace?: string; limit?: number }.",
|
||||
)
|
||||
}
|
||||
const request = input as { query?: unknown; namespace?: unknown; limit?: unknown }
|
||||
if (request.query !== undefined && typeof request.query !== "string") {
|
||||
throw new ToolRuntimeError(
|
||||
"InvalidToolInput",
|
||||
"tools.$codemode.search query must be a string when provided.",
|
||||
)
|
||||
}
|
||||
if (request.namespace !== undefined && typeof request.namespace !== "string") {
|
||||
throw new ToolRuntimeError(
|
||||
"InvalidToolInput",
|
||||
"tools.$codemode.search namespace must be a string when provided.",
|
||||
)
|
||||
}
|
||||
if (
|
||||
request.limit !== undefined &&
|
||||
(typeof request.limit !== "number" || !Number.isSafeInteger(request.limit) || request.limit <= 0)
|
||||
) {
|
||||
throw new ToolRuntimeError(
|
||||
"InvalidToolInput",
|
||||
"tools.$codemode.search limit must be a positive safe integer when provided.",
|
||||
)
|
||||
}
|
||||
const query = typeof request.query === "string" ? request.query : ""
|
||||
const namespace = typeof request.namespace === "string" ? request.namespace : undefined
|
||||
const index = yield* recordAndObserve(request)
|
||||
return yield* observeEnd(
|
||||
Effect.try({
|
||||
try: () => {
|
||||
const limit = typeof request.limit === "number" ? request.limit : defaultSearchLimit
|
||||
const scoped =
|
||||
namespace === undefined ? searchIndex : searchIndex.filter((entry) => entry.namespace === namespace)
|
||||
// A query that names one tool path exactly (canonical path or rendered
|
||||
// JavaScript expression) is a lookup, not a search: return that tool alone.
|
||||
const trimmed = query.trim()
|
||||
const pathQuery = trimmed.startsWith("tools.") ? trimmed.slice("tools.".length) : trimmed
|
||||
const exact =
|
||||
pathQuery === ""
|
||||
? undefined
|
||||
: scoped.find(
|
||||
(entry) =>
|
||||
entry.description.path === pathQuery || toolExpression(entry.description.path) === trimmed,
|
||||
)
|
||||
const terms = tokenize(query).map(termForms)
|
||||
// Additive field-weighted scoring, summed across terms: exact path or path
|
||||
// segment (20) > path substring (8) > description substring (4) > any
|
||||
// searchable text, incl. input parameter names/descriptions (2). Each term
|
||||
// matches a field when any of its forms (the term or a singular variant)
|
||||
// does. An empty query browses everything, alphabetical by path.
|
||||
const ranked =
|
||||
exact !== undefined
|
||||
? [exact]
|
||||
: scoped
|
||||
.map((entry) => {
|
||||
const path = entry.description.path.toLowerCase()
|
||||
const description = entry.description.description.toLowerCase()
|
||||
const score = terms.reduce(
|
||||
(total, forms) =>
|
||||
total +
|
||||
(forms.some((form) => path === form || path.endsWith(`.${form}`)) ? 20 : 0) +
|
||||
(forms.some((form) => path.includes(form)) ? 8 : 0) +
|
||||
(forms.some((form) => description.includes(form)) ? 4 : 0) +
|
||||
(forms.some((form) => entry.searchText.includes(form)) ? 2 : 0),
|
||||
0,
|
||||
)
|
||||
return { entry, score }
|
||||
})
|
||||
.filter(({ score }) => terms.length === 0 || score > 0)
|
||||
.sort(
|
||||
(left, right) =>
|
||||
right.score - left.score ||
|
||||
left.entry.description.path.localeCompare(right.entry.description.path),
|
||||
)
|
||||
.map(({ entry }) => entry)
|
||||
// Result paths are rendered as JavaScript expressions so each `path` is
|
||||
// directly usable as the call site (`await tools.github.list({ ... })` or
|
||||
// `await tools.ns["dashed-name"]({ ... })`). The signature is the pretty,
|
||||
// JSDoc-annotated form (schema descriptions and constraints ride along as
|
||||
// field comments).
|
||||
const items = ranked.slice(0, limit).map(({ description, signature }) => ({
|
||||
...description,
|
||||
path: toolExpression(description.path),
|
||||
signature,
|
||||
}))
|
||||
return copyIn({ items, total: ranked.length }, "Result from tool '$codemode.search'")
|
||||
},
|
||||
catch: (cause) => cause,
|
||||
}),
|
||||
{ index, name, input: request },
|
||||
)
|
||||
}
|
||||
|
||||
const tool = resolve(tools, path, searchEnabled)
|
||||
const tool = resolve(callableTools, path)
|
||||
let describedInput: unknown
|
||||
if (isDefinition(tool)) {
|
||||
if (externalArgs.length !== 1)
|
||||
|
||||
@@ -244,9 +244,9 @@ const renderSchema = (
|
||||
if (properties.length === 0 && indexType === undefined) return "{}"
|
||||
const pad = " ".repeat(depth + 1)
|
||||
const lines = properties.map(
|
||||
(entry) => `${jsdoc(entry[1].description, docTags(entry[1]), pad)}${pad}${field(entry)}`,
|
||||
(entry) => `${jsdoc(entry[1].description, docTags(entry[1]), pad)}${pad}${field(entry)},`,
|
||||
)
|
||||
if (indexType !== undefined) lines.push(`${pad}[key: string]: ${indexType}`)
|
||||
if (indexType !== undefined) lines.push(`${pad}[key: string]: ${indexType},`)
|
||||
return `{\n${lines.join("\n")}\n${" ".repeat(depth)}}`
|
||||
}
|
||||
return "unknown"
|
||||
|
||||
@@ -426,7 +426,7 @@ describe("CodeMode schema flexibility", () => {
|
||||
{
|
||||
path: "adapter.call",
|
||||
description: "Call an adapter-described tool",
|
||||
signature: "tools.adapter.call(input: { id: string; count?: number }): Promise<unknown>",
|
||||
signature: "tools.adapter.call(input: {\n id: string,\n count?: number,\n}): Promise<unknown>",
|
||||
},
|
||||
])
|
||||
|
||||
@@ -459,7 +459,7 @@ describe("CodeMode schema flexibility", () => {
|
||||
{
|
||||
path: "users.lookup",
|
||||
description: "Look up a user",
|
||||
signature: "tools.users.lookup(input: { login: string }): Promise<{ login: string; id: number }>",
|
||||
signature: "tools.users.lookup(input: {\n login: string,\n}): Promise<{\n login: string,\n id: number,\n}>",
|
||||
},
|
||||
])
|
||||
|
||||
@@ -475,7 +475,7 @@ describe("CodeMode schema flexibility", () => {
|
||||
run: () => Effect.succeed("pong"),
|
||||
})
|
||||
const runtime = CodeMode.make({ tools: { net: { ping } } })
|
||||
expect(runtime.catalog()[0]?.signature).toBe("tools.net.ping(input: { host: string }): Promise<unknown>")
|
||||
expect(runtime.catalog()[0]?.signature).toBe("tools.net.ping(input: {\n host: string,\n}): Promise<unknown>")
|
||||
|
||||
const result = await Effect.runPromise(runtime.execute(`return await tools.net.ping({ host: "example.test" })`))
|
||||
expect(result.ok).toBe(true)
|
||||
@@ -512,19 +512,19 @@ describe("CodeMode public contract", () => {
|
||||
{
|
||||
path: "orders.lookup",
|
||||
description: "Look up an order by ID",
|
||||
signature: "tools.orders.lookup(input: { id: string }): Promise<{ id: string; status: string }>",
|
||||
signature: "tools.orders.lookup(input: {\n id: string,\n}): Promise<{\n id: string,\n status: string,\n}>",
|
||||
},
|
||||
])
|
||||
expect(runtime.instructions()).toContain("Available tools (COMPLETE list")
|
||||
expect(runtime.instructions()).toContain("- orders (1 tool)")
|
||||
expect(runtime.instructions()).toContain(
|
||||
" - tools.orders.lookup(input: { id: string }): Promise<{ id: string; status: string }> // Look up an order by ID",
|
||||
" - tools.orders.lookup(input: {\n id: string,\n}): Promise<{\n id: string,\n status: string,\n}> // Look up an order by ID",
|
||||
)
|
||||
// A fully inlined catalog does not advertise search in the instructions...
|
||||
expect(runtime.instructions()).not.toMatch(/\$codemode/)
|
||||
|
||||
// ...but the search tool stays registered, so a speculative call still works. Search
|
||||
// results carry the pretty multiline signature; the inline catalog stays compact.
|
||||
// ...but the search tool stays registered, so a speculative call still works with the
|
||||
// same signature as the inline catalog.
|
||||
const result = await Effect.runPromise(runtime.execute(`return await tools.$codemode.search({ query: "order" })`))
|
||||
expect(result.ok).toBe(true)
|
||||
if (result.ok) {
|
||||
@@ -533,10 +533,11 @@ describe("CodeMode public contract", () => {
|
||||
{
|
||||
path: "tools.orders.lookup",
|
||||
description: "Look up an order by ID",
|
||||
signature: "tools.orders.lookup(input: {\n id: string\n}): Promise<{\n id: string\n status: string\n}>",
|
||||
signature: "tools.orders.lookup(input: {\n id: string,\n}): Promise<{\n id: string,\n status: string,\n}>",
|
||||
},
|
||||
],
|
||||
total: 1,
|
||||
remaining: 0,
|
||||
next: null,
|
||||
})
|
||||
}
|
||||
})
|
||||
@@ -554,11 +555,11 @@ describe("CodeMode public contract", () => {
|
||||
{
|
||||
path: "context7.resolve-library-id",
|
||||
description: "Resolve a library ID",
|
||||
signature: 'tools.context7["resolve-library-id"](input: { libraryName: string }): Promise<string>',
|
||||
signature: 'tools.context7["resolve-library-id"](input: {\n libraryName: string,\n}): Promise<string>',
|
||||
},
|
||||
])
|
||||
expect(runtime.instructions()).toContain(
|
||||
'tools.context7["resolve-library-id"](input: { libraryName: string }): Promise<string>',
|
||||
'tools.context7["resolve-library-id"](input: {\n libraryName: string,\n}): Promise<string>',
|
||||
)
|
||||
|
||||
const search = await Effect.runPromise(
|
||||
@@ -571,10 +572,11 @@ describe("CodeMode public contract", () => {
|
||||
{
|
||||
path: 'tools.context7["resolve-library-id"]',
|
||||
description: "Resolve a library ID",
|
||||
signature: 'tools.context7["resolve-library-id"](input: {\n libraryName: string\n}): Promise<string>',
|
||||
signature: 'tools.context7["resolve-library-id"](input: {\n libraryName: string,\n}): Promise<string>',
|
||||
},
|
||||
],
|
||||
total: 1,
|
||||
remaining: 0,
|
||||
next: null,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -588,7 +590,7 @@ describe("CodeMode public contract", () => {
|
||||
runtime.execute(`return await tools.$codemode.search({ query: 'tools.context7["resolve-library-id"]' })`),
|
||||
)
|
||||
expect(exact.ok).toBe(true)
|
||||
if (exact.ok) expect((exact.value as { total: number }).total).toBe(1)
|
||||
if (exact.ok) expect(exact.value).toMatchObject({ remaining: 0, next: null })
|
||||
})
|
||||
|
||||
test("instructions use markdown sections with placeholder-only call forms", () => {
|
||||
@@ -597,23 +599,26 @@ describe("CodeMode public contract", () => {
|
||||
// Sections in order: workflow at the top, catalog at the bottom.
|
||||
expect(instructions).toContain("## Workflow")
|
||||
expect(instructions).toContain("## Rules")
|
||||
expect(instructions).toContain("## Syntax")
|
||||
expect(instructions).toContain("## Language")
|
||||
expect(instructions.indexOf("## Workflow")).toBeLessThan(instructions.indexOf("## Rules"))
|
||||
expect(instructions.indexOf("## Rules")).toBeLessThan(instructions.indexOf("## Syntax"))
|
||||
expect(instructions.indexOf("## Syntax")).toBeLessThan(instructions.indexOf("\n## Available tools (COMPLETE list"))
|
||||
// The workflow carries the result-shape guidance; Rules only add content beyond it.
|
||||
expect(instructions).toContain(
|
||||
'`const data = typeof res === "string" ? JSON.parse(res) : res` - most tools return JSON as a string',
|
||||
expect(instructions.indexOf("## Rules")).toBeLessThan(instructions.indexOf("## Language"))
|
||||
expect(instructions.indexOf("## Language")).toBeLessThan(
|
||||
instructions.indexOf("\n## Available tools (COMPLETE list"),
|
||||
)
|
||||
expect(instructions).not.toContain("JSON.parse(res)")
|
||||
expect(instructions).toContain("Return only the fields you need")
|
||||
expect(instructions).toContain("raw payloads get truncated and waste context")
|
||||
expect(instructions).toContain("avoid returning large raw payloads")
|
||||
expect(instructions).toContain("Do not infer or normalize tool names")
|
||||
expect(instructions).toContain("bracket notation and quotes are part of the path")
|
||||
expect(instructions).toContain("surrounding agent tools are not available unless listed here")
|
||||
expect(instructions).toContain("Only tools listed here are available inside `tools`")
|
||||
expect(instructions).toContain("surrounding agent tools are not available")
|
||||
expect(instructions).toContain("Only Code Mode tools listed here and internal runtime tools")
|
||||
// Placeholders use generic namespace/tool/field names only - no fabricated real tools
|
||||
// and no real catalog tools cherry-picked into example lines.
|
||||
expect(instructions).toContain("`return { <field>: data.<field> }`")
|
||||
expect(instructions).toContain("`const result = await tools.<namespace>.<tool>(input)`")
|
||||
expect(instructions).toContain("Return only the fields you need from structured results")
|
||||
expect(instructions).toContain("check that it is a non-null object and not an array")
|
||||
expect(instructions).not.toContain("result.<field>")
|
||||
expect(instructions).not.toContain("data.<field>")
|
||||
expect(instructions).not.toContain("total_count")
|
||||
expect(instructions).not.toContain("list_issues")
|
||||
expect(instructions).not.toContain("tools.orders.lookup({")
|
||||
@@ -621,36 +626,35 @@ describe("CodeMode public contract", () => {
|
||||
expect(instructions).toContain("1. Pick a tool from the list under `## Available tools`")
|
||||
expect(instructions).not.toContain("Browse one namespace")
|
||||
|
||||
const partial = CodeMode.make({ tools, discovery: { maxInlineCatalogTokens: 0 } }).instructions()
|
||||
const partial = CodeMode.make({ tools, discovery: { catalogBudget: 0 } }).instructions()
|
||||
// PARTIAL: the workflow starts with search (with query-style guidance that is clearly
|
||||
// a query string, never a tool name) and the browse-namespace rule appears.
|
||||
expect(partial).toContain(
|
||||
'1. If the exact signature is not listed below, first search: `const { items } = await tools.$codemode.search({ query: "<intent + key nouns>" })`.',
|
||||
'1. If needed, discover tools: `return await tools.$codemode.search({ query: "<intent + key nouns>" })`.',
|
||||
)
|
||||
expect(partial).toContain("In the next execution, copy a returned path exactly")
|
||||
expect(partial).toContain(
|
||||
"Only tools listed here or returned by `tools.$codemode.search` are available inside `tools`",
|
||||
"Only Code Mode tools listed here or returned by `tools.$codemode.search` and internal runtime tools",
|
||||
)
|
||||
expect(partial).toContain(
|
||||
'- Browse one namespace: `await tools.$codemode.search({ query: "", namespace: "<name>" })`.',
|
||||
)
|
||||
expect(partial).toContain("repeat the same search with `offset: next.offset`")
|
||||
expect(partial).toContain(" limit?: number,\n offset?: number,")
|
||||
expect(partial).not.toContain("total_count")
|
||||
expect(partial).not.toContain("tools.orders.lookup({")
|
||||
})
|
||||
|
||||
test("the syntax section names what is unusual or missing, not an allowlist", () => {
|
||||
test("the language section describes the restricted runtime without overclaiming", () => {
|
||||
const instructions = CodeMode.make({ tools }).instructions()
|
||||
// Models already know JavaScript; the section leads with that.
|
||||
expect(instructions).toContain("Standard modern JavaScript works")
|
||||
expect(instructions).toContain("TypeScript type annotations are allowed and stripped before execution")
|
||||
// The not-supported list is derived from (and verified against) the interpreter.
|
||||
expect(instructions).toContain("Not supported")
|
||||
for (const missing of ["classes", "generators", "for await...of", ".then/.catch/.finally"]) {
|
||||
expect(instructions).toContain("restricted JavaScript language for calling tools")
|
||||
expect(instructions).toContain("not a general-purpose runtime")
|
||||
expect(instructions).not.toContain("Standard modern JavaScript works")
|
||||
expect(instructions).not.toContain("TypeScript type annotations")
|
||||
for (const missing of ["Modules/imports", "classes", "generators", "fetch", "promise chaining"]) {
|
||||
expect(instructions).toContain(missing)
|
||||
}
|
||||
// Implemented by the DSL-expansion pass, so no longer listed as missing.
|
||||
expect(instructions).not.toContain("instanceof Error")
|
||||
expect(instructions).not.toContain("splice")
|
||||
// The data-boundary note survives.
|
||||
expect(instructions).toContain("Use Code Mode tools for external operations")
|
||||
expect(instructions).toContain(
|
||||
"Dates serialize to ISO strings at data boundaries; Map/Set/RegExp serialize to `{}`.",
|
||||
)
|
||||
@@ -660,7 +664,7 @@ describe("CodeMode public contract", () => {
|
||||
const runtime = CodeMode.make({})
|
||||
const instructions = runtime.instructions()
|
||||
expect(instructions).toContain("No tools are currently available.")
|
||||
expect(instructions).toContain("## Syntax")
|
||||
expect(instructions).toContain("## Language")
|
||||
expect(instructions).toContain("## Available tools")
|
||||
expect(instructions).not.toContain("## Workflow")
|
||||
expect(instructions).not.toContain("## Rules")
|
||||
@@ -682,7 +686,7 @@ describe("CodeMode public contract", () => {
|
||||
})
|
||||
const runtime = CodeMode.make({
|
||||
tools: { thread: { uploadFile: upload, generateImage: generate }, orders: { lookup } },
|
||||
discovery: { maxInlineCatalogTokens: 0 },
|
||||
discovery: { catalogBudget: 0 },
|
||||
})
|
||||
expect(runtime.instructions()).toContain(
|
||||
"Available tools (PARTIAL - 0 of 3 shown; find the rest with tools.$codemode.search)",
|
||||
@@ -707,15 +711,16 @@ describe("CodeMode public contract", () => {
|
||||
{
|
||||
path: "tools.thread.uploadFile",
|
||||
description: "Upload one readable local file to the current Discord thread",
|
||||
signature: "tools.thread.uploadFile(input: {\n path: string\n}): Promise<{\n sent: boolean\n}>",
|
||||
signature: "tools.thread.uploadFile(input: {\n path: string,\n}): Promise<{\n sent: boolean,\n}>",
|
||||
},
|
||||
{
|
||||
path: "tools.thread.generateImage",
|
||||
description: "Generate an image and upload it to the current Discord thread",
|
||||
signature: "tools.thread.generateImage(input: {\n prompt: string\n}): Promise<{\n sent: boolean\n}>",
|
||||
signature: "tools.thread.generateImage(input: {\n prompt: string,\n}): Promise<{\n sent: boolean,\n}>",
|
||||
},
|
||||
],
|
||||
total: 2,
|
||||
remaining: 0,
|
||||
next: null,
|
||||
})
|
||||
expect(result.toolCalls).toStrictEqual([{ name: "$codemode.search" }])
|
||||
|
||||
@@ -761,9 +766,14 @@ describe("CodeMode public contract", () => {
|
||||
const browse = await Effect.runPromise(runtime.execute(`return await tools.$codemode.search({})`))
|
||||
expect(browse.ok).toBe(true)
|
||||
if (browse.ok) {
|
||||
const value = browse.value as { items: Array<{ path: string }>; total: number }
|
||||
const value = browse.value as {
|
||||
items: Array<{ path: string }>
|
||||
remaining: number
|
||||
next: { offset: number } | null
|
||||
}
|
||||
expect(value.items).toHaveLength(10)
|
||||
expect(value.total).toBe(14)
|
||||
expect(value.remaining).toBe(4)
|
||||
expect(value.next).toStrictEqual({ offset: 10 })
|
||||
}
|
||||
|
||||
for (const query of ["many.tool13", "tools.many.tool13"]) {
|
||||
@@ -777,10 +787,11 @@ describe("CodeMode public contract", () => {
|
||||
{
|
||||
path: "tools.many.tool13",
|
||||
description: "Numbered tool 13",
|
||||
signature: "tools.many.tool13(input: {\n id: string\n}): Promise<string>",
|
||||
signature: "tools.many.tool13(input: {\n id: string,\n}): Promise<string>",
|
||||
},
|
||||
],
|
||||
total: 1,
|
||||
remaining: 0,
|
||||
next: null,
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -807,8 +818,8 @@ describe("CodeMode public contract", () => {
|
||||
)
|
||||
expect(browse.ok).toBe(true)
|
||||
if (browse.ok) {
|
||||
const value = browse.value as { items: Array<{ path: string }>; total: number }
|
||||
expect(value.total).toBe(2)
|
||||
const value = browse.value as { items: Array<{ path: string }>; remaining: number }
|
||||
expect(value.remaining).toBe(0)
|
||||
expect(value.items.map((item) => item.path)).toStrictEqual([
|
||||
"tools.github.create_issue",
|
||||
"tools.github.list_issues",
|
||||
@@ -821,8 +832,8 @@ describe("CodeMode public contract", () => {
|
||||
)
|
||||
expect(scoped.ok).toBe(true)
|
||||
if (scoped.ok) {
|
||||
const value = scoped.value as { items: Array<{ path: string }>; total: number }
|
||||
expect(value.total).toBe(1)
|
||||
const value = scoped.value as { items: Array<{ path: string }>; remaining: number }
|
||||
expect(value.remaining).toBe(0)
|
||||
expect(value.items[0]?.path).toBe("tools.linear.list_issues")
|
||||
}
|
||||
|
||||
@@ -858,8 +869,8 @@ describe("CodeMode public contract", () => {
|
||||
)
|
||||
expect(byParameter.ok).toBe(true)
|
||||
if (byParameter.ok) {
|
||||
const value = byParameter.value as { items: Array<{ path: string }>; total: number }
|
||||
expect(value.total).toBe(1)
|
||||
const value = byParameter.value as { items: Array<{ path: string }>; remaining: number }
|
||||
expect(value.remaining).toBe(0)
|
||||
expect(value.items[0]?.path).toBe("tools.files.upload")
|
||||
}
|
||||
|
||||
@@ -869,8 +880,8 @@ describe("CodeMode public contract", () => {
|
||||
)
|
||||
expect(bySubstring.ok).toBe(true)
|
||||
if (bySubstring.ok) {
|
||||
const value = bySubstring.value as { items: Array<{ path: string }>; total: number }
|
||||
expect(value.total).toBe(1)
|
||||
const value = bySubstring.value as { items: Array<{ path: string }>; remaining: number }
|
||||
expect(value.remaining).toBe(0)
|
||||
expect(value.items[0]?.path).toBe("tools.files.upload")
|
||||
}
|
||||
})
|
||||
@@ -898,8 +909,8 @@ describe("CodeMode public contract", () => {
|
||||
)
|
||||
expect(plural.ok).toBe(true)
|
||||
if (plural.ok) {
|
||||
const value = plural.value as { items: Array<{ path: string }>; total: number }
|
||||
expect(value.total).toBe(1)
|
||||
const value = plural.value as { items: Array<{ path: string }>; remaining: number }
|
||||
expect(value.remaining).toBe(0)
|
||||
expect(value.items[0]?.path).toBe("tools.tracker.fetch_all")
|
||||
}
|
||||
|
||||
@@ -907,8 +918,8 @@ describe("CodeMode public contract", () => {
|
||||
const ranked = await Effect.runPromise(runtime.execute(`return await tools.$codemode.search({ query: "issues" })`))
|
||||
expect(ranked.ok).toBe(true)
|
||||
if (ranked.ok) {
|
||||
const value = ranked.value as { items: Array<{ path: string }>; total: number }
|
||||
expect(value.total).toBe(2)
|
||||
const value = ranked.value as { items: Array<{ path: string }>; remaining: number }
|
||||
expect(value.remaining).toBe(0)
|
||||
expect(value.items.map((item) => item.path)).toStrictEqual([
|
||||
"tools.github.list_issues",
|
||||
"tools.tracker.fetch_all",
|
||||
@@ -934,13 +945,33 @@ describe("CodeMode public contract", () => {
|
||||
const browse = await Effect.runPromise(runtime.execute(`return await tools.$codemode.search({})`))
|
||||
expect(browse.ok).toBe(true)
|
||||
if (browse.ok) {
|
||||
const value = browse.value as { items: Array<{ path: string }>; total: number }
|
||||
const value = browse.value as { items: Array<{ path: string }>; remaining: number; next: unknown }
|
||||
expect(value.items.map((item) => item.path)).toStrictEqual([
|
||||
"tools.alpha.aardvark",
|
||||
"tools.alpha.beta",
|
||||
"tools.zeta.last",
|
||||
])
|
||||
expect(value.remaining).toBe(0)
|
||||
expect(value.next).toBeNull()
|
||||
}
|
||||
|
||||
const middle = await Effect.runPromise(
|
||||
runtime.execute(`return await tools.$codemode.search({ limit: 1, offset: 1 })`),
|
||||
)
|
||||
expect(middle.ok).toBe(true)
|
||||
if (middle.ok) {
|
||||
expect(middle.value).toMatchObject({
|
||||
items: [{ path: "tools.alpha.beta" }],
|
||||
remaining: 1,
|
||||
next: { offset: 2 },
|
||||
})
|
||||
}
|
||||
|
||||
const exhausted = await Effect.runPromise(
|
||||
runtime.execute(`return await tools.$codemode.search({ limit: 1, offset: 3 })`),
|
||||
)
|
||||
expect(exhausted.ok).toBe(true)
|
||||
if (exhausted.ok) expect(exhausted.value).toStrictEqual({ items: [], remaining: 0, next: null })
|
||||
})
|
||||
|
||||
test("inlines round-robin across namespaces so one expensive namespace cannot starve the rest", () => {
|
||||
@@ -965,7 +996,7 @@ describe("CodeMode public contract", () => {
|
||||
// other namespaces from inlining (beta already got its line in the same round).
|
||||
const runtime = CodeMode.make({
|
||||
tools: { alpha: { cheap, expensive }, beta: { cheap } },
|
||||
discovery: { maxInlineCatalogTokens: 40 },
|
||||
discovery: { catalogBudget: 40 },
|
||||
})
|
||||
|
||||
const instructions = runtime.instructions()
|
||||
@@ -973,14 +1004,38 @@ describe("CodeMode public contract", () => {
|
||||
"Available tools (PARTIAL - 2 of 3 shown; find the rest with tools.$codemode.search)",
|
||||
)
|
||||
expect(instructions).toContain("- alpha (2 tools, 1 shown)")
|
||||
expect(instructions).toContain(" - tools.alpha.cheap(input: { q: string }): Promise<string> // Cheap")
|
||||
expect(instructions).toContain(" - tools.alpha.cheap(input: {\n q: string,\n}): Promise<string> // Cheap")
|
||||
expect(instructions).not.toContain("tools.alpha.expensive(")
|
||||
// Fully shown namespaces read cleanly (no "shown" annotation).
|
||||
expect(instructions).toContain("- beta (1 tool)")
|
||||
expect(instructions).toContain(" - tools.beta.cheap(input: { q: string }): Promise<string> // Cheap")
|
||||
expect(instructions).toContain(" - tools.beta.cheap(input: {\n q: string,\n}): Promise<string> // Cheap")
|
||||
expect(instructions).toMatch(/\$codemode\.search/)
|
||||
})
|
||||
|
||||
test("charges inline JSDoc against the catalog token budget", () => {
|
||||
const documented = Tool.make({
|
||||
description: "Look up a record",
|
||||
input: {
|
||||
type: "object",
|
||||
properties: {
|
||||
id: { type: "string", description: "A detailed identifier description. ".repeat(20) },
|
||||
},
|
||||
required: ["id"],
|
||||
} as const,
|
||||
run: () => Effect.succeed("ok"),
|
||||
})
|
||||
const runtime = CodeMode.make({
|
||||
tools: { records: { lookup: documented } },
|
||||
discovery: { catalogBudget: 40 },
|
||||
})
|
||||
|
||||
expect(runtime.catalog()[0]?.signature).toContain("/** A detailed identifier description.")
|
||||
expect(runtime.instructions()).toContain(
|
||||
"Available tools (PARTIAL - 0 of 1 shown; find the rest with tools.$codemode.search)",
|
||||
)
|
||||
expect(runtime.instructions()).not.toContain("tools.records.lookup(input:")
|
||||
})
|
||||
|
||||
test("decodes tool input and output before exposing either side", async () => {
|
||||
const observed: Array<unknown> = []
|
||||
const transformed = Tool.make({
|
||||
@@ -1031,17 +1086,27 @@ describe("CodeMode public contract", () => {
|
||||
expect(() => CodeMode.execute({ code: "return 1", limits: { maxToolCalls: -1 } })).toThrow(RangeError)
|
||||
expect(() => CodeMode.execute({ code: "return 1", limits: { maxOutputBytes: -1 } })).toThrow(RangeError)
|
||||
|
||||
expect(() => CodeMode.make({ tools, discovery: { maxInlineCatalogTokens: -1 } })).toThrow(RangeError)
|
||||
expect(() => CodeMode.make({ tools, discovery: { catalogBudget: -1 } })).toThrow(RangeError)
|
||||
|
||||
const result = await Effect.runPromise(
|
||||
CodeMode.make({
|
||||
tools,
|
||||
discovery: { maxInlineCatalogTokens: 0 },
|
||||
discovery: { catalogBudget: 0 },
|
||||
}).execute(`return await tools.$codemode.search({ query: "order", limit: 0.5 })`),
|
||||
)
|
||||
expect(result.ok).toBe(false)
|
||||
if (result.ok) return
|
||||
expect(result.error.kind).toBe("InvalidToolInput")
|
||||
|
||||
for (const offset of [-1, 0.5, Number.MAX_SAFE_INTEGER + 1, "1"]) {
|
||||
const invalidOffset = await Effect.runPromise(
|
||||
CodeMode.make({ tools }).execute(
|
||||
`return await tools.$codemode.search({ query: "order", offset: ${JSON.stringify(offset)} })`,
|
||||
),
|
||||
)
|
||||
expect(invalidOffset.ok).toBe(false)
|
||||
if (!invalidOffset.ok) expect(invalidOffset.error.kind).toBe("InvalidToolInput")
|
||||
}
|
||||
})
|
||||
|
||||
test("enforces the tool-call limit as a diagnostic", async () => {
|
||||
|
||||
@@ -41,7 +41,7 @@ describe("Object.keys over tool references", () => {
|
||||
const namespaces = Object.keys(tools)
|
||||
return { namespaces, count: namespaces.length }
|
||||
`),
|
||||
).toEqual({ namespaces: ["github", "memory", "playwright"], count: 3 })
|
||||
).toEqual({ namespaces: ["github", "memory", "playwright", "$codemode"], count: 4 })
|
||||
})
|
||||
|
||||
test("enumerates tool names at a nested namespace", async () => {
|
||||
@@ -52,7 +52,7 @@ describe("Object.keys over tool references", () => {
|
||||
expect(await value(`return Object.keys(tools.github.list_issues)`)).toEqual([])
|
||||
})
|
||||
|
||||
test("the virtual discovery namespace enumerates its callable surface", async () => {
|
||||
test("the internal discovery namespace enumerates its callable surface", async () => {
|
||||
expect(await value(`return Object.keys(tools.$codemode)`)).toEqual(["search"])
|
||||
})
|
||||
|
||||
@@ -137,7 +137,7 @@ describe("for...in", () => {
|
||||
).toBe("only")
|
||||
})
|
||||
|
||||
test("enumerates namespaces and tools from the host tool tree", async () => {
|
||||
test("enumerates namespaces and tools from the callable tool tree", async () => {
|
||||
expect(
|
||||
await value(`
|
||||
const names = []
|
||||
@@ -146,7 +146,13 @@ describe("for...in", () => {
|
||||
}
|
||||
return names
|
||||
`),
|
||||
).toEqual(["github.list_issues", "github.get_issue", "memory.search", "playwright.navigate"])
|
||||
).toEqual([
|
||||
"github.list_issues",
|
||||
"github.get_issue",
|
||||
"memory.search",
|
||||
"playwright.navigate",
|
||||
"$codemode.search",
|
||||
])
|
||||
})
|
||||
|
||||
test("unsupported values fail with a hint at for...of and Object.keys", async () => {
|
||||
|
||||
@@ -40,21 +40,21 @@ describe("pretty signature rendering", () => {
|
||||
[
|
||||
"{",
|
||||
" /** Repository owner */",
|
||||
" owner: string",
|
||||
" owner: string,",
|
||||
" /** Cursor from the previous response's pageInfo */",
|
||||
" after?: string",
|
||||
" after?: string,",
|
||||
" /**",
|
||||
" * Results per page",
|
||||
" * @default 30",
|
||||
" */",
|
||||
" perPage?: number",
|
||||
" perPage?: number,",
|
||||
" /**",
|
||||
" * Filter by labels",
|
||||
" * @minItems 1",
|
||||
" * @maxItems 10",
|
||||
" */",
|
||||
" labels?: Array<string>",
|
||||
' state?: "open" | "closed"',
|
||||
" labels?: Array<string>,",
|
||||
' state?: "open" | "closed",',
|
||||
"}",
|
||||
].join("\n"),
|
||||
)
|
||||
@@ -83,7 +83,7 @@ describe("pretty signature rendering", () => {
|
||||
true,
|
||||
)
|
||||
expect(pretty).toBe(
|
||||
["{", " /** Search filter */", " filter?: {", " /** Issue state */", " state?: string", " }", "}"].join(
|
||||
["{", " /** Search filter */", " filter?: {", " /** Issue state */", " state?: string,", " },", "}"].join(
|
||||
"\n",
|
||||
),
|
||||
)
|
||||
@@ -91,10 +91,10 @@ describe("pretty signature rendering", () => {
|
||||
|
||||
test("Effect Schema annotations become JSDoc on input and output fields", () => {
|
||||
expect(inputTypeScript(lookupOrder, true)).toBe(
|
||||
["{", " /** Order identifier */", " id: string", " verbose?: boolean", "}"].join("\n"),
|
||||
["{", " /** Order identifier */", " id: string,", " verbose?: boolean,", "}"].join("\n"),
|
||||
)
|
||||
expect(outputTypeScript(lookupOrder, true)).toBe(
|
||||
["{", " /** Current order status */", " status: string", "}"].join("\n"),
|
||||
["{", " /** Current order status */", " status: string,", "}"].join("\n"),
|
||||
)
|
||||
})
|
||||
|
||||
@@ -129,7 +129,7 @@ describe("pretty signature rendering", () => {
|
||||
{ type: "object", properties: { size: { type: "number", default: 1n } } },
|
||||
true,
|
||||
)
|
||||
expect(pretty).toBe(["{", " size?: number", "}"].join("\n"))
|
||||
expect(pretty).toBe(["{", " size?: number,", "}"].join("\n"))
|
||||
})
|
||||
|
||||
test("neutralizes */ inside descriptions so nothing closes the comment early", () => {
|
||||
@@ -150,7 +150,7 @@ describe("pretty signature rendering", () => {
|
||||
true,
|
||||
)
|
||||
expect(pretty).toBe(
|
||||
["{", " /**", " * First line", " *", " * Second line", " */", " query?: string", "}"].join("\n"),
|
||||
["{", " /**", " * First line", " *", " * Second line", " */", " query?: string,", "}"].join("\n"),
|
||||
)
|
||||
})
|
||||
|
||||
@@ -235,12 +235,12 @@ describe("non-identifier property names render as quoted keys", () => {
|
||||
expect(jsonSchemaToTypeScript(rawSchema, true)).toBe(
|
||||
[
|
||||
"{",
|
||||
' "123"?: number',
|
||||
' "foo-bar"?: string',
|
||||
' "@type": string',
|
||||
' "123"?: number,',
|
||||
' "foo-bar"?: string,',
|
||||
' "@type": string,',
|
||||
" /** Dotted name */",
|
||||
' "x.y"?: number',
|
||||
" plain?: boolean",
|
||||
' "x.y"?: number,',
|
||||
" plain?: boolean,",
|
||||
"}",
|
||||
].join("\n"),
|
||||
)
|
||||
@@ -259,7 +259,7 @@ describe("non-identifier property names render as quoted keys", () => {
|
||||
})
|
||||
expect(inputTypeScript(tool)).toContain('"foo-bar"?: string')
|
||||
expect(outputTypeScript(tool)).toBe('{ "content-type": string }')
|
||||
expect(outputTypeScript(tool, true)).toBe(["{", ' "content-type": string', "}"].join("\n"))
|
||||
expect(outputTypeScript(tool, true)).toBe(["{", ' "content-type": string,', "}"].join("\n"))
|
||||
})
|
||||
|
||||
test("Effect Schema structs with non-identifier field names quote too", () => {
|
||||
@@ -269,7 +269,7 @@ describe("non-identifier property names render as quoted keys", () => {
|
||||
run: () => Effect.succeed(null),
|
||||
})
|
||||
expect(inputTypeScript(tool)).toBe('{ "foo-bar": string; plain?: number }')
|
||||
expect(inputTypeScript(tool, true)).toBe(["{", ' "foo-bar": string', " plain?: number", "}"].join("\n"))
|
||||
expect(inputTypeScript(tool, true)).toBe(["{", ' "foo-bar": string,', " plain?: number,", "}"].join("\n"))
|
||||
})
|
||||
})
|
||||
|
||||
@@ -332,7 +332,7 @@ describe("union schemas render every alternative", () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe("pretty signatures in search results", () => {
|
||||
describe("JSDoc signatures in catalogs and search results", () => {
|
||||
const runtime = CodeMode.make({ tools: { github: { list_issues: listIssues }, orders: { lookup: lookupOrder } } })
|
||||
|
||||
const search = async (query: string) => {
|
||||
@@ -341,7 +341,7 @@ describe("pretty signatures in search results", () => {
|
||||
)
|
||||
expect(result.ok).toBe(true)
|
||||
if (!result.ok) throw new Error("search failed")
|
||||
return result.value as { items: Array<{ path: string; signature: string }>; total: number }
|
||||
return result.value as { items: Array<{ path: string; signature: string }>; remaining: number }
|
||||
}
|
||||
|
||||
test("a raw JSON Schema (MCP-style) tool's result signature carries field JSDoc and tags", async () => {
|
||||
@@ -351,21 +351,21 @@ describe("pretty signatures in search results", () => {
|
||||
[
|
||||
"tools.github.list_issues(input: {",
|
||||
" /** Repository owner */",
|
||||
" owner: string",
|
||||
" owner: string,",
|
||||
" /** Cursor from the previous response's pageInfo */",
|
||||
" after?: string",
|
||||
" after?: string,",
|
||||
" /**",
|
||||
" * Results per page",
|
||||
" * @default 30",
|
||||
" */",
|
||||
" perPage?: number",
|
||||
" perPage?: number,",
|
||||
" /**",
|
||||
" * Filter by labels",
|
||||
" * @minItems 1",
|
||||
" * @maxItems 10",
|
||||
" */",
|
||||
" labels?: Array<string>",
|
||||
' state?: "open" | "closed"',
|
||||
" labels?: Array<string>,",
|
||||
' state?: "open" | "closed",',
|
||||
"}): Promise<unknown>",
|
||||
].join("\n"),
|
||||
)
|
||||
@@ -379,26 +379,26 @@ describe("pretty signatures in search results", () => {
|
||||
[
|
||||
"tools.orders.lookup(input: {",
|
||||
" /** Order identifier */",
|
||||
" id: string",
|
||||
" verbose?: boolean",
|
||||
" id: string,",
|
||||
" verbose?: boolean,",
|
||||
"}): Promise<{",
|
||||
" /** Current order status */",
|
||||
" status: string",
|
||||
" status: string,",
|
||||
"}>",
|
||||
].join("\n"),
|
||||
)
|
||||
}
|
||||
})
|
||||
|
||||
test("the inline catalog line for the same tool stays single-line compact", () => {
|
||||
test("the inline catalog uses the same JSDoc signatures", async () => {
|
||||
const instructions = runtime.instructions()
|
||||
expect(instructions).toContain(
|
||||
' - tools.github.list_issues(input: { owner: string; after?: string; perPage?: number; labels?: Array<string>; state?: "open" | "closed" }): Promise<unknown> // List issues in a repository',
|
||||
)
|
||||
expect(instructions).toContain(
|
||||
" - tools.orders.lookup(input: { id: string; verbose?: boolean }): Promise<{ status: string }> // Look up an order",
|
||||
)
|
||||
expect(instructions).not.toContain("/**")
|
||||
const github = (await search("list issues repository")).items.find(
|
||||
({ path }) => path === "tools.github.list_issues",
|
||||
)!
|
||||
const orders = (await search("look up order")).items.find(({ path }) => path === "tools.orders.lookup")!
|
||||
expect(instructions).toContain(` - ${github.signature} // List issues in a repository`)
|
||||
expect(instructions).toContain(` - ${orders.signature} // Look up an order`)
|
||||
expect(instructions).toContain("/** Repository owner */")
|
||||
})
|
||||
})
|
||||
|
||||
@@ -421,7 +421,7 @@ describe("non-identifier tool paths", () => {
|
||||
const instructions = runtime.instructions()
|
||||
|
||||
expect(instructions).toContain(
|
||||
'tools.context7["resolve-library-id"](input: { query: string; libraryName: string }): Promise<unknown>',
|
||||
'tools.context7["resolve-library-id"](input: {\n query: string,\n libraryName: string,\n}): Promise<unknown>',
|
||||
)
|
||||
expect(instructions).toContain("Do not infer or normalize tool names")
|
||||
expect(instructions).toContain("bracket notation and quotes are part of the path")
|
||||
|
||||
@@ -11,18 +11,11 @@ import { Plugin } from "@/plugin"
|
||||
|
||||
export const CODE_MODE_TOOL = "execute"
|
||||
|
||||
const DESCRIPTION = [
|
||||
"Execute a JavaScript/TypeScript program that orchestrates the connected MCP tools inside a confined runtime.",
|
||||
"The full usage guide and the catalog of available tools follow below.",
|
||||
].join("\n")
|
||||
const DESCRIPTION = "Run a confined orchestration script with access to connected MCP tools."
|
||||
|
||||
export const Parameters = Schema.Struct({
|
||||
code: Schema.String.annotate({
|
||||
description: [
|
||||
"JavaScript source to execute.",
|
||||
"Inside CodeMode, `tools` contains only the MCP/CodeMode tools listed in this execute tool's description; top-level opencode tools like bash, read, or lsp are not available unless listed there.",
|
||||
"Call available tools using the exact signatures shown in this execute tool's description, compose the results, and `return` the final value.",
|
||||
].join(" "),
|
||||
description: "Script body executed by the confined interpreter.",
|
||||
}),
|
||||
})
|
||||
|
||||
|
||||
@@ -177,8 +177,10 @@ describe("code mode integration (real MCP server)", () => {
|
||||
test("the appended catalog inlines full signatures with real MCP schemas", () => {
|
||||
expect(description).toContain("Available tools (COMPLETE list")
|
||||
expect(description).toContain("- fixtures (4 tools)")
|
||||
expect(description).toContain("tools.fixtures.add(input: { a: number; b: number }): Promise<{ sum: number }>")
|
||||
expect(description).toContain("tools.fixtures.get_text(input: { name: string }): Promise<unknown>")
|
||||
expect(description).toContain(
|
||||
"tools.fixtures.add(input: {\n a: number,\n b: number,\n}): Promise<{\n sum: number,\n}>",
|
||||
)
|
||||
expect(description).toContain("tools.fixtures.get_text(input: {\n name: string,\n}): Promise<unknown>")
|
||||
expect(description).toContain("// Add two numbers and return the structured sum")
|
||||
expect(description).not.toContain("$codemode")
|
||||
expect(description).toContain("## Workflow")
|
||||
|
||||
@@ -98,6 +98,13 @@ describe("code mode execute", () => {
|
||||
const decode = Schema.decodeUnknownEffect(Parameters)
|
||||
await expect(Effect.runPromise(decode({ code: "return 1" }))).resolves.toEqual({ code: "return 1" })
|
||||
await expect(Effect.runPromise(decode({}))).rejects.toThrow()
|
||||
expect(Schema.toJsonSchemaDocument(Parameters).schema).toMatchObject({
|
||||
properties: {
|
||||
code: {
|
||||
description: "Script body executed by the confined interpreter.",
|
||||
},
|
||||
},
|
||||
})
|
||||
})
|
||||
|
||||
test("groups multi-underscore server names by longest matching prefix", () => {
|
||||
@@ -124,13 +131,13 @@ describe("code mode execute", () => {
|
||||
},
|
||||
["weather"],
|
||||
)
|
||||
expect(description).toContain("tools.weather.current(input: { city: string }): Promise<{ tempC: number }>")
|
||||
expect(description).toContain("tools.weather.current(input: {\n city: string,\n}): Promise<{\n tempC: number,\n}>")
|
||||
})
|
||||
|
||||
test("the static base description carries no catalog; the registry appends it", async () => {
|
||||
const tool = await build({ github_list_issues: mcpTool("list_issues", () => "") })
|
||||
expect(tool.id).toBe(CODE_MODE_TOOL)
|
||||
expect(tool.description).toContain("confined runtime")
|
||||
expect(tool.description).toBe("Run a confined orchestration script with access to connected MCP tools.")
|
||||
expect(tool.description).not.toContain("Available tools")
|
||||
expect(tool.description).not.toContain("list_issues")
|
||||
})
|
||||
@@ -150,7 +157,7 @@ describe("code mode execute", () => {
|
||||
expect(description).toContain("- github (2 tools)")
|
||||
expect(description).toContain("- linear (1 tool)")
|
||||
expect(description).toContain(
|
||||
"tools.github.create_issue(input: { title: string; body?: string }): Promise<unknown>",
|
||||
"tools.github.create_issue(input: {\n title: string,\n body?: string,\n}): Promise<unknown>",
|
||||
)
|
||||
expect(description).toContain("tools.github.list_issues(")
|
||||
expect(description).toContain("tools.linear.search(")
|
||||
@@ -159,9 +166,8 @@ describe("code mode execute", () => {
|
||||
expect(description).not.toContain("Browse one namespace")
|
||||
expect(description).toContain("## Workflow")
|
||||
expect(description).toContain("1. Pick a tool from the list under `## Available tools`")
|
||||
expect(description).toContain(
|
||||
'`const data = typeof res === "string" ? JSON.parse(res) : res` - most tools return JSON as a string',
|
||||
)
|
||||
expect(description).not.toContain("JSON.parse(res)")
|
||||
expect(description).toContain("check that it is a non-null object and not an array")
|
||||
expect(description).toContain("Return only the fields you need")
|
||||
expect(description).not.toContain("total_count")
|
||||
})
|
||||
@@ -180,7 +186,7 @@ describe("code mode execute", () => {
|
||||
),
|
||||
})
|
||||
expect(description).toContain(
|
||||
"tools.weather.current(input: { city: string }): Promise<{ tempC: number; summary?: string }>",
|
||||
"tools.weather.current(input: {\n city: string,\n}): Promise<{\n tempC: number,\n summary?: string,\n}>",
|
||||
)
|
||||
})
|
||||
|
||||
@@ -207,9 +213,16 @@ describe("code mode execute", () => {
|
||||
expect(description).toContain("Available tools (PARTIAL - ")
|
||||
expect(description).toMatch(/- alpha \(150 tools, \d+ shown\)/)
|
||||
expect(description).toContain("- zeta (1 tool)\n")
|
||||
expect(description).toContain("tools.zeta.only_tool(input: { topic: string }): Promise<unknown>")
|
||||
expect(description).toContain(
|
||||
"tools.zeta.only_tool(input: {\n /** Subject to look up */\n topic: string,\n}): Promise<unknown>",
|
||||
)
|
||||
expect(description).toContain("tools.$codemode.search(")
|
||||
expect(description).toContain("1. If the exact signature is not listed below, first search:")
|
||||
expect(description).toContain(" limit?: number,\n offset?: number,")
|
||||
expect(description).toContain(" remaining: number,\n next: {")
|
||||
expect(description).toContain(" offset: number,\n } | null,")
|
||||
expect(description).toContain(
|
||||
'1. If needed, discover tools: `return await tools.$codemode.search({ query: "<intent + key nouns>" })`.',
|
||||
)
|
||||
expect(description).toContain(
|
||||
'- Browse one namespace: `await tools.$codemode.search({ query: "", namespace: "<name>" })`.',
|
||||
)
|
||||
@@ -219,17 +232,18 @@ describe("code mode execute", () => {
|
||||
|
||||
const tool = await build(tools, ["alpha", "zeta"])
|
||||
const out = await Effect.runPromise(
|
||||
tool.execute({ code: "return await tools.$codemode.search({ query: 'only tool', limit: 3 })" }, ctx),
|
||||
tool.execute({ code: "return await tools.$codemode.search({ query: 'only tool', limit: 3, offset: 0 })" }, ctx),
|
||||
)
|
||||
const result = JSON.parse(out.output)
|
||||
expect(result.items.map((i: any) => i.path)).toContain("tools.zeta.only_tool")
|
||||
expect(result).toMatchObject({ remaining: 0, next: null })
|
||||
expect(result.items[0].signature).toContain("tools.")
|
||||
const signature = result.items.find((i: any) => i.path === "tools.zeta.only_tool").signature
|
||||
expect(signature).toContain("tools.zeta.only_tool(input: {\n")
|
||||
expect(signature).toContain(" /** Subject to look up */\n topic: string")
|
||||
expect(description).not.toContain("/**")
|
||||
expect(description).toContain("/** Subject to look up */")
|
||||
expect(out.metadata.toolCalls).toEqual([
|
||||
{ tool: "$codemode.search", status: "completed", input: { query: "only tool", limit: 3 } },
|
||||
{ tool: "$codemode.search", status: "completed", input: { query: "only tool", limit: 3, offset: 0 } },
|
||||
])
|
||||
})
|
||||
|
||||
@@ -240,7 +254,7 @@ describe("code mode execute", () => {
|
||||
expect(output.metadata.toolCalls).toEqual([])
|
||||
})
|
||||
|
||||
test("Object.keys(tools) enumerates the MCP server namespaces", async () => {
|
||||
test("Object.keys(tools) enumerates the MCP server and CodeMode namespaces", async () => {
|
||||
const tool = await build({
|
||||
github_list_issues: mcpTool("list_issues", () => ""),
|
||||
linear_search: mcpTool("search", () => ""),
|
||||
@@ -251,7 +265,7 @@ describe("code mode execute", () => {
|
||||
ctx,
|
||||
),
|
||||
)
|
||||
expect(JSON.parse(output.output)).toEqual({ namespaces: ["github", "linear"], count: 2 })
|
||||
expect(JSON.parse(output.output)).toEqual({ namespaces: ["github", "linear", "$codemode"], count: 3 })
|
||||
})
|
||||
|
||||
test("calls a namespaced MCP tool and flows its text result back into the program", async () => {
|
||||
|
||||
@@ -132,7 +132,7 @@ describe("tool.registry", () => {
|
||||
|
||||
expect(ids).toContain("execute")
|
||||
expect(tools.map((tool) => tool.id)).toContain("execute")
|
||||
expect(execute?.description).toContain("tools.weather.current(input: { city: string })")
|
||||
expect(execute?.description).toContain("tools.weather.current(input: {\n city: string,\n})")
|
||||
}),
|
||||
)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user