mirror of
https://github.com/openclaw/openclaw.git
synced 2026-07-21 10:16:44 +00:00
feat(nodes): add auto-discovered Ollama inference (#99234)
* feat(nodes): add local Ollama inference * fix(gateway): preserve plugin node runtime for agent turns * feat(ollama): add node inference opt-out * test(security): preserve plugin runtime exports * test(security): preserve plugin runtime exports * test(security): preserve plugin runtime exports * fix(ci): raise artifact build heap
This commit is contained in:
@@ -156,7 +156,7 @@ jobs:
|
||||
- name: Build dist on cache miss
|
||||
if: steps.dist-cache.outputs.cache-hit != 'true'
|
||||
env:
|
||||
NODE_OPTIONS: --max-old-space-size=8192
|
||||
NODE_OPTIONS: --max-old-space-size=12288
|
||||
run: pnpm build:ci-artifacts
|
||||
|
||||
- name: Build Control UI on cache miss
|
||||
|
||||
@@ -4712,6 +4712,7 @@ Do not edit it by hand; run `pnpm docs:map:gen`.
|
||||
- H3: Pair + name
|
||||
- H3: Allowlist the commands
|
||||
- H3: Point exec at the node
|
||||
- H3: Local model inference
|
||||
- H2: Invoking commands
|
||||
- H2: Command policy
|
||||
- H2: Config (openclaw.json)
|
||||
@@ -7613,6 +7614,7 @@ Do not edit it by hand; run `pnpm docs:map:gen`.
|
||||
- H2: Getting started
|
||||
- H2: Cloud models
|
||||
- H2: Model discovery (implicit provider)
|
||||
- H2: Node-local inference
|
||||
- H2: Vision and image description
|
||||
- H2: Configuration
|
||||
- H2: Common recipes
|
||||
|
||||
@@ -184,6 +184,14 @@ Related:
|
||||
- [Exec tool](/tools/exec)
|
||||
- [Exec approvals](/tools/exec-approvals)
|
||||
|
||||
### Local model inference
|
||||
|
||||
A desktop or server node can expose chat-capable models from an Ollama server
|
||||
running on that node. Agents use the Ollama plugin's `node_inference` tool to
|
||||
discover installed models and run a bounded prompt remotely; the Gateway does
|
||||
not need direct network access to Ollama. See [Ollama node-local inference](/providers/ollama#node-local-inference)
|
||||
for setup, model filtering, and direct verification commands.
|
||||
|
||||
## Invoking commands
|
||||
|
||||
Low-level (raw RPC):
|
||||
|
||||
@@ -304,6 +304,107 @@ The new model will be automatically discovered and available to use.
|
||||
If you set `models.providers.ollama` explicitly, or configure a custom remote provider such as `models.providers.ollama-cloud` with `api: "ollama"`, auto-discovery is skipped and you must define models manually. Loopback custom providers such as `http://127.0.0.2:11434` are still treated as local. See the explicit config section below.
|
||||
</Note>
|
||||
|
||||
## Node-local inference
|
||||
|
||||
Agents can delegate a short task to an Ollama model installed on a paired
|
||||
desktop or server node. The prompt and response cross the existing authenticated
|
||||
Gateway/node connection; the model request runs on the selected node against
|
||||
its standard loopback Ollama endpoint (`http://127.0.0.1:11434`).
|
||||
|
||||
<Steps>
|
||||
<Step title="Start Ollama on the node">
|
||||
Pull at least one chat model and keep Ollama running:
|
||||
|
||||
```bash
|
||||
ollama pull qwen3:0.6b
|
||||
ollama list
|
||||
```
|
||||
|
||||
</Step>
|
||||
<Step title="Connect the node host">
|
||||
On the same machine as Ollama, connect a node host to the Gateway:
|
||||
|
||||
```bash
|
||||
openclaw node run \
|
||||
--host <gateway-host> \
|
||||
--port 18789 \
|
||||
--display-name "Local inference"
|
||||
```
|
||||
|
||||
Approve the new device and its declared node commands on the Gateway host,
|
||||
then verify the node:
|
||||
|
||||
```bash
|
||||
openclaw devices list
|
||||
openclaw devices approve <deviceRequestId>
|
||||
openclaw nodes pending
|
||||
openclaw nodes approve <nodeRequestId>
|
||||
openclaw nodes status --connected
|
||||
```
|
||||
|
||||
A first connection and an upgrade that adds the Ollama commands can both
|
||||
trigger node-command approval. If the node connects without advertising
|
||||
`ollama.models` and `ollama.chat`, check `openclaw nodes pending` again.
|
||||
|
||||
</Step>
|
||||
<Step title="Ask an agent to use local inference">
|
||||
The bundled Ollama plugin exposes the `node_inference` tool. Agents first
|
||||
use `action: "discover"`, then `action: "run"` with a returned node and
|
||||
model. If exactly one capable node is connected, `run` can omit the node.
|
||||
|
||||
For example: “Discover the Ollama models on my nodes, then use the fastest
|
||||
loaded model to summarize this text.”
|
||||
|
||||
</Step>
|
||||
</Steps>
|
||||
|
||||
Discovery reads `/api/tags`, checks `/api/show` capabilities, and uses `/api/ps`
|
||||
when available to rank already-loaded models first. It returns only local
|
||||
chat-capable models: Ollama Cloud rows and embedding-only models are excluded.
|
||||
Each run asks Ollama to disable model thinking and caps output at 512 tokens
|
||||
unless the tool call requests a different `maxTokens` value. Some models, such
|
||||
as GPT-OSS, do not support disabling thinking and may still use reasoning tokens.
|
||||
|
||||
To keep Ollama running on a node without making it available to agents, set the
|
||||
following in the config used by that node host:
|
||||
|
||||
```bash
|
||||
openclaw config set plugins.entries.ollama.config.nodeInference.enabled false
|
||||
```
|
||||
|
||||
If the node uses the foreground `openclaw node run` command from the setup
|
||||
above, stop that process and run the command again. If it uses an installed node
|
||||
service, run `openclaw node restart`.
|
||||
|
||||
The node stops advertising `ollama.models` and `ollama.chat`; Ollama itself and
|
||||
the Gateway's Ollama provider remain unchanged. Set the value to `true` and
|
||||
restart the node to advertise local inference again. A changed command surface
|
||||
may require approval through `openclaw nodes pending` after reconnect.
|
||||
|
||||
You can verify the same node commands without an agent turn:
|
||||
|
||||
```bash
|
||||
openclaw nodes invoke \
|
||||
--node "Local inference" \
|
||||
--command ollama.models \
|
||||
--params '{}' \
|
||||
--invoke-timeout 90000 \
|
||||
--timeout 100000
|
||||
|
||||
openclaw nodes invoke \
|
||||
--node "Local inference" \
|
||||
--command ollama.chat \
|
||||
--params '{"model":"qwen3:0.6b","prompt":"Reply with exactly: pong","maxTokens":32,"timeoutMs":120000}' \
|
||||
--invoke-timeout 130000 \
|
||||
--timeout 140000
|
||||
```
|
||||
|
||||
Node-local inference intentionally does not reuse a remote or cloud
|
||||
`models.providers.ollama.baseUrl`. Start Ollama on the node's standard loopback
|
||||
endpoint. The node commands are available by default on macOS, Linux, and
|
||||
Windows node hosts and remain subject to the normal node pairing and command
|
||||
policy.
|
||||
|
||||
## Vision and image description
|
||||
|
||||
The bundled Ollama plugin registers Ollama as an image-capable media-understanding provider. This lets OpenClaw route explicit image-description requests and configured image-model defaults through local or hosted Ollama vision models.
|
||||
|
||||
@@ -178,6 +178,57 @@ function captureWrappedOllamaPayload(
|
||||
}
|
||||
|
||||
describe("ollama plugin", () => {
|
||||
it("registers node-local inference commands, policy, and agent tool", () => {
|
||||
const registerNodeHostCommand = vi.fn();
|
||||
const registerNodeInvokePolicy = vi.fn();
|
||||
const registerTool = vi.fn();
|
||||
|
||||
plugin.register(
|
||||
createTestPluginApi({
|
||||
id: "ollama",
|
||||
name: "Ollama",
|
||||
source: "test",
|
||||
registerNodeHostCommand,
|
||||
registerNodeInvokePolicy,
|
||||
registerTool,
|
||||
}),
|
||||
);
|
||||
|
||||
expect(registerNodeHostCommand.mock.calls.map(([entry]) => entry.command)).toEqual([
|
||||
"ollama.models",
|
||||
"ollama.chat",
|
||||
]);
|
||||
expect(registerNodeInvokePolicy).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
commands: ["ollama.models", "ollama.chat"],
|
||||
defaultPlatforms: ["macos", "linux", "windows"],
|
||||
}),
|
||||
);
|
||||
expect(registerTool).toHaveBeenCalledWith(expect.objectContaining({ name: "node_inference" }));
|
||||
});
|
||||
|
||||
it("keeps the agent tool but does not advertise node inference when disabled locally", () => {
|
||||
const registerNodeHostCommand = vi.fn();
|
||||
const registerNodeInvokePolicy = vi.fn();
|
||||
const registerTool = vi.fn();
|
||||
|
||||
plugin.register(
|
||||
createTestPluginApi({
|
||||
id: "ollama",
|
||||
name: "Ollama",
|
||||
source: "test",
|
||||
pluginConfig: { nodeInference: { enabled: false } },
|
||||
registerNodeHostCommand,
|
||||
registerNodeInvokePolicy,
|
||||
registerTool,
|
||||
}),
|
||||
);
|
||||
|
||||
expect(registerNodeHostCommand).not.toHaveBeenCalled();
|
||||
expect(registerNodeInvokePolicy).toHaveBeenCalledOnce();
|
||||
expect(registerTool).toHaveBeenCalledWith(expect.objectContaining({ name: "node_inference" }));
|
||||
});
|
||||
|
||||
it("does not preselect a default model during provider auth setup", async () => {
|
||||
const provider = registerProvider();
|
||||
|
||||
|
||||
@@ -58,6 +58,11 @@ import {
|
||||
} from "./src/embedding-provider.js";
|
||||
import { ollamaMediaUnderstandingProvider } from "./src/media-understanding-provider.js";
|
||||
import { ollamaMemoryEmbeddingProviderAdapter } from "./src/memory-embedding-adapter.js";
|
||||
import {
|
||||
createOllamaNodeHostCommands,
|
||||
createOllamaNodeInferenceTool,
|
||||
createOllamaNodeInvokePolicy,
|
||||
} from "./src/node-inference.js";
|
||||
import { readProviderBaseUrl } from "./src/provider-base-url.js";
|
||||
import {
|
||||
createConfiguredOllamaCompatStreamWrapper,
|
||||
@@ -435,12 +440,19 @@ export default definePluginEntry({
|
||||
name: "Ollama Provider",
|
||||
description: "Bundled Ollama provider plugin",
|
||||
register(api: OpenClawPluginApi) {
|
||||
const startupPluginConfig = (api.pluginConfig ?? {}) as OllamaPluginConfig;
|
||||
if (api.registrationMode === "full") {
|
||||
void checkWsl2CrashLoopRisk(api.logger);
|
||||
}
|
||||
api.registerMemoryEmbeddingProvider(ollamaMemoryEmbeddingProviderAdapter);
|
||||
api.registerMediaUnderstandingProvider(ollamaMediaUnderstandingProvider);
|
||||
const startupPluginConfig = (api.pluginConfig ?? {}) as OllamaPluginConfig;
|
||||
if (startupPluginConfig.nodeInference?.enabled !== false) {
|
||||
for (const command of createOllamaNodeHostCommands()) {
|
||||
api.registerNodeHostCommand(command);
|
||||
}
|
||||
}
|
||||
api.registerNodeInvokePolicy(createOllamaNodeInvokePolicy());
|
||||
api.registerTool(createOllamaNodeInferenceTool(api));
|
||||
const resolveCurrentPluginConfig = (config?: OpenClawConfig): OllamaPluginConfig => {
|
||||
const runtimePluginConfig = resolvePluginConfigObject(config, "ollama");
|
||||
if (runtimePluginConfig) {
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
"id": "ollama",
|
||||
"icon": "https://cdn.simpleicons.org/ollama",
|
||||
"activation": {
|
||||
"onStartup": false
|
||||
"onStartup": true
|
||||
},
|
||||
"enabledByDefault": true,
|
||||
"providers": ["ollama", "ollama-cloud"],
|
||||
@@ -155,6 +155,7 @@
|
||||
},
|
||||
"contracts": {
|
||||
"memoryEmbeddingProviders": ["ollama"],
|
||||
"tools": ["node_inference"],
|
||||
"webSearchProviders": ["ollama"]
|
||||
},
|
||||
"configSchema": {
|
||||
@@ -167,6 +168,13 @@
|
||||
"properties": {
|
||||
"enabled": { "type": "boolean" }
|
||||
}
|
||||
},
|
||||
"nodeInference": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"enabled": { "type": "boolean" }
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
@@ -178,6 +186,14 @@
|
||||
"discovery.enabled": {
|
||||
"label": "Enable Discovery",
|
||||
"help": "When false, OpenClaw keeps the Ollama plugin available but skips implicit startup discovery of ambient local or remote Ollama models."
|
||||
},
|
||||
"nodeInference": {
|
||||
"label": "Node Inference",
|
||||
"help": "Controls whether this node host advertises its local Ollama models to agents."
|
||||
},
|
||||
"nodeInference.enabled": {
|
||||
"label": "Enable Node Inference",
|
||||
"help": "When false, this node host does not advertise or accept Ollama node-inference commands."
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -24,6 +24,9 @@ export type OllamaPluginConfig = {
|
||||
discovery?: {
|
||||
enabled?: boolean;
|
||||
};
|
||||
nodeInference?: {
|
||||
enabled?: boolean;
|
||||
};
|
||||
};
|
||||
|
||||
type OllamaDiscoveryContext = {
|
||||
|
||||
@@ -0,0 +1,330 @@
|
||||
// Ollama node inference tests cover local discovery, chat, and agent tool routing.
|
||||
import { createServer, type IncomingMessage, type ServerResponse } from "node:http";
|
||||
import { createTestPluginApi } from "openclaw/plugin-sdk/plugin-test-api";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import {
|
||||
createOllamaNodeHostCommands,
|
||||
createOllamaNodeInferenceTool,
|
||||
createOllamaNodeInvokePolicy,
|
||||
OLLAMA_CHAT_COMMAND,
|
||||
OLLAMA_MODELS_COMMAND,
|
||||
} from "./node-inference.js";
|
||||
|
||||
async function readBody(request: IncomingMessage): Promise<unknown> {
|
||||
const chunks: Buffer[] = [];
|
||||
for await (const chunk of request) {
|
||||
chunks.push(Buffer.from(chunk));
|
||||
}
|
||||
return JSON.parse(Buffer.concat(chunks).toString("utf8"));
|
||||
}
|
||||
|
||||
async function withOllamaServer<T>(
|
||||
run: (
|
||||
baseUrl: string,
|
||||
chatRequests: Record<string, unknown>[],
|
||||
showRequests: string[],
|
||||
) => Promise<T>,
|
||||
): Promise<T> {
|
||||
const chatRequests: Record<string, unknown>[] = [];
|
||||
const showRequests: string[] = [];
|
||||
const handleRequest = async (request: IncomingMessage, response: ServerResponse) => {
|
||||
response.setHeader("Content-Type", "application/json");
|
||||
if (request.url === "/api/tags") {
|
||||
response.end(
|
||||
JSON.stringify({
|
||||
models: [
|
||||
{
|
||||
name: "remote:cloud",
|
||||
size: 1,
|
||||
remote_host: "https://ollama.com",
|
||||
details: {},
|
||||
},
|
||||
{
|
||||
name: "chat:small",
|
||||
size: 500,
|
||||
modified_at: "2026-07-01T00:00:00Z",
|
||||
details: {
|
||||
family: "small",
|
||||
parameter_size: "0.5B",
|
||||
quantization_level: "Q4_K_M",
|
||||
},
|
||||
},
|
||||
{ name: "chat:large", size: 5000, details: { family: "large" } },
|
||||
{ name: "embedding:latest", size: 100, details: { family: "embed" } },
|
||||
{ name: "unknown:latest", size: 50, details: { family: "unknown" } },
|
||||
],
|
||||
}),
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (request.url === "/api/ps") {
|
||||
response.end(JSON.stringify({ models: [{ name: "chat:large" }] }));
|
||||
return;
|
||||
}
|
||||
if (request.url === "/api/show") {
|
||||
const body = (await readBody(request)) as { name?: string };
|
||||
if (body.name) {
|
||||
showRequests.push(body.name);
|
||||
}
|
||||
if (body.name === "unknown:latest") {
|
||||
response.statusCode = 500;
|
||||
response.end(JSON.stringify({ error: "show failed" }));
|
||||
return;
|
||||
}
|
||||
const embedding = body.name === "embedding:latest";
|
||||
response.end(
|
||||
JSON.stringify({
|
||||
capabilities: embedding ? ["embedding"] : ["completion", "tools"],
|
||||
model_info: embedding ? {} : { "test.context_length": 32768 },
|
||||
}),
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (request.url === "/api/chat") {
|
||||
const body = (await readBody(request)) as Record<string, unknown>;
|
||||
chatRequests.push(body);
|
||||
response.end(
|
||||
JSON.stringify({
|
||||
model: body.model,
|
||||
message: { content: "local answer" },
|
||||
done_reason:
|
||||
(body.options as { num_predict?: unknown } | undefined)?.num_predict === 1
|
||||
? "length"
|
||||
: "stop",
|
||||
prompt_eval_count: 8,
|
||||
eval_count: 3,
|
||||
load_duration: 2_500_000,
|
||||
total_duration: 12_750_000,
|
||||
}),
|
||||
);
|
||||
return;
|
||||
}
|
||||
response.statusCode = 404;
|
||||
response.end(JSON.stringify({ error: "not found" }));
|
||||
};
|
||||
const server = createServer((request: IncomingMessage, response: ServerResponse) => {
|
||||
void handleRequest(request, response);
|
||||
});
|
||||
await new Promise<void>((resolve) => {
|
||||
server.listen(0, "127.0.0.1", resolve);
|
||||
});
|
||||
const address = server.address();
|
||||
if (!address || typeof address === "string") {
|
||||
throw new Error("test server did not expose a TCP address");
|
||||
}
|
||||
try {
|
||||
return await run(`http://127.0.0.1:${address.port}`, chatRequests, showRequests);
|
||||
} finally {
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
server.close((error) => {
|
||||
if (error) {
|
||||
reject(error);
|
||||
return;
|
||||
}
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
function commandByName(baseUrl: string, command: string) {
|
||||
const entry = createOllamaNodeHostCommands({ baseUrl }).find(
|
||||
(candidate) => candidate.command === command,
|
||||
);
|
||||
if (!entry) {
|
||||
throw new Error(`missing ${command} test command`);
|
||||
}
|
||||
return entry;
|
||||
}
|
||||
|
||||
describe("Ollama node host inference", () => {
|
||||
it("discovers local chat models and ranks loaded models first", async () => {
|
||||
await withOllamaServer(async (baseUrl) => {
|
||||
const result = JSON.parse(await commandByName(baseUrl, OLLAMA_MODELS_COMMAND).handle()) as {
|
||||
provider: string;
|
||||
models: Array<Record<string, unknown>>;
|
||||
};
|
||||
|
||||
expect(result.provider).toBe("ollama");
|
||||
expect(result.models.map((model) => model.name)).toEqual(["chat:large", "chat:small"]);
|
||||
expect(result.models[0]).toMatchObject({ loaded: true, contextWindow: 32768 });
|
||||
expect(result.models[1]).toMatchObject({
|
||||
loaded: false,
|
||||
family: "small",
|
||||
parameterSize: "0.5B",
|
||||
quantization: "Q4_K_M",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it("runs bounded chat and returns compact usage", async () => {
|
||||
await withOllamaServer(async (baseUrl, chatRequests, showRequests) => {
|
||||
const result = JSON.parse(
|
||||
await commandByName(baseUrl, OLLAMA_CHAT_COMMAND).handle(
|
||||
JSON.stringify({
|
||||
model: "chat:small",
|
||||
prompt: "Summarize this",
|
||||
system: "Be concise",
|
||||
maxTokens: 64,
|
||||
temperature: 0.2,
|
||||
}),
|
||||
),
|
||||
);
|
||||
|
||||
expect(chatRequests).toEqual([
|
||||
{
|
||||
model: "chat:small",
|
||||
messages: [
|
||||
{ role: "system", content: "Be concise" },
|
||||
{ role: "user", content: "Summarize this" },
|
||||
],
|
||||
stream: false,
|
||||
think: false,
|
||||
options: { num_predict: 64, temperature: 0.2 },
|
||||
},
|
||||
]);
|
||||
expect(showRequests).toEqual(["chat:small"]);
|
||||
expect(result).toEqual({
|
||||
provider: "ollama",
|
||||
model: "chat:small",
|
||||
response: "local answer",
|
||||
usage: { promptTokens: 8, completionTokens: 3 },
|
||||
timings: { loadMs: 2.5, totalMs: 12.75 },
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects remote and non-chat models before inference", async () => {
|
||||
await withOllamaServer(async (baseUrl, chatRequests) => {
|
||||
await expect(
|
||||
commandByName(baseUrl, OLLAMA_CHAT_COMMAND).handle(
|
||||
JSON.stringify({ model: "remote:cloud", prompt: "hello" }),
|
||||
),
|
||||
).rejects.toThrow("is not a local chat model");
|
||||
await expect(
|
||||
commandByName(baseUrl, OLLAMA_CHAT_COMMAND).handle(
|
||||
JSON.stringify({ model: "embedding:latest", prompt: "hello" }),
|
||||
),
|
||||
).rejects.toThrow("is not a local chat model");
|
||||
expect(chatRequests).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects a token-limited partial answer", async () => {
|
||||
await withOllamaServer(async (baseUrl) => {
|
||||
await expect(
|
||||
commandByName(baseUrl, OLLAMA_CHAT_COMMAND).handle(
|
||||
JSON.stringify({ model: "chat:small", prompt: "long answer", maxTokens: 1 }),
|
||||
),
|
||||
).rejects.toThrow("reaching maxTokens (1)");
|
||||
});
|
||||
});
|
||||
|
||||
it("registers a desktop and server pass-through policy", async () => {
|
||||
const policy = createOllamaNodeInvokePolicy();
|
||||
const invokeNode = vi.fn(async () => ({ ok: true as const, payload: { ok: true } }));
|
||||
|
||||
expect(policy.commands).toEqual([OLLAMA_MODELS_COMMAND, OLLAMA_CHAT_COMMAND]);
|
||||
expect(policy.defaultPlatforms).toEqual(["macos", "linux", "windows"]);
|
||||
await expect(policy.handle({ invokeNode } as never)).resolves.toEqual({
|
||||
ok: true,
|
||||
payload: { ok: true },
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("node_inference agent tool", () => {
|
||||
it("discovers models through the connected node runtime", async () => {
|
||||
const invoke = vi.fn(async () => ({
|
||||
payload: { provider: "ollama", models: [{ name: "chat:small", loaded: true }] },
|
||||
}));
|
||||
const api = createTestPluginApi({
|
||||
runtime: {
|
||||
nodes: {
|
||||
list: async () => ({
|
||||
nodes: [
|
||||
{
|
||||
nodeId: "node-1",
|
||||
displayName: "Desk",
|
||||
connected: true,
|
||||
commands: [OLLAMA_MODELS_COMMAND, OLLAMA_CHAT_COMMAND],
|
||||
},
|
||||
],
|
||||
}),
|
||||
invoke,
|
||||
},
|
||||
} as never,
|
||||
});
|
||||
|
||||
const result = await createOllamaNodeInferenceTool(api).execute("call-1", {
|
||||
action: "discover",
|
||||
});
|
||||
|
||||
expect(invoke).toHaveBeenCalledWith({
|
||||
nodeId: "node-1",
|
||||
command: OLLAMA_MODELS_COMMAND,
|
||||
params: {},
|
||||
timeoutMs: 90_000,
|
||||
scopes: ["operator.write"],
|
||||
});
|
||||
expect(result.details).toEqual({
|
||||
nodes: [
|
||||
{
|
||||
nodeId: "node-1",
|
||||
displayName: "Desk",
|
||||
ok: true,
|
||||
provider: "ollama",
|
||||
models: [{ name: "chat:small", loaded: true }],
|
||||
},
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
it("routes a run to the sole capable node", async () => {
|
||||
const invoke = vi.fn(async () => ({
|
||||
payload: { provider: "ollama", model: "chat:small", response: "done" },
|
||||
}));
|
||||
const api = createTestPluginApi({
|
||||
runtime: {
|
||||
nodes: {
|
||||
list: async () => ({
|
||||
nodes: [
|
||||
{
|
||||
nodeId: "node-1",
|
||||
connected: true,
|
||||
commands: [OLLAMA_MODELS_COMMAND, OLLAMA_CHAT_COMMAND],
|
||||
},
|
||||
],
|
||||
}),
|
||||
invoke,
|
||||
},
|
||||
} as never,
|
||||
});
|
||||
|
||||
const result = await createOllamaNodeInferenceTool(api).execute("call-2", {
|
||||
action: "run",
|
||||
model: "chat:small",
|
||||
prompt: "answer fast",
|
||||
maxTokens: 32,
|
||||
});
|
||||
|
||||
expect(invoke).toHaveBeenCalledWith({
|
||||
nodeId: "node-1",
|
||||
command: OLLAMA_CHAT_COMMAND,
|
||||
params: {
|
||||
model: "chat:small",
|
||||
prompt: "answer fast",
|
||||
maxTokens: 32,
|
||||
timeoutMs: 120_000,
|
||||
},
|
||||
timeoutMs: 130_000,
|
||||
scopes: ["operator.write"],
|
||||
});
|
||||
expect(result.details).toMatchObject({
|
||||
nodeId: "node-1",
|
||||
provider: "ollama",
|
||||
model: "chat:small",
|
||||
response: "done",
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,550 @@
|
||||
// Ollama node inference exposes local models to agents through paired node hosts.
|
||||
import { jsonResult } from "openclaw/plugin-sdk/channel-actions";
|
||||
import {
|
||||
readFiniteNumberParam,
|
||||
readPositiveIntegerParam,
|
||||
readStringParam,
|
||||
} from "openclaw/plugin-sdk/param-readers";
|
||||
import type {
|
||||
AnyAgentTool,
|
||||
OpenClawPluginApi,
|
||||
OpenClawPluginNodeHostCommand,
|
||||
OpenClawPluginNodeInvokePolicy,
|
||||
} from "openclaw/plugin-sdk/plugin-entry";
|
||||
import {
|
||||
readProviderJsonResponse,
|
||||
readResponseTextLimited,
|
||||
} from "openclaw/plugin-sdk/provider-http";
|
||||
import { fetchWithSsrFGuard } from "openclaw/plugin-sdk/ssrf-runtime";
|
||||
import { Type } from "typebox";
|
||||
import { OLLAMA_DEFAULT_BASE_URL } from "./defaults.js";
|
||||
import {
|
||||
buildOllamaBaseUrlSsrFPolicy,
|
||||
enrichOllamaModelsWithContext,
|
||||
fetchOllamaModels,
|
||||
resolveOllamaApiBase,
|
||||
} from "./provider-models.js";
|
||||
|
||||
export const OLLAMA_NODE_INFERENCE_CAPABILITY = "local-inference";
|
||||
export const OLLAMA_MODELS_COMMAND = "ollama.models";
|
||||
export const OLLAMA_CHAT_COMMAND = "ollama.chat";
|
||||
export const OLLAMA_NODE_INFERENCE_COMMANDS = [OLLAMA_MODELS_COMMAND, OLLAMA_CHAT_COMMAND] as const;
|
||||
|
||||
const DEFAULT_INFERENCE_TIMEOUT_MS = 120_000;
|
||||
const DEFAULT_MAX_TOKENS = 512;
|
||||
const DISCOVERY_TRANSPORT_TIMEOUT_MS = 90_000;
|
||||
const INFERENCE_TRANSPORT_GRACE_MS = 10_000;
|
||||
const MAX_INFERENCE_TIMEOUT_MS = 10 * 60_000;
|
||||
const MAX_TOKENS = 8192;
|
||||
const MAX_PROMPT_CHARS = 128_000;
|
||||
const MAX_SYSTEM_PROMPT_CHARS = 32_000;
|
||||
const MAX_DISCOVERED_MODELS = 200;
|
||||
const MAX_ERROR_BODY_BYTES = 500;
|
||||
|
||||
type NodeModel = {
|
||||
name: string;
|
||||
size?: number;
|
||||
modifiedAt?: string;
|
||||
family?: string;
|
||||
parameterSize?: string;
|
||||
quantization?: string;
|
||||
contextWindow?: number;
|
||||
capabilities?: string[];
|
||||
loaded: boolean;
|
||||
};
|
||||
|
||||
type OllamaModelsPayload = {
|
||||
provider: "ollama";
|
||||
models: NodeModel[];
|
||||
};
|
||||
|
||||
type OllamaChatPayload = {
|
||||
provider: "ollama";
|
||||
model: string;
|
||||
response: string;
|
||||
usage?: {
|
||||
promptTokens?: number;
|
||||
completionTokens?: number;
|
||||
};
|
||||
timings?: {
|
||||
loadMs?: number;
|
||||
totalMs?: number;
|
||||
};
|
||||
};
|
||||
|
||||
type NodeSummary = Awaited<
|
||||
ReturnType<OpenClawPluginApi["runtime"]["nodes"]["list"]>
|
||||
>["nodes"][number];
|
||||
|
||||
function asRecord(value: unknown): Record<string, unknown> | null {
|
||||
return value && typeof value === "object" && !Array.isArray(value)
|
||||
? (value as Record<string, unknown>)
|
||||
: null;
|
||||
}
|
||||
|
||||
function readNodeCommandParams(paramsJSON?: string | null): Record<string, unknown> {
|
||||
if (!paramsJSON) {
|
||||
return {};
|
||||
}
|
||||
const parsed = asRecord(JSON.parse(paramsJSON));
|
||||
if (!parsed) {
|
||||
throw new Error("node inference params must be a JSON object");
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function errorMessage(error: unknown): string {
|
||||
return error instanceof Error && error.message ? error.message : String(error);
|
||||
}
|
||||
|
||||
function durationMs(value: unknown): number | undefined {
|
||||
if (typeof value !== "number" || !Number.isFinite(value) || value < 0) {
|
||||
return undefined;
|
||||
}
|
||||
return Math.round((value / 1_000_000) * 100) / 100;
|
||||
}
|
||||
|
||||
function optionalNumber(value: unknown): number | undefined {
|
||||
return typeof value === "number" && Number.isFinite(value) ? value : undefined;
|
||||
}
|
||||
|
||||
async function requestOllamaJson<T>(params: {
|
||||
baseUrl: string;
|
||||
path: string;
|
||||
timeoutMs: number;
|
||||
init?: RequestInit;
|
||||
}): Promise<T> {
|
||||
const apiBase = resolveOllamaApiBase(params.baseUrl);
|
||||
let response: Response;
|
||||
let release: (() => Promise<void>) | undefined;
|
||||
try {
|
||||
const guarded = await fetchWithSsrFGuard({
|
||||
url: `${apiBase}${params.path}`,
|
||||
init: {
|
||||
...params.init,
|
||||
signal: AbortSignal.timeout(params.timeoutMs),
|
||||
},
|
||||
policy: buildOllamaBaseUrlSsrFPolicy(apiBase),
|
||||
auditContext: `ollama-node-inference${params.path}`,
|
||||
});
|
||||
response = guarded.response;
|
||||
release = guarded.release;
|
||||
} catch (error) {
|
||||
throw new Error(`Ollama is unavailable at ${apiBase}: ${errorMessage(error)}`, {
|
||||
cause: error,
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
if (!response.ok) {
|
||||
const body = (await readResponseTextLimited(response, MAX_ERROR_BODY_BYTES)).trim();
|
||||
let detail = body;
|
||||
try {
|
||||
const parsed = asRecord(JSON.parse(body));
|
||||
detail = typeof parsed?.error === "string" ? parsed.error : body;
|
||||
} catch {
|
||||
// Keep the bounded response text when Ollama returns a non-JSON error.
|
||||
}
|
||||
throw new Error(
|
||||
`Ollama ${params.path} failed (HTTP ${response.status})${detail ? `: ${detail}` : ""}`,
|
||||
);
|
||||
}
|
||||
return await readProviderJsonResponse<T>(response, `ollama-node-inference${params.path}`);
|
||||
} finally {
|
||||
await release();
|
||||
}
|
||||
}
|
||||
|
||||
async function fetchLoadedModelNames(baseUrl: string): Promise<Set<string>> {
|
||||
try {
|
||||
const data = await requestOllamaJson<{ models?: Array<{ name?: unknown; model?: unknown }> }>({
|
||||
baseUrl,
|
||||
path: "/api/ps",
|
||||
timeoutMs: 5000,
|
||||
});
|
||||
return new Set(
|
||||
(data.models ?? [])
|
||||
.map((model) =>
|
||||
typeof model.name === "string"
|
||||
? model.name.trim()
|
||||
: typeof model.model === "string"
|
||||
? model.model.trim()
|
||||
: "",
|
||||
)
|
||||
.filter(Boolean),
|
||||
);
|
||||
} catch {
|
||||
// Model discovery still works against Ollama versions without /api/ps.
|
||||
return new Set();
|
||||
}
|
||||
}
|
||||
|
||||
export async function discoverOllamaNodeModels(
|
||||
baseUrl = OLLAMA_DEFAULT_BASE_URL,
|
||||
): Promise<OllamaModelsPayload> {
|
||||
const apiBase = resolveOllamaApiBase(baseUrl);
|
||||
const discovered = await fetchOllamaModels(apiBase);
|
||||
if (!discovered.reachable) {
|
||||
throw new Error(`Ollama is not running at ${apiBase}`);
|
||||
}
|
||||
const localModels = discovered.models
|
||||
.filter((model) => !model.remote_host?.trim())
|
||||
.slice(0, MAX_DISCOVERED_MODELS);
|
||||
const [models, loadedNames] = await Promise.all([
|
||||
enrichOllamaModelsWithContext(apiBase, localModels),
|
||||
fetchLoadedModelNames(apiBase),
|
||||
]);
|
||||
const rows = models
|
||||
// Nodes advertise only models Ollama positively identifies as chat-capable.
|
||||
// Failed /api/show probes must not turn embedding models into runnable choices.
|
||||
.filter((model) => model.capabilities?.includes("completion") === true)
|
||||
.map((model): NodeModel => {
|
||||
const details = model.details;
|
||||
const row: NodeModel = {
|
||||
name: model.name,
|
||||
loaded: loadedNames.has(model.name),
|
||||
};
|
||||
if (typeof model.size === "number") {
|
||||
row.size = model.size;
|
||||
}
|
||||
if (typeof model.modified_at === "string") {
|
||||
row.modifiedAt = model.modified_at;
|
||||
}
|
||||
if (details?.family) {
|
||||
row.family = details.family;
|
||||
}
|
||||
if (details?.parameter_size) {
|
||||
row.parameterSize = details.parameter_size;
|
||||
}
|
||||
if (details?.quantization_level) {
|
||||
row.quantization = details.quantization_level;
|
||||
}
|
||||
if (typeof model.contextWindow === "number") {
|
||||
row.contextWindow = model.contextWindow;
|
||||
}
|
||||
if (model.capabilities) {
|
||||
row.capabilities = model.capabilities;
|
||||
}
|
||||
return row;
|
||||
})
|
||||
.toSorted((left, right) => {
|
||||
if (left.loaded !== right.loaded) {
|
||||
return left.loaded ? -1 : 1;
|
||||
}
|
||||
const sizeDelta =
|
||||
(left.size ?? Number.MAX_SAFE_INTEGER) - (right.size ?? Number.MAX_SAFE_INTEGER);
|
||||
return sizeDelta || left.name.localeCompare(right.name);
|
||||
});
|
||||
return { provider: "ollama", models: rows };
|
||||
}
|
||||
|
||||
async function runOllamaNodeChat(params: {
|
||||
baseUrl: string;
|
||||
model: string;
|
||||
prompt: string;
|
||||
system?: string;
|
||||
temperature?: number;
|
||||
maxTokens: number;
|
||||
timeoutMs: number;
|
||||
}): Promise<OllamaChatPayload> {
|
||||
const apiBase = resolveOllamaApiBase(params.baseUrl);
|
||||
const discovered = await fetchOllamaModels(apiBase);
|
||||
const localModel = discovered.models.find(
|
||||
(model) => model.name === params.model && !model.remote_host?.trim(),
|
||||
);
|
||||
const [model] = localModel ? await enrichOllamaModelsWithContext(apiBase, [localModel]) : [];
|
||||
if (!discovered.reachable || model?.capabilities?.includes("completion") !== true) {
|
||||
throw new Error(
|
||||
`Ollama model ${JSON.stringify(params.model)} is not a local chat model; discover models first`,
|
||||
);
|
||||
}
|
||||
const messages = [
|
||||
...(params.system ? [{ role: "system", content: params.system }] : []),
|
||||
{ role: "user", content: params.prompt },
|
||||
];
|
||||
const data = await requestOllamaJson<{
|
||||
model?: unknown;
|
||||
message?: { content?: unknown };
|
||||
done_reason?: unknown;
|
||||
prompt_eval_count?: unknown;
|
||||
eval_count?: unknown;
|
||||
load_duration?: unknown;
|
||||
total_duration?: unknown;
|
||||
}>({
|
||||
baseUrl: params.baseUrl,
|
||||
path: "/api/chat",
|
||||
timeoutMs: params.timeoutMs,
|
||||
init: {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
model: params.model,
|
||||
messages,
|
||||
stream: false,
|
||||
think: false,
|
||||
options: {
|
||||
num_predict: params.maxTokens,
|
||||
...(params.temperature !== undefined && { temperature: params.temperature }),
|
||||
},
|
||||
}),
|
||||
},
|
||||
});
|
||||
const response = typeof data.message?.content === "string" ? data.message.content : undefined;
|
||||
if (response === undefined) {
|
||||
throw new Error("Ollama /api/chat response did not contain message.content");
|
||||
}
|
||||
if (data.done_reason === "length") {
|
||||
throw new Error(
|
||||
`Ollama stopped after reaching maxTokens (${params.maxTokens}); retry with a larger maxTokens value`,
|
||||
);
|
||||
}
|
||||
const promptTokens = optionalNumber(data.prompt_eval_count);
|
||||
const completionTokens = optionalNumber(data.eval_count);
|
||||
const loadMs = durationMs(data.load_duration);
|
||||
const totalMs = durationMs(data.total_duration);
|
||||
return {
|
||||
provider: "ollama",
|
||||
model: typeof data.model === "string" && data.model.trim() ? data.model : params.model,
|
||||
response,
|
||||
...(promptTokens !== undefined || completionTokens !== undefined
|
||||
? { usage: { promptTokens, completionTokens } }
|
||||
: {}),
|
||||
...(loadMs !== undefined || totalMs !== undefined ? { timings: { loadMs, totalMs } } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
export function createOllamaNodeHostCommands(options?: {
|
||||
baseUrl?: string;
|
||||
}): OpenClawPluginNodeHostCommand[] {
|
||||
const baseUrl = options?.baseUrl ?? OLLAMA_DEFAULT_BASE_URL;
|
||||
return [
|
||||
{
|
||||
command: OLLAMA_MODELS_COMMAND,
|
||||
cap: OLLAMA_NODE_INFERENCE_CAPABILITY,
|
||||
handle: async () => JSON.stringify(await discoverOllamaNodeModels(baseUrl)),
|
||||
},
|
||||
{
|
||||
command: OLLAMA_CHAT_COMMAND,
|
||||
cap: OLLAMA_NODE_INFERENCE_CAPABILITY,
|
||||
handle: async (paramsJSON) => {
|
||||
const params = readNodeCommandParams(paramsJSON);
|
||||
const model = readStringParam(params, "model", { required: true });
|
||||
const prompt = readStringParam(params, "prompt", { required: true, trim: false });
|
||||
const system = readStringParam(params, "system", { trim: false });
|
||||
const maxTokens =
|
||||
readPositiveIntegerParam(params, "maxTokens", {
|
||||
max: MAX_TOKENS,
|
||||
message: `maxTokens must be an integer between 1 and ${MAX_TOKENS}`,
|
||||
}) ?? DEFAULT_MAX_TOKENS;
|
||||
const timeoutMs =
|
||||
readPositiveIntegerParam(params, "timeoutMs", {
|
||||
max: MAX_INFERENCE_TIMEOUT_MS,
|
||||
message: `timeoutMs must be an integer between 1 and ${MAX_INFERENCE_TIMEOUT_MS}`,
|
||||
}) ?? DEFAULT_INFERENCE_TIMEOUT_MS;
|
||||
const temperature = readFiniteNumberParam(params, "temperature", {
|
||||
min: 0,
|
||||
max: 2,
|
||||
message: "temperature must be between 0 and 2",
|
||||
});
|
||||
if (prompt.length > MAX_PROMPT_CHARS) {
|
||||
throw new Error(`prompt exceeds ${MAX_PROMPT_CHARS} characters`);
|
||||
}
|
||||
if (system && system.length > MAX_SYSTEM_PROMPT_CHARS) {
|
||||
throw new Error(`system exceeds ${MAX_SYSTEM_PROMPT_CHARS} characters`);
|
||||
}
|
||||
return JSON.stringify(
|
||||
await runOllamaNodeChat({
|
||||
baseUrl,
|
||||
model,
|
||||
prompt,
|
||||
system,
|
||||
temperature,
|
||||
maxTokens,
|
||||
timeoutMs,
|
||||
}),
|
||||
);
|
||||
},
|
||||
},
|
||||
];
|
||||
}
|
||||
|
||||
export function createOllamaNodeInvokePolicy(): OpenClawPluginNodeInvokePolicy {
|
||||
return {
|
||||
commands: [...OLLAMA_NODE_INFERENCE_COMMANDS],
|
||||
defaultPlatforms: ["macos", "linux", "windows"],
|
||||
handle: async (ctx) => await ctx.invokeNode(),
|
||||
};
|
||||
}
|
||||
|
||||
function findNode(nodes: NodeSummary[], query: string): NodeSummary {
|
||||
const normalized = query.trim().toLowerCase();
|
||||
const matches = nodes.filter(
|
||||
(node) =>
|
||||
node.nodeId.toLowerCase() === normalized || node.displayName?.toLowerCase() === normalized,
|
||||
);
|
||||
if (matches.length === 0) {
|
||||
throw new Error(`node ${JSON.stringify(query)} is not connected with Ollama inference support`);
|
||||
}
|
||||
if (matches.length > 1) {
|
||||
throw new Error(`node ${JSON.stringify(query)} is ambiguous; use its nodeId`);
|
||||
}
|
||||
return matches[0];
|
||||
}
|
||||
|
||||
function parseInvokePayload(raw: unknown): Record<string, unknown> {
|
||||
const result = asRecord(raw);
|
||||
let payload = asRecord(result?.payload);
|
||||
if (!payload && typeof result?.payloadJSON === "string") {
|
||||
payload = asRecord(JSON.parse(result.payloadJSON));
|
||||
}
|
||||
if (!payload) {
|
||||
throw new Error("node returned an invalid Ollama inference payload");
|
||||
}
|
||||
return payload;
|
||||
}
|
||||
|
||||
async function invokeNode(
|
||||
api: OpenClawPluginApi,
|
||||
nodeId: string,
|
||||
command: string,
|
||||
params: Record<string, unknown>,
|
||||
timeoutMs: number,
|
||||
): Promise<Record<string, unknown>> {
|
||||
const raw = await api.runtime.nodes.invoke({
|
||||
nodeId,
|
||||
command,
|
||||
params,
|
||||
timeoutMs,
|
||||
scopes: ["operator.write"],
|
||||
});
|
||||
return parseInvokePayload(raw);
|
||||
}
|
||||
|
||||
export const ollamaNodeInferenceToolDefinition = {
|
||||
name: "node_inference",
|
||||
label: "Node Inference",
|
||||
description:
|
||||
"Discover and run chat-capable Ollama models installed on paired desktop/server nodes. Use action=discover first, then action=run with a node and model from that result. Inference stays on the selected node.",
|
||||
parameters: Type.Object(
|
||||
{
|
||||
action: Type.Union([Type.Literal("discover"), Type.Literal("run")]),
|
||||
node: Type.Optional(
|
||||
Type.String({ description: "Connected node id or display name. Required when ambiguous." }),
|
||||
),
|
||||
model: Type.Optional(
|
||||
Type.String({ description: "Exact local model name returned by discover." }),
|
||||
),
|
||||
prompt: Type.Optional(Type.String({ description: "Prompt for action=run." })),
|
||||
system: Type.Optional(Type.String({ description: "Optional system prompt for action=run." })),
|
||||
temperature: Type.Optional(Type.Number({ minimum: 0, maximum: 2 })),
|
||||
maxTokens: Type.Optional(Type.Integer({ minimum: 1, maximum: MAX_TOKENS })),
|
||||
timeoutMs: Type.Optional(Type.Integer({ minimum: 1, maximum: MAX_INFERENCE_TIMEOUT_MS })),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
),
|
||||
} as const;
|
||||
|
||||
export function createOllamaNodeInferenceTool(api: OpenClawPluginApi): AnyAgentTool {
|
||||
return {
|
||||
...ollamaNodeInferenceToolDefinition,
|
||||
execute: async (_toolCallId, args) => {
|
||||
const params = asRecord(args) ?? {};
|
||||
const action = readStringParam(params, "action", { required: true });
|
||||
const nodeQuery = readStringParam(params, "node");
|
||||
const listed = await api.runtime.nodes.list({ connected: true });
|
||||
const modelNodes = listed.nodes.filter((node) =>
|
||||
node.commands?.includes(OLLAMA_MODELS_COMMAND),
|
||||
);
|
||||
|
||||
if (action === "discover") {
|
||||
const targets = nodeQuery ? [findNode(modelNodes, nodeQuery)] : modelNodes;
|
||||
const nodes = await Promise.all(
|
||||
targets.map(async (node) => {
|
||||
try {
|
||||
const payload = await invokeNode(
|
||||
api,
|
||||
node.nodeId,
|
||||
OLLAMA_MODELS_COMMAND,
|
||||
{},
|
||||
DISCOVERY_TRANSPORT_TIMEOUT_MS,
|
||||
);
|
||||
const result: Record<string, unknown> = { nodeId: node.nodeId, ok: true };
|
||||
if (node.displayName) {
|
||||
result.displayName = node.displayName;
|
||||
}
|
||||
return Object.assign(result, payload);
|
||||
} catch (error) {
|
||||
const result: Record<string, unknown> = {
|
||||
nodeId: node.nodeId,
|
||||
ok: false,
|
||||
error: errorMessage(error),
|
||||
};
|
||||
if (node.displayName) {
|
||||
result.displayName = node.displayName;
|
||||
}
|
||||
return result;
|
||||
}
|
||||
}),
|
||||
);
|
||||
return jsonResult({
|
||||
nodes,
|
||||
...(modelNodes.length === 0 && {
|
||||
hint: "No connected node advertises Ollama inference. Start Ollama and `openclaw node run` on the target machine, then approve any request shown by `openclaw nodes pending`.",
|
||||
}),
|
||||
});
|
||||
}
|
||||
|
||||
if (action !== "run") {
|
||||
throw new Error("action must be discover or run");
|
||||
}
|
||||
const chatNodes = modelNodes.filter((node) => node.commands?.includes(OLLAMA_CHAT_COMMAND));
|
||||
const node = nodeQuery
|
||||
? findNode(chatNodes, nodeQuery)
|
||||
: chatNodes.length === 1
|
||||
? chatNodes[0]
|
||||
: undefined;
|
||||
if (!node) {
|
||||
throw new Error(
|
||||
chatNodes.length === 0
|
||||
? "no connected node advertises Ollama inference"
|
||||
: "multiple nodes advertise Ollama inference; specify node",
|
||||
);
|
||||
}
|
||||
const model = readStringParam(params, "model", { required: true });
|
||||
const prompt = readStringParam(params, "prompt", { required: true, trim: false });
|
||||
const maxTokens =
|
||||
readPositiveIntegerParam(params, "maxTokens", { max: MAX_TOKENS }) ?? DEFAULT_MAX_TOKENS;
|
||||
const timeoutMs =
|
||||
readPositiveIntegerParam(params, "timeoutMs", { max: MAX_INFERENCE_TIMEOUT_MS }) ??
|
||||
DEFAULT_INFERENCE_TIMEOUT_MS;
|
||||
const system = readStringParam(params, "system", { trim: false });
|
||||
const temperature = readFiniteNumberParam(params, "temperature", { min: 0, max: 2 });
|
||||
const commandParams: Record<string, unknown> = {
|
||||
model,
|
||||
prompt,
|
||||
maxTokens,
|
||||
timeoutMs,
|
||||
};
|
||||
if (system !== undefined) {
|
||||
commandParams.system = system;
|
||||
}
|
||||
if (temperature !== undefined) {
|
||||
commandParams.temperature = temperature;
|
||||
}
|
||||
const result = await invokeNode(
|
||||
api,
|
||||
node.nodeId,
|
||||
OLLAMA_CHAT_COMMAND,
|
||||
commandParams,
|
||||
// The command validates the selected model before starting its chat timeout.
|
||||
// Keep that bounded preflight outside the inference budget seen by users.
|
||||
timeoutMs + INFERENCE_TRANSPORT_GRACE_MS,
|
||||
);
|
||||
return jsonResult({
|
||||
nodeId: node.nodeId,
|
||||
...(node.displayName && { displayName: node.displayName }),
|
||||
...result,
|
||||
});
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -1,8 +1,8 @@
|
||||
// Ollama provider module implements model/runtime integration.
|
||||
import { createHash } from "node:crypto";
|
||||
import { readProviderJsonResponse } from "openclaw/plugin-sdk/provider-http";
|
||||
import type { ModelProviderConfig } from "openclaw/plugin-sdk/provider-model-shared";
|
||||
import type { ModelDefinitionConfig } from "openclaw/plugin-sdk/provider-onboard";
|
||||
import { readProviderJsonResponse } from "openclaw/plugin-sdk/provider-http";
|
||||
import { fetchWithSsrFGuard } from "openclaw/plugin-sdk/ssrf-runtime";
|
||||
import {
|
||||
OLLAMA_DEFAULT_BASE_URL,
|
||||
@@ -22,6 +22,7 @@ export type OllamaTagModel = {
|
||||
details?: {
|
||||
family?: string;
|
||||
parameter_size?: string;
|
||||
quantization_level?: string;
|
||||
};
|
||||
};
|
||||
|
||||
|
||||
@@ -2184,6 +2184,15 @@ describe("CLI attempt execution", () => {
|
||||
expect(embeddedArg.suppressLiveStreamOutput).toBe(false);
|
||||
});
|
||||
|
||||
it("forwards Gateway plugin runtime binding to embedded runs", async () => {
|
||||
const embeddedArg = await runOpenClawEmbeddedAttemptForTest({
|
||||
opts: { allowGatewaySubagentBinding: true },
|
||||
runId: "gateway-plugin-runtime-binding",
|
||||
});
|
||||
|
||||
expect(embeddedArg.allowGatewaySubagentBinding).toBe(true);
|
||||
});
|
||||
|
||||
it("suppresses live stream output for hidden internal runs", async () => {
|
||||
const embeddedArg = await runOpenClawEmbeddedAttemptForTest({
|
||||
opts: { lane: "subagent", sessionEffects: "internal" },
|
||||
|
||||
@@ -97,9 +97,7 @@ const ACP_TRANSCRIPT_USAGE = {
|
||||
const GOOGLE_GEMINI_CLI_PROVIDER_ID = "google-gemini-cli";
|
||||
const GOOGLE_PROVIDER_ID = "google";
|
||||
|
||||
function shouldSuppressEmbeddedLiveStreamOutput(params: {
|
||||
opts: AgentCommandOpts;
|
||||
}): boolean {
|
||||
function shouldSuppressEmbeddedLiveStreamOutput(params: { opts: AgentCommandOpts }): boolean {
|
||||
return params.opts.sessionEffects === "internal" && params.opts.deliver !== true;
|
||||
}
|
||||
|
||||
@@ -824,6 +822,7 @@ export function runAgentAttempt(params: {
|
||||
disableMessageTool: params.opts.disableMessageTool,
|
||||
streamParams: params.opts.streamParams,
|
||||
agentDir: params.agentDir,
|
||||
allowGatewaySubagentBinding: params.opts.allowGatewaySubagentBinding,
|
||||
allowTransientCooldownProbe: params.allowTransientCooldownProbe,
|
||||
cleanupBundleMcpOnRunEnd: params.opts.cleanupBundleMcpOnRunEnd,
|
||||
oneShotCliRun: params.opts.oneShotCliRun,
|
||||
|
||||
@@ -152,6 +152,8 @@ export type AgentCommandOpts = {
|
||||
cleanupCliLiveSessionOnRunEnd?: boolean;
|
||||
/** Mark explicit one-shot local CLI runs so plugin tools can release resources promptly. */
|
||||
oneShotCliRun?: boolean;
|
||||
/** Gateway-owned runs can late-bind plugin subagent and node runtime helpers. */
|
||||
allowGatewaySubagentBinding?: boolean;
|
||||
/** Internal local CLI callers can annotate result metadata before JSON/text output. */
|
||||
resultMetaOverrides?: AgentCommandResultMetaOverrides;
|
||||
/** Called when the actual run model is selected, including fallback retries. */
|
||||
|
||||
@@ -8,7 +8,11 @@ import {
|
||||
} from "../../packages/gateway-protocol/src/client-info.js";
|
||||
import type { OpenClawConfig } from "../config/types.openclaw.js";
|
||||
import { createEmptyPluginRegistry } from "../plugins/registry-empty.js";
|
||||
import { resetPluginRuntimeStateForTest, setActivePluginRegistry } from "../plugins/runtime.js";
|
||||
import {
|
||||
pinActivePluginChannelRegistry,
|
||||
resetPluginRuntimeStateForTest,
|
||||
setActivePluginRegistry,
|
||||
} from "../plugins/runtime.js";
|
||||
import {
|
||||
isForegroundRestrictedPluginNodeCommand,
|
||||
isNodeCommandAllowed,
|
||||
@@ -37,6 +41,7 @@ describe("gateway/node-command-policy", () => {
|
||||
},
|
||||
});
|
||||
setActivePluginRegistry(registry);
|
||||
return registry;
|
||||
}
|
||||
|
||||
it("normalizes declared node commands against the allowlist", () => {
|
||||
@@ -109,6 +114,34 @@ describe("gateway/node-command-policy", () => {
|
||||
expect(allowlist.has("canvas.present")).toBe(true);
|
||||
});
|
||||
|
||||
it("keeps plugin node defaults from the pinned Gateway registry", () => {
|
||||
const startupRegistry = installCanvasPluginDefaults();
|
||||
pinActivePluginChannelRegistry(startupRegistry);
|
||||
const transientRegistry = createEmptyPluginRegistry();
|
||||
const startupPolicy = startupRegistry.nodeInvokePolicies?.[0];
|
||||
if (!startupPolicy) {
|
||||
throw new Error("expected canvas node policy");
|
||||
}
|
||||
(transientRegistry.nodeInvokePolicies ??= []).push({
|
||||
...startupPolicy,
|
||||
pluginId: "transient",
|
||||
policy: {
|
||||
...startupPolicy.policy,
|
||||
commands: ["transient.read"],
|
||||
},
|
||||
});
|
||||
setActivePluginRegistry(transientRegistry);
|
||||
|
||||
const allowlist = resolveNodeCommandAllowlist({} as OpenClawConfig, {
|
||||
platform: "macos",
|
||||
deviceFamily: "Mac",
|
||||
});
|
||||
|
||||
expect(allowlist.has("canvas.snapshot")).toBe(true);
|
||||
expect(allowlist.has("canvas.present")).toBe(true);
|
||||
expect(allowlist.has("transient.read")).toBe(false);
|
||||
});
|
||||
|
||||
it("does not grant host command defaults for platform prefix aliases", () => {
|
||||
const cfg = {} as OpenClawConfig;
|
||||
const cases = [
|
||||
|
||||
@@ -8,7 +8,7 @@ import {
|
||||
NODE_SYSTEM_NOTIFY_COMMAND,
|
||||
NODE_SYSTEM_RUN_COMMANDS,
|
||||
} from "../infra/node-commands.js";
|
||||
import { getActiveRuntimePluginRegistry } from "../plugins/active-runtime-registry.js";
|
||||
import { getActivePluginGatewayNodePolicyRegistry } from "../plugins/runtime.js";
|
||||
import { normalizeDeviceMetadataForPolicy } from "./device-metadata-normalization.js";
|
||||
import type { NodeSession } from "./node-registry.js";
|
||||
|
||||
@@ -221,7 +221,7 @@ function normalizePlatformId(platform?: string, deviceFamily?: string): Platform
|
||||
}
|
||||
|
||||
export function listDangerousPluginNodeCommands(): string[] {
|
||||
const registry = getActiveRuntimePluginRegistry();
|
||||
const registry = getActivePluginGatewayNodePolicyRegistry();
|
||||
if (!registry) {
|
||||
return [];
|
||||
}
|
||||
@@ -237,7 +237,7 @@ export function listDangerousPluginNodeCommands(): string[] {
|
||||
}
|
||||
|
||||
function listDefaultPluginNodeCommands(platformId: PlatformId): string[] {
|
||||
const registry = getActiveRuntimePluginRegistry();
|
||||
const registry = getActivePluginGatewayNodePolicyRegistry();
|
||||
if (!registry) {
|
||||
return [];
|
||||
}
|
||||
@@ -252,7 +252,7 @@ function listDefaultPluginNodeCommands(platformId: PlatformId): string[] {
|
||||
}
|
||||
|
||||
export function isForegroundRestrictedPluginNodeCommand(command: string): boolean {
|
||||
const registry = getActiveRuntimePluginRegistry();
|
||||
const registry = getActivePluginGatewayNodePolicyRegistry();
|
||||
if (!registry) {
|
||||
return false;
|
||||
}
|
||||
|
||||
@@ -1,12 +1,18 @@
|
||||
/**
|
||||
* Node invoke plugin-policy regression tests.
|
||||
*/
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import {
|
||||
MAX_PLUGIN_APPROVAL_TIMEOUT_MS,
|
||||
type PluginApprovalRequestPayload,
|
||||
} from "../infra/plugin-approvals.js";
|
||||
import { createEmptyPluginRegistry } from "../plugins/registry-empty.js";
|
||||
import type { PluginRegistry } from "../plugins/registry-types.js";
|
||||
import {
|
||||
pinActivePluginChannelRegistry,
|
||||
resetPluginRuntimeStateForTest,
|
||||
setActivePluginRegistry,
|
||||
} from "../plugins/runtime.js";
|
||||
import type { OpenClawPluginNodeInvokePolicyContext } from "../plugins/types.js";
|
||||
import { ExecApprovalManager } from "./exec-approval-manager.js";
|
||||
import { applyPluginNodeInvokePolicy } from "./node-invoke-plugin-policy.js";
|
||||
@@ -17,14 +23,6 @@ const DEMO_PLUGIN_ID = "demo";
|
||||
const DEMO_COMMAND = "demo.read";
|
||||
const DEMO_PARAMS = { path: "/tmp/x" };
|
||||
|
||||
const registryState = vi.hoisted(() => ({
|
||||
current: null as PluginRegistry | null,
|
||||
}));
|
||||
|
||||
vi.mock("../plugins/active-runtime-registry.js", () => ({
|
||||
getActiveRuntimePluginRegistry: () => registryState.current,
|
||||
}));
|
||||
|
||||
function createNodeSession(): NodeSession {
|
||||
return {
|
||||
nodeId: "node-1",
|
||||
@@ -133,22 +131,25 @@ function createApprovalRequestPolicy(params?: {
|
||||
}
|
||||
|
||||
function setDangerousDemoCommandRegistry(policies: NodeInvokePolicyRegistration[] = []) {
|
||||
registryState.current = {
|
||||
nodeHostCommands: [
|
||||
{
|
||||
pluginId: DEMO_PLUGIN_ID,
|
||||
command: {
|
||||
command: DEMO_COMMAND,
|
||||
dangerous: true,
|
||||
handle: async () => "{}",
|
||||
},
|
||||
source: "test",
|
||||
},
|
||||
],
|
||||
nodeInvokePolicies: policies,
|
||||
} as unknown as PluginRegistry;
|
||||
const registry = createEmptyPluginRegistry();
|
||||
(registry.nodeHostCommands ??= []).push({
|
||||
pluginId: DEMO_PLUGIN_ID,
|
||||
command: {
|
||||
command: DEMO_COMMAND,
|
||||
dangerous: true,
|
||||
handle: async () => "{}",
|
||||
},
|
||||
source: "test",
|
||||
});
|
||||
(registry.nodeInvokePolicies ??= []).push(...policies);
|
||||
setActivePluginRegistry(registry);
|
||||
}
|
||||
|
||||
function createPolicyRegistry(handle: NodeInvokePolicyHandler): PluginRegistry {
|
||||
const registry = createEmptyPluginRegistry();
|
||||
(registry.nodeInvokePolicies ??= []).push(createDemoPolicy(handle));
|
||||
return registry;
|
||||
}
|
||||
async function invokeDemoPolicy(
|
||||
context: GatewayRequestContext,
|
||||
client: GatewayClient | null = null,
|
||||
@@ -189,7 +190,11 @@ async function expectApprovalResolution(
|
||||
|
||||
describe("applyPluginNodeInvokePolicy", () => {
|
||||
beforeEach(() => {
|
||||
registryState.current = null;
|
||||
resetPluginRuntimeStateForTest();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
resetPluginRuntimeStateForTest();
|
||||
});
|
||||
|
||||
it("fails closed for dangerous plugin node commands without a policy", async () => {
|
||||
@@ -227,6 +232,25 @@ describe("applyPluginNodeInvokePolicy", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("uses a matching policy from the pinned Gateway registry after an active swap", async () => {
|
||||
const gatewayRegistry = createPolicyRegistry((ctx) => ctx.invokeNode());
|
||||
setActivePluginRegistry(gatewayRegistry);
|
||||
pinActivePluginChannelRegistry(gatewayRegistry);
|
||||
setActivePluginRegistry(
|
||||
createPolicyRegistry(async () => ({
|
||||
ok: false,
|
||||
code: "TRANSIENT_POLICY",
|
||||
message: "agent-scoped policy must not shadow Gateway policy",
|
||||
})),
|
||||
);
|
||||
const { context, invoke } = createContext();
|
||||
|
||||
const result = await invokeDemoPolicy(context);
|
||||
|
||||
expect(result).toStrictEqual({ ok: true, payload: { ok: true, value: 1 }, payloadJSON: null });
|
||||
expect(invoke).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it("binds plugin policy approval requests to the invoking client", async () => {
|
||||
const manager = new ExecApprovalManager<PluginApprovalRequestPayload>();
|
||||
const visibleConnIds = new Set(["conn-owner-approval"]);
|
||||
@@ -288,10 +312,7 @@ describe("applyPluginNodeInvokePolicy", () => {
|
||||
});
|
||||
|
||||
it("leaves commands without a dangerous plugin registration to normal allowlist handling", async () => {
|
||||
registryState.current = {
|
||||
nodeHostCommands: [],
|
||||
nodeInvokePolicies: [],
|
||||
} as unknown as PluginRegistry;
|
||||
setActivePluginRegistry(createEmptyPluginRegistry());
|
||||
const { context } = createContext();
|
||||
|
||||
const result = await applyPluginNodeInvokePolicy({
|
||||
|
||||
@@ -4,8 +4,8 @@ import { randomUUID } from "node:crypto";
|
||||
import { normalizeOptionalString } from "@openclaw/normalization-core/string-coerce";
|
||||
import type { PluginApprovalRequestPayload } from "../infra/plugin-approvals.js";
|
||||
import { resolvePluginApprovalTimeoutMs } from "../infra/plugin-approvals.js";
|
||||
import { getActiveRuntimePluginRegistry } from "../plugins/active-runtime-registry.js";
|
||||
import type { PluginRegistry } from "../plugins/registry-types.js";
|
||||
import { getActivePluginGatewayNodePolicyRegistry } from "../plugins/runtime.js";
|
||||
import type {
|
||||
OpenClawPluginNodeInvokePolicyContext,
|
||||
OpenClawPluginNodeInvokePolicyResult,
|
||||
@@ -128,7 +128,7 @@ export async function applyPluginNodeInvokePolicy(params: {
|
||||
timeoutMs?: number;
|
||||
idempotencyKey?: string;
|
||||
}): Promise<OpenClawPluginNodeInvokePolicyResult | null> {
|
||||
const registry = getActiveRuntimePluginRegistry();
|
||||
const registry = getActivePluginGatewayNodePolicyRegistry();
|
||||
const entry = registry?.nodeInvokePolicies?.find((candidate) =>
|
||||
candidate.policy.commands.includes(params.command),
|
||||
);
|
||||
|
||||
@@ -4,12 +4,6 @@ import fs from "node:fs/promises";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { ErrorCodes } from "../../../packages/gateway-protocol/src/index.js";
|
||||
import type { readAcpSessionMeta } from "../../acp/runtime/session-meta.js";
|
||||
import {
|
||||
onDiagnosticEvent,
|
||||
resetDiagnosticEventsForTest,
|
||||
waitForDiagnosticEventsDrained,
|
||||
type DiagnosticEventPayload,
|
||||
} from "../../infra/diagnostic-events.js";
|
||||
import {
|
||||
registerExecApprovalFollowupRuntimeHandoff,
|
||||
resetExecApprovalFollowupRuntimeHandoffsForTests,
|
||||
@@ -20,6 +14,12 @@ import {
|
||||
resetSubagentRegistryForTests,
|
||||
testing as subagentRegistryTesting,
|
||||
} from "../../agents/subagent-registry.js";
|
||||
import {
|
||||
onDiagnosticEvent,
|
||||
resetDiagnosticEventsForTest,
|
||||
waitForDiagnosticEventsDrained,
|
||||
type DiagnosticEventPayload,
|
||||
} from "../../infra/diagnostic-events.js";
|
||||
import {
|
||||
getDetachedTaskLifecycleRuntime,
|
||||
resetDetachedTaskLifecycleRuntimeForTests,
|
||||
@@ -2374,6 +2374,29 @@ describe("gateway agent handler", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("enables Gateway-bound plugin runtimes for ingress agent runs", async () => {
|
||||
primeMainAgentRun({ cfg: mocks.loadConfigReturn });
|
||||
mocks.agentCommand.mockClear();
|
||||
|
||||
await invokeAgent(
|
||||
{
|
||||
message: "plugin runtime check",
|
||||
agentId: "main",
|
||||
sessionKey: "agent:main:main",
|
||||
idempotencyKey: "test-gateway-plugin-runtime-binding",
|
||||
},
|
||||
{
|
||||
reqId: "gateway-plugin-runtime-binding",
|
||||
client: backendGatewayClient(),
|
||||
},
|
||||
);
|
||||
|
||||
expect(
|
||||
(await waitForAgentCommandCall<{ allowGatewaySubagentBinding?: boolean }>())
|
||||
.allowGatewaySubagentBinding,
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("rejects public transcriptMessage overrides", async () => {
|
||||
primeMainAgentRun({ cfg: mocks.loadConfigReturn });
|
||||
mocks.agentCommand.mockClear();
|
||||
|
||||
@@ -77,13 +77,13 @@ import {
|
||||
import { hasProviderOwnedSession } from "../../config/sessions/entry-freshness.js";
|
||||
import { resolveMaintenanceConfigFromInput } from "../../config/sessions/store-maintenance.js";
|
||||
import type { OpenClawConfig } from "../../config/types.openclaw.js";
|
||||
import { emitDiagnosticEvent } from "../../infra/diagnostic-events.js";
|
||||
import {
|
||||
assertAgentRunLifecycleGenerationCurrent,
|
||||
claimAgentRunContext,
|
||||
clearAgentRunContext,
|
||||
getAgentEventLifecycleGeneration,
|
||||
} from "../../infra/agent-events.js";
|
||||
import { emitDiagnosticEvent } from "../../infra/diagnostic-events.js";
|
||||
import { formatUncaughtError, readErrorName } from "../../infra/errors.js";
|
||||
import {
|
||||
resolveAgentDeliveryPlanWithSessionRoute,
|
||||
@@ -2908,6 +2908,9 @@ export const agentHandlers: GatewayRequestHandlers = {
|
||||
spawnedBy: spawnedByValue,
|
||||
sessionEntry,
|
||||
}),
|
||||
// Plugin tools created for Gateway-owned turns must resolve the live
|
||||
// Gateway subagent and node runtimes, not standalone placeholders.
|
||||
allowGatewaySubagentBinding: true,
|
||||
allowModelOverride,
|
||||
},
|
||||
runId,
|
||||
|
||||
@@ -46,6 +46,7 @@ const EXPECTED_BUNDLED_STARTUP_PLUGIN_IDS = [
|
||||
"llm-task",
|
||||
"lobster",
|
||||
"memory-wiki",
|
||||
"ollama",
|
||||
"openshell",
|
||||
"phone-control",
|
||||
"policy",
|
||||
@@ -62,6 +63,7 @@ const EXPECTED_EMPTY_CONFIG_GATEWAY_STARTUP_PLUGIN_IDS = [
|
||||
"device-pair",
|
||||
"file-transfer",
|
||||
"memory-core",
|
||||
"ollama",
|
||||
"phone-control",
|
||||
"talk-voice",
|
||||
] as const;
|
||||
|
||||
@@ -351,6 +351,17 @@ export function getActivePluginGatewayCommandRegistry(): PluginRegistry | null {
|
||||
return pinnedChannelRegistry ?? pinnedHttpRouteRegistry ?? activeRegistry;
|
||||
}
|
||||
|
||||
export function getActivePluginGatewayNodePolicyRegistry(): PluginRegistry | null {
|
||||
// Node allowlists and invoke guards are Gateway security policy. Agent-scoped
|
||||
// registry swaps must not add commands or shadow the pinned startup policy.
|
||||
return (
|
||||
(state.channel.pinned ? asPluginRegistry(state.channel.registry) : null) ??
|
||||
(state.httpRoute.pinned ? asPluginRegistry(state.httpRoute.registry) : null) ??
|
||||
(state.sessionExtension.pinned ? asPluginRegistry(state.sessionExtension.registry) : null) ??
|
||||
asPluginRegistry(state.activeRegistry)
|
||||
);
|
||||
}
|
||||
|
||||
export function requireActivePluginChannelRegistry(): PluginRegistry {
|
||||
const existing = getActivePluginChannelRegistry();
|
||||
if (existing) {
|
||||
|
||||
@@ -15,9 +15,13 @@ vi.mock("../plugins/channel-plugin-ids.js", () => ({
|
||||
resolveConfiguredChannelPluginIdsMock(...args),
|
||||
}));
|
||||
|
||||
vi.mock("../plugins/runtime.js", () => ({
|
||||
getActivePluginRegistry: (...args: unknown[]) => getActivePluginRegistryMock(...args),
|
||||
}));
|
||||
vi.mock("../plugins/runtime.js", async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import("../plugins/runtime.js")>();
|
||||
return {
|
||||
...actual,
|
||||
getActivePluginRegistry: (...args: unknown[]) => getActivePluginRegistryMock(...args),
|
||||
};
|
||||
});
|
||||
|
||||
vi.mock("../plugins/runtime/metadata-registry-loader.js", () => ({
|
||||
loadPluginMetadataRegistrySnapshot: (...args: unknown[]) =>
|
||||
|
||||
Reference in New Issue
Block a user