node-tools is a module beside mesh-tools: the runtime as a bundle, and serve is the console (hq ADR 0175, to-be 38 WP3) #31

Merged
mesh-admin merged 1 commits from feat/wp3-node-tools into main 2026-10-02 19:57:25 +00:00
37 changed files with 181 additions and 72 deletions
+1
View File
@@ -1,2 +1,3 @@
node_modules/ node_modules/
dist/ dist/
.mesh-build/
+25 -16
View File
@@ -1,5 +1,8 @@
ARG NODE_BASE=node:22-bookworm-slim ARG NODE_BASE=node:22-bookworm-slim
# Three stages, two published images: the one modules are COMPILED in, and the one they RUN in. # The mesh-tools module: the two images every TypeScript module is COMPILED in and may RUN in. The
# runtime itself ships as the node-tools module's bundle (node-tools/, novox/hq ADR 0175, to-be 38
# WP3); these images are the toolchain for TypeScript bundles and the base a module's own service
# may still be built on. They are no longer how tools reach a node.
# #
# **They were the same image, and that was a mistake.** A module's recipe starts from this and # **They were the same image, and that was a mistake.** A module's recipe starts from this and
# invokes the compiler out of it, so the compiler had to be here — and because the same image was # invokes the compiler out of it, so the compiler had to be here — and because the same image was
@@ -19,36 +22,42 @@ RUN apt-get update \
&& apt-get install -y --no-install-recommends git ca-certificates \ && apt-get install -y --no-install-recommends git ca-certificates \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
WORKDIR /app WORKDIR /app
COPY package.json ./ COPY node-tools/package.json ./
# The builder writes .npmrc into the build context; it authenticates to the mesh's package registry # The builder writes .npmrc into the build context; it authenticates to the mesh's package registry
# for the @novox scope, which is where @novox/mesh-sdk resolves. This stage is not published, so the # for the @novox scope, which is where @novox/mesh-sdk resolves. This stage is not published, so the
# credential travels no further than here. Development dependencies included: the compiler is one. # credential travels no further than here. Development dependencies included: the compiler is one.
COPY .npmrc ./.npmrc COPY .npmrc ./.npmrc
RUN npm install --no-audit --no-fund RUN npm install --no-audit --no-fund
# ---- toolchain: what a module is compiled in, WITHOUT the credential -------------------------- # ---- compiling: the runtime's own code built, WITHOUT the credential -------------------------
FROM ${NODE_BASE} AS toolchain FROM ${NODE_BASE} AS compiling
WORKDIR /app WORKDIR /app
COPY package.json ./ COPY node-tools/package.json ./
# The resolved libraries, but not the .npmrc that resolved them. # The resolved libraries, but not the .npmrc that resolved them.
COPY --from=deps /app/node_modules ./node_modules COPY --from=deps /app/node_modules ./node_modules
# The toolkit arrives compiled. It used to arrive as sources, and this compiled it by hand — the COPY node-tools/tsconfig.json ./
# hook that builds it on install was running all along, and the result was then packed out of the COPY node-tools/src ./src
# package, because with no explicit file list npm falls back to .gitignore and that ignores the
# build output. Fixed where it belonged, in the toolkit.
COPY tsconfig.json ./
COPY src ./src
RUN npm run build RUN npm run build
# ---- what the running image needs, and nothing else ------------------------------------------- # ---- what a running bundle needs, and nothing else -------------------------------------------
# Its own stage so the toolchain image keeps its build tools while the runtime image does not. # Its own stage so the toolchain image keeps its build tools while the runtime image does not.
FROM toolchain AS lean FROM compiling AS lean
RUN npm prune --omit=dev RUN npm prune --omit=dev
# ---- runtime: what a module runs in ----------------------------------------------------------- # ---- toolchain: what a TypeScript bundle is compiled in ---------------------------------------
# Beside the compiler, at /app/runtime, what every TypeScript bundle runs with: the production
# dependencies the SDK and the runtime need, and a package.json saying the compiled files are ES
# modules. The builder copies this directory whole into a compiled bundle (novox/hq ADR 0188 §5),
# so a bundle unpacked on a machine starts — a `.js` without that package.json is read as
# CommonJS, and an import of `nats` without node_modules beside it resolves to nothing.
FROM compiling AS toolchain
COPY --from=lean /app/node_modules /app/runtime/node_modules
RUN printf '{"type":"module","private":true}\n' > /app/runtime/package.json
# ---- runtime: what a module's own service may run in ------------------------------------------
FROM ${NODE_BASE} AS runtime FROM ${NODE_BASE} AS runtime
WORKDIR /app WORKDIR /app
COPY package.json ./ COPY node-tools/package.json ./
COPY --from=lean /app/node_modules ./node_modules COPY --from=lean /app/node_modules ./node_modules
COPY --from=toolchain /app/dist ./dist COPY --from=compiling /app/dist ./dist
ENTRYPOINT ["node", "dist/main.js"] ENTRYPOINT ["node", "dist/main.js"]
+13 -7
View File
@@ -1,10 +1,15 @@
# mesh-tools # mesh-tools
The Novox Mesh **tool runtime** — the one process per node that makes every assigned module's tools Two modules in one repository (novox/hq ADR 0069), one piece of software:
actually serve (novox/hq ADR 0175).
A module ships its tools as a bundle (built on [`@novox/mesh-sdk`](https://git.novox.be/novox/mesh-sdk)); - **`node-tools`** (`node-tools/`) — the node's **tool runtime** as a module (ADR 0175, to-be 38
this runtime is what loads them and puts them on the mesh. It: WP3): one process per machine the host runs from this bundle, serving every assigned module's tools
and every held seat's verbs on the bus, and answering MCP on the machine's loopback — the console
(design 34). The code, its tests and the `mesh` client all live there.
- **`mesh-tools`** (this directory) — the two images TypeScript bundles are compiled in and a module's
own *service* may still run in. Built from the same code; no longer how tools reach a node.
The runtime:
1. connects the mesh bus on the node's credential — a concrete implementation of the sdk's `Broker` 1. connects the mesh bus on the node's credential — a concrete implementation of the sdk's `Broker`
contract; contract;
@@ -20,8 +25,8 @@ marked executable — is **launched** rather than imported (novox/hq ADR 0188):
as a child with its own environment and speaks MCP over stdio to it, `tools/list` once and as a child with its own environment and speaks MCP over stdio to it, `tools/list` once and
`tools/call` per call. A tool it lists as `<seat>.<verb>` is the seat's implementation. A child that `tools/call` per call. A tool it lists as `<seat>.<verb>` is the seat's implementation. A child that
exits is named in the log and started again on its next call. So a tools bundle may be written in exits is named in the log and started again on its next call. So a tools bundle may be written in
any language; the mesh's SDK for each is the stdio loop and nothing more (`src/launch.ts` is the any language; the mesh's SDK for each is the stdio loop and nothing more (`node-tools/src/launch.ts`
runtime's side of it). is the runtime's side of it).
Everything hard — dispatch, collection, duplicate-name safety — is the sdk's. This is the thin Everything hard — dispatch, collection, duplicate-name safety — is the sdk's. This is the thin
wrapper that binds the bus and loads the modules. Keeping the bus client here, out of the sdk, is wrapper that binds the bus and loads the modules. Keeping the bus client here, out of the sdk, is
@@ -43,7 +48,8 @@ MESH_BROKER_URL a plain URL instead of the credential, for the bootstrap
``` ```
`node dist/main.js`. On a node the controller composes the variables and the host supervises the `node dist/main.js`. On a node the controller composes the variables and the host supervises the
process like any other host-side workload (novox/hq to-be 38). The container (`Dockerfile`) is how process like any other host-side workload (novox/hq to-be 38). As `node-tools` the same process is
the console: MCP on `127.0.0.1:4270` (or `MESH_CONSOLE_LISTEN`). The container (`Dockerfile`) is how
a module's own *service* may still be built; it is no longer how tools reach a node. a module's own *service* may still be built; it is no longer how tools reach a node.
## `mesh` — the tools for whoever is on a machine ## `mesh` — the tools for whoever is on a machine
-48
View File
@@ -1,48 +0,0 @@
import amqp from "amqplib";
const PORT = process.argv[2];
const MPORT = process.argv[3];
const B = `http://127.0.0.1:${MPORT}`;
const AUTH = "Basic " + Buffer.from("guest:guest").toString("base64");
async function api(method, path, body) {
const r = await fetch(B + path, {
method,
headers: { "content-type": "application/json", authorization: AUTH },
body: body ? JSON.stringify(body) : undefined,
});
if (r.status >= 300 && r.status !== 404) throw new Error(`${method} ${path} -> ${r.status}`);
}
await api("PUT", "/api/exchanges/%2f/mesh.events.dead", { type: "topic", durable: true });
await api("PUT", "/api/users/al", { password: "s", tags: "" });
const Q = "anchor.al.events";
const D = "mesh.events.dead";
const q = Q.replace(/\./g, "\\.");
const d = D.replace(/\./g, "\\.");
// configure, write, read patterns per grant on the dead exchange
const combos = {
"none": { configure: `^${q}$`, write: `^${q}$`, read: `^${q}$` },
"read-dead": { configure: `^${q}$`, write: `^${q}$`, read: `^(${q}|${d})$` },
"write-dead": { configure: `^${q}$`, write: `^(${q}|${d})$`, read: `^${q}$` },
"configure-dead": { configure: `^(${q}|${d})$`, write: `^${q}$`, read: `^${q}$` },
"read+write-dead": { configure: `^${q}$`, write: `^(${q}|${d})$`, read: `^(${q}|${d})$` },
};
let i = 0;
for (const [label, perms] of Object.entries(combos)) {
await api("PUT", "/api/permissions/%2f/al", perms);
const queue = `${Q}.${i++}`; // fresh each time
try {
const c = await amqp.connect(`amqp://al:s@127.0.0.1:${PORT}/`);
const ch = await c.createChannel();
ch.on("error", () => {});
await ch.assertQueue(queue, { durable: true, deadLetterExchange: D });
console.log(`${label}: declare-with-DLX OK`);
await c.close();
} catch (e) {
console.log(`${label}: FAIL - ${String(e.message).slice(0, 70)}`);
}
}
+11
View File
@@ -0,0 +1,11 @@
# node-tools
The node's tool runtime as a module (novox/hq ADR 0175, to-be 38 WP3). Assigned to a machine, it is
one process the host runs from this bundle, as the operator's account: it serves every assigned
module's tools and every held seat's verbs on the bus, and answers MCP on the machine's loopback —
the console (design 34). The controller composes the process (which bundles to load, where the
credential is, whose machine it is); this manifest says only what the machine must have for it: the
interpreter, a place for the credential, the loopback port, and leave to call every tool.
The code is the `mesh-tools` package in this directory; the module at the repository root,
`mesh-tools`, builds the images TypeScript bundles are compiled in. See the repository README.
+45
View File
@@ -0,0 +1,45 @@
{
"module": "node-tools",
"version": "1",
"slug": "node-tools",
"invokes": [
"*"
],
"own-secrets": {
"broker": "${dir:mesh-state}/broker"
},
"listens": [
{
"name": "mcp",
"port": 4270,
"protocol": "tcp",
"from": "machine",
"why": "the mesh's tools for whoever is on this machine, over MCP on loopback; the machine's login is the authority (novox/hq ADR 0152, 0175)"
}
],
"resources": [
{
"id": "mesh-state",
"type": "directory",
"mode": "0755",
"place": "mesh"
},
{
"id": "interpreter",
"type": "package",
"package": "nodejs"
}
],
"build": {
"artifacts": [
{
"name": "runtime",
"kind": "bundle",
"language": "typescript",
"entrypoints": [
"src/main.js"
]
}
]
}
}
View File
+23 -1
View File
@@ -1,7 +1,7 @@
// The runnable entrypoint. Three modes: // The runnable entrypoint. Three modes:
// //
// mesh-tools serve — bind the broker and serve the assigned modules until // mesh-tools serve — bind the broker and serve the assigned modules until
// stopped. A module entrypoint that subscribes to events (on("#")) // stopped; as the node-tools module, also the console on loopback. A module entrypoint that subscribes to events (on("#"))
// starts consuming as it is imported, so this also runs consumers. // starts consuming as it is imported, so this also runs consumers.
// mesh-tools emit TYPE [JSON] emit one event onto the mesh and exit — an operable primitive, // mesh-tools emit TYPE [JSON] emit one event onto the mesh and exit — an operable primitive,
// and what an events test uses to put a message on the wire. // and what an events test uses to put a message on the wire.
@@ -30,6 +30,7 @@ import { readFileSync } from "node:fs";
import { pathToFileURL } from "node:url"; import { pathToFileURL } from "node:url";
import { connectNats, fatalBrokerReason as fatalNatsReason, type Credential } from "./broker-nats.js"; import { connectNats, fatalBrokerReason as fatalNatsReason, type Credential } from "./broker-nats.js";
import { runTools, type ServedModule } from "./runtime.js"; import { runTools, type ServedModule } from "./runtime.js";
import { serveMcpHttp, type Listening } from "./http.js";
/** The credential this process connected with, for what it says beyond the connection (ADR 0159). */ /** The credential this process connected with, for what it says beyond the connection (ADR 0159). */
let lastCredential: Credential | undefined; let lastCredential: Credential | undefined;
@@ -150,14 +151,35 @@ export function servedModulesFrom(spec: string, own: string | undefined): { serv
return { serves: [...serves].map(([module, entrypoints]) => ({ module, entrypoints })), moduleEntrypoints }; return { serves: [...serves].map(([module, entrypoints]) => ({ module, entrypoints })), moduleEntrypoints };
} }
/** The module that is the node's tool runtime (novox/hq ADR 0175, to-be 38 WP3): on its credential,
* `serve` is also the console — MCP on the machine's loopback (design 34). */
export const RUNTIME_MODULE = "node-tools";
/** Where the console listens when the runtime is node-tools and nothing says otherwise: the port
* the module's manifest declares `from: machine`. MESH_CONSOLE_LISTEN overrides it either way. */
const CONSOLE_LISTEN = "127.0.0.1:4270";
async function serve(): Promise<void> { async function serve(): Promise<void> {
const broker = await connectBrokerPatiently(); const broker = await connectBrokerPatiently();
// Parsed after connecting: a bare entrypoint belongs to the module the credential names. // Parsed after connecting: a bare entrypoint belongs to the module the credential names.
const { serves, moduleEntrypoints } = servedModulesFrom(process.env.MESH_TOOL_MODULES ?? "", lastCredential?.module); const { serves, moduleEntrypoints } = servedModulesFrom(process.env.MESH_TOOL_MODULES ?? "", lastCredential?.module);
const stop = await runTools({ broker, serves, moduleEntrypoints, credential: lastCredential }); const stop = await runTools({ broker, serves, moduleEntrypoints, credential: lastCredential });
// The console is this runtime's serving mode (ADR 0175 §6): as node-tools, or wherever the
// listen address is given, the same process answers MCP on loopback for whoever is on the
// machine. A module's own runtime in a container on the machine's network does not — two of
// them on one port would be the fault, and the console is one per machine.
const listen = process.env.MESH_CONSOLE_LISTEN ?? (lastCredential?.module === RUNTIME_MODULE ? CONSOLE_LISTEN : "");
let consoleUp: Listening | undefined;
if (listen) {
const who = `${lastCredential?.node ?? "?"}.${lastCredential?.module ?? RUNTIME_MODULE}`;
consoleUp = await serveMcpHttp(broker, who, listen);
console.log(`mesh console listening on http://${consoleUp.address}/mcp as ${who}`);
}
const shutdown = async (): Promise<void> => { const shutdown = async (): Promise<void> => {
stop(); stop();
await consoleUp?.close();
await broker.close(); await broker.close();
process.exit(0); process.exit(0);
}; };
+63
View File
@@ -0,0 +1,63 @@
/**
* The runtime as the node-tools module (novox/hq ADR 0175 §6, to-be 38 WP3): started the way the
* host starts it — `main.js` with the node's credential and MESH_TOOL_MODULES — it serves the bundles
* AND answers MCP on loopback as the console, through which a tool it serves can be called.
*
* docker run -d --rm --name t -p 14232:4222 nats:2.10-alpine -js
* MESH_TEST_NATS=nats://127.0.0.1:14232 node --test --experimental-strip-types test/node-tools-serve.test.ts
*/
import assert from "node:assert/strict";
import { test } from "node:test";
import { spawn, type ChildProcess } from "node:child_process";
import { mkdtemp, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { connectNats } from "../dist/broker-nats.js";
const url = process.env.MESH_TEST_NATS;
const fixture = (name: string) => fileURLToPath(new URL(`./fixtures/${name}`, import.meta.url));
async function post(endpoint: string, body: unknown): Promise<any> {
const res = await fetch(endpoint, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify(body) });
return res.json();
}
test("as node-tools, serve loads the bundles and is the console on loopback", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
// The catalogue, for discovery; the node credential names the runtime module and no claims.
const catalogue = await connectNats({ url, module: "mesh-catalog" });
await catalogue.handle("catalog_modules", async () => ({ modules: [{ module: "alpha" }] }));
t.after(() => catalogue.close());
const dir = await mkdtemp("/tmp/node-tools-");
const credential = join(dir, "broker");
await writeFile(credential, JSON.stringify({ url, node: "desk", module: "node-tools", user: "desk.node-tools", password: "x" }));
const child: ChildProcess = spawn(process.execPath, ["dist/main.js"], {
env: {
...process.env,
MESH_BROKER_FILE: credential,
MESH_TOOL_MODULES: `alpha=${fixture("many-alpha.mjs")}`,
MESH_CONSOLE_LISTEN: "127.0.0.1:0",
},
stdio: ["ignore", "pipe", "pipe"],
});
t.after(() => {
child.kill("SIGTERM");
});
const endpoint = await new Promise<string>((resolve, reject) => {
let out = "";
let err = "";
child.stdout!.on("data", (d) => {
out += d.toString();
const m = /listening on (http:\/\/[^/]+\/mcp) as desk\.node-tools/.exec(out);
if (m) resolve(m[1]!);
});
child.stderr!.on("data", (d) => (err += d.toString()));
child.on("exit", (code) => reject(new Error(`serve exited ${code}: ${err}`)));
});
const listed = await post(endpoint, { jsonrpc: "2.0", id: 1, method: "tools/list" });
assert.deepEqual(listed.result.tools.map((x: any) => x.name).filter((n: string) => n.startsWith("alpha.")), ["alpha.one", "alpha.two"]);
// A tool the same process serves on the bus, called through the console it also is.
const called = await post(endpoint, { jsonrpc: "2.0", id: 2, method: "tools/call", params: { name: "alpha.one", arguments: { node: "desk" } } });
assert.deepEqual(JSON.parse(called.result.content[0].text), { alpha: 1 });
});