node-tools is a module beside mesh-tools: the runtime as a bundle, and serve is the console (hq ADR 0175, to-be 38 WP3)

One repository, two modules (ADR 0069). `node-tools/` holds the runtime — its code, tests, package
and the manifest of the module the controller composes a process for on every machine it is
assigned to: a bundle of `src/main.js`, the interpreter as a package, a place for the node's
credential, the loopback port the console declared, and leave to call every tool. Nothing about
how it runs: which bundles to load, where the credential is and whose machine it is are the
controller's to compose (WP2). The root module `mesh-tools` keeps the two images TypeScript
bundles are compiled in and a module's own service may run in; it is no longer how tools reach a
node.

As node-tools, `serve` is also the console (ADR 0175 §6): the same process answers MCP on
loopback for whoever is on the machine, through which the tools it serves can be called. A
module's own runtime in a container keeps serving without a listener.

The toolchain image now carries /app/runtime — a package.json saying the compiled files are ES
modules and the production node_modules — for the builder to copy into every TypeScript bundle,
so a bundle unpacked on a machine starts (ADR 0188 §5; the builder's side is the controller's).
Proven here by compiling node-tools with the toolchain's exact flags and starting the result.
The AMQP probe script is gone with the bus it probed.
This commit is contained in:
jochen
2026-10-02 21:43:37 +02:00
parent 2746fd31f0
commit c46f9502ee
37 changed files with 181 additions and 72 deletions
+1
View File
@@ -1,2 +1,3 @@
node_modules/
dist/
.mesh-build/
+25 -16
View File
@@ -1,5 +1,8 @@
ARG NODE_BASE=node:22-bookworm-slim
# Three stages, two published images: the one modules are COMPILED in, and the one they RUN in.
# The mesh-tools module: the two images every TypeScript module is COMPILED in and may RUN in. The
# runtime itself ships as the node-tools module's bundle (node-tools/, novox/hq ADR 0175, to-be 38
# WP3); these images are the toolchain for TypeScript bundles and the base a module's own service
# may still be built on. They are no longer how tools reach a node.
#
# **They were the same image, and that was a mistake.** A module's recipe starts from this and
# invokes the compiler out of it, so the compiler had to be here — and because the same image was
@@ -19,36 +22,42 @@ RUN apt-get update \
&& apt-get install -y --no-install-recommends git ca-certificates \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY package.json ./
COPY node-tools/package.json ./
# The builder writes .npmrc into the build context; it authenticates to the mesh's package registry
# for the @novox scope, which is where @novox/mesh-sdk resolves. This stage is not published, so the
# credential travels no further than here. Development dependencies included: the compiler is one.
COPY .npmrc ./.npmrc
RUN npm install --no-audit --no-fund
# ---- toolchain: what a module is compiled in, WITHOUT the credential --------------------------
FROM ${NODE_BASE} AS toolchain
# ---- compiling: the runtime's own code built, WITHOUT the credential -------------------------
FROM ${NODE_BASE} AS compiling
WORKDIR /app
COPY package.json ./
COPY node-tools/package.json ./
# The resolved libraries, but not the .npmrc that resolved them.
COPY --from=deps /app/node_modules ./node_modules
# The toolkit arrives compiled. It used to arrive as sources, and this compiled it by hand — the
# hook that builds it on install was running all along, and the result was then packed out of the
# package, because with no explicit file list npm falls back to .gitignore and that ignores the
# build output. Fixed where it belonged, in the toolkit.
COPY tsconfig.json ./
COPY src ./src
COPY node-tools/tsconfig.json ./
COPY node-tools/src ./src
RUN npm run build
# ---- what the running image needs, and nothing else -------------------------------------------
# ---- what a running bundle needs, and nothing else -------------------------------------------
# Its own stage so the toolchain image keeps its build tools while the runtime image does not.
FROM toolchain AS lean
FROM compiling AS lean
RUN npm prune --omit=dev
# ---- runtime: what a module runs in -----------------------------------------------------------
# ---- toolchain: what a TypeScript bundle is compiled in ---------------------------------------
# Beside the compiler, at /app/runtime, what every TypeScript bundle runs with: the production
# dependencies the SDK and the runtime need, and a package.json saying the compiled files are ES
# modules. The builder copies this directory whole into a compiled bundle (novox/hq ADR 0188 §5),
# so a bundle unpacked on a machine starts — a `.js` without that package.json is read as
# CommonJS, and an import of `nats` without node_modules beside it resolves to nothing.
FROM compiling AS toolchain
COPY --from=lean /app/node_modules /app/runtime/node_modules
RUN printf '{"type":"module","private":true}\n' > /app/runtime/package.json
# ---- runtime: what a module's own service may run in ------------------------------------------
FROM ${NODE_BASE} AS runtime
WORKDIR /app
COPY package.json ./
COPY node-tools/package.json ./
COPY --from=lean /app/node_modules ./node_modules
COPY --from=toolchain /app/dist ./dist
COPY --from=compiling /app/dist ./dist
ENTRYPOINT ["node", "dist/main.js"]
+13 -7
View File
@@ -1,10 +1,15 @@
# mesh-tools
The Novox Mesh **tool runtime** — the one process per node that makes every assigned module's tools
actually serve (novox/hq ADR 0175).
Two modules in one repository (novox/hq ADR 0069), one piece of software:
A module ships its tools as a bundle (built on [`@novox/mesh-sdk`](https://git.novox.be/novox/mesh-sdk));
this runtime is what loads them and puts them on the mesh. It:
- **`node-tools`** (`node-tools/`) — the node's **tool runtime** as a module (ADR 0175, to-be 38
WP3): one process per machine the host runs from this bundle, serving every assigned module's tools
and every held seat's verbs on the bus, and answering MCP on the machine's loopback — the console
(design 34). The code, its tests and the `mesh` client all live there.
- **`mesh-tools`** (this directory) — the two images TypeScript bundles are compiled in and a module's
own *service* may still run in. Built from the same code; no longer how tools reach a node.
The runtime:
1. connects the mesh bus on the node's credential — a concrete implementation of the sdk's `Broker`
contract;
@@ -20,8 +25,8 @@ marked executable — is **launched** rather than imported (novox/hq ADR 0188):
as a child with its own environment and speaks MCP over stdio to it, `tools/list` once and
`tools/call` per call. A tool it lists as `<seat>.<verb>` is the seat's implementation. A child that
exits is named in the log and started again on its next call. So a tools bundle may be written in
any language; the mesh's SDK for each is the stdio loop and nothing more (`src/launch.ts` is the
runtime's side of it).
any language; the mesh's SDK for each is the stdio loop and nothing more (`node-tools/src/launch.ts`
is the runtime's side of it).
Everything hard — dispatch, collection, duplicate-name safety — is the sdk's. This is the thin
wrapper that binds the bus and loads the modules. Keeping the bus client here, out of the sdk, is
@@ -43,7 +48,8 @@ MESH_BROKER_URL a plain URL instead of the credential, for the bootstrap
```
`node dist/main.js`. On a node the controller composes the variables and the host supervises the
process like any other host-side workload (novox/hq to-be 38). The container (`Dockerfile`) is how
process like any other host-side workload (novox/hq to-be 38). As `node-tools` the same process is
the console: MCP on `127.0.0.1:4270` (or `MESH_CONSOLE_LISTEN`). The container (`Dockerfile`) is how
a module's own *service* may still be built; it is no longer how tools reach a node.
## `mesh` — the tools for whoever is on a machine
-48
View File
@@ -1,48 +0,0 @@
import amqp from "amqplib";
const PORT = process.argv[2];
const MPORT = process.argv[3];
const B = `http://127.0.0.1:${MPORT}`;
const AUTH = "Basic " + Buffer.from("guest:guest").toString("base64");
async function api(method, path, body) {
const r = await fetch(B + path, {
method,
headers: { "content-type": "application/json", authorization: AUTH },
body: body ? JSON.stringify(body) : undefined,
});
if (r.status >= 300 && r.status !== 404) throw new Error(`${method} ${path} -> ${r.status}`);
}
await api("PUT", "/api/exchanges/%2f/mesh.events.dead", { type: "topic", durable: true });
await api("PUT", "/api/users/al", { password: "s", tags: "" });
const Q = "anchor.al.events";
const D = "mesh.events.dead";
const q = Q.replace(/\./g, "\\.");
const d = D.replace(/\./g, "\\.");
// configure, write, read patterns per grant on the dead exchange
const combos = {
"none": { configure: `^${q}$`, write: `^${q}$`, read: `^${q}$` },
"read-dead": { configure: `^${q}$`, write: `^${q}$`, read: `^(${q}|${d})$` },
"write-dead": { configure: `^${q}$`, write: `^(${q}|${d})$`, read: `^${q}$` },
"configure-dead": { configure: `^(${q}|${d})$`, write: `^${q}$`, read: `^${q}$` },
"read+write-dead": { configure: `^${q}$`, write: `^(${q}|${d})$`, read: `^(${q}|${d})$` },
};
let i = 0;
for (const [label, perms] of Object.entries(combos)) {
await api("PUT", "/api/permissions/%2f/al", perms);
const queue = `${Q}.${i++}`; // fresh each time
try {
const c = await amqp.connect(`amqp://al:s@127.0.0.1:${PORT}/`);
const ch = await c.createChannel();
ch.on("error", () => {});
await ch.assertQueue(queue, { durable: true, deadLetterExchange: D });
console.log(`${label}: declare-with-DLX OK`);
await c.close();
} catch (e) {
console.log(`${label}: FAIL - ${String(e.message).slice(0, 70)}`);
}
}
+11
View File
@@ -0,0 +1,11 @@
# node-tools
The node's tool runtime as a module (novox/hq ADR 0175, to-be 38 WP3). Assigned to a machine, it is
one process the host runs from this bundle, as the operator's account: it serves every assigned
module's tools and every held seat's verbs on the bus, and answers MCP on the machine's loopback —
the console (design 34). The controller composes the process (which bundles to load, where the
credential is, whose machine it is); this manifest says only what the machine must have for it: the
interpreter, a place for the credential, the loopback port, and leave to call every tool.
The code is the `mesh-tools` package in this directory; the module at the repository root,
`mesh-tools`, builds the images TypeScript bundles are compiled in. See the repository README.
+45
View File
@@ -0,0 +1,45 @@
{
"module": "node-tools",
"version": "1",
"slug": "node-tools",
"invokes": [
"*"
],
"own-secrets": {
"broker": "${dir:mesh-state}/broker"
},
"listens": [
{
"name": "mcp",
"port": 4270,
"protocol": "tcp",
"from": "machine",
"why": "the mesh's tools for whoever is on this machine, over MCP on loopback; the machine's login is the authority (novox/hq ADR 0152, 0175)"
}
],
"resources": [
{
"id": "mesh-state",
"type": "directory",
"mode": "0755",
"place": "mesh"
},
{
"id": "interpreter",
"type": "package",
"package": "nodejs"
}
],
"build": {
"artifacts": [
{
"name": "runtime",
"kind": "bundle",
"language": "typescript",
"entrypoints": [
"src/main.js"
]
}
]
}
}
View File
+23 -1
View File
@@ -1,7 +1,7 @@
// The runnable entrypoint. Three modes:
//
// mesh-tools serve — bind the broker and serve the assigned modules until
// stopped. A module entrypoint that subscribes to events (on("#"))
// stopped; as the node-tools module, also the console on loopback. A module entrypoint that subscribes to events (on("#"))
// starts consuming as it is imported, so this also runs consumers.
// mesh-tools emit TYPE [JSON] emit one event onto the mesh and exit — an operable primitive,
// and what an events test uses to put a message on the wire.
@@ -30,6 +30,7 @@ import { readFileSync } from "node:fs";
import { pathToFileURL } from "node:url";
import { connectNats, fatalBrokerReason as fatalNatsReason, type Credential } from "./broker-nats.js";
import { runTools, type ServedModule } from "./runtime.js";
import { serveMcpHttp, type Listening } from "./http.js";
/** The credential this process connected with, for what it says beyond the connection (ADR 0159). */
let lastCredential: Credential | undefined;
@@ -150,14 +151,35 @@ export function servedModulesFrom(spec: string, own: string | undefined): { serv
return { serves: [...serves].map(([module, entrypoints]) => ({ module, entrypoints })), moduleEntrypoints };
}
/** The module that is the node's tool runtime (novox/hq ADR 0175, to-be 38 WP3): on its credential,
* `serve` is also the console — MCP on the machine's loopback (design 34). */
export const RUNTIME_MODULE = "node-tools";
/** Where the console listens when the runtime is node-tools and nothing says otherwise: the port
* the module's manifest declares `from: machine`. MESH_CONSOLE_LISTEN overrides it either way. */
const CONSOLE_LISTEN = "127.0.0.1:4270";
async function serve(): Promise<void> {
const broker = await connectBrokerPatiently();
// Parsed after connecting: a bare entrypoint belongs to the module the credential names.
const { serves, moduleEntrypoints } = servedModulesFrom(process.env.MESH_TOOL_MODULES ?? "", lastCredential?.module);
const stop = await runTools({ broker, serves, moduleEntrypoints, credential: lastCredential });
// The console is this runtime's serving mode (ADR 0175 §6): as node-tools, or wherever the
// listen address is given, the same process answers MCP on loopback for whoever is on the
// machine. A module's own runtime in a container on the machine's network does not — two of
// them on one port would be the fault, and the console is one per machine.
const listen = process.env.MESH_CONSOLE_LISTEN ?? (lastCredential?.module === RUNTIME_MODULE ? CONSOLE_LISTEN : "");
let consoleUp: Listening | undefined;
if (listen) {
const who = `${lastCredential?.node ?? "?"}.${lastCredential?.module ?? RUNTIME_MODULE}`;
consoleUp = await serveMcpHttp(broker, who, listen);
console.log(`mesh console listening on http://${consoleUp.address}/mcp as ${who}`);
}
const shutdown = async (): Promise<void> => {
stop();
await consoleUp?.close();
await broker.close();
process.exit(0);
};
+63
View File
@@ -0,0 +1,63 @@
/**
* The runtime as the node-tools module (novox/hq ADR 0175 §6, to-be 38 WP3): started the way the
* host starts it — `main.js` with the node's credential and MESH_TOOL_MODULES — it serves the bundles
* AND answers MCP on loopback as the console, through which a tool it serves can be called.
*
* docker run -d --rm --name t -p 14232:4222 nats:2.10-alpine -js
* MESH_TEST_NATS=nats://127.0.0.1:14232 node --test --experimental-strip-types test/node-tools-serve.test.ts
*/
import assert from "node:assert/strict";
import { test } from "node:test";
import { spawn, type ChildProcess } from "node:child_process";
import { mkdtemp, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { connectNats } from "../dist/broker-nats.js";
const url = process.env.MESH_TEST_NATS;
const fixture = (name: string) => fileURLToPath(new URL(`./fixtures/${name}`, import.meta.url));
async function post(endpoint: string, body: unknown): Promise<any> {
const res = await fetch(endpoint, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify(body) });
return res.json();
}
test("as node-tools, serve loads the bundles and is the console on loopback", async (t) => {
if (!url) return t.skip("MESH_TEST_NATS unset");
// The catalogue, for discovery; the node credential names the runtime module and no claims.
const catalogue = await connectNats({ url, module: "mesh-catalog" });
await catalogue.handle("catalog_modules", async () => ({ modules: [{ module: "alpha" }] }));
t.after(() => catalogue.close());
const dir = await mkdtemp("/tmp/node-tools-");
const credential = join(dir, "broker");
await writeFile(credential, JSON.stringify({ url, node: "desk", module: "node-tools", user: "desk.node-tools", password: "x" }));
const child: ChildProcess = spawn(process.execPath, ["dist/main.js"], {
env: {
...process.env,
MESH_BROKER_FILE: credential,
MESH_TOOL_MODULES: `alpha=${fixture("many-alpha.mjs")}`,
MESH_CONSOLE_LISTEN: "127.0.0.1:0",
},
stdio: ["ignore", "pipe", "pipe"],
});
t.after(() => {
child.kill("SIGTERM");
});
const endpoint = await new Promise<string>((resolve, reject) => {
let out = "";
let err = "";
child.stdout!.on("data", (d) => {
out += d.toString();
const m = /listening on (http:\/\/[^/]+\/mcp) as desk\.node-tools/.exec(out);
if (m) resolve(m[1]!);
});
child.stderr!.on("data", (d) => (err += d.toString()));
child.on("exit", (code) => reject(new Error(`serve exited ${code}: ${err}`)));
});
const listed = await post(endpoint, { jsonrpc: "2.0", id: 1, method: "tools/list" });
assert.deepEqual(listed.result.tools.map((x: any) => x.name).filter((n: string) => n.startsWith("alpha.")), ["alpha.one", "alpha.two"]);
// A tool the same process serves on the bus, called through the console it also is.
const called = await post(endpoint, { jsonrpc: "2.0", id: 2, method: "tools/call", params: { name: "alpha.one", arguments: { node: "desk" } } });
assert.deepEqual(JSON.parse(called.result.content[0].text), { alpha: 1 });
});